Promising Practices Promising PracticesPromising Practices
A forum for government's best ideas and most innovative leaders.

How Do You Measure Performance You Can’t See?

ARCHIVES
Image via Novelo/Shutterstock.com

Law enforcement has it tough when it comes time to measure their performance. Their effectiveness can’t be just measured on what is reported. After all, the goal of most law violators is to not be caught! So how can we measure what we can’t observe?

Cracking the challenge of measuring unobserved behaviors – drug smuggling, tax fraud, counterfeiting, and illegal immigration – is key to developing better strategies and targeting resources in the right places to catch and deter illegal actions.

Interestingly, observes Dr. John Whitley in a new IBM Center report, “the local law enforcement community has been a pioneer in measuring and reporting performance and in using these data to drive strategy development and manage execution.” He points to successes in places like New York City.

But he also observes that “many areas of federal law enforcement do not systematically collect, use, or report basic data on crime rates within their jurisdictions.” This is in part because federal crimes – like tax evasion or drug smuggling – are less likely to be reported than local crimes – like muggings or break-ins.

Dr. Whitley offers a useful tutorial about measurement and statistics. He says that “when outcomes are hard to measure, proxy variables are often used,” such as the number of arrests for various crimes. He notes that “arrests are an important element in the projection of law enforcement that should be measured, but they are not the outcome.” In support, he quotes Washington, DC police chief Cathy Lanier: “Arresting people is not a measure of success. Less crime is a measure of success.”

Five Methods for Estimating Unobserved Events

Dr. Whitley says that, in order to “effectively manage federal law enforcement activities, officials and policy-makers in charge must have an idea of what is happening.” He describes five statistical methods for estimating data on unobserved events:

  • Method 1: Use Administrative Records. Agencies collect reams of administrative data and this should be the first place for analysts to look. For example, the US Border Patrol collects information of all apprehensions of illegal border-crossers. Analyzing how many of these people try to cross again can provide insight into which deterrence strategies work better than others.
  • Method 2: Use Surveys. Surveys can be costly, especially if they have to be statistically valid for small geographic areas. And surveys can be suspect, especially if there are concerns about the truthfulness of respondents. But using existing surveys, such as the National Crime Victimization Survey, can be a cost-effective way to understand how to understand and measure potential undercounting.
  • Method 3: Inspections, Investigations, and Audits. Typically, law enforcement prioritizes its investigations. But if this approach is to be used to estimate the impact of law enforcement, it would need to be done randomly in order to be statistically valid. This would not make sense for the most part, but if employed in a limited way, it could provide valuable data. A good example is the IRS’s National Research Program which does annual samples of 13,000 taxpayers and is used to estimate the size of the tax gap – the amount of taxes not being paid.
  • Method 4: Experimental Methods. Adding or modifying routine law enforcement activities can help estimate crime rates, as well. For example, Customs and Border Protection conducts random secondary screening of passengers at ports of entry (e.g., the border or airport) to evaluate the effectiveness of primary screening processes.
  • Method 5: Technical Measurement. Well-known methods of technical measurement include red-light and speeding cameras. The Secret Service also uses technical measures to identify counterfeit money

Conclusions

Dr. Whitley cautions that getting the measures right is critical. If the wrong measures are selected, then it could result in decision-makers making bad decisions or wasting money. But he says the law enforcement community has some inherent advantages because their performance management systems are “relatively straightforward,” even though technical challenges exist because “outcomes are often unobserved and cannot be directly measured.” But if approached thoughtfully, “it is possible to bring the radical reforms that have been seen in state and local law enforcement to the federal level.”

(Image via Novelo/Shutterstock.com)

John M. Kamensky is a Senior Research Fellow for the IBM Center for the Business of Government. He previously served as deputy director of Vice President Gore's National Partnership for Reinventing Government, a special assistant at the Office of Management and Budget, and as an assistant director at the Government Accountability Office. He is a fellow of the National Academy of Public Administration and received a Masters in Public Affairs from the Lyndon B. Johnson School of Public Affairs at the University of Texas at Austin.

FROM OUR SPONSORS
JOIN THE DISCUSSION
Close [ x ] More from GovExec
 
 

Thank you for subscribing to newsletters from GovExec.com.
We think these reports might interest you:

  • Going Agile:Revolutionizing Federal Digital Services Delivery

    Here’s one indication that times have changed: Harriet Tubman is going to be the next face of the twenty dollar bill. Another sign of change? The way in which the federal government arrived at that decision.

    View
  • Cyber Risk Report: Cybercrime Trends from 2016

    In our first half 2016 cyber trends report, SurfWatch Labs threat intelligence analysts noted one key theme – the interconnected nature of cybercrime – and the second half of the year saw organizations continuing to struggle with that reality. The number of potential cyber threats, the pool of already compromised information, and the ease of finding increasingly sophisticated cybercriminal tools continued to snowball throughout the year.

    View
  • Featured Content from RSA Conference: Dissed by NIST

    Learn more about the latest draft of the U.S. National Institute of Standards and Technology guidance document on authentication and lifecycle management.

    View
  • GBC Issue Brief: The Future of 9-1-1

    A Look Into the Next Generation of Emergency Services

    View
  • GBC Survey Report: Securing the Perimeters

    A candid survey on cybersecurity in state and local governments

    View
  • The New IP: Moving Government Agencies Toward the Network of The Future

    Federal IT managers are looking to modernize legacy network infrastructures that are taxed by growing demands from mobile devices, video, vast amounts of data, and more. This issue brief discusses the federal government network landscape, as well as market, financial force drivers for network modernization.

    View
  • eBook: State & Local Cybersecurity

    CenturyLink is committed to helping state and local governments meet their cybersecurity challenges. Towards that end, CenturyLink commissioned a study from the Government Business Council that looked at the perceptions, attitudes and experiences of state and local leaders around the cybersecurity issue. The results were surprising in a number of ways. Learn more about their findings and the ways in which state and local governments can combat cybersecurity threats with this eBook.

    View

When you download a report, your information may be shared with the underwriters of that document.