Promising Practices Promising PracticesPromising Practices
A forum for government's best ideas and most innovative leaders.

How Do You Measure Performance You Can’t See?

Image via Novelo/

Law enforcement has it tough when it comes time to measure their performance. Their effectiveness can’t be just measured on what is reported. After all, the goal of most law violators is to not be caught! So how can we measure what we can’t observe?

Cracking the challenge of measuring unobserved behaviors – drug smuggling, tax fraud, counterfeiting, and illegal immigration – is key to developing better strategies and targeting resources in the right places to catch and deter illegal actions.

Interestingly, observes Dr. John Whitley in a new IBM Center report, “the local law enforcement community has been a pioneer in measuring and reporting performance and in using these data to drive strategy development and manage execution.” He points to successes in places like New York City.

But he also observes that “many areas of federal law enforcement do not systematically collect, use, or report basic data on crime rates within their jurisdictions.” This is in part because federal crimes – like tax evasion or drug smuggling – are less likely to be reported than local crimes – like muggings or break-ins.

Dr. Whitley offers a useful tutorial about measurement and statistics. He says that “when outcomes are hard to measure, proxy variables are often used,” such as the number of arrests for various crimes. He notes that “arrests are an important element in the projection of law enforcement that should be measured, but they are not the outcome.” In support, he quotes Washington, DC police chief Cathy Lanier: “Arresting people is not a measure of success. Less crime is a measure of success.”

Five Methods for Estimating Unobserved Events

Dr. Whitley says that, in order to “effectively manage federal law enforcement activities, officials and policy-makers in charge must have an idea of what is happening.” He describes five statistical methods for estimating data on unobserved events:

  • Method 1: Use Administrative Records. Agencies collect reams of administrative data and this should be the first place for analysts to look. For example, the US Border Patrol collects information of all apprehensions of illegal border-crossers. Analyzing how many of these people try to cross again can provide insight into which deterrence strategies work better than others.
  • Method 2: Use Surveys. Surveys can be costly, especially if they have to be statistically valid for small geographic areas. And surveys can be suspect, especially if there are concerns about the truthfulness of respondents. But using existing surveys, such as the National Crime Victimization Survey, can be a cost-effective way to understand how to understand and measure potential undercounting.
  • Method 3: Inspections, Investigations, and Audits. Typically, law enforcement prioritizes its investigations. But if this approach is to be used to estimate the impact of law enforcement, it would need to be done randomly in order to be statistically valid. This would not make sense for the most part, but if employed in a limited way, it could provide valuable data. A good example is the IRS’s National Research Program which does annual samples of 13,000 taxpayers and is used to estimate the size of the tax gap – the amount of taxes not being paid.
  • Method 4: Experimental Methods. Adding or modifying routine law enforcement activities can help estimate crime rates, as well. For example, Customs and Border Protection conducts random secondary screening of passengers at ports of entry (e.g., the border or airport) to evaluate the effectiveness of primary screening processes.
  • Method 5: Technical Measurement. Well-known methods of technical measurement include red-light and speeding cameras. The Secret Service also uses technical measures to identify counterfeit money


Dr. Whitley cautions that getting the measures right is critical. If the wrong measures are selected, then it could result in decision-makers making bad decisions or wasting money. But he says the law enforcement community has some inherent advantages because their performance management systems are “relatively straightforward,” even though technical challenges exist because “outcomes are often unobserved and cannot be directly measured.” But if approached thoughtfully, “it is possible to bring the radical reforms that have been seen in state and local law enforcement to the federal level.”

(Image via Novelo/

John M. Kamensky is a Senior Research Fellow for the IBM Center for the Business of Government. He previously served as deputy director of Vice President Gore's National Partnership for Reinventing Government, a special assistant at the Office of Management and Budget, and as an assistant director at the Government Accountability Office. He is a fellow of the National Academy of Public Administration and received a Masters in Public Affairs from the Lyndon B. Johnson School of Public Affairs at the University of Texas at Austin.

Close [ x ] More from GovExec

Thank you for subscribing to newsletters from
We think these reports might interest you:

  • Sponsored by G Suite

    Cross-Agency Teamwork, Anytime and Anywhere

    Dan McCrae, director of IT service delivery division, National Oceanic and Atmospheric Administration (NOAA)

  • Data-Centric Security vs. Database-Level Security

    Database-level encryption had its origins in the 1990s and early 2000s in response to very basic risks which largely revolved around the theft of servers, backup tapes and other physical-layer assets. As noted in Verizon’s 2014, Data Breach Investigations Report (DBIR)1, threats today are far more advanced and dangerous.

  • Federal IT Applications: Assessing Government's Core Drivers

    In order to better understand the current state of external and internal-facing agency workplace applications, Government Business Council (GBC) and Riverbed undertook an in-depth research study of federal employees. Overall, survey findings indicate that federal IT applications still face a gamut of challenges with regard to quality, reliability, and performance management.

  • PIV- I And Multifactor Authentication: The Best Defense for Federal Government Contractors

    This white paper explores NIST SP 800-171 and why compliance is critical to federal government contractors, especially those that work with the Department of Defense, as well as how leveraging PIV-I credentialing with multifactor authentication can be used as a defense against cyberattacks

  • Toward A More Innovative Government

    This research study aims to understand how state and local leaders regard their agency’s innovation efforts and what they are doing to overcome the challenges they face in successfully implementing these efforts.

  • From Volume to Value: UK’s NHS Digital Provides U.S. Healthcare Agencies A Roadmap For Value-Based Payment Models

    The U.S. healthcare industry is rapidly moving away from traditional fee-for-service models and towards value-based purchasing that reimburses physicians for quality of care in place of frequency of care.

  • GBC Flash Poll: Is Your Agency Safe?

    Federal leaders weigh in on the state of information security


When you download a report, your information may be shared with the underwriters of that document.