Promising Practices Promising PracticesPromising Practices
A forum for government's best ideas and most innovative leaders.

How Do You Measure Performance You Can’t See?

ARCHIVES
Image via Novelo/Shutterstock.com

Law enforcement has it tough when it comes time to measure their performance. Their effectiveness can’t be just measured on what is reported. After all, the goal of most law violators is to not be caught! So how can we measure what we can’t observe?

Cracking the challenge of measuring unobserved behaviors – drug smuggling, tax fraud, counterfeiting, and illegal immigration – is key to developing better strategies and targeting resources in the right places to catch and deter illegal actions.

Interestingly, observes Dr. John Whitley in a new IBM Center report, “the local law enforcement community has been a pioneer in measuring and reporting performance and in using these data to drive strategy development and manage execution.” He points to successes in places like New York City.

But he also observes that “many areas of federal law enforcement do not systematically collect, use, or report basic data on crime rates within their jurisdictions.” This is in part because federal crimes – like tax evasion or drug smuggling – are less likely to be reported than local crimes – like muggings or break-ins.

Dr. Whitley offers a useful tutorial about measurement and statistics. He says that “when outcomes are hard to measure, proxy variables are often used,” such as the number of arrests for various crimes. He notes that “arrests are an important element in the projection of law enforcement that should be measured, but they are not the outcome.” In support, he quotes Washington, DC police chief Cathy Lanier: “Arresting people is not a measure of success. Less crime is a measure of success.”

Five Methods for Estimating Unobserved Events

Dr. Whitley says that, in order to “effectively manage federal law enforcement activities, officials and policy-makers in charge must have an idea of what is happening.” He describes five statistical methods for estimating data on unobserved events:

  • Method 1: Use Administrative Records. Agencies collect reams of administrative data and this should be the first place for analysts to look. For example, the US Border Patrol collects information of all apprehensions of illegal border-crossers. Analyzing how many of these people try to cross again can provide insight into which deterrence strategies work better than others.
  • Method 2: Use Surveys. Surveys can be costly, especially if they have to be statistically valid for small geographic areas. And surveys can be suspect, especially if there are concerns about the truthfulness of respondents. But using existing surveys, such as the National Crime Victimization Survey, can be a cost-effective way to understand how to understand and measure potential undercounting.
  • Method 3: Inspections, Investigations, and Audits. Typically, law enforcement prioritizes its investigations. But if this approach is to be used to estimate the impact of law enforcement, it would need to be done randomly in order to be statistically valid. This would not make sense for the most part, but if employed in a limited way, it could provide valuable data. A good example is the IRS’s National Research Program which does annual samples of 13,000 taxpayers and is used to estimate the size of the tax gap – the amount of taxes not being paid.
  • Method 4: Experimental Methods. Adding or modifying routine law enforcement activities can help estimate crime rates, as well. For example, Customs and Border Protection conducts random secondary screening of passengers at ports of entry (e.g., the border or airport) to evaluate the effectiveness of primary screening processes.
  • Method 5: Technical Measurement. Well-known methods of technical measurement include red-light and speeding cameras. The Secret Service also uses technical measures to identify counterfeit money

Conclusions

Dr. Whitley cautions that getting the measures right is critical. If the wrong measures are selected, then it could result in decision-makers making bad decisions or wasting money. But he says the law enforcement community has some inherent advantages because their performance management systems are “relatively straightforward,” even though technical challenges exist because “outcomes are often unobserved and cannot be directly measured.” But if approached thoughtfully, “it is possible to bring the radical reforms that have been seen in state and local law enforcement to the federal level.”

(Image via Novelo/Shutterstock.com)

John M. Kamensky is a Senior Research Fellow for the IBM Center for the Business of Government. He previously served as deputy director of Vice President Gore's National Partnership for Reinventing Government, a special assistant at the Office of Management and Budget, and as an assistant director at the Government Accountability Office. He is a fellow of the National Academy of Public Administration and received a Masters in Public Affairs from the Lyndon B. Johnson School of Public Affairs at the University of Texas at Austin.

FROM OUR SPONSORS
JOIN THE DISCUSSION
Close [ x ] More from GovExec
 
 

Thank you for subscribing to newsletters from GovExec.com.
We think these reports might interest you:

  • Sponsored by G Suite

    Cross-Agency Teamwork, Anytime and Anywhere

    Dan McCrae, director of IT service delivery division, National Oceanic and Atmospheric Administration (NOAA)

    Download
  • Data-Centric Security vs. Database-Level Security

    Database-level encryption had its origins in the 1990s and early 2000s in response to very basic risks which largely revolved around the theft of servers, backup tapes and other physical-layer assets. As noted in Verizon’s 2014, Data Breach Investigations Report (DBIR)1, threats today are far more advanced and dangerous.

    Download
  • Sponsored by One Identity

    One Nation Under Guard: Securing User Identities Across State and Local Government

    In 2016, the government can expect even more sophisticated threats on the horizon, making it all the more imperative that agencies enforce proper identity and access management (IAM) practices. In order to better measure the current state of IAM at the state and local level, Government Business Council (GBC) conducted an in-depth research study of state and local employees.

    Download
  • Sponsored by Aquilent

    The Next Federal Evolution of Cloud

    This GBC report explains the evolution of cloud computing in federal government, and provides an outlook for the future of the cloud in government IT.

    Download
  • Sponsored by LTC Partners, administrators of the Federal Long Term Care Insurance Program

    Approaching the Brink of Federal Retirement

    Approximately 10,000 baby boomers are reaching retirement age per day, and a growing number of federal employees are preparing themselves for the next chapter of their lives. Learn how to tackle the challenges that today's workforce faces in laying the groundwork for a smooth and secure retirement.

    Download
  • Sponsored by Hewlett Packard Enterprise

    Cyber Defense 101: Arming the Next Generation of Government Employees

    Read this issue brief to learn about the sector's most potent challenges in the new cyber landscape and how government organizations are building a robust, threat-aware infrastructure

    Download
  • Sponsored by Aquilent

    GBC Issue Brief: Cultivating Digital Services in the Federal Landscape

    Read this GBC issue brief to learn more about the current state of digital services in the government, and how key players are pushing enhancements towards a user-centric approach.

    Download

When you download a report, your information may be shared with the underwriters of that document.