Jennifer Vinciguerra / Shutterstock.com

‘Quality Review’ Isn’t Fixing Veterans’ Claims System

Public reports have understated due process mistakes and legal errors in the veterans' appeals process, research finds.

A new study turns the spotlight on governance issues that have plagued a cornerstone of the nation’s administrative system for years: rampant errors and a backlog of appeals cases involving veterans’ benefits.

The volume of veterans’ appeals—the vast majority of which relate to disability compensation claims—is huge. Some 90 judges in the Board of Veterans’ Appeals (BVA) historically decided about 50,000 cases per year, with an inventory of over 425,000 cases pending.

It takes an average of seven years for a veteran’s disputed claim to get resolved. In fact, the Inspector General’s Office at the Department of Veterans Affairs estimated that in one quarter of 2016, 7 percent of cases were deemed “resolved” at the Veterans Benefits Administration because the veterans had died while waiting.

A 'Shocking' Finding

Part of the problem, the study suggests, stems from a mismanaged trade-off between quantity and quality. Thousands of cases that should not necessarily go up the chain of appeal end up on appeal. That prolongs the wait for decisions and increases the backlog of cases. And a program meant for quality review that was intended to reduce the rate of appeals and erroneous decisions largely failed to do so, the study finds.

“What was shocking about our findings is that the quality review program had none of its intended effects to reduce errors,” says Daniel Ho, professor of law at Stanford University and a senior fellow at the Stanford Institute for Economic Policy Research (SIEPR).

“When a veteran challenges the denial of disability benefits, Supreme Court cases require accurate decision-making as a matter of due process,” Ho says. “Based on our study, it’s hard to believe the BVA is meeting that goal of accuracy. Or it is overstating it in pretty dramatic terms.”

A Constitutional Problem?

The stakes go beyond veterans’ lives, too.

The study raises broader questions about government oversight and the value of these kinds of review systems that have become a management linchpin for federal agencies, Ho says. Under the Government Performance and Results Act of 1993, federal agencies are required to provide performance measures and many agencies have instituted quality assurance systems to improve and measure quality of service delivery. The deficiency of BVA opinions may even rise to the level of a constitutional problem, the authors note.

“Our results paint a sobering picture about the ability for an agency to internally develop such quality assurance initiatives,” the study states. And the BVA’s quality review program, as it is structured today, will “unlikely address the longstanding quality problems in (veterans’) adjudication.”

The working paper on the findings will appear in a future issue of the Journal of Law, Economics, and Organization and a companion paper on the more general crisis in mass adjudication is forthcoming in the Stanford Law Review.

This research is the first to rigorously examine the effectiveness of quality assurance systems, long espoused by scholars and policymakers, using data on nearly 600,000 veterans’ appeals cases from 2003 to 2016 that had never been accessible before by outside researchers. Ho and colleagues also interviewed a wide range of officials and secured a rich set of internal records using the Freedom of Information Act.

Ho’s coauthors are David Ames, former chief of the Office of Quality Review at the US Department of Veterans Affairs; David Marcus, a law professor at the University of California, Los Angeles; and Cassandra Handan-Nader, a doctoral candidate in political science at Stanford.

Ames developed concerns about the effectiveness of quality review during his time overseeing the process. “I found it increasingly difficult to shake the suspicion that our work was not benefiting veterans,” he says.

Quality Review

At issue, the researchers say, are decision errors, including those based on legally inadequate explanations, inaccurate documentation, and due process mistakes. Their analysis also shows evidence of discrepancies, or inconsistent judgments between similar cases. The BVA established its quality review program in 1998 to try to fix some of these problems, but with no substantial effect toward that end, they found.

Under the program, 5 percent of appeals cases were randomly selected to undergo an additional layer of review by an elite slate of staff attorneys to detect and correct any errors before a decision moved forward. Random selection enabled researchers to test whether such review in fact reduced subsequent appeals and remands—the rate at which disputed BVA decisions are sent back to the agency for further review.

Yet the appeals rate and remand rates remained indistinguishable—despite the quality review efforts that were supposed to catch and deter errors to begin with.

“The caseload makes it difficult to guarantee no errors, but intensive review by an elite set of attorneys to correct errors had little effect,” Ho says.

False Stats?

The reasons for the ineffectiveness, Ho explains, are the cross-purposes of quality review. The same agency charged with its own quality review faces a competing interest in trying to keep case numbers and accuracy high for its performance measures, which in turn affect its funding allocations. For over a decade, the BVA has published and touted its “accuracy rate” to Congress and the public as being between 91 and 95 percent.

“…THE QUALITY REVIEW PROCESS WAS USED TO GENERATE THE APPEARANCE OF EFFECTIVENESS [RATHER] THAN TO ACTUALLY IMPROVE PERFORMANCE.”

In their analysis, however, researchers found that BVA deployed an extremely deferential way of counting errors, inflating the agency’s measure of accuracy. When the quality review team deemed the decision error-free and the case was appealed further, it was still remanded—sent back to the agency—nearly three-fourths of the time.

“It is well-known in the social science literature that creating your own performance measures poses conflicts of interest,” Ho says. “We found that over time, the quality review process was used to generate the appearance of effectiveness [rather] than to actually improve performance.”

The findings also bolster criticisms that the VA’s Office of General Counsel and others have raised in public records and internal documents, according to the study.

Back in 2010, the general counsel questioned, for instance, the BVA’s reported accuracy rates, in light of high remand rates. And some 100 staff attorneys submitted a loss-of-confidence statement to congressional committees in 2017, contending how the BVA’s increased production quota, “gross mismanagement,” and inadequate training have failed to deliver accurate decisions to veterans.

Despite increasing the output to 85,000 cases over the last year, the BVA continues to tout a 94 percent accuracy rate.

The study concludes that this accuracy statistic is inaccurate.

“We were wasting some of the Board’s most talented attorneys on producing an essentially arbitrary number that glossed over quality problems, when those same attorneys could have been proactively working to reduce errors and inefficiencies,” Ames says.

Source: May Wong for Stanford University

This article was originally published in Futurity. Edits have been made to this republication. It has been republished under the Attribution 4.0 International license.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.