The Pay Gap Analysis is a House of Cards

What are you going to believe, the statistics or your lying eyes? That 32 percent pay gap between what federal employees make versus their private sector counterparts outlined in the latest Federal Salary Council report is based on a staggeringly complex methodology that cannot be validated.

One of the two Bureau of Labor Statistics surveys used in the annual gap analysis, the Occupational Employment Statistics survey, provide average pay levels across the country for about 800 occupations and approximately 415 industry classifications—and data for 82 million workers—but occupation by occupation, the numbers do not show that feds make 32 percent less than comparable private sector workers.

I did not invest the time to check all 800 occupations but here is a mixed sample of 10 jobs, comparing the average federal salary (from Fedscope) with the OES average.

A partial explanation, which has been important in studies of the Congressional Budget Office, is that federal employees tend to be older and better educated. That affects the averages. It’s likely the OES data includes employees ranging from recent community college grads to PhDs close to retirement.

The OES data includes a number of additional weaknesses as well. Employers provide only a listing of salaries for full and part time employees by occupation; there is no information about what employees actually do. The data fails to include performance-based incentives integral to the compensation of executives, managers and professionals in other sectors; and the averages are based on six separate surveys, each with a changing list of participating employers, conducted over three years. (National data masks locality differentials.)

In a description of the survey, BLS states it does not “encourage the use of OES data for time-series analysis”—that is to say, comparing data from one year to the next! Further, the survey was “not designed for use in comparing federal and private sector pay.”  

The second BLS survey is the National Compensation Survey. Shortly after the Federal Employee Pay Comparability Act was enacted in 1990, BLS started modifying its surveys. BLS reports use the word “experimental” in discussing efforts to improve their methodology. All the changes make any year-to-year comparison suspect. Combining the data from two surveys with a multivariate statistical model was introduced in 2011.

A non-technical description of the methodology used to generate the wage estimates used in the gap analysis is available at the BLS Modeled Wage Estimate Home page. The MWE produces “annual estimates of average hourly wages for occupations by selected job characteristics . . .  including bargaining status (union and nonunion), part- and full-time work status, incentive- and time-based pay, and work levels by occupation.” Note the focus on estimated hourly wages—not what workers actually earn.

Missing, and highly significant, is any information about what employees in the organizations surveyed actually do. Nurses, for example, work in several different roles and settings, each with distinct pay patterns. That’s also true for psychologists, lawyers, accountants, librarians and many other occupations not on the list. Salary planning is based on understanding market pay levels for similar jobs.  

In a technical discussion of the methodology, BLS economists commented, “the regression method does appear to be capable of estimating reasonable-looking pay gaps” with greater precision than does the current approach. [Italics added.] Despite the potential impact of the gap analysis on 2 million federal employees and their careers and billions of dollars, “reasonable-looking” is apparently good enough.  Further, as late as 2015, BLS was not ready to report the usual standard errors used to assess the accuracy of regression estimates, which means BLS cannot validate the accuracy of the results.

The most prominent weakness is that the methodology does not report market pay levels for specific jobs or career stages. The analyses cannot identify federal jobs where noncompetitive salaries are likely to be a barrier for staffing. That’s essential for hiring, as an obvious example, those cybersecurity specialists. In other sectors, that is the primary purpose in analyzing survey data.

The bottom line is that the gap analysis is a house of cards, with estimates stacked upon estimates. Rube Goldberg would be pleased.

It’s interesting that after developing data for the 800 occupations, in the end the data are summarized in “five broad occupational groups”: Professional, Administrative, Technical, Clerical and Officer. And then the gap is reported as a single number so all the supposed accuracy and specificity is forgotten.  

In fairness to BLS, the analyses completed by the Congressional Budget Office and the think tanks also fail to provide useful statistics. Their analyses fail to tell us what, if any, changes in federal pay are needed.  They should know their focus on education and/or years of experience is not relevant in salary planning. Those decisions are based solidly on what an employee is expected to do—their job responsibilities.

It’s also important to highlight the cost of the surveys. This is speculation, since the cost of the two surveys is buried in the BLS budget. However, in an appendix to the 2012 Council report, the estimated cost for full restoration of the National Compensation Survey (budget cuts forced cutbacks) was $9.8 million in additional funding. If that’s the cost of “fixing” the NCS, the overall cost of the surveys has to be at least $20 million.

If a similar market analysis and gap estimate was completed by a consulting firm, my guess is the cost would not exceed $500,000—and if government HR analysts helped, the cost would be less. I am certain the results would be more credible and easily understood.

Equally important, government decision makers need reliable, credible data to plan for salary increases.  At no time since 1990 Federal Employees Pay Comparability Act was enacted—almost 30 years—has a president recommended salary increases to close the gap. After all these years, leaders generally ignore the gap. In June, as a typical rationale, a Senate panel recommended a 1.9 percent increase as “a well-deserved pay adjustment for federal civilian workers.” The logic as reported was the comparison with the White House’s pay recommendation for members of the military, 2.4 percent in 2019. The gap was reported only in passing by the media.

If this is to be the time for civil service reform, replacing the General Schedule and the classification system has to be a priority. It’s essential that employees feel they are paid fairly and agencies can once again compete for qualified talent. It’s also essential for the public to be assured any increases are warranted.