igor.stevanovic / Shutterstock.com

Tracking Individual Feds Over Time Could Improve Employee Viewpoint Survey

Academics look at the methodology of the annual survey gauging the attitudes of federal workers.

This story has been updated.

The annual Federal Employee Viewpoint Survey would be more valuable if it tracked individual respondents over time, gauging how their attitudes toward management and agency leadership changed from one year to the next, according to a new study.

A group of researchers from four universities assessed the strengths and weaknesses of the FEVS from an information-gathering and methodology standpoint. FEVS, which the Office of Personnel Management first administered in 2002, doesn’t clearly link individual respondents’ feedback over time, which would help researchers reduce bias in their causal estimates, the academics said.

“If tracing all the respondents from year to year is impossible, or undesirable, OPM could select a subsample of federal employees and track them over time to create a panel,” the study recommended. OPM could address privacy and confidentiality concerns “through a combination of randomly assigned identification codes” and “restricted data best practices” from agencies like the Education Department, the researchers said.

Linking respondents’ feedback over a period of time in what is known as a “longitudinal survey” also requires consistency in questions asked and sampling methods, the study said. “Unfortunately, there have been instances when OPM has added and deleted questions from one survey to the next, changed the wording of questions, or changed the response categories.” The study, conducted by academics from Indiana University, the University of Southern California, the University of Georgia and Haverford College, was published in March in the Public Administration Review, a professional journal in the field of public administration research, theory and practice.

OPM reviewed the academic paper earlier this month, calling it “thoughtful” and saying it “broadened” the agency’s perspective on its methodology. OPM noted that it redesigned its sampling strategy in 2013 to improve representation and enable HR managers to receive employee feedback. In addition to the annual FEVS, the agency has developed online tools to allow managers to do customized analyses of the data.

Kimya Lee, who is OPM’s technical expert on research and evaluation, responded to the March paper with her own paper published in April in PAR. She said the suggestion to trace individual respondents over time was “a good suggestion that may be possible with some careful thought and planning,” offering possible statistical scenarios of how it that could be accomplished. But she also said “extreme caution is needed as we discuss the possibility of linking survey results to other administrative data. Although it may further our research endeavors, it may be to the detriment of the survey program, specifically if employees feel that their responses are no longer confidential.”

The academics studied FEVS because so many researchers have used the survey in separate work exploring public management and governance. FEVS is a source of data “for empirical studies of public management and public organizations,” the publication said. Agencies, lawmakers and other stakeholders have increasingly looked to FEVS to gauge the morale, commitment, and job satisfaction of the federal workforce.

The researchers praised OPM for continuing to increase participation from the federal workforce, in addition to including a broad range of questions, exploring different areas such as employee empowerment, diversity management, job satisfaction and leadership styles. But casting such a wide net when it comes to topics has also limited the survey’s ability to dig deeper, the study said.

“We noted that the wide range of phenomena captured by the FEVS is one of the strengths of the survey,” the study said. “However, it appears a trade-off was made between breadth and quality of measurement. Many concepts are measured in the FEVS using just one or a handful of items, making it difficult for researchers to show that their measurement approach captures all or most of the key dimensions that make up a concept’s content.”

The study recommended that OPM create some more tailored questions that reveal, for example, the specific actions agencies take to increase diversity and how greater diversity affects job satisfaction and respondents’ perceptions of work quality.

The academics also suggested that OPM refine the wording of some questions, avoiding “double-barreled” constructions that end up measuring two separate ideas simultaneously, like motivation and commitment. Using “strongly agree” and “strongly disagree” as response options in surveys also injects too much ambiguity, the publication said. “When a respondent is asked to provide his or her level of agreement with a statement such as, ‘Managers communicate the goals and priorities of the organization,’ is a ‘strongly agree’ response an indication that the respondent believes this behavior to be extremely important (intensity) or that the respondent is highly confident in his or her manager’s respective behavior in this area (strong position)?” 

The study urged OPM, the Government Accountability Office and other stakeholders to put together a working group that includes researchers who can help improve the design and implementation of FEVS.

Lee said in her response that OPM was “committed” to working with the research community inside and outside government. “The utility of the survey grows at a seemingly exponential pace within the milieu of the federal government, and with this article and associated commentary, we hope to witness an equally expansive growth in academic capacity.”

(Image via  / Shutterstock.com)

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.