Data Compatibility Issues: How to Prevent Miscoding and Dropped Observations When Using U.S. Office of Personnel Management Data Sets

Abstract

A critical comparison of the agency identifier codes in the Federal Employee Viewpoint Survey (FEVS) and FedScope data sets reveals three distinct types of issues will occur when researchers attempt to merge the data sets: (a) a single agency is assigned different codes across data sets; (b) a single code is assigned to different agencies across data sets; and (c) a single code is assigned to two or more agencies in the FEVS data set and a separate agency in the FedScope data set. Between 2013 and 2016, these issues are present in almost all major federal departments. Compatibility issues between the agency identifiers could cause the user to drop observations unnecessarily or unknowingly combine two different agencies’ data improperly. If uncorrected, these issues will distort the analysis of studies that rely on this combination of data. However, researchers can correct for this issue and still use Office of Personnel Management (OPM) identifiers to combine data across multiple data sets.

Keywords

data issues FedScope Federal Employee Viewpoint Survey survey research panel data

The U.S. Office of Personnel Management (OPM) is one of the key sources of data on human resources and performance-related metrics for federal employees. In addition to serving as the human resources and policy guide for the U.S. federal government, OPM also routinely collects demographic and survey data on all federal employees. These data are invaluable for both researchers studying federal issues and for practitioners working to provide data-driven responses to issues impacting their organization. However, a critical comparison of the agency identifier codes in the Federal Employee Viewpoint Survey (FEVS) and FedScope data sets reveals that three distinct types of issues will occur when researchers attempt to merge the data sets:

An agency is assigned one agency identifier code in the FEVS data set and a different code in the FedScope data set;

The same agency identifier code is used for one agency in the FEVS data set and another agency in the FedScope data set; and

A single agency identifier code is assigned to two or more agencies in the FEVS data set and a different agency in the FedScope data set.

If these issues are not corrected, this will result in observations for multiple agencies being dropped from analysis because not all variables have a value assigned. It will also result in the researcher inadvertently combining data from two different agencies into a single observation. If uncorrected, these issues represent a substantial threat to the validity of research produced when researchers combine the FEVS and FedScope data sets.

Despite the fact that the FEVS is used as a data source in dozens of peer-reviewed articles and other publications, until Fernandez et al. (2015), Somers (2018), and Resh et al. (2019) articles, few have critically evaluated this data set. In their article, Fernandez et al. (2015) state that one of the “advantage[s] of using the FEVS data is that respondents are coded by agency in a manner that allows researchers to easily merge the survey data with many other sources in the federal government” (p. 388). This is enabled because “[e]ach survey response is coded according to the federal government agency in which he or she works . . . [and] this agency coding scheme, or ones very much like it, are used regularly by federal agencies such as OPM (e.g., FedScope) . . .” and others “to gather, analyze, and report statistics” (Fernandez et al., 2015, p. 388). However, as this article illustrates, although the data sets appear to use the same coding scheme, the codes are not the same between the FEVS and FedScope data sets.

The FEVS data has most commonly been used by public management researchers to conduct cross-sectional research. However, “[m]any critiques in the current public management literature have been levied against cross-sectional survey research” (Stritch, 2017, p. 221). Recently, researchers have begun to think more critically about some of issues inherent in using cross-sectional data and develop approaches to address the potential for introducing threats to their findings from both common methods (Jakobsen & Jensen, 2015) and common source bias (Favero & Bullock, 2015; Meier & O’Toole, 2013). However, these do not address the fundamental issue that “[r]esearchers cannot use static data to directly test dynamic theories” (Stritch, 2017, p. 222, citing Chan, 1998). Given this, recently increased attention has been devoted to longitudinal and panel data because both may allow researchers to identify change and causal relationships (for a discussion of additional issues to be aware of when using longitudinal and panel data, see Stritch, 2017).

Researchers can use the FEVS to conduct panel data research using OPM’s survey weights to aggregate the individual employee responses by organizational unit and analyze these units over several years. The questions that can be explored using the FEVS data are further expanded when researchers combine this data set with other information such as the data provided by FedScope (e.g., Caillier, 2016; Cohen et al., 2016; S. Lee, 2018; S. Lee et al., 2018; Moon, 2018), the Merit Service Protection Board (Rubin & Kellough, 2011), the NoFEAR Act (Rubin & Alteri, 2019), the Central Personnel Data File (Lewis & Pitts, 2018), and others. Combining data sets helps the researcher to avoid single source bias, a common problem with research that relies upon the FEVS. However, to combine the FEVS data set with others, researchers must either find a variable common to the data sets or hand code identifiers into their data sets. There is a coding scheme that is common to FEVS data and other OPM data sets. However, as this research note illustrates, this agency-specific code cannot be used without correcting for the discrepancies in how individual codes are used across the data sets.

Accordingly, first this research note describes the FEVS and FedScope OPM data sets. Second, I discuss the discrepancies in how agency identifier codes are used across the data sets and describe the impact of this issue. Finally, I describe the steps a researcher can take to ensure that their analyses are correct.

OPM Data Sets

OPM is one of the central managers of data concerning the U.S. federal workplace. Although OPM maintains a number of raw data sets, two of the most frequently used data sets are the annual results of the FEVS and workforce data available on the FedScope website. Although each data set can be used independently, it is common for researchers to combine the FEVS and FedScope data sets. Recently, this combination of data sets has been used to explore whether turnover intentions are a good predictor of future turnover (Cohen et al., 2016), the impact of inclusive management on effectively managing diversity (Moon, 2018), and the impact of representative hiring on employee perceptions and discrimination complaints (Alteri, 2018). Given the usefulness of combining these data sets to examine a wide range of management and personnel questions, it is imperative that researchers understand the problems they may encounter when doing so.

FEVS Data

The FEVS data sets contain individual-level responses to a survey that is collected annually by OPM. Although this survey is voluntary, approximately 40% of all federal employees participate each year. This data set is widely used by researchers, practitioners, and individuals assessing the organizational culture and health of the federal workforce. Specifically, researchers have used the FEVS to examine perceptions of equity and fairness based on the federal employee’s sexual orientation (Lewis & Pitts, 2017), procedural justice perceptions in the Department of Defense (Rubin & Weinberg, 2016), the impact of leadership styles on organizational commitment (Moldogaziev & Silvia, 2015), and performance-based human resource management (H. W. Lee, 2017), among others.

Each FEVS data set includes stratified random sample of federal employee opinions on a variety of workplace issues including personal work experiences, work unit, agency, supervisor, leadership, satisfaction, work/life, and demographics (OPM, 2016, p. 1). To ensure that the federal employees responding to the survey are representative, the OPM (2016) has designed the FEVS so

that the resulting estimates of perceptions are statistically reliable not only at the overall Federal workforce (i.e., government wide) level but also at the level of pre-identified work units and senior leader status (i.e., whether a member of the Senior Executive Service (SES) or equivalent) (p. 1).

For discussion on other strengths and limitations of the FEVS, see Fernandez et al. (2015), Somers (2018), and Resh et al. (2019).

When these data sets are downloaded, a 4-digit alpha-numeric code is attached to the observation. This code is automatically assigned by OPM and identifies the department and agency that the observation is tied to, for example, AG07 means that the observation is for the Department of Agriculture (AG) and the agency is the Food and Nutrition Service (+07) (2016 FEVS). This 4-digit code is named “plevel1” in the FEVS data sets. There is also a second agency identifier called “plevel2” that is also included in data sets prior to 2016.

FedScope Data

FedScope is OPM’s repository for workforce data on federal employees. FedScope draws data from Enterprise Human Resources Integration-Statistical Data Mart (EHRI-SDM) and is available online at www.fedscope.opm.gov. It includes department and agency-level metrics on employee ascensions, length of service, salary, education level, federal grade level, duty location, occupation, pay, supervisory status, work schedule, and a wealth of demographic data on employee ages, ethnicities, races, and genders. Although the data does not identify individual employees, it does include the most comprehensive summary data available on federal employees.

FedScope contains a user-friendly interface that you can use to look up data on specific agencies or departments. It also contains a function that allows the user to download data sets to be used in analysis. When you use this function, like the FEVS data, a 4-digit alpha-numeric code is automatically attached to the agency-level observation. Although FedScope data sets also include a title of the agency, this field is more difficult to use across data sets and most researchers rely on the 4-digit code when working with multiple observations.

Compatibility Issue Between OPM Data Sets

When you download the data sets, both the FEVS and FedScope include a 4-digit code used to identify the agency or department. The codes are determined by OPM and cannot be customized by the user when the data set is downloaded. The presumption is that these 4-digit codes are the same across all OPM data sets. Researchers use these codes to match or join data sets because matching these codes is an efficient way to combine multiple data sources, while ensuring that you are matching like organizational units. Matching or joining data sets using these codes has been a more reliable way to ensure that you are correctly matching a given agency or department’s data across multiple data sets. Other methods, such as hand entering data into a master data set or matching based on the agency or department name are more time consuming and subject to keying error.

For many agencies, the code is the same in both the FedScope and FEVS data. However, for a substantial number of agencies, the codes do not match across the FedScope and FEVS data sets. For those agencies that do not have identical codes in both data sets, one of three issues is present (see Table 1). The first issue occurs when an agency is assigned one 4-digit code in the FEVS data set and a different code in the FedScope data set. Uncorrected, this issue will result in data for both agencies being dropped from the analysis because not all variables included in the analysis have a value assigned. The second issue occurs when the same 4-digit code is used for one agency in the FEVS data set and another agency in the FedScope data set. If this issue is not corrected, then the data for two different agencies will be combined when the data sets are merged, distorting the results of the analysis. These issues are not limited to departments that have reorganized. Between 2013 and 2016, issue Types 1 and 2 occurred in almost every major department within the federal government, including the Departments of Agriculture, the Army, Commerce, Justice, Labor, Energy, Education, Health and Human Services, Homeland Security, Housing and Urban Development, Interior, Transportation, Treasury, Veterans Administration, as well as the Environmental Protection Agency, Securities and Exchange Commission.

Table 1.

2015 Examples of Coding Mismatch Between the Federal Employee Viewpoint Survey and FedScope Data Sets.

Description of code mismatch	FEVS code (agency name)	FedScope code (agency name)	Impact of mismatch
One agency has different codes across data sets	AG14 (Agricultural Marketing Service)	AG02 (Agricultural Marketing Service)	Data for the agency will be dropped from analysis because not all variables have values assigned
Code is assigned to two different agencies across data sets	AG07 (Food, Nutrition and Consumer Services)	AG07 (Rural Housing Service)	Data for two different agencies will be combined when the data sets are merged, distorting the results
Code is assigned to multiple agencies within the master data set and a different agency in the secondary data set	AG01 (Farm and Foreign Agriculture Service) and AG01 (Farm Service Agency)	AG01 (Office of the Secretary of the Agriculture)	Data set may not be able to be merged because the data in the master file is not unique; observations cannot be used until the correct match is identified

Note. The italicized print indicates a mismatch. FEVS = Federal Employee Viewpoint Survey.

The third issue occurs when a single 4-digit code is assigned to two or more agencies in the FEVS data set and a different agency in the FedScope data set. This issue is partially due to the fact that prior to 2016, OPM used a secondary agency identifier for smaller organizational units, a variable called “plevel2.” Although this means there are unique identifiers within the FEVS data, if a researcher is not familiar with this change and they do not correct for it, then the data sets may not be able to be merged because the data in the variable in the master file you are using to merge or join by is not unique. After the duplication is corrected in the FEVS data, the researcher then needs to correct for the discrepancy between the codes in the FEVS and FedScope data sets. Between 2013 and 2015, the third issue occurred in agencies in the Departments of Agriculture, Health and Human Services, Housing and Urban Development, Interior, Treasury, and the National Aeronautics and Space Administration.

One of the issues inherent to the FEVS is “the aggregation of data at the subagency or agency level results in a significant loss of information” (Fernandez et al., 2015, p. 385). This loss of data is further compounded when the FEVS data sets are combined with other data sets that because all data sets must contain the same panel of agencies or subagencies for comparison. Given this, it is imperative that researchers take steps to ensure that observations are not dropped from their final data sets unnecessarily and that they are not inadvertently combining data from two agencies into the same observation.

Steps to Correct for Data Set Compatibility Issues

To correct for the issues described above, the user should select a master data set and then check each observation in the secondary data sets to ensure that each agency uses the same codes. If the codes do not match, the user must replace the code in the secondary data set with the code that is assigned to that agency in the master data set. As this is a time-consuming process that will likely need to be repeated several times before the data sets are correctly matched, I recommend capturing these commands in some type of reproducible file (e.g., a STATA do-file).

To better illustrate the steps needed to correct for this issue, I describe the process used in a recent project. First, I examined the data with which I was working, in this case the FEVS and FedScope data sets. In this project, I was using a panel data methodology to analyze aggregate FEVS data at the agency-level and included FedScope data as control variables. A quick comparison of these two data sets revealed that there were substantially more agency and subagency level observations in the Fedscope data than in the FEVS data (524 observations, as compared with 219 observations in 2016). Given this, I used the FEVS data as the master file because I would not be able to use the FedScope observations that did not match to FEVS data points and coding according to the smaller population was more efficient.

Second, I used the FEVS agency codebook to compare the agency name assigned to each 4-digit code, with the agency name assigned to the same code in the FedScope data. Note, when downloaded the FedScope data set returns a variable named “Employmentasvalues” that is a combination of OPM’s 4-digit code (named Agency_ID in the example below) and the agency name. This variable can be separated into two separate variables using the following Stata commands:

rename Employmentasvalues Agency

split Agency, parse(-) limit(2)

rename Agency1 Agency_ID

drop Agency

rename Agency2 Agency_Name

drop if Agency_ID==“All Agencies”

A review of these data sets reveals that there are three separate issues that will either cause the researcher to drop observations unnecessarily or improperly combine the data for two different agencies (see Table 1).

The first type of issue occurs when one agency has different codes across data sets. For example, in 2015, the agency Agricultural Marketing Service was assigned the 4-digit code “AG14” in the FEVS and “AG02” in FedScope. This means that, if not corrected, the data for the agency Agricultural Marketing Service will be dropped from the analysis because not all variables have values assigned. The second type of issue occurs when the 4-digit code is assigned to two different agencies. If this happens, then the researcher will inadvertently combine the data for two different agencies when they merge the data sets and the coding software will not return an error message when the data for the two agencies is merged. For example, in 2015, the code “AG07” was assigned to the Food, Nutrition and Consumer Services Agency in the FEVS data and assigned to the Rural Housing Service in FedScope.

Finally, the third type of issue occurs because for a select number of agencies the same code is applied to multiple agencies within the same FEVS data set and that code is assigned to a wholly separate agency in the FedScope data set. This issue occurs because, prior to 2016, the OPM included an additional agency identifier in the data set called “plevel2.” For most agencies, this value in the data set is blank, but for a select number of agencies, OPM assigns a value for both “plevel1” and “plevel2” (e.g., 11 agencies in 2016 had more than one organizational unit assigned to them, impacting a total of 25 organizational units). Depending on the way the researcher chooses to combine the data sets, the coding software may return an error if this is not corrected prior to merging because the 4-digit code in the master file is not unique.

To illustrate this issue type, in 2015, the code AG10 is assigned to both the Farm and Foreign Agriculture Service and Farm Service Agency in the FEVS data, and to the Office of the Secretary of the Agriculture in the FedScope data. For one of the FEVS observations, Farm Service Agency, there is a value populated in the “plevel2” field, but for the other FEVS observation assigned to this code, there is no value in the “plevel2” field. This means that there are always unique identifiers in the FEVS data, but that if you use data sets prior to 2016, you will need to ensure that these unique identifiers all appear in the same variable. There are a number of ways to correct for this issue, but for those agencies that had a value in the secondary identifier, “plevel2,” I first replaced the 4-digit code (“plevel1”) with the “plevel2.” Doing so ensured that I had a unique agency identifier for all of the FEVS observations. After doing so, I then made changes to FedScope data to ensure that the agencies matched prior to merging.

For all agencies whose 4-digit codes do not match across the master (FEVS) and secondary (FedScope) data sets, you must replace the code in the secondary data set with the code in the master data set. To make this replacement, you should use the agency name in the FedScope data set to uniquely identify the variable. As the spelling or abbreviations for particular agencies change over time in FedScope, this may mean using several replace commands for the same agency. For example, Department of Education agencies that are not otherwise classified may be labeled “Dept Ed,” “Department of Education—Other,” “Education Other,” and so on. This means that you will need to run the “replace” command illustrated above with each variation of the agency’s name that is included in your secondary data set. You use the following Stata commands to replace these codes:

replace Agency_ID=“AG14” if Agency_Name==“AGRICULTURAL MARKETING SERVICE”

Note that the third type of error will require at least two edits before the data are matched correctly.

Despite these limitations, the 4-digit code still represents the best way of combining data for agencies across multiple data sets. The OPM data sets are an unmatched source of workforce data on federal employees. However, researchers should take caution to ensure that they are not unnecessarily excluding observations or matching agencies’ data incorrectly, thereby distorting their results.

Conclusion

The FEVS and FedScope data sets were not created explicitly for academic researchers (Fernandez et al., 2015). Instead, they were created for use by managers and administrators within the federal government. Given this, perhaps the reason for the coding discrepancies between the two OPM data sets discussed in this article is that they were not intended to be combined. They were created for different agency-specific audiences and with different uses in mind. However, these issues do not and should not preclude researchers from combining OPM data sets in different ways than the agency originally intended. An invaluable contribution that the academy can make to the world of practice is to approach this information in new and different ways. However, we must also realize that we are using this data differently than the purpose it was created for and take active steps to ensure that we are combining these data sets appropriately.

Embracing existing agency data also requires us to continue to critically evaluate our data sources and the assumptions that we as researchers make about them. As this research note has highlighted, we cannot simply use agency data without accounting for its intended use and the nuances that accompany this design. If we fail to do so, then the disconnect between what the agency data was intended for and what we, as researchers, are using it for represents a substantial threat to the validity of research we produce.

Footnotes

Acknowledgements

I wish to thank the reviewers for their insight and comments, as well as Dr. Ellen Rubin for encouraging me to write this paper.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ashley M. Alteri

Author Biography

Dr. Ashley Alteri, J.D., is an assistant professor of Human Resources in the College of Business at Tennessee State University. Dr. Alteri’s research interests focus on labor and employment issues within public and private sector workplaces. Her work has been published in the Review of Public Personnel Administration and Conflict Resolution Quarterly.

References

Alteri

A. M.

(2018). The impact of representative hiring on employee perceptions and discrimination complaints over time. In Academy of management proceedings (Vol. 2018, No. 1, p. 18371). Academy of Management. https://journals.aom.org/doi/10.5465/AMBPP.2018.18371abstract

Caillier

J. G.

(2016). Does satisfaction with family-friendly programs reduce turnover? A panel study conducted in U.S. Federal Agencies. Public Personnel Management, 45(3), 284–307. https://doi.org/10.1177/0091026016652424

Chan

(1998). The conceptualization and analysis of change over time: An integrative approach incorporating longitudinal mean and covariance structures analysis (LMACS) and multiple indicator latent growth modeling (MLGM). Organizational Research Methods, 1(4), 421–483.

Cohen

Blake

R. S.

Goodman

(2016). Does turnover intention matter? Evaluating the usefulness of turnover intention rate as a predictor of actual turnover rate. Review of Public Personnel Administration, 36(3), 240–263.

Favero

Bullock

J. B.

(2015). How (not) to solve the problem: An evaluation of scholarly responses to common source bias. Journal of Public Administration Research and Theory, 25(1), 285–30.

Fernandez

Resh

W. G.

Moldogaziev

Oberfield

Z. W.

(2015). Assessing the past and promise of the Federal Employee Viewpoint Survey for public management research: A research synthesis. Public Administration Review, 75(3), 382–394.

Jakobsen

Jensen

(2015). Common method bias in public management studies. International Public Management Journal, 18(1), 3–30.

Lee

H. W.

(2017). Performance-based human resource management and federal employee’s motivation: Moderating roles of goal-clarifying intervention, appraisal fairness, and feedback satisfaction. Review of Public Personnel Administration, 39, 323–348.

Lee

(2018). Employee turnover and organizational performance in U.S. Federal Agencies. The American Review of Public Administration, 48(6), 522–534. https://doi.org/10.1177/0275074017715322

10.

Lee

Fernandez

Chang

(2018). Job scarcity and voluntary turnover in the U.S. Federal Bureaucracy. Public Personnel Management, 47(1), 3–25. https://doi.org/10.1177/0091026017732798

11.

Lewis

G. B.

Pitts

(2018). Deciding to retire from the federal service. Review of Public Personnel Administration, 38(1), 49–82. https://doi.org/10.1177/0734371X16639110

12.

Lewis

G. B.

Pitts

D. W.

(2017). LGBT-heterosexual differences in perceptions of fair treatment in the federal service. American Review of Public Administration, 47(5), 574–587. https://doi.org/10.1177/0275074015605378

13.

Meier

K. J.

O’Toole

L. J.

Jr. (2013). I think (I am doing well), therefore I am: Assessing the validity of administrators’ self-assessments of performance. International Public Management Journal, 16(1), 1–27.

14.

Moldogaziev

T. T.

Silvia

(2015). Fostering affective organizational commitment in public sector agencies: The significance of multifaceted leadership roles. Public Administration, 93(3), 557–575.

15.

Moon

K. K.

(2018). Examining the relationships between diversity and work behaviors in U.S. federal agencies: Does inclusive management make a difference? Review of Public Personnel Administration, 38(2), 218–247.

16.

Office of Personnel Management. (2016). Federal Employee Viewpoint Survey Technical Report. https://www.opm.gov/fevs/reports/technical-reports/technical-report/technical-report/2016/2016-technical-report.pdf

17.

Resh

Moldogaziev

Fernandez

Leslie

C. A.

(2019). Reversing the lens: Assessing the use of federal employee viewpoint survey in public administration research. Review of Public Personnel Administration. Advanced online publication. https://doi.org/10.1177/0734371X19865012

18.

Rubin

E. V.

Alteri

A. M.

(2019). Discrimination complaints in the U.S. Federal Government: Reviewing progress under the No FEAR Act. Review of Public Personnel Administration, 39(4), 511–522. https://doi.org/10.1177/0734371X17744864

19.

Rubin

E. V.

Kellough

J. E.

(2011). Does civil service reform affect behavior? Linking alternative personnel systems, perceptions of procedural justice, and complaints. Journal of Public Administration Research and Theory, 22(1), 121–141.

20.

Rubin

E. V.

Weinberg

S. E.

(2016). Does changing the rules really matter? Assessing procedural justice perceptions under civil service reform. Journal of Public Administration Research & Theory, 26(1), 129–141. https://doi.org/10.1093/jopart/muu048

21.

Somers

M. J.

(2018). Strategies for improving measurement models for secondary data in public administration research: Illustrations from the Federal Employee Viewpoint Survey. Public Administration Review, 78(2), 228–239.

22.

Stritch

J. M.

(2017). Minding the time: A critical look at longitudinal design and data analysis in quantitative public management research. Review of Public Personnel Administration, 37(2), 219–244.