Using the Annual Social and Economic Supplement as part of a Current Population Survey panel

2.1 CPS supplements

In addition to the Basic Monthly CPS, supplements to the CPS are frequently fielded. CPS supplements vary widely in scope and type (see Table∼11-1 [1] for a complete list of CPS supplements) and usually contain only individuals who also complete the Basic Monthly Survey in the month the supplement is fielded.2 For example, the Voting and Registration supplement is fielded biennially and administered only to respondents from the November Basic Monthly CPS. Eligibility for participation in the supplement varies, however, meaning that some respondents to a Basic Monthly CPS will not receive the supplemental questionnaire fielded in that month. The Displaced Worker supplement is asked of workers 20 years of age and older who were displaced from their jobs and who were interviewed in the January Basic Monthly CPS. The ASEC is an exception to the rule about supplements being fielded only for respondents who participate in a specific month; this supplement is administered during the March Basic Monthly CPS, but also includes CPS participants from other months who are not scheduled to receive the March Basic Monthly CPS. This unique aspect of the ASEC requires special handling in the process of linking it to other CPS data files.

The most popular CPS supplement is the “March Supplement.” Introduced in 1947 as the Annual Demographic File (ADF), the technical name for this supplement since 2003 has been the Annual Social and Economic Supplement (ASEC). Between 1947 and 1955, the ASEC was administered in April and included households from the April Basic Monthly CPS (see Table 2). After 1955, the ASEC was implemented in March and began being commonly referred to as the “March Supplement.” Between 1956 and 1975, the ASEC consisted only of respondents from the March Basic Monthly CPS. Over time, the ASEC has expanded to improve the reliability of information about certain subpopulations (i.e., persons of Spanish [Hispanic] origin and low-income children who do not have health insurance coverage). Currently, the ASEC contains basic monthly demographic and labor force data as well as supplementary data on work experience, income, noncash benefits, and migration [3]. Given the expansion and implementation of the ASEC – drawing the ASEC sample from the March Basic Monthly CPS and households from non-March Basic Monthly CPS samples – there are complications for longitudinal linking of the ASEC with Basic Monthly CPS files [1].

The ASEC oversampling scheme has important ramifications for researchers who want to link CPS respondents across time. Though all ASEC respondents participate in the Basic Monthly CPS, only ASEC households that were administered the March Basic Monthly CPS can be easily matched. Linking ASEC oversample respondents to their Basic Monthly CPS observations is extremely tedious and labor intensive at best, and, in some cases, impossible. We therefore focus our efforts on matching the March Basic Monthly CPS to the ASEC. Figure 1 graphs the size of the ASEC oversample from 1989 to 2017 and shows that in each year the ASEC is larger than the March Basic Monthly CPS with larger differences when the SCHIP (State Children’s Health Insurance Program) oversample is introduced (details on this below). MARBASECID has been created for the 1989 to 2017 period and will be added annually for the most recent ASEC and backward in time (to the 1976 ASEC) so that CPSID can be made available via IPUMS CPS.

OPEN IN VIEWER

From March 1976 through 2000 the Census Bureau increased the reliability of estimates for people of “Spanish origin” by conducting additional interviews with November households (from the previous year) that contained one or more persons of Spanish origin [15]. The ASEC oversample of people of Spanish origin is commonly referred to as the “ November Hispanic oversample.” The November Hispanic oversample increased the size of the ASEC by about 2,500 households (see the 1989–2000 range in Fig. 1). Because of the CPS rotation pattern (4-8-4), all of the Spanish households identified in November are out of the CPS sample when the March Basic Monthly CPS is conducted; no households interviewed in November are eligible for the March interview based on the 4-8-4 CPS rotation pattern. For example, a household in MIS 1 in November will be MIS 4 in February and thus out of the CPS in March (similarly, MIS 5 households in November will be MIS 8 in February). Because the oversample households would not have otherwise been in the ASEC, we refer to these extra visits as MIS 9 and 10, respectively; in the data, however, the Hispanic oversample cases are assigned MIS values between one and eight, making the oversample cases more complicated to identify.3 During the extra interviews, the November Hispanic oversample receives both the March Basic Monthly CPS and the ASEC [1]. Thus, the November Hispanic oversample results in additional cases in the ASEC from other months of the CPS that would not have otherwise been in the March Basic Monthly CPS.

The second CPS sample expansion in 2001 was funded by a Congressional allocation of $10 million annually to the Census Bureau, which included both a general expansion and an additional oversample to the ASEC. The expansion was motivated by an interest in producing reliable state-level estimates on low-income children without health insurance and to measure the effects of the State Children’s Health Insurance Program (SCHIP) established by Congress in 1997 [7]. The general expansion added 12,000 units to the sample monthly [7]. The Basic Monthly CPS sample size increases were completed between September 2000 and July 2001 as is evident by the increasing sample sizes during this period in Fig. 1. While the sample increases were completed in July 2001, the expansion is not evident in the March Basic Monthly CPS until 2002 [7].

In contrast to the November Hispanic oversample, the SCHIP oversample is drawn using two strategies: “split-path” assignment and month-in-sample 9 (MIS 9) assignment. The “split-path” strategy selects respondents from the February Basic Monthly CPS and April Basic Monthly CPS (adjacent months to the March Basic Monthly CPS). February households with MIS 4 and 8 that contain children (18 or younger) or non-White household members complete the ASEC at the time of the February Basic Monthly CPS interview. Similarly, April households in MIS 1 and 5 that include children (18 or younger) or non-White household members receive the ASEC during the April CPS Basic interview. Neither the “split-path” eligible respondents from February or April would have otherwise participated in the ASEC because of the survey’s rotation pattern. The term “split-path” thus refers to these February and April cases that would normally have received the supplements assigned for February and April but instead are “split” into the ASEC.

The second set of households in the SCHIP oversample – the MIS 9 households – is administered an extra interview [7].4 These cases are contacted for a ninth interview in either February or April. From 2001 to 2003, these households were drawn from the November Basic Monthly CPS of the previous year if they were in MIS 6, 7, or 8 in November and they were not part of the November Hispanic oversample and they were not Hispanic and either had at least one child 18 years or younger or a non-White member. Note that these households would have completed all eight interviews of the CPS rotation pattern by January at the latest (for MIS 6). Starting in 2004, the MIS 9 oversample was chosen from August (MIS 8), September (MIS 8) and October (MIS 8); the same condition applies as before of either having at least one child 18 years or younger or a non-White member and being non-Hispanic [7].

In 2014, the ASEC included a series of redesigned income and health insurance questions. Three-eighths of the 2014 ASEC sample received the redesigned income questions while the other five-eighths of the sample received the “traditional” income questions [16]. The redesigned health insurance questions were asked of the entire 2014 ASEC sample. To later evaluate the effects of the redesigned health insurance questions, the Census Bureau used a split-path assignment to randomly select about 6,000 households from the 2016 and 2017 March Basic Monthly sample to answer the complete pre-2014 ASEC questionnaire [17]. As a result, we cannot locate a subset of March Basic respondents in the ASEC for 2016 and 2017 (see Table 3).

3.1 CPSID

The ASEC is unique among CPS data, as described above, and that has implications for linking, which we describe in the next section. To lay the foundation for our work on MARBASECID, we first outline the procedures for linking CPS Basic Monthly data. Using linking keys available on all public use Basic Monthly CPS files, researchers can link observations over time to create short sixteen-month panels with up to eight observations per person. This work, however, is cumbersome and expensive for each individual researcher to perform independently. The several obstacles researchers face in linking Basic Monthly CPS observations, including recycled identifiers, changing linking keys, and the household rather than the person as a sampling unit, are detailed elsewhere [2].

As a service to users, IPUMS CPS delivers a single unique identifier, CPSID, that lowers the barrier to using repeated observations of individuals from Basic Monthly CPS files as a panel. CPSID uses the original linking keys provided by CPS to match records over time, accounts for the complex CPS rotation pattern, and assigns a new unique identifier to each record in the Basic Monthly CPS. However, CPSID was not initially created for ASEC files because the ASEC lacks all of the linking keys required for matching records to other Basic Monthly CPS files.

3.2 MARBASECID

To make CPSID available on the ASEC, we create MARBASECID, a variable we use and deliver to users to match individuals in the ASEC to the March Basic Monthly CPS. MARBASECID is a 10-digit variable on both the March Basic Monthly CPS and ASEC files. MARBASECID consists of two digits (either 00 or 11), a two-digit year, and a six-digit sequence number. Individuals who appear in both the March Basic Monthly CPS and the ASEC in the same year are assigned 11 as the first two digits in MARBASECID; the six-digit sequence number begins at 000001 and increments by one for each additional person who is in both files. For a matched observation in the 1989 March Basic Monthly CPS, MARBASECID is 11 + 89 + six-digit sequence number. The two-digit number for unlinked observations in both the March Basic Monthly CPS and the ASEC is 00. For unlinked March Basic Monthly CPS individuals, MARBASECID takes the form: 00 + two-digit year + six-digit sequence number starting at 000001 and incrementing by one for each unlinked March Basic Monthly CPS person. For unlinked ASEC individuals, MARBASECID is a concatenation of 00, two-digit year, and a six-digit number starting at 500,000 and incrementing by one for each unlinked ASEC observation. This method ensures that MARBASECID is unique within and across years. For example, a MARBASECID value of 1100012345 is decoded as follows: “11” refers to the individual who is in both the March Basic Monthly CPS and ASEC; “00” refers to the ASEC survey year of 2000; “012345” refers to that household being given the randomly sequenced order number of 12345. Similarly, a MARBASECID of 0098000012 refers to an unlinked person from the March Basic of 1998 while 0098500012 refers to an unlinked person from the 1998 ASEC.

We created MARBASECID using Stata 13 for the 1989 to 2013 period. Given the ease of matching in later years as described below, we matched ASEC and Basic Monthly CPS files and constructed MARBASECID for 2014 forward using a program written in Perl. All of our code for the construction of MARBASECID from 1989 to 2017 is available on GitHub (https://github.com/mnpopcenter/cps-march-asec-linking).

3.3 Creating MARBASECID

The creation of MARBASECID is a critical step in the process of attaching CPSID, a unique IPUMS-created identifier, to the ASEC. As discussed previously, CPSID allows researchers to easily and reliably link data across CPS months, including the ASEC. The creation of MARBASECID eliminates the need for individual researchers to perform the tedious and cumbersome process of linking the March Basic Monthly Survey and the ASEC, which is complicated for two primary reasons. First, the variables required to link the ASEC to CPS monthly files are not available for all years on the ASEC. As a result of omitted linking keys and the ASEC oversample, duplicate and false matches are problematic. Second, we speculate, despite the absence of technical documentation to be certain, that the Census Bureau transition to a computer-based interview resulted in more prominent data quality issues for linking across months even if they did not compromise the integrity of each individual month of data.

The algorithm matching the March Basic Monthly CPS to ASEC overcomes these difficulties and allows us to put CPSID on the ASEC for easy linkages to other CPS monthly data. With CPSID on the ASEC, opportunities for using the CPS as a panel multiply since the ASEC is the premier CPS supplement. Theoretically, Census-provided household and person identifiers should be sufficient to link the March Basic Monthly CPS and ASEC files. If that were the case, researchers would have many demographic variables with which to check the validity of matches. Practically, however, the linking keys that should uniquely identify records do not always [9]. Table 3 details by year (1989–2017) the variables used to link the March Basic Monthly CPS and ASEC, the number of persons in the March Basic Monthly CPS, the number of matches/non-matches to the ASEC, and the number of invalidated matches.5 From 1994 forward, we validate matches based on AGE, SEX, and RACE (we follow the evaluation of validity using age, sex and race in line with [8]), and we find high validation rates for links made between the March Basic Monthly CPS and ASEC files.

Our strategy for creating MARBASECID depended on the types of problems we encountered linking the March Basic Monthly CPS and ASEC files. From 2005 to 2017, the matching algorithm is very simple. Using the variables listed in Table 3, we can uniquely identify all March Basic Monthly CPS respondents and ASEC respondents. March Basic Monthly CPS observations are easily located in the ASEC except, as described above, for the split-path households in the 2016 and 2017 ASEC.

From 1996 to 2004, the matching algorithm is more complicated because of duplicate records caused by non-unique linking keys. Our strategy for handling the duplicate records is as follows. During the first stage, duplicate records based on the first stage linking keys in Table 3 are identified and flagged in both the March Basic Monthly CPS and the ASEC. Then, March Basic Monthly CPS records that are not uniquely identified are dropped from the file. Within a pair of ASEC duplicates, we keep the duplicate with the lowest H_SEQ (the household identifier created by Census Bureau that is unique within a given survey month) value since these records are part of the March Basic Monthly CPS rather than an ASEC oversample [18]. We then merge the pruned March Basic Monthly CPS and ASEC files using the first stage linking keys in Table 3. The second stage of work uses the observations from the duplicate record file (i.e., the “pruned” observations) and the non-matches from the first round of matching. We link records using as few variables as possible. Even then, the data sometimes require a close analysis of a few observations in order to find the correct match.

The period 2001 to 2004 was especially problematic because of the SCHIP expansion of the ASEC oversample. Though CPS documentation details the variables researchers should use for linking, these variables do not uniquely identify records, thus complicating the process [10]. The 1996 to 2000 period was also problematic for unknown reasons. We detail complications we encountered in both periods in Appendix A.

The greatest challenges in creating MARBASECID occur prior to 1996 when data quality problems (e.g., duplicate records based on linking keys and missing observations) are more common. In 1995, we employed the two-stage matching approach and were unable to match 951 observations from the March Basic Monthly CPS to the ASEC. It is possible that the observations are missing from the ASEC, though we have yet to find documentation about this specific issue. To link the 1994 March Basic Monthly CPS and ASEC files, we employ first and second stage matching (Table 3) and also make additional adjustments. The most important adjustment is that the ASEC file must contain a corrected version of HRHHID (the originally released 1994 ASEC file contained an error in HRHHID resulting from the program that created the variable [19]; the corrected version of the file is available via IPUMS CPS); matching is impossible without the corrected version of HRHHID. Several other minor adjustments must be made, including harmonizing age for a few observations (e.g., an observation may have age of 81 in the March Basic Monthly CPS but 80+ in the ASEC) and handling duplicate observations; our handling of these issues is detailed in Appendix B.6

4.1 Substantive example #1: Child Tax Credit receipt and food security

The link between food insecurity and low income in the United States has been widely documented [20]. Using the CPS, a researcher could examine the relationship between receiving the Child Tax Credit (CTC) and food security. The CTC reduces the amount of taxes families pay (depending on adjusted gross income) by $1,000 dollars per qualifying child. This tax credit increases the disposable annual income of low-income families, potentially reducing their food insecurity.

Investigating this relationship using CPS requires linking ASEC and December Food Security files since tax credit questions are only asked in the ASEC and food security is only assessed in the Food Security Supplement. For illustration purposes, we link the 2005, 2006, 2007, and 2008 Food Security Supplement respondents (from December) to their ASEC records in the following year (2006–2009). We use CPSID to link the Food Security Supplement to the March Basic Monthly CPS and MARBASECID to link March Basic Monthly CPS and ASEC observations (see Table 4). Only one quarter of December Basic Monthly CPS respondents (those in MIS1 and 5) are eligible to be linked to the March Basic Monthly CPS using CPSID (see Column 3 of Table 4). We match 90% of eligible observations between the December and March Basic Monthly surveys (or 22% of the entire December Basic Monthly CPS sample [see Table 4, Column 5]). The linkage rates are consistent with other observations four months apart [2].

After linking December and March Basic Monthly CPS observations using CPSID, we use MARBASECID to link to the March Basic Monthly CPS to the ASEC. The March Basic Monthly CPS to ASEC linkage is perfect, but the number of records with both food security and tax responses is slightly lower because the CTC variable is only available for persons 15 and older (Table 4, Column 7). The final linked and eligible sample is about 15% of the December Basic Monthly CPS (Table 4, Column 8) and about 62% of the total MIS 1 and 5 observations from the December Basic Monthly CPS (Table 4, Column 9). Nonetheless, sample sizes for examining the relationship between CTC receipt and food security are large in each of the four years ( ∼ 20,000 respondents).8

One may also use CPS data to examine how the CTC has affected population food security over time. This requires extending the December (MIS 1) to ASEC linkage (MIS 4) forward in time to include the subsequent December (MIS 5) and ASEC (MIS 8) observations. For example, individuals from MIS 1 in December 2005 are linked first to the 2006 March Basic Monthly CPS using CPSID (and then to the 2006 ASEC using MARBASECID); CPSID is then used again to link to December 2006 (when respondents are MIS 5) and then to March 2007 (via CPSID and then to the 2007 ASEC via MARBASECID). About 5% of respondents from each of the December Basic Monthly CPS surveys from 2005–2008, with both food security and child tax credit data in both years, may be linked in this way (see Table 4, Column 11). Of those eligible (MIS 1 in December of a given year), about 70% are linked (see Table 4, Column 12), and the resulting samples are sizeable ( ∼ 7,000 observations for each year).

A popular set of employment questions have been asked only of the outgoing rotation groups (ORG) from MIS 4 and 8 of each Basic Monthly Survey. This set of variables is commonly known as the ORG questions or the Earner Study questions. Information is collected on topics such as usual hours worked, hourly wage rate, usual weekly earnings, union membership, class of worker, and multiple job holdings (hereafter referred to as the “earner study” questions). Earnings data from the earner study refer to a usual week in the last month while the earnings data collected in the ASEC refer to the “past year”. Given these differences, researchers may want to use these variables in combination with ASEC variables. For example, any research question looking at unionization alongside taxes paid or poverty will require the use of both the ORG and the ASEC. Using the earner study variables from the March Basic Monthly CPS with the ASEC, researchers encounter a substantial reduction in observations – typically about 20% of the March Basic Monthly CPS has earnings data. This limitation may be overcome by linking ASEC and ORG data from surrounding non-March months. By linking the ASEC to subsequent ORG responses from the Basic Monthly CPS in April, May, and June, the number of cases is nearly quadrupled, increasing the power for combining earnings from the ASEC with information about union participation from the Basic Monthly CPS.

Because not all earner study variables are part of the ASEC file, using the full set of earner study variables with the ASEC data requires, at the very least, linking the March Basic Monthly CPS with the ASEC; MARBASECID drastically simplifies this effort. About 25% of March Basic Monthly CPS respondents would have this weekly wage data available since only the outgoing rotations (MIS 4 and 8) respond to the earner study questions. Using CPSID to link ORG data from different months to the ASEC, researchers can easily increase their sample sizes. Table 5 shows for the years 2005–2009 the number of cases in the March Basic, the number of linkages to other months to get ORG data, and the number of cases with earnings data collected for individuals9 in ORGs. Researchers can leverage the power of the short panel aspect of the CPS and use CPSID to link from March to April, May, and June and MARBASECID to link from March to the ASEC; the resulting sample size for 2005 is 100,042 compared to 26,503 if only ORG data from March are used (Table 5, Column 4). Larger sample sizes, for example, allow for detailed subgroup analyses which would otherwise be limited.

Researchers may also make these kinds of linkages across years to get ASEC data combined with earnings data from two points in time. The process just described

would be performed for two points in time, year x and year x+1. Year x and year x+1 are then linked together using CPSID. As Table 5 shows, this is possible for just over one quarter of the March Basic Monthly CPS respondents (26% in 2005),10 which is substantially higher than if we only linked individuals from MIS 4 of the 2005 March Basic Monthly CPS to 2006, which would be 7% (or 9,548 individuals) of the 2005 March Basic Monthly CPS sample (see Table 5, Column 7). Including individuals in the March Basic Monthly whose earnings data come from April, May, or June increases the total sample size to 35,030 in 2005 (see Table 5, Column 6). Patterns are similar for the 2006–2009 period.

The value of MARBASECID and CPSID is evident in recent research on unionization and poverty [22]. Doe et al. replicate a previous study using the ASEC union status variable. The limitations of the analysis are acknowledged: “the CPS asks the union membership question only for one-fourth of the sample (the two outgoing rotation groups). As a result the CPS samples are much smaller” [22, p. 886]. The substantial loss of observations the authors lament could easily be overcome with MARBASECID and CPSID. Using MARBASECID to link the March Basic Monthly CPS and the ASEC and then CPSID to link the ASEC to other monthly surveys, the authors could have combined earnings from the ASEC with union membership information from April, May, and June and retained a much larger number of observations (roughly four times as many).