Abstract
A new criticism of the variety in measures serving to identify the disabled population in the American Community Survey (ACS) has been raised by Burkhauser, Houtenville, and Tennant. That criticism identifies the lack of a participation component, specifically a measure of work limitation, as creating bias resulting in an underestimate of the size of the working-age population with disabilities. The purpose of this article is to provide another perspective on the relationship of the current ACS measure and a work limitation measure demonstrating the complications introduced by combining measures that represent two different metrics and introducing an unmeasured environmental element. The relationship of the measures with receipt of Social Security Disability Insurance or Supplemental Security Insurance is also examined and discussed.
It is clear that research about disability, both academic and policy related, can be complicated by the variety of measures available to represent the population in question and the concepts underlying these measures. An article by McDermott and Turk (2011) is one of the more recent publications to raise this issue. McDermott and Turk base their concerns on the multiple perspectives created by the medical, functional, and social theoretical approaches that make the comparisons across different conceptual measurements difficult and to them “distorting of the overall understanding of disability” (McDermott & Turk, 2011, p. 1). They argue for a need for parsimony in identifying persons with disabilities with an emphasis on a more precise measure offered by a focus on impairment.
A new criticism of the variety in measures serving to identify the disabled population has been raised by Burkhauser, Houtenville, and Tennant (2012). Their concern addresses the questions used rather than the theoretical approaches to measurement. They focus on the questions in the Current Population Survey (CPS) data, specifically the set of six questions on physical, mental, and emotional functioning difficulties adopted from the American Community Survey (herein referred to as the “ACS measure” or the “ACS question set”) and the single work limitation question that has been regularly administered since 1981 in the CPS-Annual Social and Economic Supplement (ASEC), a March supplement to the CPS used to collect income, poverty, and health insurance data. The concern about these two very different measures indicates that either measure used alone appears to underestimate the size of the population with disabilities, particularly estimates of the population receiving financial benefits for disability, specifically Social Security Disability Insurance (SSDI) and Supplemental Security Insurance (SSI). An additional concern is that use of either measure alone will create biases in estimates of program participation, employment, and economic well-being (Burkhauser et al., 2012). The purpose of this article is to provide an alternative to the interpretations that Burkhauser et al. bring to their analysis of the questions used in the CPS to represent disability.
The questions being compared in Burkhauser et al. (2012) capture two very different conceptual components of measurement of the respondent’s experience, and therefore should not be considered equivalent. The ACS set of questions represents a brief set of functional limitations and the ability to act independently that the person brings to any type of social role activity (work being one social role). The work limitation question identifies a person who is unable or limited in the kind or amount of work they can do; however, the question does not indicate whether the inability or limitation is associated with functional limitations at the person level or environmental conditions or discrimination associated with the attempt to work.
When examining various operationalizations of measures that reflect the different conceptual elements of theoretical models of disability, it is important to keep in mind that different measures not only reflect a short hand for the conceptual elements but they also represent different aspects of the disability process. They represent a combination of personal characteristics (an individual’s impairment or function limitation) as well as person/environment interaction characteristics (inability or limitation in social role participation that can be associated either with limitations that the functioning disparities or the type of environment create). Although a respondent may indicate a positive response on one of the elements of the disability process, that does not mean he or she will experience all of the elements and will give a positive response to questions on all those elements. Within the large general population of persons with disabilities, there are different smaller populations captured by one measure, but not all measures (Altman & Gulley, 2009). People can experience single or multiple functional limitations, without having major restrictions in participation, depending on their personal resources and the nature of their environments. In a similar way, people with impairments do not necessarily have functional limitations although over time their impairments can worsen enough to create problems with functioning. In many instances, individuals with one functional problem can maintain their involvement in daily activities, but when they develop additional conditions or impairments, the combination of the impact of the associated limits in functioning can reduce participation or even limit it completely. However, for a person to indicate that he or she is unable or limited to accomplish an outcome activity, such as work—without also indicating some functional limitation—chances are that it is the environmental context and/or skill level that is preventing such activity rather than any serious impairment. If the person reported the inability or limitation in work and also the receipt of disability benefits without having personal functional limitations, this requires further examination, but one cannot assume the benefit receipt as a “gold standard” identifying disability.
This raises a question about whether the combined elements of the disablement process or work limitation alone are necessary to measure and define disability as implied by Burkhauser et al. (2012, p. 2). While the recognition that the restrictions in participation are the ultimate outcomes of the disablement process as demonstrated in Brandt (1997), if we want to predict those outcomes for our population, where do we measure? What forms the basis of the population with disabilities—those with serious physical, mental, and emotional functional limitations who, depending on their personal resources and environmental context, can be “at risk” of the negative outcomes or only those who experience the negative outcomes?
Background
The term disability is a shorthand expression or as explained in the International Classification of Functioning, Disability and Health (ICF) an umbrella term (World Health Organization [WHO], 2001). It is a term that has been used very generally to identify everything from the impairments that result from disease and injury; the physical, mental, or emotional functional limitations or difficulties a person has as a result of the impairments; the limitation in particular activities (such as driving or acquisition of goods or services); or participation restriction they experience when the environment is not supportive. At the person level, disability reflects the point at which a person exhibits a recognizable difference of reduced functional capacity/or performance from others or from the person’s own previous capacity or performance. It is the physical, mental, or emotional functioning capacity that the person brings to their attempts to accomplish life activities (participation) and puts them “at risk” for not being able to accomplish those activities. When the research focus is participation, “disability” usually represents the point at which the person is restricted or prevented entirely from taking part in a role activity. This element is based entirely on the type of social roles a person seeks to participate in and the physical and social environment in which they attempt to play those roles. That point at which disability is defined may change or fluctuate based not only on the dynamics of the condition causing the problem and the resulting functional limitation but also on the availability of assistive devices as well as changes in the physical or social environmental context. The functioning approach to measurement is locating the “risk of disability” at the person level and identifying the functional limitation (based on a condition or impairment) in association with the person’s capacity to act. Asking about inability or limitation in work also tries to locate the problem within the person, but in reality, it is an interactional measure because it is dependent on the norms and culture of the society, which create the work situations and makes those situations accessible or inaccessible to the person with a functional problem. The interaction component of the measure is not often included because at the moment, we do not field questions that can describe environmental factors.
Figure 1 shows the relationship of the conceptual areas of measurement defined by several models of disability in use today (Nagi, 1965; Verbrugge & Jette, 1994; WHO, 2001). The representation of pathology, impairment, and functional limitations are measures at the person level. They represent the kind of health condition (diabetes, cancer, birth defect, amputation, etc.) that can create the individual’s impairment, as well as the types of functional limitations that are created by the impairments. For example, a functional limitation in walking or climbing stairs can result from numerous different pathologies or impairments, including limb amputation or paralysis, vision loss, or heart/lung deficiencies. It is a much more concise or economical measure of risk at the person level than measurement of the pathology or impairment that can include persons with pathology who do not go on to develop impairments or persons with impairments who do not go on to develop functional limitations.
Relationship of conceptual areas of measurement.
If we consider chronic conditions, injuries, birth defects, and other health problems as causes of impairments that can result in functional difficulties, there appears to be a general sequencing of the disablement process. In general, we expect that the impairments/functional difficulties precede participation restrictions in most cases. If this is a basic assumption of an investigation of disablement, we can then accept functional limitation as a preceding factor in the development of participation restrictions. Altman and Bernstein (2008) examined just that relationship between basic actions (measured in terms of functioning limitations) and complex activities (measured in terms of participation in social and work activities) and demonstrated that among persons with reported basic action difficulties, including vision, hearing, mobility, emotional and cognitive functions, or any combination of those functional limitations, 41.9% also reported a participation restriction either in work, social activities, or self-care and maintaining independence. There is not a one-to-one relationship between basic action difficulties and complex activity limitations or restrictions, although functional difficulties can put a person “at risk” to have those kinds of limitations or restrictions. In fact, many people have a variety of functional difficulties or even multiple functional difficulties without necessarily experiencing complex activity limitations. It is difficult to explain that relationship completely without incorporating personal characteristics (age, gender, income) and environmental conditions, but the analysis also revealed one further interesting factor. Among persons who do not report any of the measured functional limitations, 2.7% report a limitation in complex activities (Altman & Bernstein, 2008). The interpretation of that finding was that although they measure functional limitations fairly well in the National Health Interview Survey (NHIS), there is still a small group for whom they have either not captured the nature of their functional difficulty that contributes to their complex activity limitation or some people report complex activity limitations (predominantly work limitations) sans functional limitations. It is apparent from the Altman and Bernstein (2008) analysis that the measurement of functional limitations, or basic actions, provides the broadest base for measuring limitations with or without accommodation. It is this group that may be most affected by cuts in transportation budgets or other cuts in supportive services that allow them to maintain their participation, particularly in work. Those with functional limitations who manage to maintain their life roles in the current environment cannot be ignored. They represent the bulk of the iceberg that remains under water that will overwhelm our SSDI and SSI support systems if environments change in a nonsupportive way.
Although the “disability” may be best defined at the participation level as modeled by the Institute of Medicine work on this topic (Brandt, 1997), this type of outcome is the result of what the person brings to the participation choice (including not only the functional limitation but age, gender, race, and other personal characteristics) as well as what the environment allows. It can be measured as full, limited, or nonparticipation, but without some environmental measurement to represent the intervening factors, its attribution is unknown. Without that, it is difficult to ascertain where remedies should be applied. At an earlier point in time when many jobs and occupations required substantial physical activity, it was easier to ignore the environmental components as the inability to lift, walk, or see could be the overriding factor influencing the inability to participate in work. However, jobs and occupations are substantially different today and assistive technology has provided mechanical or programmatic substitutes for mobility, vision, and upper body strength. Today, nonparticipation can often be attributed to the lack of these devices, lack of accessible transportation or other forms of accommodation, or discrimination in the workplace as much or more so than to the person’s functioning limitations.
Burkhauser et al.’s (2012) definition of disability adapted from the ICF 1 implies that a person must have an impairment, activity limitation, and participation restriction; or that a participation restriction is sufficient to be identified as disabled. This definition excludes those who have an activity limitation 2 but can participate because of accommodations or a more accessible environment. It also contradicts the purpose of the Americans With Disabilities Act (ADA) and the UN Convention on Disability, which identify the population with disabilities as a protected class—not just the ones who cannot participate, but all. By defining disability as a combination of the personal impairment, activity limitations (of undefined levels), and participation, the definition totally excludes persons with impairments and activity limitations (of whatever level), but without participation restrictions as part of the population with disability. This has been a common problem for many years with the use of the work limitation type questions alone and has resulted in the exclusion of persons with functional limitations but with jobs as part of the population with disabilities (Kirchner, 1996).
A more methodological issue associated with measurement is the restriction created by the need to limit the number of questions asked of the respondent. The six questions about functioning that comprise the ACS measure do not allow for the complete coverage of functioning problems that more questions would allow, thus limiting coverage to the most commonly occurring functional limitations. The six questions ask about the four most common areas of functional limitation that can be measured in one question and also asks two questions about independence, one on self-care and one about getting around outside the home. In the same way, one question about work restriction does not capture the variety of work experience, nor does it capture the environmental factors that restrict participation. Limitations in the kind or amount of work a person may be able to accomplish are different from being totally unable to work. Moreover, the nature of the type of work in which one is involved is also an important consideration. Factors such as availability of types of work, environment barriers or supports, and even individual choice are rarely considered in development of these measures or in the subsequent analyses.
From a social or policy perspective, we are concerned about identifying these disabling representations so that we can provide social policy, education, or changes to improve the individual’s ability to enact their roles. Definition and measurement of disability then become tools for improvement. Another complicating factor in definition and measurement arises because there is no clearly marked separation between those with disabilities and those without disability. Although we try to treat disability as a dichotomous status, either a person is disabled or not, it is not the reality. People may be disabled in one situation and not another depending on what is required of them, what the environmental conditions are or what the purpose or objective of the definition used to describe the situation. The result is measurement that can be interpreted as fuzzy along the edges of the dividing context which can be influenced as much by the person’s mood when answering the question, the weather on day the question is asked or the effectiveness of the pain pill taken an hour before the interview.
Except for persons with very severe limitations who would be identified as disabled on almost any type of measurement, prevalence estimates of disability are subject to the basis of the measurement (what types of functioning, activities, or participation are the point of focus), the person’s environmental situation at the point/time/place of measurement (are the weather, building structure, lighting, or many other characteristics of the environment supportive or restrictive), and the intent of the purpose for which the estimate is required (Are we attempting to ascertain the number of persons who will need fiscal support because they cannot work or are we identifying a group whose civil rights to access may be restricted or prevented?). This latter requirement is what forces us to create a dichotomy for policy purposes. However, the dichotomy will be very different depending on the purpose for which the measure is being developed—exactly what aspects of the population we are trying to define and measure. The nature of the dichotomy will also be influenced by the categories of response we are offering the respondent. If we are asking them to choose between a “yes” or “no” answer (as required in the ACS questions and the CPS work limitation question) to describe a function that is changeable, we are likely to get fewer positive responses. In the same way, if we ask a “yes” or “no” question about participation in a variety of roles, we do not know whether the functional limitation precludes consideration of the activity or if other individual or environmental factors are preventing the activity. In effect, we do not know whether the “disability” measured reflects personal limitations, choice, environmental restrictions, or some combination of these.
Examples of some of the many purposes for which we make disability prevalence estimates are numerous, but here are two very different general examples. Civil rights purposes seek to affect a broadly defined group, anyone with a disability; whereas, benefit programs are organized for a specific group defined by their age, location, a specific characteristic (such as unemployment) or a specific type of condition or impairment. Different purposes highlight different conceptual elements and focus the measurement process on those highlighted elements. In the past, the measurement of inability to work to predict use of benefit packages may have been one of several appropriate approaches to the purpose. However, today, within the context of the 1990, Americans With Disabilities Act that extends civil rights and equalization of opportunity to the population with disabilities (whether they are able to work or not) and has broadened the conceptualization of disability policy issues beyond the need for specific programmatic support; a more appropriate measure of disability is required.
While I consider the definitional limitations, the most important issue of concern presented in the Burkhauser et al.’s (2012) article, there are several other areas which require comment. The first is the nature of the comparisons made from Table 1. In response to the question that underlies this work, the issue driving the analysis is about adding the work limitation question to the six-question ACS question set to include the respondents in Subset C, those who do not indicate a functional limitation or problems with independence but do indicate a work problem. If that is correct, we would want to compare the characteristics of those in Group C with the characteristics of those in Group A + B—all the persons who answer the ACS questions. Comparing Group C with Group A alone is comparing persons with identified functional limitations who can work (A) 3 with persons who do not identify a functional limitation who either cannot work or are limited in the kind or amount of work they can do (C). Those in Group B identify themselves with a functional limitation and also indicate they cannot work or are limited in the kind or amount of work they can do. Both question sets capture their presence although only one question set is needed to do that.
Comparison of Prevalence Rate, Percentage Employed, Poverty Rate, and Program Participation for Work-Activity limitation and Six-Question Subset and Six-Question Sequence and Work-Activity Limitation Subset.
Source. Adapted from Table 1 in Burkhauser, Houtenville, and Tennant (2012).
If the reader agrees that it is appropriate to include in a definition of general disability persons with functional limitations who do not report a work limitation along with those who do report a work limitation, then the comparisons should be between Subset C and the A + B groups who answer the six-question sequence. In Table 1, using a limited set of numbers from Burkhauser et al.’s (2012) Table 1, I make comparisons to see whether the addition of Group C remedies a bias in the six-question sequence, Group A + B. First, we see that the percentage employed is 22.8% compared with 30.8%, a difference of 8%. Note that the members of Group C are employed at a proportion 6.2% greater than the full population that answers the work limitation question (16.6%, 22.8%). As there are no standard errors available, we do not know whether the differences are significant. The comparison of the three groups (C), (A + B), and (B + C) on percentage in the labor force is similar to that found with employment.
If we compare Group C with Groups A + B on proportion of receiving SSDI or SSI (receipt of SSDI and SSI are put forward as the “gold standard” for identifying true disability), we see that 25.5% of Group A + B receive SSDI compared with 23.9% of Group C, whereas 16.8% of Group A + B receive SSI compared with 15.9% of Group C. Both programs seem to be well represented in the six-question sequence and addition of the work limitation question appears not to add any population that is not represented. However, because the measurement of Group A + B includes persons with functional limitations who currently do not have a work limitation, the poverty rate is somewhat lower (25.6% compared with 28.6%)—but again without standard errors, it is hard to know whether that is a significant difference. As we do not know the functional limitations of persons in Group C, it is possible that they represent limitations that are not captured in the six-question sequence. It would be useful to investigate that detail and add a question about the missing limitations rather than to include a larger group—part of which may not be relevant.
If we do the same comparison to see whether adding Group A to the group defined by the work limitation question, Group B + C, we find a much more marked difference. Those in Group A define a population with less work limitation, less poverty, and less programmatic support from SSDI and SSI. If the reader can accept that not all persons with disabilities are limited in work, then the work limitation question has been providing only a subsample of the population with disabilities for all these years. The underlying assumption of this approach that persons with disabilities cannot or do not work reflects an assumption of dependence within this group which is no longer valid.
Another area of comment reflects on the authors’ comparisons of the earlier and later ACS questions. The questions prior to 2006 (which included the work limitation question) were focused on conditions, or conditions that lasted 6 months or more as a cause of certain activity limitations. The revised ACS questions, which were developed to better represent the intentions of the ADA, focused less on the conditions (although physical, mental, or emotional conditions causing the difficulties were used in two of the six questions) and more on having serious difficulties with the functioning of the individual. All of the new ACS questions were more direct without the long lead-ins and the multiple parts. Please see the two sets of questions in the appendix. The 2006 ACS Content Test demonstrated that the new questions provided better response rates and equal or greater reliability than the earlier ACS question set (Brault, Stern, & Raglin, 2007).
One Final Note
While the authors hypothesize that the location of the work limitation question at the end of the ACS sequence in 2007 may cause repetition of the previous questions by the enumerator as noted in the text (Burkhauser et al., 2012, p.10), this is probably not the case as the 2007 ACS questions (as with all previous and subsequent ACS surveys) are self-response paper/pencil mode.
Although approximately 50% required computer-assisted telephone interview (CATI) or computer-assisted personal interview (CAPI) follow-up for completion in 2007, the other 50% were completed without an enumerator (http://www.census.gov/acs/www/Downloads/survey_methodology/acs_design_methodology_ch07.pdf).
Discussion
Because of the limitation in space allowed by the Census, the ACS six-question sequence does not capture the absolutely complete population with disability. However, at any point in time that population is a moving target due to temporary disabilities, or just good days and bad days for persons with impairments that result in functional limitations. The same could be said about the work limitation measure advocated in the Burkhauser et al.’s (2012) article because of the economic situation or environmental changes. Therefore, the question becomes “how do we best represent the population with disability for policy purposes?”
The measures that best represent the population with disability are dictated by the analytic question that is being asked. However, I would make several points: (a) There are usually some gaps in the measurements used to represent the population with disabilities, but filling those gaps requires identifying the missing components and developing questions with a similar metric to capture those gaps; (b) combining measures that use two different measurement metrics (functioning limitation or participation) is like combining apples and oranges to represent all citrus fruits—it is not particularly reasonable to substitute measures dependent on environmental contexts for the missing physical, mental, or emotional functioning components among those who do not report functional difficulties; and (c) the use of receipt of services or stipends from a benefit program as the gold standard of defining “true disability” ignores the reality of the differences among demographic groups and the possibility that some individuals with similar levels of functional limitations to those receiving benefits may choose not to apply for benefits because of more favorable environmental or resource circumstances or they may be ineligible. Are they any less or differently “disabled” (based on either functioning or employment status) than benefit recipients, or does the “gold standard” also have gaps? In addition, the combination of being “unable to work” or “limited in the kind or amount of work” into the wording of a single question prevents us from understanding whether being unable or limited has a different impact on response to the question and relationship with employment, application for benefits, or the severity of the functional limitations associated with the measure (Altman & Weeks, 2012). Adding those with limitations in kind or amount of work, who may still be working, would possibly bring as many unrelated respondents into the group as those who are missed, again distorting the data.
It would appear that Burkhauser et al.’s (2012) data, if viewed from a different perspective than was taken in their article, show that adding the “C” group to the ACS questions will not significantly change the representation of the sample except possibly in the area of unemployment. On the other hand, the group defined by the ACS questions (A + B) includes persons who have functional limitations and are unemployed along with persons who have functional limitations who are employed, the latter group which has been ignored in the CPS since 1981. Although I would agree that it would be important to capture all persons with work limitations who also have a functional limitation, the C group should be examined in more detail to ascertain what kinds of different functioning problems they may represent. Once documented, it would be important to add those functional difficulties to the ACS set if feasible. However, abandoning the A + B group, simply because a larger proportion is employed, would be misrepresenting the population with disability in the current milieu. There is also a group of SSDI/SSI recipients who do not answer either the ACS questions or the work limitation question, which raises the possibility that eligibility for and application and receipt of SSDI is more complicated than just identifying disability using six (or seven) general survey questions. Ideally, that is another group for which there should be some accounting.
Footnotes
Appendix
Comparison of ACS Questions Asked Prior to and Post-2008
| ACS Questions Before 2008 | ACS Questions 2008 and After |
|---|---|
|
Blindness, deafness, or a severe vision or hearing impairment? A condition that substantially limits one or more basic physical activities such as walking, climbing stairs, reaching, lifting, or carrying? Yes; No |
Yes; No |
Yes; No |
|
| Answer ONLY IF person is 5 years old or over | |
|
Learning, remembering, or concentrating? Yes; No Dressing, bathing, or getting around inside the home? (Answer if this person is 16 YEARS OLD OR OVER.) Going outside the home alone to shop or visit a doctor’s office? (Answer if this person is 16 YEARS OLD OR OVER.) Working at a job or business? |
Yes; No |
Yes; No |
|
Yes; NoAnswer ONLY IF this person is 15 years or over Yes; No |
Note. ACS = American Community Survey.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.