Piloting an empirical approach to link the international classification of functioning,disability,and health to job demand classification

Abstract

BACKGROUND:

The Social Security Administration has a thorough disability claims process, though one goal is to process claims more efficiently. This pilot described the creation and trial of a web-based tool to assist this process.

OBJECTIVE:

To empirically link the International Classification of Functioning, Disability and Health (ICF) model to the Occupational Information Network (O*NET) with a self-reported activity measure (physical domain).

METHODS:

ICF Activity domains and item difficulty calibrations were obtained from a self-reported ICF-Activity Measure. Three work/disability experts matched measurement constructs, job names, and item difficulties with job demands. Item difficulties were linked for “Positioning and Transfer” with O*NET data values of “Stamina”; “Trunk Strength”, and “Walking and Moving” with O*NET data values of “Stamina.” The system was then pilot tested with 14 adults with more than one activity challenges.

RESULTS:

An average total of 613 jobs were generated per participant and each job was categorized into one of 16 job clusters. Person ability measures and constructs were significant predictors for numbers of job (R² = 0.92, p < 0.05). Participants demonstrated moderate satisfaction with program usability, and reported capability in performing jobs.

CONCLUSIONS:

The system provides a feasible means to assist disability examiners and claimants in identifying relevant job abilities and options.

Keywords

Social security self-report disability evaluation employment assessment outcome assessment (healthcare)

1 Introduction

Based on the definition of disability adopted in the Social Security Amendments of 1967: “An individualshall be determined to be under a disability only if his physical or mental impairment or impairments are of such severity that he is not only unable to do previous work… (but also) engage in any other kind of substantial gainful activity…”, the US Social Security Administration has established a thorough, five-step sequential disability claim process used by more than two million applicants annually [1, 2]. This broad evaluation process determines if the disability claimant is engaging in substantial gainful activity, and if necessary, determines the claimants’ capacity to work in alternative occupations [1, 2]. The Social Security Administration’s five-step evaluation process begins with determining whether the claimant is engaging in any substantial gainful activity (Step 1), leading to the determination of the claimants’ capacity to perform jobs (Step 2) (Fig. 1).

Fig.1

The Five Stages of Disability Determination*. *: This figure was adapted from Stobo, McGeary, Barnes, & Institute of Medicine, 2007; received rights & permission from The National Academies Press.

Due to high number of claimants and the multiple stages of the process, one of the Social Security Administration’s primary goals is to evaluate claimants quickly and accurately [3]. To provide a means to facilitate Steps 4 and 5 of the disability determination process, which is to gauge the capacity of claimants for accomplishing past and alternate jobs, we developed a conceptual and empirical link between self-reported function and publicly surveyed job demands, to identify potential occupations that matches with the job-related ability of the Social Security claimants.

To measure health/disability at individual and population levels, the International Classification of Functioning Disability and Health (ICF) model was proposed by the World Health Organization [4, 5] and defines “activity” as the execution of a task or action by an individual. Thus, “activity limitation” is defined as difficulties an individual may have in executing activities, which is of particular relevance to work capacity evaluation [4, 5]. We focused on “mobility,” one of five “activity” subdomains in this study as mobility subcategories is mostly relevant to claimant’s basic capacity to accomplish a job. For example, a claimant may need to change or maintain body position or handling objects in order to perform a job task.

To provide guidelines of job placement and classification, the Dictionary of Occupational Titles (DOT) has been the standard reference [6, 7]. While the last revision of the DOT was over 20 years ago, it has been partially superseded by the Occupational Information Network (O*NET), an online database created in 1987. Similar as DOT, the O*NET is developed for career exploration and job analysis through public and private sectors and can be publicly accessed and completely downloadable via the web [7, 8]. Both DOT and O*NET provide fundamental information of job demands and job classifications for public and Social Security examiners. However, the feasibility and usefulness of DOT and O*NET systems remain unknown.

Although the ICF contains components that represent claimant abilities (e.g., maintaining body position, carrying, fine motor) and are related to O*NET/DOT job demands (e.g., trunk strength, dynamic strength, arm hand steadiness), an empirical link between the two is lacking. The primary purpose of this study is to empirically link ICF-based claimant ability measures to O*NET/DOT job demands. Once the ability and job demands were linked, we developed an automated computerized system that matches the ability level of the claimant to a list of potential occupations for that particular claimant. Finally, we pilot tested usability and user satisfaction of the system with individuals with disability.

2 Materials and methods

2.1 Participants

For piloting testing the developed system, we recruited 14 community-dwelling adults who reported one or more physical disability. All fourteen participants underwent an IRB-approved informed consent and were fully anonymized. The Institutional Review Board for Human Research at the University of Florida approved the study protocol (U-745-2013).

2.2 Data sources

2.2.1 Occupational Information Network (O*NET)

Since DOT components are included or updated and can be linked back to the O*NET system, to avoid redundancy, we only described O*NET components in this study. O*NET includes a numerical “data value,” a data point reflects the job demand, ranging from 0–7; with 0 represents the lowest level of job ability and 7 represents the highest level. O*NET provides each job with a data value on each job element (i.e. “element names”), including: arm-hand steadiness, manual dexterity, finger dexterity, control movement abilities, control precision, physical strength abilities, static strength, explosive strength, dynamic strength, trunk strength, stamina, and gross body coordination. In addition, the O*NET classified jobs into a number of “career clusters,” defined as “occupations in the same field of work that require similar skills” [9] (Table 1). A total of 1846 jobs were divided across 16 career clusters in the O*NET system.

Table 1
O*NET job clusters

No. Job cluster

1 Agriculture, Food, and Natural Resources

2 Architecture and Construction

3 Arts, Audio/Video Technology and Communications

4 Business Management and Administration

5 Education and Training

6 Finance

7 Government and Public Administration

8 Health Sciences

9 Hospitality and Tourism

10 Human Services

11 Information Technology

12 Law, Public Safety, Corrections and Security

13 Manufacturing

14 Market, Sales and Service

15 Science, Technology, Engineering and Mathematics

16 Transportation Distribution and Logistics

No.	Job cluster
1	Agriculture, Food, and Natural Resources
2	Architecture and Construction
3	Arts, Audio/Video Technology and Communications
4	Business Management and Administration
5	Education and Training
6	Finance
7	Government and Public Administration
8	Health Sciences
9	Hospitality and Tourism
10	Human Services
11	Information Technology
12	Law, Public Safety, Corrections and Security
13	Manufacturing
14	Market, Sales and Service
15	Science, Technology, Engineering and Mathematics
16	Transportation Distribution and Logistics

2.2.2 ICF Activity Measure (ICF-AM)

Velozo and colleagues [10] developed a self-reported ICF-Activity Measure (ICF-AM) to assess physical ability in individuals with physical disabilities. Using the Rasch model, the ICF-AM shows a specific probability pattern of the respondents about functional challenges related to job demands. Under the “mobility” subdomain, Velozo and colleagues [10] applied qualitative (focus groups and cognitive interviews) and quantitative (factor analysis and item response theory analyses) methodologies to develop item banks with six physical function constructs:1) Transfer and Positioning, 2) Gross Upper Extremity, 3) Fine Hand, 4) Walking and Moving, 5) Moving around Using a Wheelchair, and 6) Activities of Daily Living [10, 11]. For each construct, the ICF-AM includes items at different difficulty levels. Table 2 demonstrated an example of item hierarchy in the construct of Positioning and Transfers of ICF-AM. For example, scooting up/back into chair is the easiest item while squatting 1-2 minutes is the most difficulty item. The ICF-AM is available as a web-based computerized adaptive test (CAT) measure witan advantage of only selecting the most relevant items to measure each individual[10 –12].

Table 2
Example of item hierarchy of ICF-activity measure and job demands of ONET/DOT

ICF Activity Measure ONET/DOT

Positioning and Transfers Trunk Strength

Item Measure (0–100) Client ability/Job demands Data Value (0–3.5) DOT Job Title DOT Level of Strength DOT Code O*NET SOC Code

Scooting up/back into chair 25.45 0.12 Registration Clerk 205.367-042 43-4111.00

Change position standing to sitting chair 30.18 Easy 0.25 Customer Service Representatives Sedentary 191.167-022 43-4051.00

Seated 10–20 minutes 31.10 0.50 Monotype-Keyboard Op 650.582-014 51-5111.00

Change positions 34.14 0.88 Computer Programmer 030.162-010 15-1134.00

Standing 10–20 minutes 36.83 Medium 1.12 Bus Driver Light 913.463-010 53-3022.00

Shift sitting chair without armrests 37.58 2.12 Inspector, Building 168.167-030 47-4011.00

Change position standing to squatting 40.14 2.25 Guide, Sightseeing 353.363-010 39-7011.00

Change position standing to kneeling 40.70 Difficult 2.38 Air-Compressor Operator Medium 950.685-010 51-8021.00

Squatting 1-2 minutes 45.67 2.50 Cargo Checker 222.367-010 43-5111.00

ICF Activity Measure	O*NET/DOT
Scooting up/back into chair	25.45		0.12	Registration Clerk		205.367-042	43-4111.00
Change position standing to sitting chair	30.18	Easy	0.25	Customer Service Representatives	Sedentary	191.167-022	43-4051.00
Seated 10–20 minutes	31.10		0.50	Monotype-Keyboard Op		650.582-014	51-5111.00
Change positions	34.14		0.88	Computer Programmer		030.162-010	15-1134.00
Standing 10–20 minutes	36.83	Medium	1.12	Bus Driver	Light	913.463-010	53-3022.00
Shift sitting chair without armrests	37.58		2.12	Inspector, Building		168.167-030	47-4011.00
Change position standing to squatting	40.14		2.25	Guide, Sightseeing		353.363-010	39-7011.00
Change position standing to kneeling	40.70	Difficult	2.38	Air-Compressor Operator	Medium	950.685-010	51-8021.00
Squatting 1-2 minutes	45.67		2.50	Cargo Checker		222.367-010	43-5111.00

2.3 Study procedures

2.3.1 Connecting ICF-AM with O*NET/DOT

We used two steps to connect the ICF-AM with O*NET/DOT system: (a) matching ICF-AM constructs with O*NET job element names, and (b) matching the ICF-AM item difficulties with the DOT/O*NET job demands. Three independent and doctoral-trained work/disability experts were recruited in this study. The three experts are assistant, associate and full professors at the Department of Occupational Therapy and the Department of Behavioural Science and Community Health from two research-intensive universities (one public and one private). Each expert had more than five-years of experience working in vocational rehabilitation and were randomly selected as a convenient expert sample in our study. The experts matched ICF constructs,O*NET job element names and the ICF item difficulties with three levels of job demands. First, four ICF-AM constructs (Positioning and Transfers, Gross Upper Extremity, Fine Hand and Walking and Moving) were matched with 13 selected O*NET job element names (arm-hand steadiness, manual dexterity, finger dexterity, control movement abilities, control precision, physical strength abilities, static strength, explosive strength, dynamic strength, trunk strength, stamina, and gross body coordination). For example, the ICF-AM construct “Positioning and Transfers” was connecting to the O*NET element name “Stamina” based on 100% agreement across experts (Supplementary Table S1).

Secondly, we determined the anchor points on the ICF-AM scale and O*NET data value based on 100% agreement across experts in matching item difficulties with the job demand data values (Supplementary Table S2). For example, the item “seated 10–20 minutes” from the ICF-AM construct “Positioning and Transfers” was connecting with the “sedentary” job demand, as defined based on three job descriptions: manicurist, circulation clerk and receptionist (Supplementary Table S3). Only 100% agreement constructs/items were included in this study.

If multiple matches occurred for a job demand, the average ICF-AM item difficulty was calculated as the anchor point. The two anchor points (lowest and highest) were matched to the two anchor points of O*NET data values (sedentary and medium). Finally, a simple scaling method was used to match the range of four ICF-AM constructs and O*NET data values (Supplementary Table 3).

Table 3
Demographic characteristics of participants

Count Percentage (%)

Gender

Male 7 50.0

Female 7 50.0

Mobility

Manual Wheelchair 5 35.7

Electric Wheelchair 3 21.4

Both Manual and Electric Wheelchair 3 21.4

No Wheelchair 3 21.2

Spinal Cord Injury

Yes 4 28.5

No 10 71.4

	Count	Percentage (%)
Gender
Male	7	50.0
Female	7	50.0
Mobility
Manual Wheelchair	5	35.7
Electric Wheelchair	3	21.4
Both Manual and Electric Wheelchair	3	21.4
No Wheelchair	3	21.2
Spinal Cord Injury
Yes	4	28.5
No	10	71.4

2.3.2 Developing an automated ICF-AM/O*NET system

After the conceptual connections were empirically established, a software programmer converted the ICF-AM online CAT measure to generate job listing. To individualize the results for both the examiners and the respondents, we used 16 job clusters to generate the final job listing while optimizing the number of potential jobs (Table 1). That is, the examiner or claimant first chose the most relevant job cluster (e.g., Agriculture, Food, and Natural Resources), based on this preference, the potential jobs were presented only for that cluster (e.g., Farm and Ranch Managers; Agricultural Technicians). Figure 2 presents the example of the link between ICF-AM item difficulty hierarchy and O*NET/DOT job demands.

Fig.2

Components of the ICF-AM/O*NET System.

2.3.3 Pilot testing of usability

The usability of the ICF-AM/O*NET system was examined at three levels: (a) descriptive analyses (e.g., time to complete the assessment, number of questions answered, number of job titles output), (b) the responses to the System Usability Scale (SUS) and (c) linear regression to examine the percent of variance of number of jobs explained by item difficulty and ICF-AM construct. The SUS is a short, reliable, and valid self-reported satisfaction questionnaire for measuring system usability with a Likert Scale of 1–5 with a rating scale of 1 representing strong disagreement and 5 representing strong agreement [13]. A total score of 10 items of the SUS is converted to 0–100 scale13. Higher score represents better system usability. We used a score above 50.9 as a cut-off score to present an acceptable usability of the system, as the SUS score at 35.7 or below indicate “poor” or “worst” usability [14]. We added one item to the scale based on our study purpose, narrated as “I’m capable of performing reported jobs,” which was not included in the final scorecalculation.

3 Results

3.1 Demographics

For the 14 pilot testing sample, the gender ratio was even (seven males and females). Seventy-nine percent of the participants used either manual or electric wheelchair. Four of the participants (28.5%) had the diagnosis of spinal cord injury (Table 3). The participants were randomly recruited and each participant was part of the larger research project that investigated assisted living for the older adults.

3.2 Connecting ICF-AM with O*NET/DOT

The expert panel review resulted in four constructs in the ICF-AM matching with the O*NET jobelement names. The ICF-AM construct, “Positioning and Transfers” was matched with the O*NET element names, “Trunk Strength and Stamina”; the ICF-AM construct, “Gross Upper Extremity” matched with the O*NET element name “Physical Strength Abilities”; the ICF-AM construct, “Fine Hand” matched with the O*NET element name “Manual Dexterity.” Lastly, the ICF-AM construct, “Walking and Moving” matched with the O*NET element name “Stamina.”

Based on the agreement from the expert panel review, we only included the constructs with 100% agreement and excluded one construct, “Fine Hand” (with “Manual Dexterity”) from all further analysis. We also excluded the “Gross Upper Extremity” construct because in the O*NET content model, the Physical Strength construct does not have data values and is used as a combination of Static Strength and Explosive Strength. Since Static Strength and Explosive Strength both have data values, and using averaged data values would not accurately reflect of Physical Strength, we also excluded the Physical Strength construct from all further analysis.

3.3 Automated online ICF-AM/O*NET system

Figure 2 demonstrated the process of integrating ICF-AM online measure and O*NET to generate an automated online ICF-AM/O*NET system. Based on the connecting results above, we used the anchored values to link the ICF-AM item difficulty measure with the O*NET system data values of job demands for each job.

3.4 Usability of ICF-AM/O*NET system

The averaged test time of the system was 4.5 minutes (SD = 4.8) and the mean number of items tested from ICF-AM was 10 (SD = 3.6) (Table 4). The mean person measure (ability score) of ICF-AM was 46.6 (SD = 7.2) with a range of 32–57. The mean total number of jobs was 613 (SD = 275) with a range from 0 to 906 (Table 4).

Table 4
Descriptive statistics of disability determination process testing

Variable Mean (SD) Range

Test duration (minutes) 4.5 (4.8) 1–17

Number of CAT items 10 (3.6) 5–18

Person Measure 46.6 (7.2) 32–57

Total Number of Jobs 613 (275). 0–906

Number of Jobs in a Cluster 58 (18)† 0–67†

136 (23)†† 0–64††

Variable	Mean (SD)	Range
Test duration (minutes)	4.5 (4.8)	1–17
Number of CAT items	10 (3.6)	5–18
Person Measure	46.6 (7.2)	32–57
Total Number of Jobs	613 (275).	0–906
Number of Jobs in a Cluster	58 (18)†	0–67†
	136 (23)††	0–64††

Note. †, Job titles were selected from Business, Management and Administration Cluster; ††, Job titles were selected from Agriculture, Food and Natural Resource Cluster.

While the number of total jobs was high, a more manageable list of jobs was presented per cluster. The number of jobs were divided according to the 16 O*NET job clusters as the output of ICF-AM/O*NET system. For example, for one of our participants, the mean number of jobs (in a cluster) was 58 (SD = 18) for “Business, Management and Administration”; and 35 (SD = 23) for “Agriculture, Food and Natural Resource” (Table 4).

The total mean SUS score in this sample was 53.6 (SD = 22), which is above the acceptable cutoff score (of 50.9). Since the SUS item regarding claimant ability to perform reported jobs was not originally part of the scale, it was calculated independently, and produced mean scores of 3.21 (SD = 1.25) with a range of 1–5. One of 14 participants (7%) received zero job output. Person ability measure and ICF-AM construct were significant predictors of number of jobs (R² = 0.92, p < 0.05).

3.5 SUS response

With a moderate level of agreement (≥3 on ranges of 1–5 or 2–5), the respondents indicated the system was easy to use (Mean = 3.4, SD = 1.3), well integrated with various functions (Mean = 3.3, SD = 0.9) and they could learn the system quickly (Mean = 3.4, SD = 1.1). Alternatively, the respondents also considered the system was too complex (Mean = 3.2, SD = 1.4) and inconsistent (Mean = 3.1, SD = 1.2), and may need technical assistance (Mean = 3.0, SD = 1.5). However, the respondents also reported they would like to use this system frequently (Mean = 2.9, SD = 1.2) and felt very confident using the system (Mean = 2.9, SD = 1.5) (Table 5). Item hierarchy for the SUS is displayed in Table 6

Table 5
System usability scale response: Mean score of each item

Variable Mean (SD) Range

Use Site Frequently 2.9 (1.2) 1–5

Site too Complex 3.2 (1.4) 1–5

Easy to Use 3.4 (1.3) 1–5

Need Assistance with Site 3.0 (1.5) 1–5

Site Functions Integrated 3.3 (0.9) 2–5

Inconsistency in Site 3.1 (1.2) 1–5

Learn Site Quickly 3.4 (1.1) 2–5

Site Cumbersome/Awkward 2.6 (1.0) 1–5

Confidence in Use 2.9 (1–5) 1–5

Need for Prior Learning 2.7 (1.0) 1–4

Variable	Mean (SD)	Range
Use Site Frequently	2.9 (1.2)	1–5
Site too Complex	3.2 (1.4)	1–5
Easy to Use	3.4 (1.3)	1–5
Need Assistance with Site	3.0 (1.5)	1–5
Site Functions Integrated	3.3 (0.9)	2–5
Inconsistency in Site	3.1 (1.2)	1–5
Learn Site Quickly	3.4 (1.1)	2–5
Site Cumbersome/Awkward	2.6 (1.0)	1–5
Confidence in Use	2.9 (1–5)	1–5
Need for Prior Learning	2.7 (1.0)	1–4

1: strong disagreement; 5: strong agreement. Higher score represents better system usability.

Table 6

System usability scale: Scaled hierarchy (items ranked easiest to most difficult)

Variable	Mean (SD)	Range	Difficulty Index
Easy to Use	3.4 (1.3)	1–5	0.68
Learn Site Quickly	3.4 (1.1)	2–5	0.68
Need for Prior Learning	2.7 (1.0)	1–4	0.68
Site too Complex	3.2 (1.4)	1–5	0.64
Inconsistency in Site	3.1 (1.2)	1–5	0.62
Need Assistance with Site	3.0 (1.5)	1–5	0.60
Use Site Frequently	2.9 (1.2)	1–5	0.58
Confidence in Use	2.9 (1–5)	1–5	0.58
Site Functions Integrated	3.3 (0.9)	2–5	0.56
Site Cumbersome/Awkward	2.6 (1.0)	1–5	0.52

1: strong disagreement; 5: strong agreement. Higher score represents better system usability. Difficulty index = item mean divided by total item score 1 .

4 Discussion

Our study demonstrated the ICF-AM/O*NET system successfully automated the process of connecting claimants’ self-reported ability with job listings. The developed ICF-AM/O*NET showed acceptable usability by the participants, implying the system can potentially be used in the disability determination process. The most unique feature of the developed ICF-AM/O*NET system is the ability to generate a comprehensive list of personalized potential job recommendations automatically. Overall, the participants reported of being capable to perform the generated jobs.

The expert panel reviewed and identified the best match (100% agreement) between ICF-AM constructs/item difficulties and O*NET element names, by eliminating constructs/elements that did not match well, conceptually and empirically. The Fine Hand construct failed to match the O*NET element. This could be because the ICF-AM was originally developed for the individuals with disabilities. Thus, the ability level measured in the ICF-AM may only match with relatively lower-level difficulty. We suggest future studies of this kind include more difficult fine hand items, to further match job difficulty listed in the O*NET.

While 100% agreement was obtained for ICF-AM constructs/item difficulties and O*NET names/job demands, experts also expressed some conceptual challenges in making these connections. For example, Positioning and Transfers were not directly a measure of Stamina. Expert qualitative feedback suggested the need of a self-reported measure of job demands (e.g. a self-report measure of stamina) in order to increase the accuracy of the measured job demand.

ICF-based ability measures were authenticated against O*NET/DOT job demands successfully. By allowing participants to select jobs by cluster, generated jobs appeared to be reduced to a reasonable number. For example, for the cluster of Business Management and Administration, an average of job listing was reduced to 58 jobs compared to originally more than 600 jobs. However, one participant could not perform any jobs in this cluster. This may, at times, be the case as the easiest items in the construct are “position shift in chair with armrest” (person measure 36.47) and “scooting up/back into chair” (person measure 37.12); this participant’s person measure was 32. That is, this participant’s person ability was too low to accomplish any demands of the job titles in this cluster. This result may not automatically deem the participant eligible for disability compensation, as the disability examiner may review additional information to judge this participant’s eligibility and capability; however, providing such screening result of the respondent’s job ability level can be critically helpful for the examiners.

The satisfaction with the system was moderate, which is less than expected, but above an acceptable level. Most of the respondents reported being capable to perform the suggested jobs from the system, indicating the ICF-AM/O*NET generated feasible results. Respondents also reported that they would like to use this system more frequently and the majority of them considered operating this system relatively easy. However, this system may also be too complex for some with less experience with technology, as one respondent indicated a need to further simplify the ICF-AM/O*NET system.

4.1 Study limitations

Our findings are consistent with a growing body of work regarding the application of metrological frameworks to physical and social sciences [15, 16]. Despite positive findings, some limitations exist. First, due to a conceptual limitation and the original design purpose of the selected tool, the ICF-AM/O*NET system may only be feasible for the participants who are able to perform certain levels of gross motor ability independently. If the individual was either unable to or required significant assistance in performing positioning and transferring, then the generated job listing for this person could be limited. Second, only three work and disability assessment experts matched the constructs, which may be a threat to validity in this study. We suggest future studies of this kind include more experts to ensure the concepts are validly linked. However, perhaps the most significant limitation is that the Social Security Administration’s disability definition is work based by statute, which can only be altered through congressional action. Additionally, disability examiners have mandated limits on the data they may collect and use for the determination process, limiting the full potential of products using this, and similar designs. We are also aware of, by isolating each construct may fail to see a ‘whole picture’ of the respondent’s work ability; thus, we suggest future studies connect separate constructs to generate an overall work profile for therespondents.

In summary, this study successfully modified the O*NET database to generate O*NET/DOT job selections from claimant ICF-based ability measures based on empirically expert review and methodological linkage. While this study has only been based on physical function ability/demands applied to job classifications available in O*NET, the methodologies may be highly transferrable to other functional abilities (i.e., cognition) and future developments of job classification systems beyond O*NET. Expanding this system with other ICF constructs will be a next step to evaluate this system. Additional research may also be warranted to develop ratings or predictive construct theory that can evaluate the predictive power of expected performance levels [17, 18].

5 Conclusion

This study addresses a priority topic highlighted by the Social Security Administration and serves as an example of an expedited, automated system that may assist disability examiners in identifying potential occupations for claimants. The developed ICF-AM/O*NET is a free online system that can be easily and publicly accessed via the internet. We successfully incorporated the ICF-based client ability measures (ICF-AM) and the O*NET database structure into the ICF-AM/O*NET system. This study represents a promising approach for programmatic applications to achieve a variety of meaningful disability determination process, and serves as a demonstration of what is possible in a future system with a self-report measure that closely connects abilities to job demands. Further applications may streamline the complex process of disability determination to assist both claimants and examiners alike. Evidence-based future intuitive automation may equip disability examiners to more quickly and accurately process claims, improving the system for all users.

Conflict of interest

No potential conflict of interest was reported by the authors.

Funding

This work was supported for two years by the by the US Social Security Administration’s Disability Determination Process Improving Disability Determination Process Small Grant Program awards (PI: Arthur), managed by Policy Research Incorporated (2013-2015).

Footnotes

Difficulty index was calculated based on Portney LG, Watkins MP. Foundations of clinical research: Application to practice. Stamford, USA: Appleton & Lange. 1993..

Acknowledgments

We would like to recognize Mr. Shankar Manamalkav for his work in system construction as well as our participants in each phase of the project.

The Supplementary material is available in the electronic version of this article: .

References

US Social Security Administration. US Social Security Act 223(d)(2)(A). Code of Federal Regulations: Definition of Disability. 404.1505. Basic definition of disability. US Social Security Administration; 2003 [cited 2013 March 1]. Available from: http://www.ssa.gov/OP_Home/cfr20/404/404-1505.html.

US Social Security Administration. DI 24510.001. Residual functional capacity (RFC) assessment – Introduction; 2014 [cited 2014 June 2]. Available from: https://secure.ssa.gov/poms.nsf/lnx/0424510001.

Stobo

, McGeary

, David

, Institute of Medicine (US). Committee on Improving the Disability Design Process: SSA’s Listing of Impairments and Agency Access to Medical Expertise. Improving the social security disability decision process; 2007. Washington, D.C.: National Academies Press.

World Health Organization. International classification of functioning, disability and health: ICF. Geneva: World Health Organization; 2001.

World Health Organization. Towards a Common Language for Functioning, Disability and Health (ICF); 2002 [cited 2014 May 21]. Available from: http://www.who.int/classifications/icf/training/icfbeginnersguide.pdf.

Photius Coutsoukis and Information Technology Associates. Dictionary of Occupational Titles: Introduction; 2003 [cited 2013 March 1]. Available from http://www.occupationalinfo.org/front_148.html.

US Department of Labor. Employment & Training Administration. O*NET - beyond information – intelligence; 2004 [cited 2013 March 1]. Available from: http://www.doleta.gov/programs/onet/.

National Center for O*NET Development. O*NET Resource Center. O*NET Questionnaires; 1998 [cited 2013 March 1]. Available from: http://www.onetcenter.org/questionnaires.html.

Peterson

, Mumford

, Borman

, Jeanneret

, Fleishman

. Development of Prototype Occupational Information Network (O* NET) Content Model. Volume I: Report [and] Volume II: Appendices; 1995.

10.

Velozo

, Gray

. ICF activity measure: A computer adaptive survey designed to better understand the experiences of individuals with injuries and disabilities; 2004 [cited 2013 March 1]. Available from: http://icfmeasure.phhp.ufl.edu/.

11.

Velozo

, Wang

, Lehman

, Wang

. Utilizing Rasch measurement models to develop a computer adaptive self-report of walking, climbing, and running. Disabil Rehabil. 2008;30(6):458–67. doi:787634494

12.

Velozo

, Choi

, Zylstra

, Santopoalo

. Measurement qualities of a self-report and therapist-scored functional capacity instrument based on the Dictionary of Occupational Titles. J Occup Rehabil. 2006;16(1):109–22. doi: 10.1007/s10926-005-9014-z

13.

Brooke

. System usability scale. Reading, England: Digital Equipment Corporation. 1986.

14.

Bangor

, Kortum

, Miller

. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies. 2009;4(3):114–23.

15.

Cano

, Melin

, Fisher

, Stenner

, Pendrill

. Patient-centred cognition metrology. In Journal of Physics: Conference Series. 2018;1065(7):072033. IOP Publishing.

16.

Pendrill

, Fisher

Jr . Quantifying Human Response: Linking metrological and psychometric characterisations of Man as a Measurement Instrument. Journal of Physics: Conference Series. 2013;459(1):012057. IOP Publishing.

17.

Best

. A Rasch model of the Crohn’s Disease Activity Index (CDAI): Equivalent levels of ranked attribute and continuous variable scales. Crohn’s disease: Etiology, Pathogenesis and Interventions. 2008.

18.

Stenner

, Fisher

Jr , Stone

, Burdick

. Causal Rasch models. Frontiers in Psychology. 2013;4:536.

ICF Activity Measure			O*NET/DOT
Positioning and Transfers			Trunk Strength
Item	Measure (0–100)	Client ability/Job demands	Data Value (0–3.5)	DOT Job Title	DOT Level of Strength	DOT Code	O*NET SOC Code
Scooting up/back into chair	25.45		0.12	Registration Clerk		205.367-042	43-4111.00
Change position standing to sitting chair	30.18	Easy	0.25	Customer Service Representatives	Sedentary	191.167-022	43-4051.00
Seated 10–20 minutes	31.10		0.50	Monotype-Keyboard Op		650.582-014	51-5111.00
Change positions	34.14		0.88	Computer Programmer		030.162-010	15-1134.00
Standing 10–20 minutes	36.83	Medium	1.12	Bus Driver	Light	913.463-010	53-3022.00
Shift sitting chair without armrests	37.58		2.12	Inspector, Building		168.167-030	47-4011.00
Change position standing to squatting	40.14		2.25	Guide, Sightseeing		353.363-010	39-7011.00
Change position standing to kneeling	40.70	Difficult	2.38	Air-Compressor Operator	Medium	950.685-010	51-8021.00
Squatting 1-2 minutes	45.67		2.50	Cargo Checker		222.367-010	43-5111.00

Piloting an empirical approach to link the international classification of functioning,disability,and health to job demand classification

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSIONS:

Keywords

1 Introduction

2.1 Participants

2.2 Data sources

2.2.1 Occupational Information Network (O*NET)

2.3.1 Connecting ICF-AM with O*NET/DOT

Table 3 Demographic characteristics of participants Count Percentage (%) Gender Male 7 50.0 Female 7 50.0 Mobility Manual Wheelchair 5 35.7 Electric Wheelchair 3 21.4 Both Manual and Electric Wheelchair 3 21.4 No Wheelchair 3 21.2 Spinal Cord Injury Yes 4 28.5 No 10 71.4

3 Results

3.1 Demographics

3.2 Connecting ICF-AM with O*NET/DOT

3.3 Automated online ICF-AM/O*NET system

3.4 Usability of ICF-AM/O*NET system

4.1 Study limitations

5 Conclusion

Conflict of interest

Funding

Footnotes

Acknowledgments

References

Table 3
Demographic characteristics of participants

Count Percentage (%)

Gender

Male 7 50.0

Female 7 50.0

Mobility

Manual Wheelchair 5 35.7

Electric Wheelchair 3 21.4

Both Manual and Electric Wheelchair 3 21.4

No Wheelchair 3 21.2

Spinal Cord Injury

Yes 4 28.5

No 10 71.4