Abstract
BACKGROUND:
Individuals with disabilities experience disruptions in life participation at the onset or exacerbation of a disability. The multiple dimensions of functioning impacted go beyond the symptoms of a disabling condition and assessing an individual’s level of functioning is a critical first step of a strengths-based rehabilitation approach. With functioning playing an important role in the vocational rehabilitation process, it would be important to have an assessment tool that can be used to measure an individual’s level of functioning.
OBJECTIVE:
The purpose of this study was to examine the psychometric properties of the Illinois Brief Functioning Inventory (IBFI), a scale developed to measure the multi-dimensional nature of functioning as it relates to vocational rehabilitation, career development, and employment of people with disabilities.
METHODS:
Factor analysis, including exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), was used to determine and validate the underlying dimension of IBFI.
RESULTS:
Overall, the study findings indicate strong psychometrics for a 26-item instrument comprised of five meaningful subscales identifying functioning across physical, cognitive, and emotional dimensions.
CONCLUSION:
The results of this study provide initial psychometric support regarding the IBFI as an instrument that can be used to identify an individual’s current level of functioning.
Introduction
As many as 65 million Americans are currently living with one or more disabilities (Smart, 2020), and that prevalence figure is expected to rise steadily over the next 30 years as the 73 million members of the Baby Boomer generation continue to age into retirement and experience age-related health conditions in large numbers (Koch & Rumrill, 2017). Experts have long held that the onset or exacerbation of a disability is one of the most stressful circumstances in the human experience (Smart, 2020). The injury or illness that results in a disability may threaten the individual’s perceived ability to live a fulfilling life (Livneh et al., 2019).
Disability is also apparent to others in one’s family, social network, and employment setting. Research indicates that others view disabilities as stigmatizing conditions associated with stereotypical perceptions of the person as contagious, a safety risk, unproductive, dependent, and unstable (Smart, 2020). These stereotypes may cause family members to emphasize what the person cannot do, friends to avoid social contact with the person, and employers to initiate actions that cause the person to resign or retire (Kosciulek, 2007; Vickers, 2012). These psychosocial features of the disability experience, combined with the physical, sensory, neurological, and mental health effects of disability on the individual, often make it difficult for people with disabilities to function in desired personal and social roles and to pursue their educational, community living, and employment goals (Rubin et al., 2016).
In their text on the medical and psychological aspects of disability, Andrew and Andrew (2017) asserted that it is not the etiology or symptoms of disabling conditions per se that primarily thwart people’s participation in desired activities and roles, but the impact that disability has on functioning. Functioning is a matter of what people can and cannot do, and functional abilities are determined partly by disability-related limitations. However, functioning can be enhanced by medical interventions, counseling and psychotherapy, allied health services such as occupational or physical therapy, education and training, effective health practices, assistive technology and environmental modifications, prosthetics and orthotics, and personal supports such as interpreters and personal care attendants (Andrew & Andrew, 2017).
With vocational rehabilitation counselors playing an important role in promoting employment and community inclusion of people with chronic health conditions and disabilities, assessing an individual’s level of functioning is a critical first step of a strengths-based rehabilitation approach (Chou et al., 2013). The strengths-based rehabilitation counseling approach is consistent with other areas of rehabilitation worldwide that utilize the International Classification of Functioning (ICF; World Health Organization, 2001) that identifies functioning, activity, participation, personal and environmental factors as foundational to the rehabilitation process (Ditchman et al., 2013). For vocational rehabilitation counselors working within the State/Federal Vocational Rehabilitation system in the United States, the assessment of functioning is a central part of determining eligibility for services and plan development, with individuals with multiple functional limitations receiving the highest priority for vocational rehabilitation services (Chan et al., 2019).
With functioning playing an important role in the vocational rehabilitation process, it would be important to have an assessment tool that can be used to measure an individual’s level of functioning, specifically as it applies to the State/Federal Vocational Rehabilitation system. To date numerous instruments have been developed to measure functioning within rehabilitation. The most widely used measure is the World Health Organization Disability Assessment Schedule II (WHODAS 2.0) that assesses the impact of a health conditions or disabilities in terms of functioning based on the ICF (Üstün et al., 2010). The WHODAS 2.0 is made up of 36 items related to the following six domains of common daily activities: moving and getting around; self-care; getting along with people; life activities; and participation in society. Although this scale provides relevant information related to disability, the application to vocational rehabilitation may be limited because it does not provide detailed information regarding specific dimensions of functioning that have been found to impact career development and employment (Strauser, 2021). Given that the multi-dimensional nature of functioning is often mutable, vocational rehabilitation professionals have a vested interest in having access to psychometrically sound measurement instruments that can gauge a person’s functioning at any given point in time (Rubin et al., 2016). Clarity in the assessment of functioning will help identify assets and barriers related to career development and employment process and facilitate the identification of appropriate supports and interventions. Therefore, the purpose of this study was to examine the psychometric properties of the Illinois Brief Functioning Inventory (IBFI), a scale developed to measure the multi-dimensional nature of functioning as it relates to vocational rehabilitation, career development, and employment of people with disabilities. Specifically, the following three research questions guided this study: Can meaningful dimensions be identified among items on the IBFI, which would then be defined as instrument subscales? Do the subscales show adequate internal consistency? Is there a relationship between scores on the IBFI and the SF-36, DS14, and the WHODAS 2.0 communication subscale?
Materials and method
Procedures
Human subjects’ approval for this project was granted from the University’s Institutional Review Board. Participants were recruited using the crowdsourcing data collection tools Amazon Mechanical Turk (MTurk) and TurkPrime. MTurk is used by thousands of researchers to obtain sample participants while TurkPrime provides a more user-friendly interface and additional options for data collection (see Litman et al., 2017; Sheehan, 2018 for a description of these tools). Potential participants who had previously identified as having a disability or chronic health condition were offered an opportunity to participate in the study through MTurk and were administered the survey online via Qualtrics. Individuals who completed the survey received a $15.00 incentive payment. Participants were informed that their participation was voluntary and that they were free to withdraw at any time. No individual identifying information tying the subjects to their results was retained with the data.
Participants
The sample consisted of 738 U.S. adults living with a disability or chronic health condition. The sample population included only people with disabilities and chronic health conditions as the aim of this study was to understand how the IBFI would perform in rehabilitation settings. The mean age of participants was 43.48 years (SD = 12.51, Range = 20-77), with 30.4% male, 69.1% female, and 0.5% other. A total of 82.2% identified as Caucasian, 8.1% as Black or African American, 3.3% as Latino/Hispanic, 2.6% as Asian, 0.4% as Native American/Alaskan Native, and 3.4% as other/multiple races. Highest level of education for the individuals in the sample was 0.3% with some high school, 8.8% with a high school diploma, 3.5% with a General Education Diploma (GED), 28.9% with some post-secondary schooling without a degree, 15.7% with an associate’s degree, 30.6% with a bachelor’s degree, and 12.2% with a graduate degree. At the time of the study, 41.9% of our sample was employed full-time, 14% was employed part-time, 16.7% worked contract, freelance, or temporary work, 22.9% were unemployed, and 4.6% selected other. Finally, when asked which disability category (or categories) best describe their current functioning, 74.8% selected physical, 24.3% neurological, 55.7% psychiatric/mental health, 5.0% cognitive, 3.8% deaf/hard of hearing, 4.9% vision impairment/blind, and 4.1% other. Table 1 provides a summary of the demographic characteristics of study participants.
Demographic characteristics of the participants. (N = 738)
Demographic characteristics of the participants. (N = 738)
Note: N = 738. Percentages may not add up to exactly 100 due to multiple responses allowed or rounding error.
The IBFI is a 26-item brief measure designed to gather data related to functional limitations across five areas: physical, psychosomatic, cognitive, negative coping, and social isolation. The IBFI measure was developed to identify levels of physical, emotional, and cognitive functioning across dimensions of both severity and intensity. Participants provided self-report evaluations of symptom on their ability to work in each of the five subscales. A 6-point Likert scale is used to obtain reported experiences of both severity and frequency on items such as “Do you experience problems standing/walking?” and “Do you experience difficulty focusing/concentrating”.
The RAND 36-item Health Survey (SF-36) is a multipurpose, short-form health survey with only 36 questions (Ware & Sherbourne, 1992). It is a generic measure of health status as opposed to one that targets a specific age, disease, or treatment group. It yields an 8-scale profile of scores measuring physical functioning, bodily pain, role limitations due to physical health problems, role limitations due to personal or emotional problems, emotional well-being, social functioning, energy/fatigue, and general health perceptions. The items are scored and summed according to a standardized protocol on a scale from 0-100 for each of the eight subscales with higher scores being representative of better health status (Ware et al., 1994). The following six subscales were utilized in the current study with their corresponding reliability coefficients of Cronbach’s alpha values: physical functioning (α= 0.93), bodily pain (α= 0.87), role limitations due to physical health problems (α= 0.86), role limitations due to personal or emotional problems (α= 0.83), emotional well-being (α= 0.88), and social functioning (α= 0.84).
The DS14 is a 14-item self-report measure, consisting of two subscales, negative affectivity (e.g., “I often feel unhappy”) and social inhibition (e.g., “I am a closed person”) (Spindler et al., 2009). Responses are indicated on a 5-point Likert scale from 0 (false) to 4 (true) with scores ranging from 0-28 for each subscale. Higher scores on the two subscales of the DS14 indicates higher scores on the personality traits of negative affectivity and social inhibition. Reliability coefficients for each of the subscales were calculated for this sample with a Cronbach’s alpha value of 0.92 for negative affectivity and 0.90 for social inhibition.
The WHODAS 2.0 is built around the ICF framework and captures an individual’s level of functioning in six major life domains: understanding and communicating; getting around; self-care; getting along with people; life activities; and participation in society (Üstün et al., 2010). Respondents are asked to indicate their level of difficulty experienced in these areas of functioning during the past 30 days on a 5-point Likert scale ranging from “none” to “extreme or cannot do”. Higher scores on the WHODAS 2.0 subscale scores and overall score are indicative of greater functioning difficulties. Only the understanding and communicating subscale which consists of 6 items (e.g., “Remembering to do important things”) was utilized for the current study as a comparison measure of cognitive functioning. Reliability coefficients using Cronbach’s alpha for the present sample was found to be 0.86.
Data analysis
Factor analysis, including exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), was used to determine and validate the underlying dimension of IBFI. The full sample (N = 738) was first randomly split into two independent groups –training (N = 369) and test groups (N = 369). Training set was used to train the factor model and test set was used to test the trained model. Mplus (Muthén & Muthén, 2017) was used to fit these models.
For training group, an EFA was conducted to determine the most appropriate factor structure for IBFI. Geomin rotation was used so the correlations between factors were estimated. The optimal number of factors was selected based on the eigenvalues (greater than 1) and scree plot, items cross-loadings, and theoretical interpretability. Only items with factor loadings of an absolute value greater than 0.35 were retained, which would explain around 16% of the variance in the variable. Items were assigned to factors on which they loaded most highly. Items that were not loaded or cross-loaded on any factor were removed from further analysis.
For test group, CFA within the framework of structural equation modeling was performed to confirm the factor structure determined in EFA by training group. Model fit was assessed. The overall model fit was examined using various absolute fit indices and incremental fit indices. Absolute fit indices included normed chi-square, root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR). Normed chi-square is defined as the ratio of chi-square and the degree of freedom in the model. Schumacker and Lomax (2004) recommended a value less than 2 for normed chi-square for adequate model fit. RMSEA is a parsimony-adjusted index which is based on the non-centrality parameter. The RMSEA estimates the lack of fit in a model compared to a saturated model and the threshold value for an acceptable model fit is 0.08 (Browne & Cudeck, 1993). SRMR is the square-root of the difference between the residuals of the sample covariance matrix and the hypothesized model. Values SRMR as high as.08 are deemed acceptable (Hu & Bentler, 1999). We also considered two incremental fit indices: Tucker-Lewis index (TLI), and comparative fit index (CFI). Values of these indices over.90 represent a satisfactory model fit (Awang, 2012). After the selection of the final best-fitting model, internal consistency reliability for each factor using the full sample was determined. Convergent validity were also conducted. Pearson’s product-moment correlations were examined between scores on the IBFI and scores on the SF-36, DS14 and the WHODAS 2.0 communication subscale.
Results
EFA was performed with 37 IBFI items on training data. One- to eight-factor models were tested to determine the optimal number of factors to retain. Eigenvalue, scree plot and item factor loadings in each model were examined. The result indicated a solution of five factors due to fewer cross-loaded items and higher factor loadings compared to other factor models. With the exception of 11 items, all remaining 26 items had their highest loadings on their assigned factors (see Table 2). Factor 1 included 5 items and was represented as “Physical”. Factor 2 was comprised of 10 items and is represented as “Psychosomatic”. Factor 3 included 5 items which comprised “Cognitive dimension”. Factor 4 included 3 items comprised as Negative coping and Factor 5 is a dimension of Social isolation, which included 3 items.
EFA of the IBFI
EFA of the IBFI
Note. N = 369. Factor loadings ≥.35 bolded. *p≤.05. α= Cronbach’s Alpha. An oblique Geomin rotation was used. IBFI = Illinois Brief Functioning Inventory.
CFA based on the data from test group, 5-factor model was acceptable (Table 3). Fit indices of the model indicated a χ2/df=3.79, RMSEA = 0.087, CFI = 0.86, TLI = 0.84, SRMR = 0.08. While some of the fit indices fall below the recommended values, this 5-factor structure is the best simple structure in the pattern of relationships between the items that also makes sense theoretically (Kline, 2015). All items were significantly loaded on relevant factors. In addition, five factors were positively related to each other. The significant correlation coefficients ranged from 0.15 to 0.52 (see Table 2).
CFA model of the IBFI
Note. N = 369. Significant estimates bolded (p < .05).
The internal consistency reliability coefficients of each subscale assessed by Cronbach’s alpha was reported in Table 2. All five subscale scores for IBFI showed very strong reliability with a mean of 0.84 (ranged from 0.74 to 0.91). High Cronbach’s alpha for Psychiatric subscale (= 0.91) indicated a strong relationship among items as a group. The internal consistency for Negative coping subscale (= 0.74) was weaker than the other subscales, it nonetheless was considered as fairly acceptable.
Pearson’s product-moment correlations were examined between scores on each IBFI subscale and scores on the SF-36, DS14 and the WHODAS 2.0 communication subscale. Correlations between the five factors of the IBFI and the SF-36, DS14, and WHODAS 2.0 provide convergent validity evidence (Table 4). Specifically, Factor 1 correlated -.746 (p < .001) with SF-36 Pain and -.752(<.001) with SF-36 Physical functioning. Factor 2 correlated -.783 (p < .001) with SF-36 Emotional well-being, .727 (p < .001) with DS14 Negative affectivity. Factor 3 correlated .725 (p < .001) with WHODAS 2.0 communication. Although Factor 4 & 5 did not correlate with any external correlate greater than .70, the subscales had meaningful correlations with appropriate external correlates ranging from .39 to .63. Finally, all 5 factors had significant and meaningful correlations with all of the external correlates providing support to a bio-psycho-social conceptualization of chronic health and disability.
IBFI correlations with SF-36, DS14, and WHODAS 2.0 communication
Note. *p ≤ 0.05. RoleLimit=“Role limitation due to . . . ”.
The primary purpose of this study was to develop a brief psychometrically sound instrument. The IBFI can be used to determine the level of functioning to assist community agencies in making appropriate referrals to the State/Federal VR program, assist VR counselors in plan development, and increase vocational rehabilitation outcomes. The IBFI developed in this study contains 26 items and is comprised of of five meaningful subscales identifying functioning across physical, cognitive, and emotional dimensions (Table 5). The subscales identified show adequate internal consistency providing evidence of both subscale item interrelatedness and stability. Significant and meaningful correlations with the construct of with the SF-36, DS14 and communication subscale of the WHODAS 2.0 provide initial evidence regarding scale and subscale construct validity. Overall, the study findings indicate strong psychometrics for a 26-item instrument with the brevity of the scale increasing potential application and utilization in both research and clinical settings.
Illinois Brief Functioning Inventory items retained after EFA/CFA
Illinois Brief Functioning Inventory items retained after EFA/CFA
Note. Respondents are asked “Do you experience any of the following as they relate to any physical, cognitive, or mental difficulties you have that may impact your work?” and rate themselves on a 6-point Likert scale (0-5) for both severity and frequency.
Regarding research question one the results of the exploratory and confirmatory factor analyses identified five factors. Factor 1 consists of five items that appear to be related to physical functioning. Research in the area of vocational rehabilitation, career development and employment has consistently found that individual’s levels of physical functioning impact an individual’s ability to participate and benefit from vocational rehabilitation and impacts vocational outcomes. Importantly, research regarding employment and functioning has found that the individual’s physical functioning is related to one’s willingness to engage in the necessary activities critical in identifying, acquiring and maintaining involvement with appropriate education and employment activities and outcomes (Strauser, 2021).
Factor 2 consists of ten items that address psychosomatic functioning. Items making up this subscale cover a range of emotional based areas of functioning such as depression and anxiety as well as somatic factors such as pain and problems with sleep. Psychosomatic factors have a significant influence on career and vocational outcomes and present as significant barriers that impact participation in VR activities and negatively impact outcomes. Despite their significance, many times these factors may not be identified by community agencies as disabling conditions and also tend to be overlooked by VR counselors in developing vocational rehabilitation plans. This subscale would assist community agencies in identifying individuals with psychosomatic issues that may benefit from VR services and assist VR counselors in identifying individuals who may benefit from the provision of psychological services to help facilitate and maximize career and vocational outcomes.
Factor 3 consists of five items that address areas of cognitive functioning. Cognition is implicated in a variety of chronic health and disabling conditions and has been found to significantly limit career development and vocational outcomes (Peterson et al., 1991). Importantly, the results of this study highlight the broad impact of cognition on other areas of functioning that are related to positive outcomes. Specifically, cognition (Factor 3) was found to have meaningful significant correlations over .40 with SF-36 Social functioning, SF-36 Emotional well-being, SF-36 Role limitations due to emotional problems, DS14 Negative affectivity, and had the highest relationship with communication.
Factor 4 consists of three items and addresses issues related to the negative dimensions of coping. Engaging in risky behaviors, problems with drugs and alcohol, and aggressive acting out are behaviors and coping strategies that tend to be comorbid with psychological conditions such as depression and may also impact individual’s ability to communicate and meet the social demands of work (Sampson et al., 1996; Strauser & O’Sullivan, 2009). This subscale may be of particular value to VR counselors in the plan development process and may suggest the need for psychological and work adjustment services. In addition, positive responses on this subscale may suggest the need for a more comprehensive evaluation regarding substance and alcohol abuse.
Items making up factor 5 address issues related to social isolation or an individual’s preference to not interact with other individuals and is comprised of three items. Like the psychosomatic factor, social isolation is an area of functioning that has broader implications for vocational functioning and typically goes unaddressed in vocational rehabilitation planning (Strauser et al., 2015). However, recent research has suggested that social isolation negatively impacts career development of both individuals with and without disabilities (Strauser, 2021). In addition, social isolation is often a co-morbid trait with a variety of psychiatric conditions ranging from autism to depression and anxiety (Fitzgerald et al., 2015).
Findings related to research questions two and three provide initial support regarding the structure of the IBFI. The internal consistencies for the five subscales, .85 = Factor 1, .91 = Factor 2, .86 = Factor 3, .74 = Factor 4, and .83 = Factor 5 (Table 2) suggest that the within subscale items are highly related and demonstrate adequate reliability. In addition, each of the five factors had significant and meaningful correlations with appropriate external correlates providing adequate initial construct validity evidence (Table 4). An important finding is that all five subscales of the IBFI had significant correlations across the related measures providing evidence that the IBFI covers the physical, emotional and cognitive dimensions of health.
Overall, the results of this study provide good initial evidence that the IBFI has the potential to be a useful screener for community agencies to identify individuals for referral to VR as well as an effective tool for VR counselors to use in the plan development process. It is also important to indicate what the IBFI is not designed to address. First, the IBFI is not designed to determine if an individual is eligible for VR services. Determining eligibility for VR services is the role of the VR counselor and will vary depending on state, case load size, order of selection, and other state and system specific factors. Second, the IBFI is not a diagnostic instrument. Results of the IBFI may provide some information or guidance to both community-based organizations and VR counselors regarding potential issues related to functioning that may need to be addressed through specific services or the potential need for additional diagnostics such as neuropsychological, psychological, sensory, or orthopedic evaluations.
Although the results of this study are promising, additional research is needed to examine the factorial structure of the scale and generate additional construct validity evidence with a sample that may be more representative of the individuals served by the State/Federal Vocational Rehabilitation system in the United States. Examining the predictive validity of the scale would appear to be particularly important. Collaborative research is also needed with community-based organizations and VR counselors to further examine the clinical utility of the scale. Specific questions that should be addressed to both groups should inquire if the IBFI is assessable and easy to use and whether or not agency staff find the results easy to interpret and usable.
Several potential limitations should be considered when interpreting the results of the study. First, while the sample was comprised of individuals with disabilities and chronic health conditions, participants may not represent all individuals with disabilities. It is likely that sample characteristics may not mirror those of individuals who are typically served through vocational rehabilitation. Moreover, the sample drawn in this study was primarily female, well educated, and employed, which differs from the general demographic of individuals with disabilities, thereby limiting the generalizability of the study findings. Additionally, the sample was limited in that cross-sectional and self-report data was obtained through an online format.
Conclusion
The results of this study provide initial psychometric support regarding the IBFI as an instrument that can be used in both clinical and research settings to identify an individual’s current level of functioning. The instrument is made up of five subscales that measure physical, psychosomatic, cognitive, coping, and social dimensions of functioning. Clinically, the scale was designed to can be used identify individuals who may benefit from VR services and to assist VR counselors in the plan development process. The IBFI also provides researchers with a psychometrically sound instrument that can be used to help guide research related to functioning, career development and employment. However, more research is needed to confirm the factorial structure and construct validity of the scale. In addition, more research is needed to understand the potential clinical applications of the scale.
Footnotes
Acknowledgments
None to report.
Conflict of interest
The authors declare that they have no conflict of interest.
Ethics statement
All procedures performed in this study were in accordance with the ethical standards of the University of Illinois at Urbana-Champaign and/or national research committee with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Human subjects research approval for this project was provided by the institutional review board of the University of Illinois at Urbana-Champaign (# 19879).
Funding
This paper was developed with support from the Vocational Rehabilitation Technical Assistance Center for Targeted Communities (VRTAC-TC or Project E3) at the Department of Rehabilitation and Disability Studies, Southern University, Baton Rouge, LA funded by the US Department of Education, Rehabilitation Services Administration (Grant number H264F15003). The ideas, opinions, and conclusions expressed, however, are those of the authors, and do not represent recommendations, endorsements, or policies of the US Department of Education.
Informed consent
Prior to enrolment, participants were informed that their participation was voluntary, were free to withdraw at any time, and that they would receive a small honorarium for completing the survey. Informed consent was obtained from each participant.