Abstract
Introduction
Persons with moderate to severe TBI are at increased risk for unintentional injury or harm (UIH) when transitioning from medical inpatient and outpatient rehabilitation to home and community settings (Seel, et al., 2007). Four TBI registry studies show that unintentional injury is a leading cause of death (18– 20%) in the year following discharge, and the standardized mortality rate from UIH is 3 to 36 times greater for people with TBI compared to the general population (Harrison-Felix, et al., 2006; McMillan & Teasdale, 2007; Selassie, et al., 2005; Ventura, et al., 2010). Unintentional injuries also cause high rates of emergency room (ER) visits and hospitalizations. Carlson et al. (2012) found that 32% of TBI participants (n = 504) had unintentional injuries that resulted in 228 ER visits or hospitalizations from 3-months to several years following discharge for a TBI. These unintentional injuries were similar to those found in the general population, such as falls; motor vehicle crashes; pedestrian, bicycle or motorcycle crashes; improper use of electrical equipment; fire and burns; being struck by or against an object; poisoning; and firearm-related incidents (Carlson, et al., 2012). Additional risk of harm may occur due to victimization, loss of money or valuables, property damage, medication errors, and inappropriately responding to emergency situations or managing medical conditions (J. S. Kreutzer, et al., 2009; Reichard, et al., 2007; Tyson, Pham, Brown, & Mayer, 2012). Caregivers and payers often manage these risks by providing part- or full-time supervision, which have high economic and social costs.
Self-managing activities that increase UIH risk is challenging for persons who have sustained TBI. Impaired attention, memory, visuomotor skills, awareness and self-regulation affect individuals’ abilities to perform everyday activities, recognize risk, take preventive actions, learn safety strategies, and apply compensatory skills (Riley, Brennan, & Powell, 2004; Tyson, et al., 2012). Persons with TBI who report difficulties performing activities often experience anxiety, frustration, loss of confidence, and depression that result in activity avoidance and diminished life quality (Riley et al., 2004; Seel, Macciocchi, & Kreutzer, 2010; Selassie et al., 2008). Conversely, persons with TBI who highly value their independence may dismiss family and provider concerns about safety, which can lead to confrontation, conflict, power struggles, and strained relations (Durgin, 2000).
For family members, balancing supervisory needs, independent return to activities, and safety is complicated. Families often rely on rehabilitation providers’ determinations of supervision needs (Cooney, Kennedy, Hawkins, & Hurme, 2004; Galski, Ehle, McDonald, & Mackevich, 2000; Macciocchi & Stringer, 2001; Tyson et al., 2012) but short rehabilitation stays often do not allow providers to assess patient safety on instrumental activities of daily living (IADLs) in the home and community. Further, little time is available to train families how to manage high risk situations. Providers often recommend high levels of supervision and activity restrictions, which may unnecessarily limit clients’ autonomy (Banja, 1994; Macciocchi, 2009; Macciocchi & Stringer, 2001; Ruchinskas, Macchiocchi, Howe, & Newton, 2001). While many persons with TBI-related impairments show improvement in physical and cognitive functioning over time, some families continue to follow providers’ initial supervision recommendations or may not know how or when to help survivors transition to independent and safe engagement in IADLs. Other families may have limited support or may need to return to work, which may force them to leave TBI survivors on their own and in environments that present hazards and increased risk (Pai, Zadov, & Hickman, 2012). Consequently, many people with TBI who are at high risk for UIH are under-supervised while others who have low risk are overly restricted (Durgin, 2000).
Currently, there is no established measure to guide family members, rehabilitation providers, and payers in assessing safety risk in the home and community for persons who have sustained TBI. An empirically validated safety assessment measure would improve rehabilitation practice by providing observable, quantified evidence that can improve caregiver knowledge of activities and factors associated with unsafe events in the home and community following TBI. Such a measure would provide an evidence basis to guide the level of family involvement required to keep persons that have TBI-related impairments safe within the least restrictive environment. A safety assessment measure may also provide an evidence basis to justify additional healthcare service provision including outpatient rehabilitation, home health/community supports, or life skills coaching to facilitate self-management of safety-related activities in high risk TBI survivors.
Using a contemporary, state-of-the-science, mixed qualitative-quantitative methods approach within the framework of modern measurement theory would facilitate the development of a safety assessment measure that could enhance clinical practice and research. Qualitative methods that incorporate stakeholder input and feedback to develop an ecologic and content valid item pool have become a standard initial phase in the outcome measurement development process (Administration, 2009; Cella et al., 2007; Lasch et al., 2010; Magasi et al., 2012; Velozo, et al., 2012). A safety assessment measure based on item response theory (IRT) methods would possess a hierarchical, progressive set of items that could serve as a behavioral diagnostic for treatment planning, modifying home environments, and implementing technological supports. An empirically valid, interval level, safety assessment measure would also provide clinicians and researchers with a reliable instrument that could evaluate longitudinal changes in safety risk, including outcomes for clinical trials that evaluate interventions to improve safety.
This study is the first phase of a three phase scale development project to develop and evaluate the Safety Assessment Measure (SAM) for persons who have sustained moderate or severe TBI. The SAM is a set of scales that captures a wide range of safety issues that persons with TBI face in their everyday living situations in the home and the community. The SAM is intended for use following inpatient rehabilitation discharge with the primary raters of safety and risk being family and non-family caregivers, as well as clinicians and paraprofessionals in outpatient rehabilitation, day program, brain injury clubhouse, home health, and neurobehavioral treatment settings. The primary objectives of this study are to define the conceptual framework and constructs that ground the SAM scales and items, describe the item pool for each proposed scale, and evaluate each scale’s content validity.
Method
Qualitative approach to develop the safety assessment measure
Based on state-of-the-science recommendations on developing health outcome measures, we undertook three qualitative steps (see Fig. 1) to develop the SAM item pool: step 1-conceptualize a theoretical model and scale constructs; step-2 develop an item pool; and step 3-evaluate and revise the item pool (Cella et al., 2007; Velozo et al., 2012). Participants included the research team, the scientific andclinical advisory boards, five clinicians from our TBI day rehabilitation program, additional TBI rehabilitation health providers who participated in focus groups, and family members of persons with TBI who provided feedback in cognitive interviews. Key methods within each step are denoted using capital letters. The resulting primary products (denoted within framed text boxes at the bottom of each step) were then used to inform proceeding steps (indicated by arrows).
Conceptualize model and constructs
Methodological framework for measurement
Item response theory (IRT) and use of Rasch methods are the state-of-the-science framework for developing and evaluating outcome measures (Administration, 2009; Cella et al., 2007; Reeve & Fayers, 2005; Velozo et al., 2012). A core principle of IRT is that the probability of rating an item with a response that indicates a higher level of functioning is dependent on “person ability,” “item difficulty,” and the item’s success in discriminating person ability on an outcome of interest (Velozo et al., 2012). Thus, a difficult item on a measurement scale should have a higher probability of being successfully performed by persons with high ability than persons with medium or low ability. Another core principle of IRT is that scale items must be unidimensional (i.e., measure a single latent construct), and be capable of detecting differences in person ability related to the construct (Reeve & Fayers, 2005; Velozo et al., 2012). Differentiation within a scale requires that items are developed in an a priori hierarchical difficulty order within each of the scales (Stone, 1997; Velozo et al., 2012).
Definition of safety risk and contexts for scale use
We defined safety as “the ability to participate in home activities and community living free from harm.” Safety risk is a state of uncertainty in which unintentional harm may arise from a future event, activity, or behavior. Temporally, the threat of a safety risk is likely or imminent. Safety risk is observable and can lead directly to unintentional harm including death, physical injury, financial loss, property damage, or incarceration. The scope of safety risk is not limited to the person with TBI-safety risk may affect other persons, animals, and property. Exposure to violent and aggressive acts is also a safety risk. Conversely, pre-meditated, intentional harmful acts to one’s self (e.g., self-mutilation, suicide), another person, or property secondary to sociopathic and psychiatric disorders are outside the scope of our assessment.
The target of evaluation is persons aged 16 and older who sustained moderate or severe TBI, required acute hospitalization for their injury, and are now returning to everyday activities of daily living in the home and community. Self-assessment by persons with attention, memory, judgment, and self-control deficits are affected by impaired awareness and meta-cognition (Cavallo, Kay, & Ezrachi, 1992; Hart, Sherer, Whyte, Polansky, & Novack, 2004; Malec, 2004; Seel, Kreutzer, & Sander, 1997). Therefore, we focused on collecting safety ratings from family members and health providers in this initial phase of measure development.
The primary use for the safety measure is clinical with an emphasis on using the measure to: (a) make empirically-based prognoses on risk of unsafe events; (b) assess current safe functioning level and identify primary person factors that drive risk; (c) inform individualized treatment planning related to prevention, supervision needs, and specific functional areas and activities to be targeted; and (d) assess change in safety risk over time, including evaluating individual treatment as well as program effectiveness. A second important use is research and includes longitudinal, prognosis and treatment efficacy and effectiveness studies. Long-term, payers and policy-makers may find epidemiological and treatment effectiveness data useful for reimbursing home health care/supervision and/or prevention education and treatment.
Model and construct development
The research team used a simple concept mapping approach to develop models and scale constructs. First, we reviewed the literature and evaluated models related to unintentional injury and prevention, and self-management of chronic medical conditions, to identify potential person-centered behavioral determinants (Sleet & Gielen, 2007; Sleet, et al., 2006). Next, we considered how TBI-related impairments – cognitive, physical, and behavioral-might play a role in safety risk. Social Cognitive Theory models provided input on person-environment-behavior factors and interactions (Bandura, 1986; Simons-Morton, 2006; Sleet et al., 2006). Then, we constructed a model based on an individual level behavior change framework with four person factor constructs and operational definitions that would serve as the basis for creating scales. Two conference calls each were conducted with the clinical and scientific advisory boards to review the proposed constructs, obtain agreement on their importance, and evaluate the operational definitions of constructs for completeness and unidimensionality.
Develop item pool
Item identification, binning and winnowing
Item pool content was developed with consideration given to the primary scales and their operational definitions, safety concerns in the general population, safety concerns specific to the TBI population, and feedback from TBI therapists and our clinical advisory board. We reviewed the research literature to identify measures and items that were relevant to the core scale constructs that would comprise the Safety Assessment Measure (SAM) (DeWalt, et al., 2007). We reviewed 11 self-report or clinician-rated measures of neurobehavioral function, decision-making, risk-taking, awareness of impairment, and safety judgment and function in rehabilitation or psychiatric populations (Blais & Weber, 2006; Borgaro & Prigatano, 2003; Chiu & Oliver, 2006; Hart, 2000; Hart et al., 2006; J. Kreutzer & Marwitz, 2000; J. S. Kreutzer et al., 2009; J. Kreutzer, Seel, & Marwitz, 1999; Morrongiello & Corbett, 2006; Oliver, Blathwayt, Brackley, & Tamaki, 1993; Patton, Stanford, & Barratt, 1995; Sherer et al., 1995; Velozo & Peterson, 2001; Whyte, Hart, Bode, & Malec, 2003; Yudofsky, et al., 1986). Only 2 measures specifically addressed safety issues in the home and community, both of which were organized predominately by activity type (Chiu & Oliver, 2006; J. Kreutzer & Marwitz, 2000). Conversely, the SAM scale constructs are predominately organized by functional abilities that would identify the underlying causes of safety risk across everyday activities and participation. From these 11 measures, we binned and winnowed items that were potentially associated with our 6 primary scale constructs (DeWalt et al., 2007) Forty-six of these extracted items were modified and included in the field test version of the SAM scale item pools.
Item revision and development
To assure full coverage of real life experiences within each scale construct, the research and clinical team developed additional items. In addition to personal experience with the TBI population, we used the Centers for Disease Control and Prevention-National Center for Injury Prevention and Control (Centers for Disease Control and Prevention, (n.d.)) injury reporting structure for the general population to assist in identifying activities in which injuries may be likely to occur (see Table 1) (Centers for Disease Control and Prevention, (n.d.)). We supplemented the CDC list, based on feedback from the clinical team, clinical advisory board and family members, with other activities and aspects of harm that are common in persons who sustain moderate to severe TBI (see Table 1, column 1). We also considered the settings in which UIH occurs including inside and outside of the home; on streets, highways, sidewalks, and parking lots; in sports facilities, recreation centers, lakes, rivers and pools; and in the community such as in public buildings. Because the focus of our scale was adults who were transitioning from basic self-care to instrumental activities of daily living, we did not create items for hospital-based, school, or workplace settings.
Several sets of administration instructions and item rating schemes were considered that emphasized family and clinician raters’ direct observation of person behavior on specific activities, controlled for the confounds of assistance or supervision during activities, and allowed for judgment in determining levels of risk (DeVellis, 2006). The SAM rating instructions read as follows: “The following Items in this questionnaire ask about behaviors that could put a person with a brain injury at increased risk for harm. Based on your best judgment of the person’s current ability and behavior, please rate the likelihood of each behavior occurring if the person with TBI did not have any help or supervision – for example, if no one was there to step in, how likely is it that the person will not be able to exit a room on their own in an emergency?”
The item rating scheme has the following four responses: 4-highly likely to occur; 3-somewhat likely to occur; 2-somewhat unlikely to occur; and 1-highly unlikely to occur. A not applicable response is offered for a subset of items with a specific reason for each not applicable response noted, e.g., for the item, “Is not responsible with firearms,” “Not applicable, person with TBI does not use firearms”.
The research team edited and standardized item wording to: (a) fit the selected rating scheme; (b) be unidirectional, e.g., not reverse scored; (c) be easy to understand with examples provided if deemed helpful; and (d) maintain no higher than a 9th grade reading level. To meet the key assumptions of IRT, the research and clinical teams re-reviewed items and the proposed scale constructs for unidimensionality (Reeve & Fayers, 2005). Each item was then rated as being of high, medium, or low difficulty to assess coverage and identify “gaps” in the scale’s hierarchical difficulty continuum including floor and ceiling effects (Stone, 1997; Velozo et al., 2012). The clinical and scientific advisory boards provided feedback on the rating scheme, item selection and editing, and the final model and scales.
Evaluate and refine the item pool
Health provider focus groups and rating of content validity
Health providers participated in 6 focus groups to discuss item content, clarity, and comprehensiveness. Participants were recruited from Shepherd Pathways Day Rehabilitation Program, Marcus Community Bridge Program, Side-by-Side Brain Injury Clubhouse, and Restore Neurobehavioral Center in Atlanta, Georgia. The 5 clinicians involved in item development did not participate in the focus groups. Focus group sizes ranged from 2–6 participants and lasted 60–70 minutes. The research coordinator led semi-structured group interviews. Following group discussion, simple concept mapping was used in which healthcare providers independently quantified the extent that each scale item represented the safety experiences of persons with TBI using a 4-point rating scale (4-critically important, 3-very important, 2-somewhat important, and 1-not important). All focus group proceedings were audio taped and transcribed with identifiers removed.
Following the completion of all focus groups, the research coordinator reviewed the transcripts, created a spreadsheet summarizing respondent comments, and entered health provider item ratings into an SPSS data file. The mean importance score for each item was calculated. An Item Level-Content Validity Indicator (I-CVI) ratio was determined by computing the percentage of critically important plus very important ratings for each item (Polit & Beck, 2006). Excellent item validity is indicated by an I-CVI ratio of ≥ 78% (Polit & Beck, 2006). The research team reviewed the focus group summaries and content validity ratings, and modified, expanded, deleted, or added items accordingly.
Cognitive interviewing of family members and final item revisions
The lead research coordinator conducted cognitive interviews with family members of persons with TBI using a semi-structured script to obtain feedback on item comprehension, meaningfulness, and comprehensiveness (DeWalt et al., 2007; Willis, 2005). Interviews were audio-taped and transcribed. Five family members were interviewed and items revised based on their feedback. Following these revisions, four more family members were interviewed. With minor changes following their feedback, the research team was satisfied that the item pool was comprehensive and understandable.
Results
Social cognitive theory framework of unsafe events following TBI
Social cognitive theory was selected as the most suitable framework for understanding person, environment, and behavior factors that represent safety risk in the home and community. We conceptualized unsafe behavior and events as the result of interactions between the person with TBI, his or her everyday environment, and behavior (see Fig. 2). Personal factors including the abilities, beliefs and expectancies of a person who has sustained moderate to severe TBI are considered the primary determinants of safety risk and behaviors that result in UIH. The 6 primary SAM scales measure person abilities (indicated by bold lettering) and are hypothesized to be the most critical factors associated with UIH. Environmental factors such as family members’ abilities, beliefs, and expectancies; the home and community settings; social relationships; and community supports play mediating roles to varying degrees in shaping personal factors and behavior. In particular, the amount and proximity of family involvement (e.g., supervision, care coordination) is a primary environmental factor that interacts with person factors to influence UIH. The behavior factors of persons with TBI include their activity levels, performance of activities, experiential learning, social interactions, emotional responses, and working alliance with family members. These behavior factors provide an observable gauge of safe and unsafe functioning and feedback that shapes person and environmental responses to behavior.
Scale constructs (Personal Ability Factors) and definitions
The proposed constructs, a summary of stakeholder and expert feedback on the constructs, and the final construct/scale names and their operational definitions are presented in Table 2. Primary decisions reflected in Table 2 include: (a) removing primarily upper extremity based activities from the Physical Capacity Scale and focusing on walking mobility as a primary construct, (b) adding Wheelchair Use as a primary construct/scale; and (c) separating activities and behaviors related to complying with medical recommendations from perceiving risk and creating separate scales. The six scales are: Cognitive Capacity, Visuomotor Capacity, Wheelchair Use, Risk Perception, Compliance Failures with Safety Recommendations, and Self-Regulation.
Risk due to substance use was developed as a stand-alone SAM scale and 19 items were generated. Following clinical and research advisory board review, there was consensus that alcohol and other drug use is not a unidimensional construct and that items would not conform to a hierarchical continuum. We decided to use existing measures to assess the presence and severity of alcohol and other drug use, and evaluate their association with safety risk. Motor vehicle operation and childcare items were initially developed and distributed across the 6 primary SAM scales. Given these activity sets are applicable to only a relatively small subset to persons with moderate or severe TBI, the clinical and research advisory boards and research team agreed that these activities would best be assessed on supplemental scales named Motor Vehicle Operation and Child Caretaking.
Safety Assessment Measure item pool and content validity ratings
Focus groups evaluated the Safety Assessment Measure item pool (n = 145 items) for the 6 primary and 2 supplemental scales. Participants were 26 TBI healthcare professionals with backgrounds in case management, nursing, occupational therapy, physical therapy, speech language pathology, therapeutic recreation, and psychology. On average, healthcare professionals were aged 40.7 (SD = 12.2), predominately female (85%), predominately white (69%) or black (19%), and were well-educated (42% with master’s and 35% with bachelor’s degrees). Health providers had a mean of 9.2 (SD = 7.9) years of TBI rehabilitation experience and reported on average 36.6 (SD = 9.5) hours weekly treating clients with TBI.
Based on focus group feedback and core research team re-examination of items, 32 items were removed from the pool primarily due to confounds (e.g., multiple activities or behaviors assessed within item content), similarity with retained items, or the activity lacked an imminent safety threat. The retained items (n = 115 items) and their content validity ratings, as well as 11 new items recommended in either the focus groups or the family member cognitive interviews are presented in Table 3, columns 1–3. Overall, the SAM item pool was rated as having high levels of content validity. Of the 115 retained SAM scale items, 106 items (92%) had Item-Content Validity Index ratios ≥ 78%, indicating excellent content validity. Sixty-nine (60%) of the retained items had I-CVI ratios ≥ 90%.
Overall, the SAM item pool covered the full range of activities, behaviors, and causes of injury in the general population and specific to persons with TBI (see Table 1, column 2.). Items related to transportation activities (24%), falls (14%), aggressive or violent behaviors (12%), and identifying, avoiding, or responding to potentially harmful others (9%) comprised 57% of the item pool content. For each of the 6 primary scales and 2 supplemental scales, item content reflected a wide range of activities, behaviors, and causes of injury (see Table 3, columns 4) as well as the people and property that could be potentially affected by harmful activities (see Table 3, columns 5).The Cognitive Capacity and Compliance Failures with Safety Recommendations scales covered the largest number of activities/behaviors within their scale content with 10 each. The Risk Perception and Self-Regulation scales had 7 and 6 different activities or behaviors, respectively, covered in their item content. The Visuomotor Capacity, Wheelchair Use, and Motor Vehicle Operation scales had more focused content.
Discussion
Summary of study findings
Family, provider, and payer understanding of the specific activities in the home and community that pose threats and the cognitive, physical, and behavioral impairments that underlie risk is essential to managing the safety of persons who have sustained TBI. Our study presents a conceptual framework for understanding safety risk in the home and community following moderate to severe TBI. Person factors were identified as the key determinants of unsafe behavior and six primary constructs-Cognitive Capacity, Visuomotor Capacity, Wheelchair Use, Risk Perception, Compliance Failures with Safety Recommendations, and Self-Regulation-represented the most critical person factors that underlie safety risk. This conceptual framework and the operational definitions for the primary person constructs served as the basis for developing the SAM scales and item pool.
The SAM assesses risk on a large number of IADLs as well as the underlying abilities/ impairments that underlie safe and unsafe performance of theseactivities. Our findings provide evidence that the SAM scales and item pool (n = 126 items) cover a wide range of real life experiences and everyday activities that place persons with TBI at risk for UIH. The SAM items cover 16 broad sets of activities, behaviors, and causes of UIH, most of which map to how the CDC monitors unintentional injury causes and activities in the general population. The SAM items also reflect the primary settings such as the home, roads, recreational centers, and the wider community in which harm occurs and the casualties of harm including persons with TBI, other people, animals, and property. The SAM items were generally viewed as being comprehensive, rated as having excellent content validity, and seen as being easily understandable based on feedback from rehabilitation providers participating in focus groups and family members who engaged in cognitive interviews.
Safety assessment measure methodological considerations and limitations
The structured qualitative methods that we used to develop the SAM scales and item pools are consistent with recommendations to develop “patient-centered” outcome measures that encompass patients’ needs, values and everyday experiences when assessing their health, function, and participation (Administration, 2009; Frank & Polkinghorne, 2010; Lasch et al., 2010; Ohman, 2005; Onwuegbuzie, Bustamante, & Nelson, 2010). Fifteen months was invested in developing a conceptual framework and item pool that reflects a wide range of activities that pose potential harm to the general population as well as persons who have sustained TBI. This time investment including feedback from clinical and family stakeholders was critical to achieving an item pool with high levels of content validity.
Our emphasis on using health provider feedback in developing SAM content reflected our decision to leverage a highly experienced group of clinicians who could reflect on and report their observations of thousands of persons who have sustained TBI in inpatient facilities, outpatient clinics, and in the home and community. We sought family member feedback when evaluating item clarity and understandability so that the SAM would be most usable to a wide range of family members making observations in the home and community. Based on our experience with developing item pools for individuals with disabilities, we used an interview format for the family caregivers, which allowed the instruction that family caregivers need to understand the review process and the individualized attention and time needed to provide comprehensive feedback. Field testing of the SAM with a large group of family raters, including 30-day follow-up interviews on unsafe events, will provide further opportunities to obtain detailed feedback on item content comprehensiveness andclarity.
Our decision to develop items that rate unsuccessful behaviors in the home and community reflects the nature of assessing risk behavior. We also received feedback from families that they found it easier to rate unsuccessful behavior when assessing levels of safety risk. A number of healthcare providers recommended increasing the specificity of causal assessment data as part of the SAM items, e.g., differentiating visual and physical components of losing balance in the Walking Mobility scale item content. Given the importance of obtaining reliable, family rated observations of behavior in the home and community, we decided that these distinctions would be difficult for family raters to make. While pre-injury abilities can play significant roles in safety risk (Pietrapiana et al., 2005), the SAM measures current ability and does not distinguish between pre-injury and injury-related abilities.
The SAM item pool addresses instrumental activities of daily living that may lead to unintentional injuries or harm in the home and community. The SAM does not assess unsafe activities in inpatient medical, school or work settings. The SAM item pool does not describe pre-meditated, intentional harmful acts to one’s self (e.g. self-mutilation, suicide), another person, or property, which are best assessed by existing psychiatric measures. The SAM also does not address the risk of persons with severe TBI being victims of violence by a parent, spouse, child, or other family caregiver in their home. This issue is not well-suited for family and caregiver self-report assessment. The SAM may not be applicable to persons who are dependent in most basic self-care activities and thus would not attempt more complex instrumental activities of daily living.
The SAM was developed for persons at least aged 16 and older who are likely to have been independent in many everyday activities prior to sustaining TBI. Young children, adolescents, or adults with significant pre-injury developmental disability would likely have different presentations, interests, and developmental expectations. A pediatric measure of safety would need to address the expectations and environments in which children participate (e.g., school), as well as consider developmental changes that reflect recovery, ongoing physical and cognitive development, and the acquisition of new versus previously learned functional skills.
Contributions of this study to the literature and future research
Our development of the SAM item pool is among the first efforts to use state-of-the-science methods to identify a broad range of specific activities and behaviors that are linked to risk of UIH and categorized by common TBI-related impairments. The SAM is unique compared to the few existing safety instruments in that it’s scales and item content are: (a) more highly connected to a theoretical framework for understanding safety risk following TBI; (b), more comprehensive and categorized by person functional ability rather than a limited number of activities; and (c) linked to how the U.S. CDC monitors activities and causes of unintentional injury. The comprehensiveness of the activities covered by the SAM scale items and their categorization by person functional ability will help clinicians and family members identify individual activities and their underlying causes that drive safety risk and prioritize those areas for treatment planning, intervention, progress monitoring, and prognosis.
The SAM is also unique compared to existing safety instruments in that our construct and scale development procedure included formally evaluating the item pool by using semi-structured focus groups with rehabilitation providers and cognitive interviewing of family members. This formal evaluation process with multiple stakeholders produced empirical, quantitative evidence that the content validity of the SAM item pool is generally perceived to be excellent, and increases the likelihood that the SAM scales and items can ultimately standardize observations and improve communication between families, clinicians and payers. Lastly, the SAM is unique in that it was developed using an item response theory framework that emphasized developing items within scales that were unidimensional and covered a range of low, medium and high activity difficulty. This will allow each SAM scale to be further evaluated using Rasch analysis.
Future research
The SAM scales and items will be field tested on a TBI development sample rated by family members and rehabilitation clinicians. Scale validation using IRT methods and Rasch analysis would evaluate the evidence of scale unidimensionality, person and item fit, and precision. Longitudinal tracking of persons with TBI unsafe behavior and events following a baseline SAM assessment would provide important data to evaluate the diagnostic and prognostic validity of the SAM scale scores. Evidence of diagnostic and prognostic validity would improve interpretation of SAM scale scores so that they could inform optimal levels of family involvement in person with TBI activities and the least restrictive environment in which persons who have sustained TBI can safely participate in activities. Health payers may find value in using SAM scale scores to identify high risk individuals who would benefit from community supports or home healthcare. The relationship between SAM scale scores and formal tests of cognitive and executive function in persons with TBI would also be of interest.
Conclusions
The Safety Assessment Measure scales and associated item pools assess a broad range of instrumental activities of daily living that can lead to unintentional injuries or harm in the home and community for adults who have sustained moderate or severe TBI. The Safety Assessment Measure scales assess risk related to diminished cognitive capacity, visuomotor capacity and/or wheelchair use, risk perception, compliance with safety recommendations, and self-regulation. Interdisciplinary rehabilitation clinicians rated most SAM items as having excellent content validity. Future research should use modern psychometric methods to examine scale unidimensionality, model fit, and precision.
Conflict of interest
We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated. We certify that all financial and material support for this research and work are clearly identified in the title page of the manuscript. The contents of this article were developed under a grant from the National Institute on Disability and Rehabilitation Research (grant no. H133G080153). However, these contents do not necessarily represent the official policy of the Department of Education or the Centers for Disease Control and Prevention. No endorsement by the Federal Government is implied.
Footnotes
Acknowledgments
We gratefully acknowledge the contributions of the clinical staff and advisory board members from the four centers/programs who participated in this study. In particular, we thank Leanne Dennis, CTRS, CCM, Greg Wright, MS, CRC, CCM, Tracey Wallace, MS, CCC-SLP, and Sary Newman, PT (Shepherd Pathways); Cindi Johnson, MA, CBIST (Side-by-Side Brain Injury Clubhouse); Virginia Martin, M.Ed., CCC-SLP, CCM (Shepherd Transition Care Program); Sara Kirkpatrick, PhD (Restore Neurobehavioral Center); and Matt Breiding at the Centers for Disease Control and Prevention.