Abstract
There is a growing recognition of the need for interventions to improve the healthcare of autistic adults. However, there is a dearth of validated measures to evaluate such interventions. Our objectives were to use a community-based participatory research approach to create an accessible set of patient- and proxy-reported instruments to measure healthcare outcomes and potential intervention targets in autistic adults and to assess the instruments’ psychometric characteristics, including content validity, construct validity, and internal consistency reliability. We administered a survey to 244 autistic adults recruited from 12 primary care clinics in Oregon and California, USA (194 participating directly and 50 participating via a proxy reporter). Community partners ensured items were easy to understand and captured the intended construct. The Academic Autism Spectrum Partnership in Research and Education (AASPIRE) Visit Preparedness Scale, Healthcare Accommodations Scale, and Patient–Provider Communication Scale were each found to have a single factor. The AASPIRE Health and Healthcare Self-Efficacy Scale had two factors: Individual Healthcare Self-Efficacy and Relationship-Dependent Healthcare Self-Efficacy. Both patient- and proxy-reported versions of all scales had good to excellent internal consistency reliability, with alphas ranging from 0.81 to 0.96. The scales were associated with the Barriers to Healthcare Checklist and the Unmet Healthcare Needs Checklist in the hypothesized directions.
Lay Abstract
Interventions to improve healthcare for autistic adults are greatly needed. To evaluate such interventions, researchers often use surveys to collect data from autistic adults (or sometimes, their supporters), but few survey measures have been tested for use with autistic adults. Our objective was to create and test a set of patient- or proxy-reported survey measures for use in studies that evaluate healthcare interventions. We used a community-based participatory research (CBPR) approach, in partnership with autistic adults, healthcare providers, and supporters. We worked together to create or adapt survey measures. Three survey measures focus on things that interventions may try to change directly: (1) how prepared patients are for visits; (2) how confident they feel in managing their health and healthcare; and (3) how well the healthcare system is making the accommodations patients feel they need. The other measures focus on the outcomes that interventions may hope to achieve: (4) improved patient–provider communication; (5) reduced barriers to care; and (6) reduced unmet healthcare needs. We then tested these measures in a survey of 244 autistic adults recruited from 12 primary care clinics in Oregon and California, USA (with 194 participating directly and 50 participating via a proxy reporter). Community partners made sure items were easy to understand and captured what was important about the underlying idea. We found the survey measures worked well in this sample. These measures may help researchers evaluate new healthcare interventions. Future research needs to assess whether interventions improve healthcare outcomes in autistic adults.
Keywords
Numerous studies have shown that autistic adults experience significant health and healthcare disparities. Specifically, autistic adults have more frequent co-occurring physical and mental health conditions (Croen et al., 2015; Fortuna et al., 2016) with higher rates of premature mortality (Hirvikoski et al., 2016) and inpatient mortality (Akobirshoev et al., 2020). They also have greater overall healthcare utilization and costs (Zerbo et al., 2019) and higher use of the emergency department (Nicolaidis et al., 2013; Vohra et al., 2016), with lower use of recommended preventive services, such as Papanicolaou smears (Nicolaidis et al., 2013; Zerbo et al., 2019). Similarly, studies focused on quality of care have found that autistic adults have greater unmet healthcare needs, lower satisfaction with patient–provider communication, and more barriers to care than non-autistic adults (Mason et al., 2019; Nicolaidis et al., 2013; Raymaker et al., 2017).
As such, there have been many calls for interventions to improve the health and healthcare of autistic adults (Barber, 2017; Hall et al., 2015; Liu et al., 2017; Nicolaidis et al., 2014; Zerbo et al., 2015). However, this field is still in its infancy, with few evidence-based interventions to improve care (Mason et al., 2019). An important early step in creating effective interventions entails identifying potential intervention targets (also known as mechanisms of action). Our prior qualitative studies and pilot intervention evaluation suggested that interventions aimed at improving healthcare for autistic adults may do so by helping patients better prepare for visits, increasing patient and provider self-efficacy, and increasing the use of necessary accommodations in healthcare (Nicolaidis et al., 2015, 2016). The next step is identifying instruments to validly measure both the desired healthcare outcomes and potential intervention targets. However, there is a dearth of validated measures for use with individuals on the autism spectrum (U.S. Department of Health and Human Services, 2017). Most existing health services measures have not been tested for use with autistic adults. The use of such measures, without adequate adaptation or testing, may significantly weaken the rigor of intervention evaluations, given autistic adults’ impressions that existing instruments may not be accessible, may not fully capture the intended constructs, and may cause confusion, anxiety, frustration, or anger (Nicolaidis et al., 2019).
Our objectives were to use a CBPR approach to create a set of accessible patient- or proxy-reported instruments to be used in research aimed at improving healthcare for autistic adults and to assess their preliminary psychometric properties, including content validity, construct validity, and internal consistency reliability. Given what is already known about the existence of healthcare disparities for autistic adults, and the urgent need for interventions, these instruments focus on healthcare outcomes—including patient–provider communication, barriers to healthcare, and unmet healthcare needs—and potential intervention targets—including healthcare self-efficacy, visit preparedness, and provider-and-staff use of accommodations.
Methods
Community–academic partnership
The Academic Autism Spectrum Partnership in Research and Education (AASPIRE; www.aapire.org) is a long-standing community–academic partnership that has been using a CBPR approach since 2006. The team includes academics, autistic adults, family members, healthcare providers, and disability services providers, some of whom have overlapping roles. Academic and community partners collaborated as equal partners throughout the project to (1) make decisions about the research questions and study design using a consensus process; (2) co-create consent and recruitment materials; (3) adapt existing instruments; (4) create new data collection instruments; (5) interpret data; and (6) co-author this article. Our community partners were instrumental in ensuring that study questions were relevant to the autistic community and that materials, including survey instruments, used accessible language, were easy to complete, fully addressed the intended construct, and were as respectful as possible. Further details about our collaboration processes are available elsewhere (Nicolaidis et al., 2011, 2019).
Conceptual model
We used our prior survey studies (Nicolaidis et al., 2013; Raymaker et al., 2017), qualitative research (Nicolaidis et al., 2015), and intervention pilot testing (Nicolaidis et al., 2016) to create a model that explains how interventions may theoretically improve care and reduce known healthcare disparities experienced by autistic adults (Figure 1). We then set out to use, adapt, or create instruments to measure each of the constructs in the model. This article focuses on the constructs that are measured by patient or proxy report. Another paper discusses constructs that are reported by healthcare providers (Nicolaidis et al, 2020). Future work will explore the relationship between these measures, healthcare utilization, and health and will assess how interventions may affect these constructs.
Theoretical model for how healthcare interventions may improve outcomes.
Setting
This analysis uses baseline data from a study to integrate the AASPIRE Healthcare Toolkit into three healthcare systems in the United States. The study took part in (1) eight primary care clinics that are part of a large integrated healthcare system in California; (2) two primary care clinics affiliated with an academic medical center in Oregon; and (3) two primary care clinics affiliated with a private health system in Oregon. The project was approved by the institutional review boards of our university and each health system.
Participants, recruitment, and data collection
Eligible participants were adults (age 18 and above) with diagnoses of autism spectrum disorder (including autistic disorder, Asperger’s disorder, and pervasive developmental disorder—not otherwise specified), who obtained primary care from the participating clinics. In cases where patients could not participate directly, even with appropriate accommodations and supports, we asked a supporter to participate as a proxy reporter with as much input as possible from the autistic patient. Each health system identified all adult patients assigned to the participating practices who had an International Classification of Diseases (ICD)-9 or ICD-10 code in their records consistent with autism (299.00, 299.8, and 299.9 or F84.0, F84.5, and F84.9). Staff from the partnering health systems sent a letter (via mail or secure electronic health record message) to potential participants inviting them to the study and followed up with telephone calls. In Oregon, where the partnering clinics only had a small number of adult autistic patients, clinic staff invited all eligible participants. In California, research staff randomly selected batches of approximately 700 participants to invite at a time, over-sampling participants of color (African American, Latinx, Asian, Native American, or mixed race). They invited additional batches of participants until our recruitment targets were met.
Our consent process followed the AASPIRE Guidelines for the Inclusion of Autistic Adults in Research (Nicolaidis et al., 2019). Participants could provide informed consent over the Internet, telephone, or in person. The consent process started with a series of questions to assess whether the person was taking part on their own, with support, or via a proxy reporter. In cases where the participant accessed and participated in the survey on their own over the Internet, we did not assess for decisional capacity, given that this study posed minimal risk and participants who use the Internet independently likely make decisions with equal or greater risk on a regular basis. In cases where they participated independently by telephone or in person, we instructed the interviewer to assess for decisional capacity if the interaction raised any concern about decisional impairment. When a supporter was involved (either to provide support or to serve as a proxy reporter), we asked a series of questions to understand whether the participant generally makes the types of decisions involved in this study themselves or via a legally authorized representative (LAR). If they made these decisions themselves, we asked the supporter to help the participant, as needed, to read and understand the information and communicate their decision to us. In cases where a participant could not offer informed consent, even with support, we asked for consent from an LAR. In some cases, the LAR was the same person who was providing support or serving as a proxy reporter; when it was not, we contacted the LAR to obtain consent prior to continuing with the survey.
We created a survey that included measurements for each of the patient- or proxy-reported constructs in our model. We created two versions of the survey—one for use with autistic adults directly (with or without help from a supporter) and another for use by proxy reporters. We administered the survey online, in person, or via telephone to 244 autistic adults (17% response rate). The vast majority participated in the survey online. One hundred ninety-four participated directly (with or without support), and 50 participated via a proxy reporter. Autistic patients had a mean age of 30 years (range, 18–72); 70% were male; and 62% were non-Hispanic white. Thirty-nine percent often or always, 24% sometimes, and 38% rarely or never needed assistance in healthcare settings. Thirty-eight percent reported a co-occurring chronic physical health condition, and 61% reported a co-occurring mental health condition. Table 1 presents characteristics of patients who participated directly (with or without help from a supporter) and those who participated via a proxy reporter.
Participant characteristics.
Measures
The baseline survey included each of the six measures discussed in this article (Table 2), as well as additional items on demographic and disability-related characteristics.
Healthcare instruments included in the survey.
AASPIRE: Academic Autism Spectrum Partnership in Research and Education.
Hyperlinks: Primary Care Provider: “Your primary care provider is your ‘regular doctor’ (though they can also be nurse practitioners or physicians assistants).” Preventive healthcare: “Preventive healthcare is healthcare that is aimed at early detection and treatment or prevention of disease. Examples of preventive healthcare may include visits where a healthcare worker performs screening tests such as pap smears, mammograms, and colonoscopies; draws blood to check a cholesterol level; counsels a patient about diet, exercise, tobacco, or alcohol; or performs a routine physical examination.”
Items shown are from the patient report version of the scales. Items from the proxy report versions available upon request.
We assessed unmet healthcare needs using a checklist we previously adapted from the 2002/2003 Joint US Canada Survey (Blackwell et al., 2004; Sanmartin et al., 2006). The item states, “During the past 6 months, there was a time when I felt that I needed the following type of healthcare, but did not receive it. (Check all that apply).” Response options include six types of healthcare (e.g. “medical care for a physical health problem” and “mental healthcare or counseling”). As we have done in prior studies (Nicolaidis et al., 2013), we dichotomized results to separate participants who did or did not have any unmet healthcare needs and created a proxy version that asked the items in the third person.
We assessed barriers to healthcare using the Barriers to Healthcare Checklist: Short Form (Raymaker et al., 2017). The checklist includes 16 barriers commonly experienced by autistic patients. It is scored as a count of the total number of barriers endorsed. As these count data were highly skewed, we dichotomized results at the top quartile to identify participants with high barriers to care (i.e. 0–3 barriers vs 4 or more barriers). The original instrument only included a patient version. We created a proxy version for use in a prior study (Nicolaidis et al., 2016).
We collected data on satisfaction with patient–provider communication using the AASPIRE Patient–Provider Communication Scale (PPCS-8). This scale is a highly adapted version of items from the 2007 Health Information National Trends Survey (HINTS; Cantor et al., 2009; Hong, 2008; Marks et al., 2010; Ok et al., 2008). Using our CBPR instrument adaptation process, we added a preface and clarified some wording to increase accessibility, made wording modifications to allow for self-report, and added two new items on expressive and receptive communication. We used the adapted scale, which asked participants to think of healthcare interactions over the past 12 months, in our healthcare survey (Nicolaidis et al., 2013). We used it again in our intervention pilot test (Nicolaidis et al., 2016), but based on community partner feedback, we changed the preface to ask participants to think about their last visit with their primary care provider and, accordingly, changed response options from the original 4-point Likert-type scale that had anchors of “never” to “always” to a 5-point Likert-type scale with anchors of “strongly disagree” to “strongly agree.” We also created a proxy report version of the scale. We used this version of the scale in this study with no further adaptations. The final 8-item scale can range from 8 to 40, with higher scores indicating higher satisfaction with patient–provider communication. A ninth item asking about overall satisfaction with healthcare was not included in these analyses.
We measured healthcare self-efficacy using the AASPIRE Health and Healthcare Self-Efficacy Scale (HHSES-21). Our community academic team had previously co-created the self-report and proxy versions of this measure, de novo, for use in our intervention pilot testing study (Nicolaidis et al., 2016). Constructs are based on a review of the literature, data from our prior survey and qualitative research, and the team’s lived experience. We made additional adaptations to that measure based on partner and participant feedback and included them in this study. The 21-item scale has response options that use a 10-point Likert-type scale with anchors of 0 = “not at all confident” to 10 = “totally confident.” The scale is scored by adding responses and then dividing the sum by the number of items. The resulting scale has a possible range of 1–10, with higher scores corresponding to higher self-efficacy. As below, factor analysis identified two subscales, so the remaining analyses were conducted using the two separate subscales.
We co-created two additional scales to cover constructs that had emerged from our prior qualitative work as potential intervention targets, namely, visit preparedness and use of healthcare accommodations. The AASPIRE Visit Preparedness Scale (VPS-6) is a 6-item measure. Response options use a 5-point Likert-type scale anchored at 1 = “strongly disagree” and 5 = “strongly agree.” The scale is scored by summing the individual items. It has a range of 6–30, with higher scores indicating higher visit preparedness. The AASPIRE Healthcare Accommodations Scale (HAS-8) measures patient’s perspectives on how well their accommodation needs are being met by healthcare providers and other clinic staff. Result options use a 5-point Likert-type scale with anchors of 1 = “strongly disagree” and 5 = “strongly agree.” Scores from the eight items are summed. The resulting scale can range from 8 to 40, with higher scores indicating higher use of accommodations. Two open-ended items ask participants to describe what accommodations they receive and what additional accommodations they would want. Each instrument has a self-report and proxy version.
Data analysis
We based our psychometric testing on the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) Initiative’s international consensus on the taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes (Mokkink et al., 2010). We assured content validity for all six instruments by working closely with our academic and community partners, including autistic adults, family members, and healthcare providers. We reviewed each draft instrument with our community partners and discussed their impressions, including whether the items were easy to understand, whether they were addressing the intended constructs, and whether they included all important aspects of the constructs. We kept written transcripts of community partner meetings, summarized recommendations, and made adaptations to draft scales as needed.
We assessed one type of reliability (internal consistency reliability) and two aspects of construct validity (structural validity and hypothesis testing about expected associations). Four of the measures included in this study (patient–provider communication, visit preparedness, use of accommodations, and healthcare self-efficacy) are multi-item scales, where individual items use Likert-type-style response options. For these measures, summary scores approximate a normal distribution. Thus, these four measures were conducive to assessment of internal consistency reliability and structural validity. The other two measures (unmet needs and barriers to care) used a checklist format, which resulted in a series of multiple dichotomous variables (6 variables indicating the presence or absence of each unmet need and 16 variables indicating the presence or absence of each barrier). Counts of unmet needs or barriers to care do not approximate a normal distribution. As such, we created two dichotomous variables—any unmet needs and four or more barriers to care. We used all six instruments in testing a priori hypotheses about expected associations, another important aspect of construct validity. We did not assess criterion validity or responsiveness to change.
We tested the structural validity for each of the four scored scales by conducting Principal Axis Factor (PAF) Analyses. We used the Kaiser–Meyer–Olkin (KMO) Measure and Bartlett’s Test of Sphericity (Green & Salkind, 2013) to verify sampling adequacy for factor analysis. We expected the KMO statistics to be 0.7 or higher and Bartlett’s test to be significant in order to proceed with factor analysis. We used three criteria to determine factor structure: Kaiser’s recommendation of eigenvalues over 1, a scree plot which we used to visualize the eigenvalues, and the interpretability of the factor communalities if the two criteria give different results (Green & Salkind, 2013). If these three criteria indicated our initial hypothesis of unidimensionality was incorrect, we rotated factors using the Varimax procedure (Green & Salkind, 2013) in order to make the factors more interpretable and to make final decisions about underlying factors. We tested two groups of study participants (i.e. those who participated directly and those who participated via proxy) separately and found no structural differences between the two groups. Thus, we conducted factor analyses with all participants combined. We only had a small amount of missing data (between 2% and 6.1% per each item), so we used listwise deletion to handle missing data. We assessed internal consistency reliability for each scored scale or subscale using Cronbach’s alpha.
We further examined construct validity for all measures by testing a priori hypotheses about expected association using pair-wise correlations and t-tests. Specifically, we hypothesized that each of the intervention targets (visit preparedness, provider/staff use of accommodations, and individual and relational healthcare self-efficacy) would be positively correlated with satisfaction with patient–provider communication. We also hypothesized that participants who reported four or more barriers to care or any unmet healthcare needs would have lower scores on each of the intervention targets that those who did not.
We conducted secondary analyses stratifying data from those who participated directly (with or without help from a supporter) and those who participated via a proxy reporter. Given the small number of patients who participated via a proxy reporter, those findings should be interpreted with caution.
Access to research materials
The full text and scoring instructions for each of the six instruments are included in Table 2. Per National Institutes of Health (NIH) guidelines, we have deposited the underlying data in the National Data Archive.
Results
Content validity
Academic and community partners carefully reviewed all instruments. They felt items were easy to understand and easy to answer, and fully captured the intended construct.
Structural validity
KMO Test scores were 0.774, 0.922, 0.943, and 0.922 for VPS-6, the Provider and Staff Use of Accommodations Scale, PPCS-8, and the Health and Healthcare Self-Efficacy Scale, respectively. KMO statistics fell into the range of good (0.7) to superb (0.9 or above), indicating that our sample size was adequate for factor analysis (Hutcheson & Sofroniou, 1999). Bartlett’s Test of Sphericity was also significant (p < .001) for each of the scales, further confirming sampling adequacy.
PAF analyses indicated that VPS-6, the Provider and Staff Use of Accommodations Scale, and PPCS-8 each had a single factor with sufficient factor loadings (0.68–0.88, 0.61–0.75, and 0.76–0.86, respectively) and explained variance of 46.94%, 64.33%, and 66.36%, respectively.
On the contrary, the assessment of eigenvalues and scree plots, as well as the extracted communalities, indicated the Health and Healthcare Self-Efficacy Scale had a two-factor structure with 49.63% of explained variance. The initial factor loadings ranged between 0.35 and 0.86. In consideration of our sample size (n = 244), all factor loadings were adequate (Stevens, 2012) for further factor analysis. Factor loadings from the Varimax rotation procedure are shown in Table 3.
Factor loadings of items from Healthcare Self-Efficacy Scale on the two-factor structure.
SE: self-efficacy.
Principal axis factor analysis was conducted with Varimax rotation procedure. The shading indicates in which scale the item was ultimately included.
These items were ultimately included in the Independent Healthcare Self-Efficacy Subscale.
These items were ultimately included in the Relationship-Dependent Healthcare Self-Efficacy Subscale.
These three items loaded similarly on the two factors. Our academic and community partners felt that, conceptually, these items were more closely related to the Relationship-Dependent Healthcare Self-Efficacy Subscale.
Academic and community partners reviewed which items loaded best on each subscale and discussed what, conceptually, may differentiate the two groups. The group concluded that while all items related to self-efficacy, one set of items primarily related to aspects of care that a patient (or supporter) could control by themselves (e.g. making an appointment, tolerating a procedure, and taking prescribed medications), while the other group of items depended not only on the patient but also on the provider, staff, or healthcare system (e.g. communicating with provider and being included in decision-making). Based on this discussion, we titled the subscales as follows: Individual Healthcare Self-Efficacy and Relationship-Dependent Healthcare Self-Efficacy. Three items loaded very similarly to both factors: taking part in healthcare decisions (loading of 0.48 on both factors); changing healthcare providers (loading of 0.35 on both factors), and getting necessary accommodation (loading of 0.49 vs 0.54 for the relationship vs individual skill-based efficacy, respectively). Our academic and community partners felt these three items were conceptually more relevant with relationship-dependent healthcare self-efficacy skills than the individual healthcare self-efficacy, so we included them with the former scale.
Internal consistency reliability
VPS-6, HAS-8, and PPCS-8 each had good to excellent internal consistency reliability (Cronbach’s alpha = 0.85, 0.93, and 0.95, respectively). Both subscales of the Health and Healthcare Self-Efficacy scale also had good to excellent reliability (Cronbach’s alpha = 0.88 for the Individual and 0.92 for the Relationship-Dependent Healthcare Self-Efficacy subscales; see Table 4).
Internal consistency reliability of multi-item scales.
Testing of hypothesized associations
Hypothesis testing demonstrated significant associations in the expected directions. Participants who reported unmet healthcare needs had lower confidence in individual and relationship-dependent healthcare self-efficacy, felt less prepared for healthcare visits, and were less likely to receive necessary accommodations. The same held true for patients who reported four or more barriers to care. There were strong positive correlations between satisfaction with patient–provider communication and individual healthcare self-efficacy, relationship-dependent healthcare self-efficacy, visit preparedness, and use of accommodations (Table 5). Secondary analyses limited to the sample of those who participated directly were very similar. In the analysis limited to people who participated via proxy, associations were qualitatively similar, but on a few occasions, they did not reach statistical significance because of the small sample size (data not shown).
Associations between healthcare outcome measures and potential intervention targets.
Discussion
Our CBPR team assembled a package of patient- or proxy-reported instruments to be used in evaluating health services interventions for autistic adults. Our preliminary psychometric testing offers support of their content validity, construct validity, and internal consistency reliability.
Patient–provider communication is a key construct throughout the health services literature, with studies in the general populations showing a correlation between effective physician–patient communication and improved patient health outcomes (Stewart, 1995). However, when the autistic members of our team reviewed existing measures of satisfaction with patient–provider communication, they consistently felt they were inaccessible or incomplete. The PPCS-8, created by collaboratively adapting items from the 2007 Health Information National Trends Survey (HINTS) (Cantor et al., 2009; Hong, 2008; Marks et al., 2010; Ok et al., 2008), allows researchers to assess this important construct in autistic adults. Our prior two studies had shown that the scale has strong internal consistency reliability in two separate samples (Nicolaidis et al., 2013, 2016) and was responsive to change (Nicolaidis et al., 2016). This study confirms the scale’s excellent internal consistency reliability in a third sample while also providing new data on its construct validity.
Similarly, the concept of patient self-efficacy is ubiquitous throughout health services research. However, most existing measures are disease-specific (Clay & Telfair, 2007; Shively et al., 2002), focus on self-management of chronic illnesses (Du & Yuan, 2010), or primarily address self-care (Callaghan, 2003). Our team did not feel that existing instruments, even with adaptations for accessibility, would capture the aspects of health and healthcare self-efficacy that were most important to autistic adults. We thus co-created a self-efficacy instrument based on our prior studies (Nicolaidis et al., 2013, 2015; Raymaker et al., 2017) and the lived experience of the autistic adults, healthcare providers, and supporters on our CBPR team. Our prior work found that an earlier version of this scale had excellent internal consistency reliability and was responsive to change (Nicolaidis et al., 2016). This study confirms the scale’s internal consistency reliability and adds support for its construct validity. Interestingly, while we originally thought of items as addressing healthcare (e.g. navigating the healthcare system or communicating with a provider) versus health (e.g. living a healthy lifestyle or managing one’s chronic medical issues), exploratory factor analyses found that the scale had two different factors: Individual Healthcare Self-Efficacy (i.e. aspects that a patient or supporter can control by themselves) and Relationship-Dependent Healthcare Self-Efficacy (i.e. aspects that depend not only on the patient but also on the provider, staff, or healthcare system). Researchers who use the scale should score these two subscales separately.
While visit preparedness may be an aspect of healthcare self-efficacy (and the HHSES-21 includes one item about it), the importance of visit preparedness was such a strong theme in our prior qualitative data (Nicolaidis et al., 2016) that our team created a separate multi-item measure for this construct. The new VPS-6 addresses aspects of visit preparedness such as if a patient brought what was needed to the visit, came prepared with a list of topics to discuss, felt ready to discuss their symptoms and answer the provider’s questions, and knew what to expect from the appointment and their provider. As opposed to self-efficacy instruments, which focus on whether a patient feels confident that they can do what is needed, this instrument asks about what actually happened at the patient’s last visit. We found the new scale had strong internal consistency reliability and construct validity. It may provide additional depth to evaluations of interventions that target patients’ visit preparedness as a mechanism for improving healthcare.
Many calls for improved healthcare for autistic adults have also focused on the need for health systems and individual providers to make necessary accommodations (Mason et al., 2019). We previously created the Autism Healthcare Accommodations Tool (AHAT) to help patients communicate their accommodation needs to healthcare providers (Nicolaidis et al., 2016). Other studies have focused on the importance of accommodations in healthcare settings for people with mobility impairments (Pharr, 2014; Pharr et al., 2019) or communication disorders (Burns et al., 2017), but such studies relied on survey data from healthcare administrators or providers or only collected qualitative data. The new HAS-8 showed strong internal consistency reliability and construct validity. To our knowledge, this is the first instrument in the literature that assesses whether patients feel their healthcare accommodation needs are actually met.
Although this article focuses primarily on the psychometric properties of the four scored scales, it also included two additional checklists, the Barriers to Healthcare Checklist and the Unmet Healthcare Needs Checklist in hypothesis testing. The concept of “barriers to care” is ubiquitous throughout the health services literature, but assessment of barriers to care requires understanding the specific issues that may be pertinent to a particular patient population or health condition. For example, disease-specific systematic reviews exist on barriers to the identification and treatment of hypertension (Khatib et al., 2014), to vaccination against human papillomavirus (Holman et al., 2014), or to testing for HIV (Deblonde et al., 2010); and population-specific systematic reviews exist on barriers to healthcare for undocumented immigrants (Hacker et al., 2015), patients in rural areas (Brundisini et al., 2013), and indigenous people with chronic diseases (Gibson et al., 2015). When we originally tried to assess barriers to general healthcare for autistic adults, there were no published studies on this topic. As such, we started with unpublished barriers to healthcare checklist that had been created for people with disabilities in general and worked with our community partners to ensure that it was accessible to autistic adults and that it included additional barriers that may be particularly relevant to autistic populations (e.g. sensory or communication-related barriers). We tested the resulting instrument, the Barriers to Healthcare Checklist–Long Form, in an online sample of autistic adults and adults with and without other disabilities (Raymaker et al., 2017); collapsed items to create the Barriers to Healthcare Checklist–Short Form; and used the Short Form in our prior pilot intervention, where we found it was responsive to change (Nicolaidis et al., 2016). This study further demonstrates the instrument’s construct validity in a separate sample. Similarly, we have now used the Unmet Healthcare Needs Checklist in three separate samples of autistic adults (Nicolaidis et al., 2013, 2016) and have found it to be easy to answer and be associated in the expected directions with other healthcare measures.
We believe our use of a CBPR approach strengthened this project in many ways. First, autistic adults, supporters, and healthcare providers helped identify what constructs were most important to measure and helped create a model for how interventions may improve healthcare for autistic adults. Second, these community partners helped ensure construct validity for all measures, with special attention on creating instruments that are easy to understand, that can be answered easily without undue participant burden or frustration, that are applicable to real-life healthcare settings, and that are comprehensive and respectful. Finally, community partners played a critical role in interpreting findings, for example, in helping understand what may conceptually differentiate the two factors on the HHSES-21.
We created this set of instruments for use in our own healthcare studies because there were no other validated measures to assess these constructs in autistic adults. However, we believe this set of measures may be useful to other autism researchers seeking instruments that have been validated with autistic populations. Use of this set of instruments may also help compare experiences of autistic adults in different health systems or countries. Researchers using these instruments with other autistic populations, especially in other countries, should assess for the need of adaptations and recheck psychometric properties.
While we created this set of instruments for use with autistic adults, the constructs themselves are not autism-specific, and none of the items specifically mention autism. Adaptations of prior instruments focused on making the language clearer and more specific or increasing completeness by adding new items. Moreover, we created the new instruments on visit preparedness and use of accommodations because we were unable to find any patient-reported measures for these constructs, even for other populations. This set of measures may potentially be useful in evaluating health services interventions in general populations or in patients with other disabilities or health conditions. However, future research is needed to establish the psychometric properties of these instruments in other populations.
Our study has several limitations. First, this analysis only uses cross-sectional data, so it is not intended to assess causative relationships or responsiveness to change. Similarly, this analysis only focuses on patient- or proxy-reported data and does not correlate findings with data from other sources. Future research is needed to assess how well these instruments capture intervention effects or to triangulate findings with healthcare utilization data. The low response rate may have introduced participation bias. Moreover, we only included a small number of participants who relied on proxy reporters. We believe that offering the survey in various modes was critical to including participants with different accommodation needs; however, we did not have a large enough number of participants taking part over the telephone or in person to allow for comparison of results by data collection mode. Additional research is needed to validate the proxy report versions of these instruments. Finally, while our study was strengthened by the inclusion of patients from three different health systems, further research is needed to validate these instruments outside of the United States or to translate the measures for non-English-speaking populations.
Conclusion
The new measures demonstrated strong content validity, construct validity, and internal consistency reliability. Future research is needed to (1) further validate these scales, especially in patients who participate via proxy; (2) assess their responsiveness to change; (3) assess whether interventions improve healthcare outcomes; and (4) test whether they do so via these hypothesized mechanisms of action.
Footnotes
Acknowledgements
We would like to thank the entire AASPIRE team for their significant contributions throughout this project. We also are grateful to the skilled research staff at Kaiser Permanente of Northern California, all the participants who volunteered to be in this study, and the people who supported them.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the National Institute of Mental Health (R34MH111536; Principal Investigator: Nicolaidis). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
