Abstract
Community-based methods for autism evaluation may be one solution for ameliorating delays in diagnosis, which are exacerbated for children from minoritized backgrounds. However, limited research has examined caregiver satisfaction with community-based models of autism evaluation. Thus, our objective was to use a mixed-methods approach to investigate caregiver satisfaction with their child's autism evaluation conducted across a statewide system of primary care autism diagnosis. Results indicated overall high satisfaction and no significant differences were found between satisfaction total scores nor caregiver stress and any child/family demographic variables. Satisfaction and stress were also not related to autism diagnostic outcome, clinician diagnostic certainty, or diagnostic accuracy. Qualitative suggestions for evaluation improvement include more thorough explanation of diagnosis and service recommendations. Overall, our findings indicate high caregiver satisfaction with multiple dimensions of community-based autism evaluation in the primary care setting, suggesting this may be a feasible and sustainable model that caregivers find acceptable.
Autism is a neurodevelopmental condition with a current United States (US) prevalence of 1 in 36 children. 1 Despite observable symptoms in the first year of life 2 and diagnostic stability in the second year of life, 3 the average age of diagnosis in the United States remains after 4 years1,4 with further delays for children from diverse racial and ethnic,5,6 socioeconomic, 7 and geographic backgrounds.8,9 Given the individual and societal impact (eg, increased lifetime care costs)10,11 of diagnostic delays there has been a push to test novel methods that promote equitable access. Community-based tiered models of autism evaluation, which equip primary care physicians and other clinicians (PCPs) with skills to conduct autism evaluations, may be one solution for ameliorating diagnostic delays.12-14 These models have been shown to be feasible, accurate, and reduce diagnostic delays.15-19 However, family satisfaction with this type of streamlined evaluation is not yet known.
Patient satisfaction is a growing area of interest for health organizations striving to identify the critical standards and values held by consumers and is one component of the Institute for Healthcare's Quadruple Aim for achieving value-based healthcare. 20 Research has identified communication, access, clinician interpersonal skills, and care coordination as strongly related to patient satisfaction. 21 Satisfaction ratings are predictive of compliance with care and clinical outcomes, 22 and influences the likelihood patients would recommend their clinician. 23 Within primary care, there has been an emphasis placed on patient satisfaction as an outcome measure alongside health status and quality of life evaluations. 24
Limited research has been conducted on satisfaction with autism diagnostic services. However, findings suggest that caregiver satisfaction varies with demographic background of the patient and factors related to the diagnostic evaluation.25,26 Specifically, age of child at diagnosis, quality of information received at diagnosis, clinician time spent on the evaluation, perceived clinician verbal and nonverbal skills, support offered following diagnosis, and parental stress during the diagnostic process have all been shown to be related to satisfaction. 26 For example, caregivers of younger children who receive more information surrounding the diagnosis and treatment resources demonstrate increased satisfaction, particularly when the process is streamlined (ie, visiting fewer providers to obtain diagnosis). 27 Further, satisfaction is associated with lower stress in caregivers of children with disabilities. 28 Given that caregivers of children with disabilities experience high levels of stress, ensuring satisfaction during primary care evaluations is of great importance. 29
To date, only one study of family satisfaction with primary care autism evaluations has been published. 30 In this study, 90% of caregivers endorsed feeling either “extremely” or “moderately” satisfied with the autism evaluation process. 30 Understanding caregiver satisfaction is essential to building and implementing successful systems of autism diagnostic care. Thus, the objective of the current study was to use a mixed-methods approach to investigate caregiver satisfaction with their child's autism evaluation conducted within a statewide system of primary care autism diagnosis. Specifically, we examined (1) quantitative ratings of caregiver satisfaction and the association between satisfaction and caregiver stress and child and clinician diagnostic factors (ie, autism outcome, clinician diagnostic certainty, and diagnostic accuracy), and (2) qualitative outcomes from open-ended responses on a caregiver satisfaction survey.
Methods
Data for the present study were collected as a part of a larger study focused on primary care autism diagnostic accuracy. The study was approved by the Indiana University Institutional Review Board (see McNally Keehn et al 31 ), and written informed consent was obtained from legal guardians of all participants.
Setting
Early Autism Evaluation (EAE) Hub System
The EAE Hub system is a statewide network of community primary care and general pediatric clinicians trained to provide streamlined diagnostic evaluations for young children ages 14 to 48 months implemented in the United States 31 (see Supplemental Material for detailed information). Following training, EAE Hub clinicians participate in a monthly learning collaborative to maintain skills. EAE Hub clinicians serve as regional consultants who receive evaluation referrals from a child's primary care clinician when the child is determined to be at increased likelihood for autism. 31 EAE Hub clinicians follow a standard clinical evaluation protocol, conducted in a 90-minute office visit, including administration/review of standard developmental and autism screening tools, a Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) focused developmental history and clinical interview, physical examination, and administration and scoring of the Screening Tool for Autism in Toddlers. 32 The EAE Hub clinician synthesizes all assessment data to make a best-estimate clinical diagnosis and provides the family with diagnostic counseling and a written summary with prioritized next step clinical recommendations. The full clinical report is disseminated to the family and referring PCP following the evaluation. Children with complex diagnostic profiles (ie, for whom EAE Hub clinicians cannot make a diagnostic decision) are referred for comprehensive autism evaluation at a specialty diagnostic center.
Participants
Participants included 98 children, ages 14 to 18 months (M = 2.61 years; SD = 0.6; White 69%, Black 8%, Latine 13%; see Table 1). Sixty-five percent (n = 64) of children received a diagnosis of autism at the EAE Hub evaluation. Demographics of children and their caregivers (M = 28 years at birth; SD = 6; 46% household income < $50 000/year; 33% ≤ high school education) are presented in Table 1.
Participant Demographics.
All data presented as n (%) unless otherwise noted. Percentages are calculated based on those children for which data are available.
Measures
Demographic Survey. Caregivers completed a brief questionnaire with items about child race and ethnicity, family annual household income, and caregiver education level.
EAE Hub Satisfaction Survey. The EAE Hub Satisfaction Survey was adapted from survey items published in 3 studies on patient and family satisfaction with autism diagnostic services.26,30,33 Survey items from across studies were selected by the EAE Hub team, consisting of a medicine-pediatrics physician, clinical psychologist, and 2 parents of adult children with disabilities, including autism. Item language was modified to fit the EAE Hub context. Caregivers completed this brief 11-item questionnaire regarding satisfaction with their child's EAE Hub evaluation (see Table 2). Items are rated on a 5-point Likert scale. A total score was calculated from satisfaction items 1 to 7, and 11 (range 8-40; higher scores indicate greater satisfaction). One item evaluated overall stress with the evaluation; 2 additional questions evaluated caregiver perception of their child's behavior during the evaluation. Finally, 2 open-ended items asked caregivers to provide suggestions to improve the EAE Hub diagnostic process.
Caregiver Satisfaction With EAE Hub Evaluations: N = 98.
EAE Hub Diagnostic Certainty Form. EAE Hub clinicians submitted data on results of the child's autism evaluation, including categorical diagnosis (autism; non-autism) and clinician diagnostic certainty (rated on a 5-point Likert scale; Not Certain at all–Completely certain).
Autism Specialist Diagnostic Certainty Form. The autism specialist submitted data on results of the child's research evaluation, including categorical diagnosis and specialist diagnostic certainty (ie, as described above).
Procedure
Seven EAE Hubs set within primary care practices across Indiana participated in a larger prospective diagnostic study from June 2019 through August 2022. 31 At study enrollment, which typically occurred within 1 to 2 weeks of the EAE Hub evaluation, caregivers were sent an electronic link with (1) demographic survey and (2) EAE Hub satisfaction survey. This allowed for caregivers to rate satisfaction with the EAE Hub evaluation while the appointment was salient and to ensure there was no bias with recall related to the research evaluation. The follow-up research evaluation was completed within 12 weeks of the EAE Hub evaluation. EAE Hub evaluation data (ie, EAE Hub diagnosis and diagnostic certainty ratings) were collected by a member of the study team. To determine EAE Hub diagnostic accuracy, the study evaluation team, consisting of a licensed clinical psychologist with expertise in evaluation of autism in young children and a clinical research technician, traveled to the EAE Hub to conduct a follow-up gold-standard autism diagnostic assessment (including measures of child developmental and adaptive functioning and autism symptom severity) within 16 weeks of the initial EAE Hub evaluation. Study personnel involved in data collection, including expert clinicians who conducted research evaluations EAE Hub diagnosis to reduce bias in diagnostic decision making. Participants were compensated with a gift card for study completion.
Statistical Analyses
A mixed methods approach was utilized for the present study. Descriptive statistics were calculated for satisfaction survey items and total score (sum of items 1-7 and 11). Pearson correlations, chi-squares tests, and independent-samples t-tests were performed to examine associations between satisfaction total scores, caregiver stress, child and family demographic characteristics, and EAE Hub diagnostic outcome and clinician certainty. Additionally, associations between total satisfaction and caregiver stress and diagnostic agreement between EAE Hub and autism specialist evaluation were examined. Version 28 of SPSS was used to conduct statistical analyses.
Analysis of 2 open-ended questions on the satisfaction survey were conducted using a constructivist grounded theory approach. 34 During initial coding, inductive codes were created by 2 members of the research team by organizing responses into categories and subcategories. During focused coding, the most salient codes (also called focused themes) were identified, and then the rest of the data was coded. Three meetings were held with a third research member to reach consensus on focused themes.
Results
Quantitative Findings
See Table 2 for item-level satisfaction survey data. Overall, results revealed high caregiver satisfaction (total score M = 35.03, SD = 5.6) with the EAE Hub evaluation process. Mean individual satisfaction item scores ranged from 4.1 to 4.6 (range 1-5; higher scores representing higher satisfaction). Highest ratings of satisfaction were on items related to caregivers’ perceptions of clinician listening skills (mean rating: 4.6; 91% agree/strongly agree) and sensitivity to family values and customs (mean rating: 4.6; 91% agree/strongly agree). Items with the lowest satisfaction ratings were related to the amount of information provided about the child's diagnosis (mean rating: 4.1; 78% agree/strongly agree), discussion about treatment options (mean rating: 4.4; 85% agree/strongly agree), and satisfaction with the overall diagnostic process (mean rating: 4.1; 69% agree/strongly agree).
Seventy-two percent of caregivers strongly agreed/agreed with the statement, “My child behaved in his/her usual way during the evaluation, showing his/her usual behaviors and skills” (mean rating: 3.9) and 84% of caregivers strongly agreed/agreed with the statement, “My child's daily routine was typical on the day of the evaluation (eg, no unusual changes in sleep or eating)” (mean rating: 4.2). Regarding stress, 77% of caregivers reported feeling “not at all stressed/slightly stressed” during their child's evaluation.
No significant associations were found between satisfaction total scores or caregiver stress and child/family demographic characteristics, including parental education, household annual income, and child race or ethnicity (all Ps > .08). Overall satisfaction and stress did not differ across autism diagnostic outcome (ie, autism; nonautism; Ps > .43) or by EAE Hub clinician diagnostic certainty (Ps > .07). Similarly, there was no significant association between total satisfaction or stress with accuracy of the EAE Hub diagnosis (ie, agreement between EAE Hub clinician and expert specialist diagnosis; Ps > .57).
Qualitative Findings
In total, 41% (n = 40) of caregivers provided a response to the 2 open-ended questions. Three main themes emerged during data analysis, which are detailed below and summarized in Table 3.
Satisfaction Thematic Analysis.
Theme 1: Evaluation Process. The first theme, Evaluation Process, encompassed 4 subthemes: Prior to evaluation; evaluation environment; building rapport; and diagnosis/feedback. Recommendations (n = 2) emerged regarding improvements to guide families on what to expect in the weeks prior to and the day of their child's evaluation. Tell families how long to expect to be there and give families tips to better prepare child for a change in routine. (Caregiver 16)
Several caregivers (n = 4) provided feedback about the clinical space in which autism evaluations took place and recommendations for creating a more conducive setting for young children navigating novel environments. We were in the same small medical room for almost 3 hours—a bigger area that [is] less “clinical” would be helpful for evaluations. (Caregiver 20)
Caregivers (n = 9) commented around the process of clinician rapport and the importance of their child feeling comfortable around the clinician. You could spend a little more time getting to know the child all the way. (Caregiver 34)
Finally, (n = 10) caregivers responded to the clinician's delivery of the diagnosis and made suggestions for improvement in how diagnostic feedback and counseling is delivered. Explain what it means to be on the spectrum. Tell me what I need to do to help my kid develop—not just therapy to send him to, but how do I need to interact with him differently on a personal level … How do I need to change my parenting? (Caregiver 19)
Theme 2: Postevaluation. The second theme, postevaluation, yielded 2 subthemes: Recommendations for next steps and follow-up. Recommendations for next steps (n = 2) included requests for more caregiver education regarding what to expect in terms of modifying interactions with their autistic children, appropriate next steps following diagnosis, and more thorough review of treatment options following an autism diagnosis. Maybe have handouts printed for parents to take home to do more research on their own once they know what to look for. This might mean you print different packets and have them ready. (Caregiver 17)
The subtheme of follow-up (n = 2) referred to caregivers’ desire to have a clinic staff member follow up with their family after the evaluation to provide additional resources or answer any lingering questions that may have arisen after caregivers have had time to adjust to the diagnosis. Follow-up (1-2 weeks post) with nurse or doc to provide additional guidance after some time to digest all that was handed to us that day. (Caregiver 31)
Theme 3: Overall Satisfaction. One subtheme, positive, emerged from the overall satisfaction theme and included general compliments and other positive feedback regarding the evaluation process as a whole. To be honest, in a difficult situation, they made it a very pleasant experience. Was very helpful, very kind and informative. (Caregiver 22)
Finally, there was no association between EAE site or clinician with the number and types of qualitative themes represented in the data (all Ps > .44).
Discussion
Patient satisfaction is a critical component of the Institute for Healthcare's Quadruple Aim of improving population health, reducing care costs, enhancing patient experience, and improving provider satisfaction. 20 Limited research to date has examined caregiver satisfaction with tiered community-based primary care models of autism diagnosis. Thus, the aim of the present study was to build on our existing knowledge about patient satisfaction with autism diagnosis by employing a mixed-methods approach to examine caregiver satisfaction with evaluations conducted across the EAE Hub system, a statewide network of community primary care and general pediatric clinicians trained to provide streamlined diagnostic evaluations for young children ages 14 to 48 months.
Caregivers reported high satisfaction with multiple dimensions of EAE Hub evaluation. Satisfaction was highest with caregivers’ perceptions of clinician listening skills and sensitivity to family values and customs, suggesting that caregivers felt heard even within the context of a 90-minute primary care evaluation. Satisfaction ratings were lower, but still moderate, for domains related to the amount of information provided during the visit, discussion about treatment options, and satisfaction with the overall diagnostic process. Despite some variability in satisfaction, it appears that caregiver satisfaction with EAE Hub evaluation is comparable to traditional evaluations for autism conducted in specialty diagnostic centers. 30 Considering that long wait times negatively impact family satisfaction, 4 and traditional specialty diagnostic centers often have waitlist that exceed a year, 35 autism evaluation models embedded within the primary care setting offer a feasible and sustainable alternative that is acceptable to families.
Satisfaction was not associated with diagnostic outcome, clinician diagnostic confidence, or diagnostic accuracy (ie, compared to expert evaluation) indicating that satisfaction is not contingent upon receiving a specific outcome during the evaluation. Caregiver stress was also not related to these diagnostic factors. Although other studies have found high levels of caregiver stress prior to an autism diagnostic assessment, 36 our findings suggest that EAE Hub clinicians and their teams can be trained and supported to address the emotional needs of caregivers during an otherwise stressful process. Further, research on association between demographic factors and caregiver satisfaction with autism evaluation has been mixed.25,26,37 In our sample of diverse children and families, child race/ethnicity, family income, and caregiver education level were not associated with caregiver satisfaction. These findings indicate that other factors, such as the amount of diagnostic information received and the communication skills of the clinician 26 may have a stronger influence on satisfaction than demographic characteristics. As such, diagnostic feedback and service recommendations for autistic children may consider individual caregiver characteristics and needs, rather than rely on general demographic characteristics.
Several takeaways for improving EAE Hub evaluations are notable from our qualitative analysis. Caregivers suggested that, when scheduling families for their evaluation, care should be taken to explain evaluation procedures and expectations. Similar research has found that most parents arrive at their evaluation appointments with limited understanding of the assessment process, creating challenges in sufficiently preparing themselves and their child. 38 Additionally, clinical teams should ensure that a child-friendly space has been prepared for the evaluation and enough time has been spent building rapport prior to beginning the assessment.
Caregivers reported a desire for more comprehensive information about autism and how they can help support their child. Previous research has found that parents appreciate feedback that is clear, direct, honest, and thorough. 39 Some families also provided feedback on improvements to follow-up care and coordination of services. This is in line with previous findings demonstrating increased caregiver satisfaction with service models that include coordination of care with community early childhood intervention providers. 40 By coordinating care across traditionally siloed service systems (eg, medical, early intervention, and educational), clinicians can more equitably support families who may struggle to navigate service provision, such as those with limited health literacy or experiencing social determinants of health risk factors. 39 Taken together, by incorporating skill building across these areas into PCP training programs, we can improve family satisfaction.
Limitations
Satisfaction ratings were generally high across items and total scores; as such it is possible that our analyses were limited by the statistical issue of range restriction. Further, it is possible that our findings may have been biased by participant and caregiver self-selection into a larger research study. Due to limited samples sizes across EAE Hub clinician and site, we cannot rule out potential site- and clinician-specific differences in satisfaction ratings. In addition, a large majority of evaluations and thus data collection occurred during the COVID-19 pandemic, potentially influencing caregiver-reported satisfaction. For example, it may be that caregivers experienced higher levels of satisfaction due to receiving the evaluation during a period in which healthcare access was very challenging. As such, our findings should be interpreted with caution and warrant replication in independent studies. Future research should include larger comparative studies of caregiver satisfaction across various autism evaluation models, as well as qualitative data on caregiver perspectives, in order to inform quality improvement efforts and refinement of evaluation models and care planning. 8 In addition, the current paper focused on caregiver satisfaction and did not address caregiver experience. Future studies also warrant investigating caregiver experience with the autism evaluation models, including how patient satisfaction factors into the overall experience as well as how patients weigh the service received against their expectations. 41
Conclusion
This study evaluated associations between caregiver satisfaction across a statewide system of autism evaluation in the primary care setting. Overall satisfaction was high with positive qualitative feedback and recommendations shared by caregivers. Improvements to communication about evaluation procedures, appropriate evaluation environments, clear and thorough diagnostic feedback, and care coordination and navigation of postevaluation recommendations are all ways in which PCPs can make targeted changes to enhance caregiver satisfaction. Although head-to-head evaluation of diverse evaluation models is needed, our findings suggest that streamlined primary care models of autism diagnosis are well accepted by caregivers.
Supplemental Material
sj-docx-1-jpx-10.1177_23743735241305531 - Supplemental material for Mixed Methods Analysis of Caregiver Satisfaction With the Early Autism Evaluation Hub System
Supplemental material, sj-docx-1-jpx-10.1177_23743735241305531 for Mixed Methods Analysis of Caregiver Satisfaction With the Early Autism Evaluation Hub System by Ann Marie Martin, Jordan Huskins, Angela Paxton, Amira Nafiseh, Mary R. Ciccarelli, Brandon Keehn and Rebecca McNally Keehn in Journal of Patient Experience
Footnotes
Abbreviations
Acknowledgments
We are grateful to Mary Delaney for her work as a practice liaison for the Indiana University School of Medicine Early Autism Evaluation Hub leadership team. We are indebted to the Early Autism Evaluation Hub clinicians for their commitment to the children and families of Indiana, our learning collaborative network, and their participation in this research. Finally, and most importantly, we are grateful to the children and caregivers who participated in this study.
Authors’ Contributions
All authors have made substantial intellectual contributions to this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Approval
The University Ethics Review Committee at Indiana University approved our study (approval: 1806262614) on August 19, 2020. Caregiver participants gave written consent for review and signature before starting the research evaluation.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by NIMH R21MH121747 (McNally Keehn, Keehn, Swigonski); UL1TR002529-Indiana Clinical and Translational Sciences, and K12TR004415 (RMK). Additional support was provided by Riley Children's Foundation and HRSA T73MC00015 (AMM).
Statement of Human and Animal Rights
Not applicable.
Statement of Informed Consent
Not applicable.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
