Caregiver Satisfaction with a Multisite Trial of Atomoxetine and Parent Training for Attention-Deficit/Hyperactivity Disorder and Behavioral Noncompliance in Children with Autism Spectrum Disorder

Abstract

Objective:

The purpose of this study was to examine caregiver satisfaction with the research experience in a randomized clinical trial of atomoxetine (ATX) and parent training (PT) for attention-deficit/hyperactivity disorder (ADHD) and behavioral noncompliance co-occurring with autism.

Methods:

The Children with Hyperactivity and Autism Research Treatment Study (CHARTS) randomly assigned 128 children 5.00–14.11 years of age to four treatment groups (ATX + PT, ATX alone, PT + placebo[PBO], and PBO). Caregivers completed an 18 item questionnaire about their satisfaction with the research experience. We summarized caregiver responses with descriptive statistics and examined whether the responses were associated with demographic variables, treatment assignment, or the child's response to treatment (positive or negative).

Results:

Ninety-three percent of caregivers (119) completed the questionnaire. When asked if they would join the study again if given the chance, 87% (103) responded “yes,” 13% (15) responded “maybe,” and 1% (1) responded “no.” When asked if they would recommend the study to other caregivers of children with similar problems, 92% (109) responded “yes” and 8% responded (10) “maybe.” Of the 59 Parent Satisfaction Questionnaire (PSQ) respondents who received PT, 75% (44) felt more confident in managing current child behaviors, 24% (14) felt that their level of confidence was unchanged, and 2% (1) felt less confident. Most caregivers expressed satisfaction with the study procedures, including the number of visits and the safety monitoring protocols.

Conclusions:

In general, caregivers were highly satisfied with their research experience. These findings may be useful for informing human subject committees and for designing study protocols that are appealing to families.

Introduction

In the context of research, patients and caregivers who are satisfied with the goals, procedures, and outcomes of a treatment may be more likely to comply with the treatment and research protocols, and patients may make larger gains (Foster and Mash 1999). In studies of children with autism spectrum disorder (ASD), patient and caregiver satisfaction is routinely assessed for one-to-one behavioral interventions (Storey and Horner 1991; Bitterman et al. 2008; Callahan et al. 2008; Strain et al. 2012) but rarely for other treatments, such as psychotropic medications or parent training (PT) (Tierney et al. 2007). The latter objective, determining satisfaction with the research experience, is important in three ways: Helping to ensure that the study is ethical, supporting recruitment, and making study participation as pleasant as possible. If participation in a study is satisfying, then study participants will likely do their best to adhere to the demands of the study. Hence, participant satisfaction benefits both the research participants and the investigators.

We are aware of four published studies of consumer satisfaction to date in the context of pharmacological trials. The first was a survey of 40 of 63 caregivers (63.5%) whose children with intellectual disability participated in pharmacological trials to treat attention-deficit/hyperactivity disorder (ADHD) symptoms (Aman and Wolford 1995). When asked if they would join the study again if they had the option to repeat, 88% of caregivers indicated that they would, 5% were uncertain, and 7% said that they would not. McAdam et al. (2002) surveyed 20 caregivers and participants with developmental disabilities who took part in a trial of placebo and two doses of risperidone (a low and high dose), as well as 52 community members who viewed videos of a subset of participants and were blind to the participants' intervention condition. All questionnaire respondents reported a positive experience in the study, and community members reported that participants on risperidone displayed fewer problem behaviors than did participants on placebo. Tierney et al. (2007), surveyed caregivers of 96 of 101 participants with ASD in a trial of risperidone and placebo for managing highly irritable and disruptive behavior. Overall satisfaction with the study was very high, as 90.5% of caregivers indicated that they would rejoin the study if they had the option to do so. Finally, Rundberg-Rivera et al. (2015) surveyed 150 of 168 caregivers (89.3%) whose typically developing children took part in the Treatment of Severe Childhood Aggression (TOSCA) study. The study compared stimulant plus parent training versus stimulant plus PT plus adjunctive risperidone. Rundberg-Rivera reported very high levels of satisfaction, with 136 caregivers (91%) stating that they would join the study again, 11 (7%) saying maybe, and 1 indicating that she would not.

Therefore, of the four studies published to date, three involved children with developmental disabilities (ASD) (Aman and Wolford 1995; McAdam et al. 2002; Tierney et al. 2006) and one involved typically developing children (Rundberg-Rivera et al. 2015). The existing evidence relating to participant satisfaction has been remarkably favorable and consistent despite varying participant diagnoses, target symptoms, intensity and duration of treatments, and modes of treatment (e.g., pharmacological, psychosocial, combination). Replicating and extending these studies can guide investigators in how to make studies as attractive and palatable as possible.

To address the need for additional treatment options to manage ADHD in children with ASD, the Children with Hyperactivity and Autism Research Treatment Study (CHARTS) examined the single and combined effects of two treatments: Atomoxetine (ATX), a nonstimulant medication approved by the United States Food and Drug Administration for treatment of ADHD, and PT in behavioral strategies to reduce disruptive behavior. CHARTS was a randomized clinical trial (RCT) conducted at three sites: The University of Pittsburgh Medical Center, the Ohio State University's Nisonger Center, and the University of Rochester Medical Center (for more information see Silverman et al. 2014).

We employed a previously used questionnaire to evaluate caregiver satisfaction in CHARTS (Tierney et al. 2007). The meaning of questionnaire items remained the same although, where necessary, we made slight modifications so that all items made sense in the context of the CHARTS study. To our knowledge, this is the first study of caregiver satisfaction in a trial that included a standardized PT program for children with ASD that also provided PT as a solo treatment. Based on previous findings, we hypothesized that caregivers of CHARTS participants would report high levels of satisfaction with the study procedures. We also predicted that caregivers whose children responded favorably to treatment (either ATX or PT, or their combination) would be more satisfied than other caregivers, and that caregivers whose children were assigned to the ATX plus PT combination would be more satisfied than caregivers whose children were randomized to other conditions. Secondary to this, we hypothesized that caregivers with a higher household income would be more satisfied than those with a lower household income. Exploratory analyses included variables such as caregiver age, ethnicity, race, and education level.

Methods

Main study design

We have previously described the background, methods, and child outcomes from CHARTS (Handen et al. 2015; Silverman et al. 2014). Briefly, CHARTS was a 10 week, parallel-groups, randomized, multisite trial. One hundred twenty-eight children were randomized to one of four treatments (n = 32 per group): ATX plus PT, ATX alone, PT plus placebo (PBO), or PBO alone. Parent training consisted of nine weekly visits with a behavior therapist. PT therapists had master's-level or higher education in psychology or a related profession and had completed training and certification in our PT program prior to conducting PT in the study. The target of PT was behavioral noncompliance. Videotaped examples of behavioral strategies were reviewed as part of the training. Study outcome measures included the Parent and Teacher Swanson, Nolan, and Pelham, Fourth Edition (SNAP-IV) questionnaire (Bussing et al. 2008) to assess ADHD symptoms, the Home Situations Questionnaire (HSQ) (Chowdhury et al. 2010) to assess behavioral compliance in daily life, and the clinician-rated Clinical Global Impressions (CGI) (Guy 1976) assessed by raters blinded to treatment assignment. CGI raters interviewed parents during scheduled visits and these raters were kept blind to all treatments throughout the study. A 24 week extension phase followed the acute trial to ensure that all study participants had the chance to try the active study treatment. We are reporting the Parent Satisfaction Questionnaire (PSQ) results from the acute trial.

Study participants

PSQ respondents were caregivers of the CHARTS participants (i.e., biological parents, adoptive parents, grandparents, step parents, and other relatives and nonrelatives). CHARTS participants were 5.00–14.11 years of age inclusive, with ASD and ADHD symptoms. To be eligible for the original study, subjects presented with an ASD according to Diagnostic and Statistical Manual of Mental Disorders, 4th ed, Text Revision (DSM-IV-TR) clinical criteria (American Psychiatric Association 2000), confirmed by the Autism Diagnostic Interview-Revised (ADI-R) (Lord et al. 1994), with a mean item score of >1.5 on the Parent SNAP-IV18 ADHD items, the nine symptom hyperactive-impulsive subscale or the nine symptom inattention subscale, and a CGI-Severity (CGI-S) score ≥4 (moderate). Any other psychotropic medicines were phased out prior to the start of study procedures (with the exception of a subset of subjects [n = 25 or 20%] who received a single dose of melatonin at night for sleep). Subjects had a mental age ≥24 months based on the Stanford-Binet Intelligence Scales-Fifth Edition (SB:5) (Roid 2003) or Mullen Scales of Early Learning (MSEL) (Mullen 1995). Subjects taking anticonvulsants were seizure free for at least 6 months prior to study enrollment and could not be prescribed more than a single antiepileptic drug (AED). One child met this criterion and was taking an AED while enrolled in the study. Participants were enrolled after agreeing to participate and signing an informed consent document approved by the human subjects review boards of each university. A Data Safety and Monitoring Board evaluated safety three times per year.

Study instruments and procedures

Parent Satisfaction Questionnaire

The PSQ consists of 18 items rated on a three point Likert scale from 1 to 3. The first nine items relate to aspects of the study design, including the frequency and duration of the diagnostic, behavioral, and safety measures. Items10–18 assess primary caregiver satisfaction with PT and were administered only to caregivers whose children were randomized to PT. The questionnaire targeted caregiver satisfaction at the molar level (e.g., “Would you participate again not knowing how your child would respond?”), as well as a more molecular level (e.g., “The videotaped examples were generally … ?”). The PSQ was administered to caregivers at the end of the acute trial (Week 10) or, for subjects who left the study early, at the time of study termination. It was the last survey provided to caregivers at the final visit. Study personnel explained the form and left the room while it was completed. To avoid possible demand characteristics, the PSQ was not administered by providers of ATX or PT, and they did not have access to the PSQ report forms.

Responder status

We evaluated the relationship between responder status (i.e., treatment responder versus nonresponder across all four treatment groups) and caregiver satisfaction. CHARTS had two primary outcomes, symptoms of ADHD (i.e., overactivity, inattention, and impulsivity) and behavioral noncompliance in the home and community. ADHD responders showed a ≥ 30% improvement on the SNAP-IV (Bussing et al. 2008) together with a rating of ≤ 2 (i.e., Much Improved or Very Much Improved) on a blinded-clinician-rated CGI–Improvement (CGI-I) scale for ADHD. Behavioral noncompliance responders showed a ≥ 30% improvement on the HSQ (Chowdhury et al. 2010; Altepeter and Breen 1989), together with ratings of ≤ 2 on a blinded clinician rated CGI-I scale for behavioral noncompliance. Because there were both ADHD responders and behavioral noncompliance responders, we evaluated each in relation to caregiver satisfaction.

Treatment assignment

We evaluated the association between the four treatment groups and caregiver satisfaction.

Participant characteristics

The Demographics Survey gathered information about gender, race, ethnicity, household income, caregiver age and education, and subject age and educational placement.

The SNAP-IV oppositional defiant disorder (ODD) subscale (Bussing et al. 2008) was used to evaluate group differences between participants whose caregivers completed the PSQ and participants whose caregivers did not complete the PSQ. Other subscales were used in this comparison as well.

Statistical analyses

Frequency counts were performed to assess satisfaction among study participants. To create subgroups large enough for statistical analyses, responses were collapsed from three to two categories. For example, the most positive category was always associated with the most satisfaction. Therefore, if the question was “How do you feel about the behavioral assessments?” and the responses included Very Important, Somewhat Important, or Not Important, the categories of Somewhat Important and Not Important were collapsed together for a larger subgroup. Once the data were recategorized, we investigated the relationship among caregiver satisfaction, treatment groups, and responder status using Fisher exact tests. The relationship between responder characteristics and satisfaction was explored using Fisher exact tests, and point biserial and rank biserial correlations. The threshold for statistical significance was set at p < 0.05, without correction for multiple comparisons.

Results

Missing data

There were nine (7%) families out of the full study sample of 128 participants who did not complete the PSQ. We compared families who completed the PSQ with those who did not, evaluating key outcomes and demographic characteristics. T tests were conducted to examine group differences. Table 1 contains key clinical and demographic data on respondent and nonrespondent families. Other key demographic data did not differ for the respondent and nonrespondent families. The children of caregivers who did not complete a PSQ had modest but significantly higher scores on the SNAP ADHD subscale and on the ODD subscale, but not on the HSQ mean scores.

Table 1.

Comparison of Parent Satisfaction Questionnaire Respondent Characteristics (n = 119) and the Missing Data Sample Characteristics (n = 9) at the Baseline Visit

Characteristic	PSQ data Mean(SD)	No PSQ data Mean(SD)	p
Child age	8.1 (2.1)	8.5 (1.2)	0.35
IQ	81.5 (24.0)	84.0 (30.0)	0.53
Father age	41.7 (7.8)	40.0 (12.3)	0.77
Mother age	39.5 (6.7)	37.0 (9.9)	0.32
SNAP-IV ADHD	2.2 (0.42)	2.5 (0.40)	0.018^*
SNAP-IV ODD	1.3 (0.67)	1.7 (0.97)	0.046^*
HSQ	3.8 (1.7)	4.3 (1.3)	0.46

p = ≤ 0.05.

IQ, intelligence quotient; PSQ, Parent Satisfaction Questionnaire; SNAP-IV, Swanson, Nolan, and Pelham, Fourth Edition; ADHD, attention-deficit/hyperactivity disorder; ODD, oppositional defiant disorder; HSQ, Home Situations Questionnaire.

PSQ respondents

Additional demographic data for the 119 (93%) children reported on in this study were as follows. Participant mean age was 8.1 ± 2.1 years, 100 (84%) were male, and 19 were female (16%). Ninety seven (82%) caregivers described their child as white; 10 children (8.4%) were described as black, 1 child (0.84%) was described as Asian, and 2 (1.7%) were described as Native American. Nine subjects (7.6%) were described as “Other.” Thirty-three study families (28%) had a household income ranging from < $20,000 to $40,000, 23 (19%) had an income of $40,001–60,000, 32 (26%) had an income of $60,001–90,000, and 31 (26%) had and income of > $90,000. Regarding the mother's education, one mother responded that she had not graduated from high school (1%), 12 mothers (10%) had graduated from high school, 38 mothers (32%) had attended some college or had a 2 year degree, 36 mothers (30%) were college graduates and 31 mothers (26%) had an advanced or professional degree. Median distance to the study site was 32.2 km. Twenty eight subjects (24%) received both ATX and PT, 31 (26%) received ATX alone, 31 (26%) received PT alone, and 29 (24%) received PBO.

PSQ

Table 2 lists items 1–9 completed by all PSQ respondents (n = 119) and shows the frequency and percent of respondents who were satisfied with the study procedures. The data indicate that caregivers were highly satisfied with all aspects of the study.

Table 2.

List of Items from the Parent Satisfaction Questionnaire (n = 119)

Items 1 through 9	Frequency of responses	Percent responses
1. How do you feel about the number of visits for monitoring medication affects?
a. Too few	1	0.8
b. Just right	110	92.4
c. Too many	8	6.7
2. How do you feel about the Autism Diagnostic Interview?
a. Too short/incomplete	0	0.0
b. Well worth the time	113	95.0
c. Too long and detailed	6	5.0
3. The medication used could cause side effects in the patient's system, so we did some blood tests to check for this. How do you feel about the blood tests?
a. Too few to ensure safety	2	1.7
b. A necessary part of treatment	113	95.0
c. Blood tests not necessary	4	3.4
4. How did you feel about the checklists you completed about your child's behavior at each visit?
a. Very important	104	87.4
b. Somewhat important	15	12.6
c. Not important	0	0.0
5. How did you feel about the side effects assessments?
a. Very important	111	93.3
b. Somewhat Important	8	6.7
c. Not important	0	0.0
6. Would you recommend this study to other parents who have children with similar problems?
a. Yes definitely	109	91.6
b. Maybe	10	8.4
c. No	0	0.0
7. If you had it to do all over again, not knowing how your child would react to the medicine(s), would you join the study again?
a. Yes definitely	103	86.6
b. Maybe	15	12.6
c. No	1	0.8
8. Were there aspects of the study that you did not like?
a. Yes	34	28.6
b. No	85	71.4
9. Were there aspects of the study that you did especially like or thought were particularly valuable?
a. Yes	97	82.2
b. No	21	17.6

Table 3 lists items 10–18 of the PSQ, completed by respondents who received PT (n = 59). The data reveal that caregivers were also highly satisfied with PT.

Table 3.

List of Items from the Parent Satisfaction Questionnaire Pertaining to Parent Training Sessions (n = 59)

Items 10 through 18	Frequency of response	Percent response
10. The number of training sessions was generally
a. Too few	1	1.7
b. Just right	54	91.5
c. Too many	4	6.8
11. The length of the training sessions was generally
a. Too short	1	1.7
b. Just right	52	88.1
c. Too long	6	10.2
12. The materials presented by the clinician were generally
a. Too easy	6	10.2
b. Just right	53	89.8
c. Too difficult	0	0.0
13. The videotape examples were
a. Very helpful	22	37.3
b. Somewhat helpful	30	50.8
c. Not helpful	7	11.9
14. The handouts were generally
a. Too easy	4	6.8
b. Just right	55	93.2
c. Too difficult	0	0.0
15. Several behavior intervention sessions were designed to help with acting out, disruption or aggressive behavior. These sessions addressed strategies such as: Attending and rewards, planned ignoring, response cost, and time out. In general these strategies were
a. Very helpful	38	64.4
b. Somewhat helpful	18	30.5
c. Not helpful	3	5.1
16. Present level of confidence: My level of confidence in managing present behaviors is
a. I am more confident since participating in this intervention program	44	74.6
b. I am about the same as when I started the program	14	23.7
c. I am less confident than I had been	1	1.7
17. Level of confidence concerning new problem behaviors: In terms of managing future behavior issues
a. I am more confident since participating in this intervention program	46	78.0
b. I am about the same as when I started the program	13	22.0
c. I am less confident than I had been	0	0.0
18. My overall recommendation: Would I recommend the behavior intervention to other parents who have a child with ASD, ADHD and noncompliance
a. Definitely yes	54	91.5
b. Maybe	5	8.5
c. probably not	0	0.0

ASD, autism spectrum disorder; ADHD, attention-deficit/hyperactivity disorder.

Aspects of the study disliked and liked by caregivers

All PSQ respondents were asked what they particularly disliked and liked about the study and were provided space on the questionnaire to comment on each. Eight (7%) caregivers thought that there were too many visits and that the sessions took too much time. Three caregivers (3%) disliked that their child missed school and/or that they missed work. Two caregivers (2%) disliked each of the following: The method of dosing, the blood collection, the drive to the site, or the delay of reimbursement for time and travel (see Table 4).

Table 4.

Elements of CHARTS Disliked by Parents

Description of study elements	Frequency count
Too many visits/too much time	8
Missed school or work	3
Medication capsules (method of dosing)	2
Blood collection	2
Drive to the center/clinic	2
Delay of payment for time and travel	2
No opportunity for child therapy or social skills training	1
Fast pace when presenting the parent training modules	1
Inflexibility when scheduling appointments	1
Parent Training homework assignments	1
Parking	1
Medication side effects	1
Unpredictability of available child care	1
Possibility of getting a placebo	1
The time it took to dispense medication	1
Lack of description of the procedures	1
Randomization (parent training should have been offered to all participants)	1
When procedures were conducted with a participant and the mother was not present	1
Rating scales	1
Pill minder (opened too easily)	1
Standard parent training (i.e., better if specific for each individual)	1
Maintaining the blind	1

CHARTS, Children with Hyperactivity and Autism Research Treatment Study.

Twenty-eight caregivers (24%) liked the thorough medical monitoring. Nineteen of 59 caregivers in PT (32%) reiterated that they liked the PT and 15 caregivers from the full sample (13%) cited caring and supportive staff members. Eleven caregivers (9%) enjoyed consulting with study personnel each week, and nine caregivers (8%) appreciated the knowledge and experience of the study personnel. Four caregivers (3%) thought the rating scales were useful. Three appreciated respectful treatment (3%), and two liked the medication and the diagnostic battery of tests conducted at the screening visit (2%) (see Table 5).

Table 5.

Elements of CHARTS Liked by Parents

Description of study elements	Frequency count
Thorough medical monitoring^a	28
Parent training (PT)^b	19
Caring supportive staff members	15
Consultations with study personnel each week	11
Knowledge and experience of study personnel	9
Rating scales	4
Felt treated with respect	3
Medication	2
Screening evaluation and diagnostic battery of tests	2
Reimbursement for time and travel	1
Students workers watching siblings	1
More flexibility in dosing	1
The development of a new medication for ADHD symptoms in ASD	1
The routine study visits	1
Teacher ratings	1
Discovery of an iron deficiency through blood tests	1
All study procedures were valuable	1

Most questions that are listed were based on the sample of 119 parents.

As there were 59 participants assigned to PT, this reflects 32% of the parents participating in PT who reiterated PT as an aspect of the study that they liked.

CHARTS, Children with Hyperactivity and Autism Research Treatment Study; ADHD, attention-deficit/hyperactivity disorder; ASD, autism spectrum disorder.

Items relating to overall impressions

Four items queried caregivers on their overall impressions of the study. PSQ respondents were asked if they would join the study again if given the chance to repeat the choice, and 103 (87%) reported that they would. Fifteen caregivers (13%) responded that they might join again and one (< 1%) reported that he or she would not do so. When asked if they would recommend the study to caregivers of children with similar problems, 109 (92%) responded that they definitely would and 10 (8%) responded that they might. Of the caregivers who received PT (n = 59), 44 (75%) felt more confident in managing their child's current behavioral problems, 24 (41%) felt that their level of confidence had not changed, and 1 (2%) felt that he or she was less confident. Also, 46 (78%) felt more confident in managing their child's future behavior problems and 13 (22%) had about the same level of confidence.

Treatment assignment, responder status, and caregiver satisfaction

Treatment assignment was not significantly associated with caregiver satisfaction on 17 of 18 items from the PSQ, including items targeting overall satisfaction (i.e., whether family would recommend the study or would join again). However, it did correlate with whether caregivers thought that the side effects assessments were important (p = 0.05) (Item # 5). Fifty-eight of 59 parents in PT thought the side effects assessments were very important (98%), compared with 53 of 60 parents (88%) who did not receive PT.

Responder status was also not significantly associated with caregiver satisfaction on 17 of 18 items from the PSQ. However, it was significantly associated with appraisal of the difficulty level of the materials provided during PT (p = 0.03) (Item # 12). Thirty-five of the 36 ADHD nonresponders thought that the materials distributed were just right (97%), but only 18 of the ADHD responders thought that the materials were just right (78%).

Exploratory analyses: Sample characteristics and caregiver satisfaction

We explored mother's age, father's age, household income, race, ethnicity, child's age, and intelligence quotient (IQ) for associations with caregiver satisfaction. Three of seven respondent characteristics were significantly associated with 3 of the 18 items on the PSQ. Higher household income and father's age were associated with greater satisfaction with the length of the PT sessions, Higher father's age was also associated with stronger level of confidence when dealing with current behavioral problems, and higher mother's age was associated with a greater likelihood of recommending PT to other caregivers of children with similar issues (see Table 6).

Table 6.

Point Biserial and Rank Biserial Correlations Between Responder Sample Characteristics and Items Taken from the Parent Satisfaction Questionnaire

Responder characteristic	PSQ items	R
Household income	Item #11 “Length of PT sessions”	0.32^*
Father's age	Item #11 “Length of PT sessions”	0.28^*
	Item #16 “Level of confidence current behavioral issues”	0.32^*
Mother's age	Item #18 “Recommend PT”	0.28^*

p < 0.05.

PT, parent training.

Discussion

Our primary objective was to determine whether the caregivers of participants in a multicenter RCT (CHARTS) were satisfied with the research experience. We found that most caregivers endorsed the goals and procedures of the study. Almost all caregivers responded that they would rejoin the study not knowing how their child would respond, suggesting that important problems experienced by their children were being addressed and that the research experience was acceptable. A large percentage of caregivers said that they would recommend the study to other caregivers, further elucidating that the goals of the study were important and the study procedures acceptable. Approximately three quarters of caregivers assigned to the PT condition responded that they were more confident that they could cope with their child's current and future behavioral issues, suggesting that the PT experience was an acceptable component of the research experience.

Items querying caregiver satisfaction with specific treatment procedures were also informative. These items targeted the acceptability of specific procedures, assessments, and evaluations, administered as part of the study. Caregivers expressed satisfaction with each of the study procedures. However, caregivers were relatively dissatisfied with the PT videotapes, as just 37% of caregivers thought the videotaped examples of PT were very helpful (see Table 3). This finding is consistent with the results reported by Rundberg-Rivera et al. (2015), indicating that we can do much more to improve these materials.

It is of concern that one caregiver responded that she or he would not rejoin the study, and another caregiver who received PT became less confident about coping with his or her child's behavioral problems. It may be that 9 weeks of PT with back-to-back homework assignments is too much to ask of some caregivers, and that this particular caregiver was overwhelmed with the number of PT strategies proffered. This was a potential concern when the study protocol was initially designed, and should be a consideration for subsequent study development. For example, when designing future studies we could allow more time for caregivers to absorb the information provided at each PT session by extending the study timeline and adding in some “flex” sessions to go over complex behavioral strategies more thoroughly.

Two open-ended questions about what caregivers liked and disliked about the study procedures revealed that approximately one third of caregivers disliked something about the study but that >90% of caregivers especially liked aspects of the study. The responses provided are potentially useful to researchers, as they help to pinpoint problem areas so that we can improve procedures in future studies. Although a few caregivers disliked certain treatment characteristics (e.g., medication capsules, side effects, pill minders, medication dispensing, and aspects of PT), more caregivers disliked study procedures that increased caregiver or participant burden (e.g., number of visits, missed school, drive to the center, delay of reimbursement, parking, childcare issues, inflexible scheduling of appointments). Notably, these are elements of the research experience that may set research apart from routine treatment (e.g., few families would willingly choose a distant care provider, whereas the distance from home to the research laboratory was an unfortunate fact of life that could not be optimized for every participant).

In general, the high level of satisfaction in the current study confirms findings in earlier studies (Aman and Wolford 1995; McAdam et al. 2002; Tierney et al. 2007; Rundberg-Rivera et al. 2015). We found, much as previous studies did, that caregiver satisfaction was not necessarily linked to perceived clinical response, as both responders and nonresponders to the study interventions were satisfied with the study as a whole.

Clinical response, assignment to treatment conditions, and degree of satisfaction

A counterintuitive finding was that caregivers of study nonresponders were more satisfied than study responders with the PT materials (i.e., handouts, homework assignments). However, this may have been a chance finding, as participants' clinical response was not significantly related to answers to any other PSQ question. The similar satisfaction across groups mitigates one potentially thorny problem in clinical trials in children; namely, the randomization of participants to inert or less desirable treatment alternatives. Clearly, participants accepted the randomization as part of the experimental process; they did not appear to resent it or show less satisfaction because of less beneficial outcomes. This may be reassuring to regulatory agencies, such as institutional review boards, that must appraise the ethics of a trial, including whether the incorporation of a placebo group is fair to subjects (Office for Human Subjects Protections 1993; Aman and Farmer 2008). A placebo group can be important in pinpointing improvement that results from study treatments as opposed to other factors, because individuals with ASD have historically shown marked placebo effects when presented with certain treatment scenarios (e.g., Sandler et al. 1999).

Sample characteristics and satisfactions ratings

Our study corroborates a previous finding (Tierney et al. 2007) that lower income families were less satisfied with the number of visits necessary to complete the study. A possibly related finding is that younger fathers also tended to be less satisfied with the length of PT sessions. This dissatisfaction with the time commitment may reflect problems faced by younger, less affluent caregivers when asking for time off from work, and should be a consideration when developing a research study.

We found that the older the father, the more confident he was that he could manage his child's current behavioral problems and that the older the mother, the more willing she was to recommend PT to other caregivers. It is possible that these findings reflect the perspective that comes with age. However, they should be interpreted with care, as we conducted many exploratory analyses and are unsure whether the statistically significant associations are meaningful or the result of chance.

Limitations

Although we obtained satisfaction data from 119 of 128 participants (93%), nine questionnaires were not completed. In most cases, this was because the participant dropped out of the study and did not return for a final assessment. Analyses of group differences on key outcome variables revealed that study participants with missing data had more severe ADHD symptoms and higher oppositional/defiant ratings by caregivers than the participants whose caregivers completed the PSQ. Caregivers of these individuals are likely to have been less satisfied than those completing all study procedures. A second possible limitation of our study is that the psychometric characteristics of the PSQ have not been scrutinized as rigorously as those of our other outcome measures. The reason is that with each RCT, the goals, procedures, and outcomes of the investigations differ, necessitating some revision. Therefore, although many of the root questions remain the same from one survey to the next, the questionnaire cannot be standardized for all possible types of intervention studies. A third issue is that study personnel and caregivers were aware that the questionnaire would be administered, which may have affected their behavior. Fourth, incentives for remaining in the study included a financial stipend to offset expenses for travel to the centers. This, in addition to the free medical and diagnostic evaluations, could have influenced the manner in which caregivers responded. Finally, multiple comparisons were run to explore associations between subject characteristics and stated satisfaction; hence it is likely that one or more of the observed “significant” associations were found by chance.

Conclusions

Caregivers who perceive a study's goals as worthwhile will be more likely to invest in the research experience, accept the study procedures, and react patiently and flexibly when assigned to control conditions. The importance of enlisting children and their caregivers as part of the research team, with appropriate respect, gratitude, and reimbursement of expenses, has been discussed in detail by Spellecy et al (2008). Our study joins others in suggesting that most caregivers are satisfied with their participation in a RCT; however, continued research in this area is necessary to design studies in ways that minimize the burden to participants. It is possible that research teams can make the experience even more acceptable to participating families. For example, teams can be as flexible as possible in setting up times for study visits, and it may be possible in some settings to arrange designated parking for research participants, which would probably be well received given how busy medical centers tend to be. Satisfaction with the research experience does not appear to be linked to achieving a positive clinical response or being assigned to active treatment conditions, at least for children with ASD and ADHD symptoms. Participating families appear to be quite sophisticated and seem able to take the “bad with the good.” These findings may be useful for informing human ethics committees in addition to strengthening the support for the research experience.

Clinical Significance

The current study findings may be useful to guide practitioners in real world settings. First, the observation that a good number of study participants were satisfied with the experience and responded that they would enroll in the study again if given a choice, demonstrated that the study procedures were acceptable in a research setting. If acceptable to parents in a setting with rigorous procedural guidelines, then it is likely that in a clinic setting similar procedures would be well tolerated. Second, all caregivers brought their children to weekly medical appointments for approximately one hour. The element most liked by parents was the thorough medical monitoring. This information should inform medical practitioners in clinic settings regarding the amount of time that is necessary to provide clear dosing instruction and medical monitoring to their patients. Third, our study results supported a previous finding that lower income families were less satisfied with the number of study visits (Tierney et al. 2007). This information may be useful to clinicians as they develop demanding treatment regimens or behavior support plans to less affluent families. Finally, caregivers who received parent training spent an extra 1-1.5 hours at the Centers each week with a behavior therapist. In order to get good quality behavioral and emotional care for their children, it appeared to be worth their time. Clearly, they tolerated well the clinical services provided as part of the study procedure.

Disclosures

Dr. Hollway has received research funding from Forest Research Institute, Supernus, Sunovion Pharmaceuticals, and Young Living. Dr. Aman has received research contracts from, consulted with, or served on advisory boards of Biomarin Pharmaceuticals, Bristol-Myers Squibb, CogState Ltd., Confluence Pharmaceutica, Coronado Bioscience, Forest Research, Hoffman LaRoche, Johnson & Johnson, MedAvante Inc., Novartis, Pfizer, and Supernus Pharmaceutica. Dr. Handen has received research funding from Curemark, Lilly, and Roche. Dr. Arnold has received research funding from CureMark, Forest, Lilly, Shire, and Young Living; advisory board honoraria from Biomarin, Novartis, Noven, Roche, Seaside Therapeutics, and Shire; consulting fees from Arbor, Gowlings, Pfizer, Tris Pharma, and travel support from Noven. The following authors have no disclosures to make: Drs. Mendoza-Burcham, Silverman, Smith, Tumuluru, and Lecavalier, and Mss. Buchan-Page and Sayre.

References

Altepeter

, Breen

: The Home Situations Questionnaire (HSQ) and the School Situations Questionnaire (HSQ): Normative data and an evaluation of psychometric properties. J Psychoeduc Assess, 7:312–322, 1989.

Aman

, Farmer

: Psychotropic medication research in children and adolescents: Empirical findings and ethical implications. J Empir Res Hum Res Ethics, 3:39–47, 2008.

Aman

, Wolford

: Consumer satisfaction with involvement in drug research: A social validity study. J Am Acad Child Adolesc Psychiatry, 34:940–945, 1995.

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Text Revision. Washington, DC: American Psychiatric Association; 2000.

Bitterman

, Daley

, Misra

, Carlson

, Markowitz

: A national sample of preschoolers with autism spectrum disorders: Special education services and parent satisfaction. J Autism Dev Disord, 38:1509–1517, 2008.

Bussing

, Fernandez

, Harwood

, Hou

, Garvan

, Eyberg

, Swanson

: Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms – psychometric properties and normative ratings from a school district sample. Assessment, 15:317–328, 2008.

Callahan

, Henson

, Cowan

. Social validation of evidence-based practices in autism by parents, teachers, and administrators. J Autism Dev Disord, 38:678–692, 2008.

Chowdhury

, Aman

, Scahill

, Sweizy

, Arnold

, Lecavalier

, Johnson

, Handen

, Stigler

, Bearss

, Sukholdosky

, McDougle

. The Home Situations Questionnaire – PDD Version: Factor structure and psychometric properties. J Intellect Disabil Res, 54:281–291, 2010.

Foster

, Mash

: Assessing social validity in clinical treatment research issues and procedures. J Consult Clin Psychol, 67:308–319, 1999.

10.

Guy

: ECEDU Assessment Manual for Psychopharmacology. Washington, DC: Department of Health, Education and Welfare, National Institutes of Health; 1976.

11.

Handen

, Aman

, Arnold

. Hyman

, Tumuluru

, Lecavalier

, Corbett-Dick

, Pan

, Hollway

, Buchan-Page

, Silverman

, Brown

, Rice

, Hellings

, Mruzek

, Sarah McAuliffe-Bellin

, Hurt

, Ryan

, Levato

, Smith

. Atomoxetine, parent training and their combination in children with autism spectrum and ADHD. J Am Acad Child Adolesc Psychiatry, 54:905–991, 2015.

12.

Lord

, Rutter

, Le Couteur

. Autism Diagnostic Interview – Revised: A revised version of a diagnostic interview for caregivers of individuals with possible developmental disorders. J Autism Dev Disord, 24:659–685, 1994.

13.

McAdam

, Zarcone

, Hellings

, Napolitano

, Schroeder

: Effects of risperdone on aberrant behavior in persons with developmental disabilities: II Social validity measures. Am J Ment Retard, 107:261–269, 2002.

14.

Mullen

: Mullen Scales of Early Learning. Circle Pines, MN: American Guidance Service Inc.; 1995.

15.

Office for Human Subjects Protections: Institutional Review Board Guidebook, 1993. Available at http://www.hhs.gov/ohrp/archive/irb/irb_guidebook.htm (Accessed on December 30, 2015 )

16.

Rundberg-Rivera

, Townsend

, Schneider

, Farmer

, Molina

, Findling

, Gadow

, Bukstein

, Arnold

, Kolko

, Buchan-Page

, McNamara

, Michel

, Austin

, Kipp

, Rice

Jr. , Aman

: Participant satisfaction in a study of stimulant, parent training, and risperidone in children with severe physical aggression. J Child Adolesc Psychopharmacol, 25:1–9, 2015.

17.

Sandler

, Sutton

, DeWeese

, Girardi

, Sheppard

, Bodfish

: Lack of benefit of a single dose of synthetic human secretin in the treatment of autism and pervasive developmental disorder. New Engl J Med, 341:1801–1806, 1999.

18.

Silverman

, Hollway

, Smith

, Aman

, Arnold

, Pan

, Li

, Handen

: A multisite trial of atomoxetine and parent training in children with autism spectrum disorders: Rationale and design challenges. Res Autism Spectr Disord, 8:899–907, 2014.

19.

Spellecy

, Arnold

, May

: Parents and children as members of the research team: Fair employment practices without a union contract. Ethics Behav, 18:1–16, 2008.

20.

Storey

, Horner

. An evaluative review of social validation research involving persons with handicaps. J Spec Educ, 25, 352–401, 1991.

21.

Strain

, Barton

, Dunlap

: Lessons learned about the utility of social validity. Educ Treat Children, 35:183–200, 2012.

22.

Tierney

, Aman

, Stout

, Pappas

, Arnold

, Vitiello

, Scahill

, McDougle

, McCracken

, Wheeler

, Martin

, Posey

, Shah

: Parent satisfaction in a multi-site acute trial of risperidone in children with autism: social validity study. Psychopharmacology, 191:149–157, 2007.

23.

Turan

, Erbas

: Social validation in special education. J Int Soc Res, 3:605–612, 2010.