Reliability and validity of the Pediatric Clinical Assessment of Modes in outpatient pediatric rehabilitation

Abstract

Introduction

Therapists’ interpersonal skills are important contributors to client participation. Providing therapists with opportunities to self-reflect on their approach to therapeutic communication can support occupational therapy best practice. The aim of this study was to evaluate the reliability and validity of the observer version of the Pediatric Clinical Assessment of Modes.

Method

The Pediatric Clinical Assessment of Modes was used to rate therapists’ overall and individual communication mode use according to the Intentional Relationship Model. Successful and unsuccessful attempts were rated separately.

Results

The observers rated 125 therapist–child interactions. The successful and unsuccessful domains of the Pediatric Clinical Assessment of Modes demonstrated appropriate internal consistency, inter-rater reliability, and structural validity for evaluating the therapist’s overall communication and individual use of the collaborating, empathizing, encouraging, instructing, and problem-solving modes. The empathizing, encouraging, and problem-solving subscales demonstrated greater than expected floor effects and could not effectively separate therapists into high and low performance groups for the unsuccessful domain. The observers reported low frequency of successful and unsuccessful communication attempts for the advocating subscale, raising concerns related to the reliability of this subscale for evaluating therapist–child interactions.

Conclusion

Study findings support the reliability and validity of the Pediatric Clinical Assessment of Modes for use in pediatric outpatient rehabilitation.

Keywords

Intentional Relationship Model IRM communication interpersonal challenges therapeutic relationship assessment psychometric testing occupational therapy

Introduction

The ability to communicate in a genuine, empathic, and intentional manner can impact child participation (King, 2009) and support the child in the process of change (King, 2017). The client–therapist relationship has been shown to contribute to client outcomes (Karver et al., 2006; King, 2017), yet there remains a lack of clarity related to how the therapist’s relational competencies can be operationally defined and systematically examined in relation to the process and outcome of therapy (Di Rezze et al., 2014; King, 2017).

The Intentional Relationship Model (IRM) explains the therapeutic use of self in occupational therapy (Taylor, 2008). The model describes a process for strengthening the therapist’s interpersonal skill base, critical self-awareness, and flexibility related to communication (Taylor, 2008), all of which are essential components of therapist competency (King et al., 2007) and contribute to participation in therapy. The IRM operationally defines therapeutic communication according to six modes: advocating, collaborating, empathizing, encouraging, instructing, and problem-solving (Taylor, 2008).

The Clinical Assessment of Modes (CAM) was developed to evaluate therapists’ use of therapeutic communication modes according to the IRM (Fan and Taylor, 2016). The CAM has demonstrated acceptable construct validity in adult inpatient and outpatient rehabilitation (Fan and Taylor, 2016). However, the reliability and validity of this assessment has not been evaluated for use in pediatric settings. The purpose of the study was to evaluate the internal consistency, inter-rater reliability, and structural validity of the observer version of the Pediatric CAM for evaluating therapists’ use of therapeutic communication modes with children receiving outpatient services.

Method

The study was conducted using a cross-sectional design and was approved by the Institutional Review Board at the University of Illinois at Chicago. Therapist and child participants were enrolled across two outpatient pediatric clinics. Written informed therapist consent, parent consent, and child assent to participate were collected. A research assistant was present during a regularly scheduled therapy session to video record the interaction between a therapist and a child. Five master’s students in occupational therapy participated as raters in the study. All raters attended a 16-week course on the IRM and participated in an intensive workshop led by the last author, which included advanced video rating opportunities to ensure inter-rater reliability.

Measures

The observer version of the Pediatric CAM (Taylor and Popova, 2019) was developed for the purpose of this study. The items on the Pediatric CAM closely resemble the original version (Fan and Taylor, 2016) designed for adult rehabilitation settings. The term client was replaced with the term child, as well as caregiver where appropriate. The Pediatric CAM contains 30 items, with five items per mode subscale (advocating, collaborating, empathizing, encouraging, instructing, problem-solving). Based on the findings from Fan and Taylor (2016), the rating scale for the Pediatric CAM was revised to a four-point scale (0 = never, 1 = rarely, 2 = occasionally, 3 = Frequently). A “not applicable” rating was added to capture instances where the item was not observed and considered as not applicable within the given clinical context by the rater. Based on the understanding that the “never” and “not applicable” categories would be used to capture instances where a specific communication attempt was not observed, both categories are given a rating of 0.

Each of the 30 items on the observer version of the Pediatric CAM is rated according to two domains: successful and unsuccessful attempts at therapeutic mode use. Communication is rated as successful when the therapist communicates in a manner that is clear, emotionally congruent, and consistent with the interpersonal needs of the child. Communication is rated as unsuccessful when the therapist communicates in a manner that: (1) is incongruent, (2) mixes two or more modes together in a manner that creates confusion related to the message communicated, or (3) is mismatched to the needs of the child. The successful and the unsuccessful domains are scored in two ways, by obtaining an average score: (1) across the 30 items as a measure of overall communication, and (2) per subscale as a measure of individual mode use.

Data analysis

The reliability and validity of the observer version of the Pediatric CAM was evaluated using classical test theory and Rasch approaches. The successful and unsuccessful domains of the Pediatric CAM were analyzed separately.

Reliability

Internal consistency and inter-rater reliability were analyzed using the IBM SPSS Statistics for Windows, Version 22.0 (IBM Corp., 2013). Cronbach’s alphas were evaluated for internal consistency, with cutoff criteria set to α: < 0.70 = poor, 0.70–0.79 = adequate, 0.80–0.90 = good, and >0.90 = excellent (Tavakol and Dennick, 2011). The intraclass correlation coefficients (ICC) using a one-way random effects model were evaluated for inter-rater reliability, with cutoff criteria set to ICC: <0.50 = poor, 0.50–0.74 = adequate, 0.75–0.90 = good, and >0.90 = excellent (Koo and Li, 2016).

Validity

Structural validity was analyzed using the Winsteps® Rasch measurement computer program Version 3.93.0 (Linacre, 2017). The Rasch model can be used to examine the structural validity of a categorical assessment by evaluating the probability of individual item responses based on: (a) the performance of the people in the sample, and (b) the difficulty of the items on the assessment (Bond and Fox, 2015). Differential item functioning (DIF), rating scale functioning, item targeting, dimensionality, item and person fit, and item and person separation were examined.

DIF was examined to evaluate potential bias in item functioning for different groups of child participants by gender. The DIF was evaluated using the Mantel–Haenszel test. Bias was suspected for items with DIF contrast (effect size) ≥0.50 and p ≤ 0.05 (Bond and Fox, 2015). It was expected that < 5% of items would demonstrate DIF.

Rating scale functioning was examined to establish whether the observers used the rating categories appropriately. It was expected that each rating category would demonstrate: (a) ≥10 responses, (b) monotonic advancement of observed averages, and (c) average outfit MnSq < 2.00 (Linacre, 2004). Item targeting was evaluated by examining assessment tendency toward ceiling or floor effects. Ceiling or floor effects were suspected if >15% of participants demonstrated an absolute maximum or minimum score (McHorney and Tarlov, 1995). Item targeting was further confirmed through visual examination of item–person hierarchy maps.

Dimensionality was evaluated to determine whether the items worked together to measure a single construct of communication. The items were expected to account for ≥50% of the raw variance (Smith and Miao, 1994). Following goodness-of-fit recommendations for Likert scales, acceptable parameters for item- and person-fit statistics were set to 0.60 < Infit MnSq < 1.40 and – 2.00 < Infit Zstd < 2.00 (Bond and Fox, 2015). It was expected that 95% of items and 90% of people would demonstrate appropriate fit to the expectations of the Rasch model for overall communication, while 80% of items and 90% of people would demonstrate appropriate fit for the individual mode subscales. Item measure scores and the associated standard errors were used to compare item difficulty for overall communication and individual mode subscales. Item and person separation were evaluated to examine the sensitivity of the assessments to different groups of item difficulty and person ability. The person separation coefficients were expected to be ≥2.00 with reliability ≥0.80 (Arnadóttir and Fisher, 2008).

Results

A total of 125 therapist–child observations were collected. The number of therapist–child observations per enrolled therapist ranged from one to nine sessions. The observations lasted between 16 and 64 minutes (M = 47.02, SD = 10.66). A parent or caregiver was present for 34% of the observations (n = 43). The Pediatric CAM was completed in full, and there was no missing data.

Participants

A convenience sample of nine therapists and 22 children were enrolled in the study. Therapist participants included occupational therapists (n = 6), physiotherapists (n = 1), and speech therapists (n = 2). All therapists identified as female, with ages ranging from 25 to 43 years old (M = 30.79, SD = 5.32). The ages for child participants ranged from three to eight years old (M = 4.81, SD = 1.68). The majority of children were male (n = 15) and were referred to outpatient rehabilitation following concerns related to autism spectrum disorder (n = 9) or sensory processing (n = 7).

Pediatric Clinical Assessment of Modes: successful domain

The reliability statistics are presented in Table 1. With the exception of the advocating and empathizing subscales, the successful domain demonstrated Cronbach’s alpha greater than 0.80 for overall communication and individual mode subscales (Table 1). The ICC was greater than 0.50 for overall communication and individual mode subscales (Table 1).

Table 1.

Reliability statistics.

Measure	α	ICC	95% CI
Measure	α	ICC	LL	UL
CAM – successful domain	0.92	0.80	0.72	0.86
Advocating	0.47	0.64	0.49	0.75
Collaborating	0.84	0.69	0.56	0.78
Empathizing	0.65	0.62	0.46	0.73
Encouraging	0.82	0.78	0.69	0.85
Instructing	0.88	0.81	0.73	0.87
Problem-solving	0.84	0.78	0.69	0.85
CAM – unsuccessful domain	0.92	0.76	0.66	0.83
Advocating	0.49	0.89	0.84	0.92
Collaborating	0.89	0.46	0.23	0.62
Empathizing	0.77	0.76	0.66	0.83
Encouraging	0.74	0.76	0.66	0.83
Instructing	0.87	0.68	0.54	0.77
Problem-solving	0.82	0.55	0.36	0.69

ICC: intraclass correlation coefficients; CI: confidence interval; LL: lower limit; UL: upper limit; CAM: Clinical Assessment of Modes.

Differential item functioning was noted for one item (3%) on the Pediatric CAM successful domain. Item 19 (“The therapist said things that made the child feel hopeful”) was more difficult to achieve a high score for therapists working with girls as compared to those working with boys. The DIF contrast score for girls was 0.88 with p = 0.02.

Table 2.

Summary of item fit statistics.

Mode/item		Overall communication								Communication per mode
		Successful domain				Unsuccessful domain				Successful domain				Unsuccessful domain
		MnSq	Zstd	IM	SE	MnSq	Zstd	IM	SE	MnSq	Zstd	IM	SE	MnSq	Zstd	IM	SE
Advocating	1. The therapist helped the child or their caregiver get access to resources or people in the community in which he or she lives.	1.3	1.0	4.0	0.3	1.1	0.4	4.0	0.6	0.9	–0.1	–2.6	0.7	0.4	–1.0	–0.6	0.9
	2. The therapist and the child or their caregiver talked about legal/educational/social rights for people with disabilities.	1.0	0.4	6.7	1.0	–	–	–	–	0.5	–0.5	2.2	1.1	–	–	–	–
	3. The therapist reminded the child or their caregiver that other people like them shared similar experiences, feelings, and/or thoughts.	1.1	0.4	5.3	0.5	1.0	0.3	5.1	1.0	1.4	1.2	0.2	0.7	1.6	0.9	1.3	1.2
	4. The therapist made the child or their caregiver aware of individuals, social groups, and/or places in the community that were not a part of the hospital or clinic.	1.1	0.4	5.3	0.5	1.8	1.4	4.0	0.6	0.8	–0.3	0.2	0.7	0.8	–0.2	–0.6	0.9
	5. The therapist helped the child or their caregiver make contact with people who had a similar experience or disability.	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–
Collaborating	6. The therapist allowed the child to choose what would happen next.	1.0	0.3	–3.2	0.2	0.9	–0.5	–1.3	0.2	0.7	–2.8	–1.4	0.2	1.0	–0.1	–1.2	0.2
	7. The therapist made sure that the child worked on what mattered most to him or her.	1.0	0.1	–2.4	0.2	0.9	–0.5	–0.5	0.2	0.8	–2.1	–0.2	0.2	0.8	–2.0	0.5	0.2
	8. The therapist improved or changed something when the child pointed out that it was not helpful.	1.1	0.6	–0.5	0.1	1.0	0.4	–0.2	0.2	1.8	5.2	3.3	0.2	1.8^a	4.5^a	1.1	0.2
	9. The therapist said or did things that made the child feel like he or she was an active participant and decision-maker in the therapy process.	1.0	–0.3	–2.7	0.2	0.7	–2.5	–0.8	0.2	0.8	–2.2	–0.6	0.2	0.7	–2.5	–0.2	0.2
	10. The therapist gave the child control over what he or she accomplished.	1.1	0.6	–3.0	0.2	0.9	–1.0	–0.8	0.2	0.9	–1.3	–1.1	0.2	0.8	–1.8	–.14	0.2
Empathizing	11. The therapist listened to the child with true interest.	0.7	–3.0	–3.2	0.2	0.7	–3.3	–1.2	0.2	0.6	–3.6	–3.9	0.2	0.6	–3.2	–2.2	0.2
	12. The therapist asked the child questions with a high level of sensitivity such that the child felt comfortable.	1.3	1.8	2.8	0.2	1.2	1.0	1.4	0.2	1.1	0.4	3.3	0.2	1.3	1.5	1.9	0.3
	13. The therapist tried to understand the child’s thoughts and feelings, no matter what they were.	2.1^a	6.9^a	0.9	0.2	1.4	2.8	–0.3	0.2	1.4	2.6	1.1	0.2	1.2	1.1	–0.7	0.2
	14. The therapist shared something about his or her personal experience so that the child did not feel alone.	1.4	1.9	3.0	0.2	1.4	1.5	2.5	0.3	1.4	1.7	3.4	0.2	1.9^a	2.5^a	3.7	0.4
	15. The therapist tried hard to understand the child’s needs by listening, observing, and/or asking as many questions as necessary.	0.7	–2.9	–3.2	0.2	0.7	–3.3	–1.5	0.1	0.6	–3.6	–3.9	0.2	0.8	–1.8	–2.7	0.2
Encouraging	16. The therapist pointed out what the child was good at doing.	0.8	–2.0	–2.9	0.2	1.1	0.8	–0.5	0.2	0.6	–3.3	–2.9	0.2	1.0	–0.2	–0.7	0.2
	17. The therapist made the child feel confident about what he or she was doing.	0.8	–1.7	–2.3	0.2	1.0	–0.1	–0.4	0.2	0.9	–0.5	–1.9	0.2	0.7	–2.0	–0.6	0.2
	18. The therapist’s positive attitude showed the child that he or she believed the child was ready to do something the child thought he or she could not do.	0.9	–1.1	0.4	0.2	1.5^a	3.3^a	–0.3	0.2	1.3	2.0	2.7	0.2	0.7	–2.0	–0.4	0.2
	19. The therapist said things that made the child feel hopeful.	1.3	1.8	2.0	0.2	1.3	1.5	1.7	0.3	1.5	2.9	5.3	0.2	1.4	2.1	2.2	0.3
	20. The therapist gave the child a compliment or other kind of reward for something he or she did.	0.7	–2.8	–3.0	0.2	1.0	–0.4	–0.3	0.2	0.6	–3.1	–3.2	0.2	0.8	–1.8	–0.5	0.2
Instructing	21. The therapist explained what was happening or told the child what would happen next.	0.8	–1.9	–2.3	0.2	0.9	–0.7	–2.6	0.1	0.9	–0.4	–1.6	0.2	0.9	–0.6	–0.8	0.2
	22. The therapist told or demonstrated to the child how to improve his or her performance or behavior.	0.8	–1.8	–1.4	0.1	0.9	–0.4	–2.3	0.1	1.1	1.0	0.3	0.2	0.8	–1.3	–0.2	0.2
	23. The therapist gave the child clear directions.	0.7	–3.1	–2.2	0.2	0.9	–1.3	–2.9	0.1	0.7	–2.5	–1.4	0.2	0.6	–3.7	–1.2	0.2
	24. The therapist showed a sense of conviction when making a recommendation.	0.7	–3.0	–1.2	0.1	1.0	–0.4	–1.8	0.1	0.9	–1.1	0.8	0.2	1.1	0.5	0.5	0.2
	25. The therapist taught the child something.	0.9	–1.0	–0.7	0.1	1.1	1.1	–1.2	0.2	1.2	1.7	1.9	0.2	1.5^a	3.7^a	1.7	0.2
Problem-solving	26. The therapist helped the child to think about a problem or activity in a different way.	0.8	–1.3	–0.3	0.1	1.0	–0.4	–0.8	0.2	0.8	–1.6	–1.6	0.2	0.7	–2.6	–1.2	0.2
	27. The therapist explained different alternatives when guiding the child to make a decision or to complete an action.	0.9	–1.2	0.2	0.2	1.0	–0.4	–0.2	0.2	0.7	–2.8	–0.9	0.2	0.8	–1.5	–0.4	0.2
	28. The therapist helped the child think about a problem in a clear-headed, non-emotional way.	1.2	1.1	2.2	0.2	1.2	1.7	0.3	0.2	1.5^a	2.8^a	2.3	0.2	1.4	2.5	0.4	0.2
	29. The therapist helped the child consider many different ways of doing things.	1.2	1.6	0.8	0.2	1.2	1.3	0.1	0.2	1.0	0.2	0.1	0.2	1.1	0.6	0.2	0.2
	30. The therapist helped the child look at a problem by breaking it down into smaller parts.	1.3	2.2	0.9	0.2	1.2	1.1	0.7	0.2	1.1	0.4	0.2	0.2	1.1	0.6	1.0	0.2

IM: item measure; SE: standard error

^aItem demonstrated misfit to the expectations of the Rasch model

n/a = item excluded from analysis due to lack of variance in responses

With a few exceptions, the successful domain met the criteria for rating scale functioning, item targeting, dimensionality, item and person fit, item and person separation for measuring overall communication, and the collaborating, empathizing, encouraging, instructing, and problem-solving subscales (Table 2 and Table 3). Greater than expected variance in responses was noted for the “rarely” rating category for overall communication (outfit MnSq = 2.3) and the encouraging subscale (outfit MnSq = 3.4). In addition, the empathizing and problem-solving subscales demonstrated lower than expected person separation and reliability (Table 3).

Table 3.

Summary of Rasch analysis findings.

Category	Successful domain							Unsuccessful domain
Category	Overall communication	Advocating	Collaborating	Empathizing	Encouraging	Instructing	Problem-solving	Overall Communication	Advocating	Collaborating	Empathizing	Encouraging	Instructing	Problem-solving
Item targeting
Ceiling effect	0%	0%	1%	0%	0%	2%	0%	1%	0%	1%	0%	0%	1%	0%
Floor effect	0%	88%^a	0%	1%	2%	1%	10%	0%	97%^a	15%	18%^a	22%^a	3%	22%^a
Dimensionality	75%	43%^a	70%	84%	81%	68%	65%	55%	43%^a	61%	70%	47%^a	65%	53%
Item fit	97%	100%	80%	100%	80%	100%	80%	97%	100%	80%	80%	100%	80%	100%
Person fit	92%	98%	90%	92%	90%	90%	94%	91%	100%	93%	94%	95%	94%	96%
Item separation
Separation coefficient	7.02	1.60^a	8.14	15.04	15.36	5.77	6.08	5.86	0.20^a	3.00	7.48	4.66	5.07	3.26
Reliability	0.98	0.72^a	0.99	1.00	1.00	0.97	0.97	0.97	0.04^a	0.90	0.98	0.96	0.96	0.91
Person separation
Separation coefficient	3.36	0.00^a	2.24	1.49^a	2.20	2.49	1.88^a	2.98	0.00^a	2.24	1.58^a	1.14^a	2.33	1.33^a
Reliability	0.92	0.00^a	0.83	0.69^a	0.83	0.86	0.78^a	0.90	0.00^a	0.83	0.71^a	0.57^a	0.84	0.64^a

^aDid not meet the predetermined cutoff criteria.

While the advocating subscale on the successful domain met the criteria for item and person fit, it did not meet the criteria for rating scale functioning, item targeting, and item and person separation (Table 2 and Table 3). There were zero responses in the “occasionally” and “frequently” rating categories, and the subscale demonstrated a significant floor effect (Table 3). The items in the subscale explained <50% of variance, and one item was omitted from analysis due to 0% variability in responses (Table 3). Both item and person separation fell below the criteria (Table 3).

Pediatric Clinical Assessment of Modes: unsuccessful domain

The reliability statistics are presented in Table 1. With the exception of the advocating subscale, the unsuccessful domain demonstrated Cronbach’s alpha greater than 0.70 for overall communication and individual mode subscales (Table 1). With an exception of the collaborating subscale, the ICC was greater than 0.50 for overall communication and individual mode subscales (Table 1).

There was no significant DIF noted for the Pediatric CAM unsuccessful domain. With a few exceptions, the unsuccessful domain met the criteria for rating scale functioning, item targeting, dimensionality, item and person fit, and item and person separation for measuring overall communication and the collaborating, empathizing, encouraging, instructing, and problem-solving subscales (Table 2 and Table 3). The collaborating, empathizing, encouraging, and problem-solving subscales demonstrated fewer than 10 responses for the “frequently” category. The empathizing subscale demonstrated greater than expected variance in responses for the “occasionally” rating category (outfit MnSq = 7.6). The empathizing mode fell slightly below the criteria for dimensionality (Table 3). The empathizing, encouraging, and problem-solving subscales demonstrated a tendency toward a floor effect, and did not meet the criteria for person separation (Table 3).

The advocating subscale on the unsuccessful domain met the criteria for item and person fit, and did not meet the criteria for rating scale functioning, item targeting, and item and person separation (Table 2 and Table 3). There were fewer than 10 responses in the “rarely,” “occasionally,” and “frequently” rating categories, and the subscale demonstrated a significant floor effect (Table 3). The items in the subscale explained < 50% of variance, and two items were omitted from analysis due to 0% variability in responses (Table 3). Both item and person separation fell below the criteria (Table 3).

Discussion

The study provides initial evidence for the reliability and validity of the Pediatric CAM for evaluating therapists’ overall and individual mode use in outpatient pediatric rehabilitation; possible threats to the reliability and validity of the advocating subscale were identified. The successful and unsuccessful domains of the Pediatric CAM demonstrated excellent internal consistency, good inter-rater reliability, and good structural validity for evaluating the therapist’s overall communication. Overall, the successful and unsuccessful domains demonstrated appropriate internal consistency, inter-rater reliability, and structural validity for evaluating the therapist’s individual use of the collaborating, empathizing, encouraging, instructing, and problem-solving modes. These findings build on those reported for the validity of the observer version of the CAM in adult rehabilitation samples (Fan and Taylor, 2016).

In the successful domain, the internal consistency for the empathizing mode fell slightly below the expected criteria, suggesting that the items may have not worked together as well as originally anticipated. Examination of item fit statistics further suggested that one item (“The therapist tried to understand the child’s thoughts and feelings, no matter what they were”) showed greater than expected misfit to the Rasch model. Future research should examine whether this item should be revised to ensure greater consistency with other items in the empathizing mode. The empathizing and problem-solving modes could not sufficiently separate therapists into high and low performance groups according to their mode use. This finding could be due to the nature of the present sample or the items included in the subscales. It is possible that greater person differentiation could be found in a larger sample of therapists. It is also possible that additional items need to be added to improve differentiation between therapists that display high and low utilization of these modes.

Consistent with the findings for the successful domain, the unsuccessful domain demonstrated adequate internal consistency, inter-rater reliability, and structural validity for evaluating the therapist’s unsuccessful attempts at the collaborating, empathizing, encouraging, instructing, and problem-solving modes. The empathizing, encouraging, and problem-solving modes demonstrated greater than expected floor effects and low person separation. Convenience sampling likely contributed to self-selection bias and it is possible that the therapists that agreed to participate in this study were more intentional in their therapeutic mode use and were less likely to experience unsuccessful attempts at communication while working with clients. On the basis of this assumption, the floor effect between 18% and 22% and the lower than expected person separation were not deemed to pose a significant threat to structural validity to the unsuccessful domain.

For both successful and unsuccessful domains, the advocating mode demonstrated low variability in responses, which posed a significant threat to reliability and validity. The observers reported a low frequency of both successful and unsuccessful attempts at the advocating mode for therapists in this study. These findings are likely related to the fact that the sample consisted of therapists that worked with children eight years and younger, and parents and caregivers were only present for a small portion of the observed sessions. The advocating mode encompasses items that capture communication that empowers children and their caregivers to get access to resources and other people in the community. Parents and caregivers often assume the role of the child’s advocate, particularly when children lack the developmental capacity to self-advocate. In these cases, the advocating mode would not be used in situations where the therapist is working one-on-one with a child in the absence of the parent or caregiver. In light of these findings, the advocating mode may not be appropriate for evaluating these types of therapist–child interactions. Additional research is warranted to evaluate the reliability and validity of the advocating subscale in clinical situations where the parents or caregivers are present and actively participating during therapy.

Limitations

The study findings are limited by convenience sampling. The present sample size met the minimum requirements for reliability and validity analysis (Linacre, 1994). However, additional research with a more robust sample is warranted. Future research should investigate clinical situations in which children demonstrate the developmental capacity to self-advocate, and parents and caregivers are actively participating and engaged in the session. The enrolled therapists were more likely to be successful than unsuccessful in their attempts at therapeutic communication. While these findings highlight clinical practices that are considered to be optimal for positive client outcomes, they do limit the conclusions that can be drawn related to the validity and the reliability of the observer version of the Pediatric CAM.

Recommendations for future research

Additional research is necessary to evaluate the reliability and validity of the other versions of the Pediatric CAM (client and therapist self-report versions). The client’s developmental capacity to reliably self-report their experience of the therapist’s communication mode should be considered. Additionally, investigation of the reliability and validity of caregiver proxy is warranted. The observer version of the Pediatric CAM may be used to evaluate a therapist’s utilization of different interpersonal modes in practice, and the relationship between the different modes of communication, client participation, and clinical outcomes.

Conclusion

The observer version of the Pediatric CAM demonstrated appropriate reliability and validity for evaluating therapists’ successful and unsuccessful attempts at overall and individual mode use in outpatient pediatric settings. The advocating subscale was found to be prone to floor effects in this sample, which raises concerns related to the reliability and validity of the advocating subscale for use in clinical situations where the therapist is working with a young child without a parent or a caregiver present.

The use of reliable and valid instruments for evaluating therapists’ communications can support the relational competencies of therapists working in pediatric rehabilitation settings. The Pediatric CAM can be used in conjunction with measures of family-centered care, such as the Measure of Processes of Care (Cunningham and Rosenbaum, 2014), or other aspects of interpersonal relating, such as the Effective Listening and Interactive Communication Scale (King et al., 2012), as part of quality improvement and professional development initiatives.

Key findings

The Pediatric CAM is a reliable and valid measure of overall, collaborating, empathizing, encouraging, instructing, and problem-solving mode use.

The Pediatric CAM was not effective at capturing advocating mode use.

What the study has added

The Pediatric CAM offers a reliable and valid approach for evaluating therapists’ successful and unsuccessful attempts at overall and individual mode use according to the Intentional Relationship Model.

Footnotes

Acknowledgments

We would like to thank our community partners Laura Mraz and Kavitha Krishnan for assisting us with participant recruitment. Additionally, we would like to thank the dedicated research assistants Jenna Colangelo, Kaitlin Ibara, Jennifer Wescott, and Ariana Rodriguez.

Research ethics

This research was approved by the Institutional Review Board at University of Illinois at Chicago. Protocol #2014-0796 was approved in 2014, with research amendment for the present study approved in 2017.

Consent

Written informed therapist consent, parent consent, and child assent to be observed and video recorded for this study was obtained upon participant enrollment.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Contributorship

Evguenia Popova, Su Ren Wong, and Renee Taylor applied for ethical approval and contributed to the research design and methodology. Evguenia Popova and Rikki Ostrowski contributed to the literature review and statistical analysis. All authors interpreted the data. Evguenia Popova wrote the first draft of the manuscript. Evguenia Popova and Renee Taylor reviewed and edited the manuscript and approved the final version.

References

Arnadóttir

Fisher

(2008) Rasch analysis of the ADL scale of the A-ONE. American Journal of Occupational Therapy 62: 51–60.

Bond

Fox

(2015) Applying the Rasch Model: Fundamental Measurement in the Human Sciences, 3rd edn. New York: Routledge.

Cunningham

Rosenbaum

(2014) Measure of processes of care: A review of 20 years of research. Developmental Medicine & Child Neurology 56(5): 445–452.

Di Rezze

Law

Eva

, et al. (2014) Therapy behaviours in paediatric rehabilitation: Essential attributes for intervention with children with physical disabilities. Disability and Rehabilitation 36(1): 16–22.

Fan

Taylor

(2016) Assessing therapeutic communication during rehabilitation: The Clinical Assessment of Modes. American Journal of Occupational Therapy 70(4): 7004280010p1–7004280010p10.

IBM Corp. Released (2013) IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp.

Karver

Handelsman

Fields

, et al. (2006) Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review 26(1): 50–65.

King

(2009) A relational goal-oriented model of optimal service delivery to children and families. Physical & Occupational Therapy in Pediatrics 29(4): 384–408.

King

(2017) The role of the therapist in therapeutic change: How knowledge from mental health can inform pediatric rehabilitation. Physical & Occupational Therapy in Pediatrics 37(2): 121–138.

10.

King

Currie

Bartlett

, et al. (2007) The development of expertise in pediatric rehabilitation therapists: Changes in approach, self-knowledge, and use of enabling and customizing strategies. Developmental Neurorehabilitation 10(3): 223–240.

11.

King

Servais

Bolack

, et al. (2012) Development of a measure to assess effective listening and interactive communication skills in the delivery of children’s rehabilitation services. Disability and Rehabilitation 34(6): 459–469.

12.

Koo

(2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine 15(2): 155–163.

13.

Linacre

(1994) Sample size and item calibration stability. Rasch Measurement Transactions 7(4): 328. Available at: www.rasch.org/rmt/rmt74m.htm (accessed 12 June 2019).

14.

Linacre

(2004) Optimizing rating scale category effectiveness. In: Smith

Smith

(eds) Introduction to Rasch Measurement: Theory, Models Application. Maple Grove: JAM Press, pp.258–278.

15.

Linacre

(2017) Winsteps® (Version 3.93.0) [Computer Software]. Beaverton: Winsteps.com. Available at: www.winsteps.com (accessed 1 January 2017).

16.

McHorney

Tarlov

(1995) Individual-patient monitoring in clinical practice: Are available health status surveys adequate? Quality of Life Research 4: 293–307.

17.

Smith

Miao

(1994) Assessing unidimensionality for Rasch measurement. In: Greenwich

(ed.) Objective Measurement: Theory into Practice (Vol. 2). Norwood: Ablex, pp.316–328.

18.

Tavakol

Dennick

(2011) Making sense of Chronbach’s Alpha. International Journal of Medical Education 2: 53–55.

19.

Taylor

(2008) The Intentional Relationship: Occupational Therapy and Use of Self. Philadelphia: F.A. Davis Co.

20.

Taylor

Popova

(2019) Pediatric Clinical Assessment of Modes – Observer, Version 3.0. Chicago: Intentional Relationship Model Clearinghouse, Department of Occupational Therapy, College of Applied Health Sciences, University of Illinois at Chicago. Available at: http://irm.ahslabs.uic.edu/assessments/ (accessed 12 June 2019).