Providers’ Reported and Actual Use of Coaching Strategies in Natural Environments

Abstract

This case study examined the agreement between reported and actual use of coaching strategies based on home visit data collected on a diverse sample of providers and families. Paired videotape and contact note data of and from providers during home visits were collected over a six month period and analyzed using structured protocols. Results of both descriptive and correlation analyses indicated providers used a range of coaching strategies, tended to underreport (under-represent) their efforts on contact notes, and demonstrated practices that were primarily collaborative and family-centered. Agreement between actual and reported use of coaching strategies was variable and did not support our anticipated finding. Implications for future research and practice are discussed.

Keywords

home visits coaching early intervention caregivers providers

There is an expectation in the field of early intervention that providers engage in practices that are family-centered and designed to strengthen the parent/caregiver–child (hereafter referred to as caregiver) relationship as a means of promoting positive child outcomes (Bruder, 2000; McCollum & Yates, 1994; National Early Childhood Technical Assistance Center [NECTAC], 2008). This expectation is asserted in both professional policy statements and recommended practice documents (NECTAC, 2008; Sandall, Hemmeter, Smith, & McLean, 2005). Family-centered practices are rooted in a blend of theoretical perspectives that recognize the value of strategies that guide, support, and scaffold caregiver learning across contexts that have meaning to the family (Brown & Palincsar, 1989; Rogoff, 1990; Vygotsky, 1978). Ideally, providers focus on working through and with the caregiver, with the expressed goal of having the parent assume a leader role and the provider acting as facilitator of positive caregiver–child interactions (McCollum & Yates, 1994; Woods & Lindeman, 2008). Early intervention providers use a variety of adult learning strategies to foster positive caregiver–child interactions, including coaching, modeling, listening, sharing feedback, prompting, and/or problem solving. These various strategies can loosely be combined under the term coaching (Peterson, Luze, Eshbaugh, Jeon, & Kantz, 2007). Despite persistent calls for providers to use collaboratively implemented intervention practices, recent evidence suggests that a substantial gap exists between expected (recommended) and actual performance among providers who deliver early intervention services in natural environments, including the child’s own home.

Indeed, studies have indicated that the vast majority (51%–83%) of home visit time is spent in provider–child, rather than caregiver–child focused interactions (Campbell & Sawyer, 2007; Hebbeler, Spiker, Morrison, & Mallik, 2008; Peterson et al., 2007) and that providers are generally not intentional about helping the caregiver assume the leader role (Campbell, Sawyer, & Muhlenhaupt, 2009). Research also has suggested that providers believe they are enacting family-centered practices during home visits even though they are not (Crais, Roy, & Free, 2006), and they engage in provider–child interactions despite espousing a philosophy of family-centered practices (Brorson, 2005). Collectively, these studies indicate that collaborative consultation practices are not being used to foster the caregiver-as-leader role during home visits. The consequence of this is that caregivers may not be receiving the training or the practice needed to build their competence and confidence to promote their child’s learning throughout the day.

Evaluations of implementation fidelity at the program and provider levels are important as the field moves forward in the use of evidence-based practices (Dunst & Trivette, 2009a; Odom, 2009). Simply put, we need to know what it is that we are trying to do and that we did what we said we were going to do, as a basis for determining if what we did worked. Measurement of fidelity involves strategies that monitor and enhance the reliability and validity of interventions, and includes many constructs such as implementation (what the program is intending to provide) and intervention (what the child and family receive from the provider). The broad range of professional practices, diverse child and family characteristics, and the often variable nature of early intervention service delivery further complicate the measurement of implementation in authentic early intervention settings.

Recent studies suggest that low fidelity of implementation at the provider level is more typical than not, and both theory and research suggest that such findings may be related to individual, pedagogical, organizational, and procedural factors (Azjen, 1991; Dunst & Trivette, 2009b; Hall & Hord, 2001; McLaughlin, 1987; Odom, 2009). The reasons for implementation problems may be related to inadequate training and generalization of what was taught in a preservice or inservice context to actual home settings, providers’ views that recommended home visiting practices are too complex for the settings in which they actually work (Dunst & Trivette, 2009b), or professional preferences for practices that are at variance with those of their employing agency (Brorson, 2005; McLaughlin, 1987). Cognitively, practitioners may perceive that what they are doing reflects recommended practices and they practice with the belief that they are meeting expectations. Alternatively, providers may intend to implement as expected, but their performance may be influenced by a perceived lack of control over the situation, the views of others, or their own attitudes about what they are being asked to do (Azjen, 1991; Salisbury, Woods, & Copeland, 2010).

The concept of doing what you said you would do (or were trained and expected to do) represents a type of implementation or intervention fidelity that is not unlike the constructs that underlie early correspondence training. In correspondence training, researchers evaluated the effects of teaching and reinforcing the match between what the individual verbalized they expected to do prior to engaging in the action and what they actually did do (say–do), or what the individual reported doing (say) with consequences for the truth value of the action taken following the actual occurrence of the behavior (do) (Isreal, 1978; Karlan & Rusch, 1982). If one looks at written documentation of what transpires in home-visit sessions (e.g., SOAP or contact notes) as the “say” portion of the correspondence equation, and direct observation of what actually transpires (e.g., videotapes of home visiting sessions) as the “do” component, it then becomes possible to evaluate the correspondence between reported and actual use of coaching strategies by early intervention providers. Secondarily, it allows us to determine whether contact notes might be a useful proxy for direct observation to study fidelity of implementation at program and provider levels.

Emerging research provides a preliminary picture of what providers do during home visits and has helped the field begin to understand the gap that exists between expected and actual use of coaching strategies. To date, there has been insufficient study of coaching strategies in early intervention and no direct investigation of what providers report doing in practice. By examining both actual and reported practices, it may be possible to move toward a deeper understanding of why and where the breakdown in implementation exists. With that knowledge in hand, Part C programs can improve professional development and support activities so there is greater correspondence between enacted and perceived practices.

The purposes of this case study were twofold: first, to measure program level fidelity of implementation across a sample of providers using video review of home visits, and second, to apply the concept of do–say correspondence to the evaluation of practitioners’ use of expected program and recommended practices. Specifically, we were interested in examining the level of agreement between actual and reported use of coaching strategies during home visits conducted in an urban setting. Two questions guided this investigation:

How often did providers in this sample use coaching strategies expected by this program during their home visits?

To what extent was there agreement between actual and reported use of coaching strategies?

Given that this study occurred in a program that intentionally promoted the use of collaborative consultation with coaching strategies, we anticipated that frequency and agreement levels would be greater than previously reported in the literature. In this report, we describe the referent home visiting process and expected provider behaviors, how we approached the collection and analysis of observed and reported behavior, and our findings and interpretations based on the correspondence analysis we undertook.

Method

Context

An exploratory case study method was selected for the analysis of providers’ home visiting practices because of the preliminary nature of the data, small sample size, and focus on a single program site. A local Part C program with federal model demonstration funding served as the context for this investigation. The unit of analysis in this study was the home visits undertaken by this program’s early intervention providers. Prior to collection of data, all providers in this program received didactic and participatory professional development over a 2-year period (six 2-day sessions) on the key principles and practices associated with the Family Guided, Routines-Based Intervention (FGRBI) approach developed by Woods and colleagues (Woods, 2005). Although all providers had had some preliminary exposure to basic concepts of family-centered practices before the onset of training and each was skilled in his or her own discipline as pediatric-oriented professionals, their initial level of knowledge and skill about FGRBI could best be described as basic. Coaching strategies and embedded interventions in typical family routines are essential elements of the FGRBI approach. As an adjunct to this core training, a home visiting process guide was developed in collaboration with the program’s providers. This guide contained essential process elements and performance indicators anchored in evidence-based and recommended practices that helped providers move through the home visiting process (Campbell & Sawyer, 2007; Sandall et al., 2005; Woods, 2005; see Table 1). Subsequent to initial professional development, providers received ongoing supervision support to ensure that practices were implemented as taught, as well as problem-based professional development on areas of concern. The inclusion of reflective supervision and problem-based learning were components added to FGRBI in the development of the Chicago Early Intervention Project (CEIP) model (Salisbury et al., 2010). Table 1 defines the components of the home visiting process used in CEIP and includes key indicators of the FGRBI practices that providers were taught and expected to exhibit during home visits. Details on the professional development process can be found in Salisbury et al. (2010) and on the website for the FGRBI project (http://tactics.fsu.edu).

Table 1.

CEIP Model Components, Home Visiting Process, and Selected Fidelity Checklist Indicators

CEIP component	Home visit process	Cross reference to fidelity checklist indicators
Responsive home visit format	• Meet family at time and location of their choice	Obtain updates about health issues, accomplishments, setbacks, concerns, etc.
	• Interact with caregiver and child unit as the “client,” not the child
	• Ensure family priorities are addressed each session
	• Establish family as decision maker on team (ongoing)
	• Share information to support understanding of role and responsibilities in CEIP
	• Include family members and informal supports as appropriate
	• Review child and family priorities, health and development updates, and concerns each session
	• Ask and answer questions to support caregiver competence and confidence throughout session
Caregiver-implemented assessment and intervention	• Arrange the environment to facilitate the caregiver–child dyad interactions	Provide instruction to teach a strategy to the caregiver
	• Step back and observe parent–child interactions to identify child skills and supports provided by caregiver	Model or demonstrate strategy with explanation for the caregiver to watch and try
	• Suggest or demonstrate intervention strategies to promote or increase child engagement or practice on outcomes desired	Provide feedback to the caregiver
	• Provide positive and constructive feedback	Provide time for the caregiver to practice strategies
	• Engage caregiver in self-reflection on the learning opportunities and supports in various routines
	• Problem-solve with caregiver to establish integration of best-fit strategies and targets for each routine
	• Develop communication and instruction style in formats most comfortable to family (e.g., verbal suggestions, modeling, joint interaction, written notes)
Family-preferred activities, routines within natural environments	• Identify and continue to expand knowledge and use of child and family typical daily routines and activities throughout the day (ongoing)	Discuss how to connect skills being learned to other routines/activities throughout the day
	• Join family in activities and routines already occurring and discuss natural learning opportunities
	• Emphasize child and family interests and preferences
	• Ensure embedding occurs in routines across the child and family’s day where natural and functional; do not make up routines—incorporate learning into what the family does
	• Connect child skills and adult strategies from one routine to another and relate to IFSP outcomes
	• Expand learning by brainstorming other environments or routines for generalization and maintenance
	• Scaffold for child and family by using familiar routines when introducing new strategies and new routines to expand use of acquired skills
Continuous progress monitoring	• Involve family members in monitoring progress using family-identified strategies (e.g., anecdotes, wipe boards)	Obtain updates about health issues, accomplishments, setbacks, concerns, etc.
	• Gather updates each session on what happened between visits	Share information about the child’s progress
	• Engage caregiver in self-reflection . . . what worked, what did not, what do you think made it work?	Evaluate progress with the caregiver
	• Plan for targets, time, and routines between sessions	Discuss and plan activities for the next visit
	• Review contact information if family wants to call with questions
	• Complete session contact notes, quarterly updates, annual reviews
Reflective supervision and mentoring	• Respond to self-evaluation questions at team meetings
	• Watch video and share implementation fidelity
	• Debrief with supervisor

Abbreviations: CEIP, Chicago Early Intervention Project; IFSP, Individualized Family Service Plan.

Participants

Two groups of individuals participated in this study and both provided informed consent. The first group was six early intervention professionals and included one occupational therapist (OT), one physical therapist (PT), two speech–language pathologists (SLPs), and two developmental therapists (DTs). All providers held at least a master’s degree, licensure, and credentialing appropriate to their respective discipline and the Illinois early intervention system. Each provider had been employed by or affiliated with this program for 2 to 12 years (M = 5.67, SD = 3.44). Half were Caucasian (n = 3), one was African American, and two were of Hispanic origin. All but one of the participants was female. To protect confidentiality of participants, we refer to all providers as “she” throughout this report.

The second group of participants included the caregivers of infants/toddlers served by this program. Two thirds of these participants were of Hispanic descent, four were Caucasian, and the rest were African American or Asian. Fifty-five percent of these families received public assistance. Additional detail about these caregivers was not available because many declined to provide information (e.g., income, education, employment status) for personal reasons. The infants/toddler children of these families received an average of 4 hr of early intervention services per month from the program, had a mean age at entry of 17 months, an average length of enrollment of 13.15 months, and evidenced either significant developmental delays (n = 19) or neurological or chromosomal disabilities (n = 2).

Sampling Strategy

Because not all families served by each provider gave consent for home visits to be videotaped, we began by determining which families by provider had consented to and completed videotape sessions during the past 2 years (n = 29). We then determined which children/families received intervention services for at least 6 months (n = 21) and identified the number of home visit sessions in which the first videotape occurred no sooner than 2 months following the child’s initial date of service. These time frames provided sufficient time for providers to build rapport with and an understanding about the child/family, and represented a reasonable period of time in which home based services were delivered to the family. Because this program employs both full- and part-time staff with differing capacities for caseloads, we then sought to identify the minimum number of videotapes (n = 15) that was common across all six providers who conducted home visits to ensure equitable representation of behavior in the videotape samples. The final number of videotapes used to code expected and actual provider behavior was 90.

Measures

Fidelity of Implementation Checklist

The extent to which providers in this sample enacted the expected process steps of a home visit was evaluated using a Fidelity of Implementation Checklist derived from materials developed by Woods and colleagues (2005). This checklist contained 18 process indicators that are rated in terms of observed frequency: 1 (observed more than half the session), 2 (some of the session), and 3 (not observed). For ease of description, Categories 1 and 2 were collapsed to create a binary scale (observed vs. not observed). For this study, we selected nine items from the checklist that most directly reflected coaching strategies. Several items were omitted because they could be accomplished in a child-focused approach, as well as through collaborative practice (e.g., uses child and family everyday materials, engages in multiple routines). Referencing both evidence-based and recommended practice information, the authors made the final determination as to which of the 18 items appeared to most closely reflect desired home visiting and coaching practices. Each provider’s level of implementation is depicted as the percentage of the nine items observed, with higher scores indicating greater fidelity with a collaborative approach to home visiting (see Table 1).

Videotape coding protocol

The Routine and Instructional Strategy Coding Protocol-IL (Salisbury, Cambray-Engstrom, Woods, & Friedman, 2008) was used to evaluate the extent to which providers evidenced the process of collaborative consultation during home visits. For this study, only items pertaining to coaching strategies were analyzed. The protocol is based on work by Woods and colleagues and has been used to analyze the frequency of provider coaching and caregiver engagement in a small sample of urban families (Friedman, 2009). The coding protocol contains nine operationally defined coaching strategies that reflect teaching and communication behaviors, and represent strategies used in the adult learning, consultation, and coaching literatures (Buysse & Wesley, 2004; Hanft, Rush, & Shelden, 2004). Items from the protocol are reflected in Table 2.

Table 2.

Contact Note, Coding Protocol, and Fidelity Checklist Items

	Contact note items
Strategies	Videotape coding protocol items	Fidelity checklist items
Conversation, information sharing	CIS = conversation, information sharing	Obtain updates about health issues, accomplishments, setbacks, concerns, etc.
		Share information about the child’s progress
Observing, data collection	OB = observing, data collection (parent works with child while provider observes without feedback)	Evaluate progress with the caregiver
Problem solving, planning	PS = problem solving (parent and provider discuss strategies to improve outcomes)	Discuss how to connect skills being learned to other routines/activities throughout the day
Joint interaction	JI = joint interaction (provider and parent work as partners with child)
Demonstration	DEM = demonstrating and modeling	Demonstrate or model strategy with explanation for the caregiver to watch and try
Caregiver practice with feedback	CPF = caregiver practice with feedback (parent leads one routine, and provider supports with positive feedback, encouragement, indirect prompts)	Provide feedback to the parent
		Provide time for the parent to practice strategies
Guided practice with feedback	GPF = guided practice with feedback (provider and parent work as partners with child to practice strategies)	Provide direct instruction to teach a strategy to the parent
		Demonstrate or model strategy with explanation for the parent to watch and try
Direct teaching	DT = direct teaching (how-to content presented to increase parent knowledge or capacity for skill mastery)	Provide direct instruction to teach a strategy to the parent
	CF = child-focused interaction. Provider works directly with child while caregiver is not present or engaged; provider makes no attempt to involve caregiver

Note: The Fidelity of Implementation Checklist reflects process items and does not contain items that expect the provider to work directly with the child without the caregiver present (child focused), nor does it contain reference to specific routines. Contact notes reflected only those coaching behaviors that the provider was expected to use and did not include video coding strategy items that either ran counter to promoting caregiver-as-leader behavior (child-focused interaction) or were too global for the provider to code (joint interaction).

Contact notes of home visits

A contact note form was developed and used by this program for billing and service accountability purposes. Consistent with recommendations from recent research on how programs can enhance the fidelity of reporting practices by practitioners (Hagermoser-Sanetti, Chafouleas, Christ, & Gritter, 2009), providers completed their contact note form either during the last 15 minutes of each home visit or on the same day of service to document what occurred during the home visit. The contact note contains four quadrants that represent caregiver or provider behaviors during the home visiting session. These quadrants contain space for clinical progress notes as well as check boxes designed to capture which coaching strategies were used during the home visit session. Specific coaching strategy boxes were checked if, during the home visit, the provider used the strategy to promote the caregiver’s knowledge and skills. Seven of the nine strategies from the video coding protocol were included on the contact note form because they reflected the coaching strategies described in the research literature as associated with promoting caregiver as leader behaviors. Child-focused and joint interaction strategies were omitted for this reason. Providers were introduced to the definitions of these strategies during initial trainings by the third author, and video examples of providers’ home visits were subsequently used as additional teaching and learning opportunities during reflective supervision sessions within CEIP. Terminology on the contact note form matched that of the items on the coding protocol. Table 2 contains items from the contact note form.

Procedures

Rating expected and actual practices

Evidence of expected and actual provider home visiting behaviors was obtained by videotaping regularly scheduled home visit sessions over the course of at least four months. In the manner described by Brady, Peters, Gamel-McCormick, and Venuto (2004) and Peterson et al. (2007), videotaping was done either by a graduate student who accompanied the provider on the home visit or the provider, who taped the session herself by setting up a tripod for the camera in the main living area and letting it run throughout the home visit session.

Graduate students from special education and occupational therapy departments (n = 4) served as video coders. Coders participated in a training process that involved a group orientation to the coding task, followed by individual time spent studying the definitions and coding process. Once coders felt confident, they practiced rating five videotapes of home visits that were not included in the study using both the Fidelity Checklist and Routine and Instructional Strategy Coding Protocol. A 30-s observe-then-code interval procedure was used to rate videotapes, and interrater reliability of coding was established by having two students independently rate the same videotapes. After coders reached 80% reliability, they then coded the videotapes used in this study. To maximize coding integrity, interrater agreement was calculated after every five videotapes to check for rater drift. Final interrater agreement was calculated on a random selection of 25% of all English and Spanish videotapes. Interrater agreement on item coding was calculated by dividing the number of agreements by the sum of the agreements and disagreements by having a second rater (whose language matched that of the caregiver) independently code the same tapes (English = 80%, Spanish = 80%).

To code the videotapes using the Fidelity Checklist, the same coders rerated a random selection of 30% of each provider’s videotapes. Each videotape was coded to determine how many of the nine items were observed. Data were aggregated across the sample of 30 videotapes, and the program level of implementation for the six providers was reflected as the percentage of items observed. Interrater agreement on scoring of individual items was calculated by dividing the number of agreements by the sum of agreements and disagreements for all Spanish and English videotapes by having a second rater (whose language matched that of the caregiver) independently code the same tapes (English = 87%, Spanish = 76%).

Rating reported practice

For each videotaped home visit, the corresponding contact note for that visit was analyzed to determine whether the strategies that providers reported using during the home visit were those that were actually observed on the videotapes. The coders who rated the videotapes identified which strategy boxes had been checked on each contact note. A random sample of 20% of the contact notes were independently coded by the raters and compared to ensure accuracy during data entry. Interrater agreement on accuracy of item-level data entry was calculated by dividing the number of agreements by the sum of agreements and disagreements (96%).

Data Analysis

For purposes of this study, the data were analyzed three ways using descriptive and/or correlation statistics to address our guiding questions. To make a direct comparison between observed and reported data, we aggregated observation data from all 90 videotapes and compared the frequency of observed behavior during each home visit to the 90 corresponding contact notes as binary variables (ever observed or not, ever reported or not). To gather a more refined view of how often coaching strategies were observed during home visits, frequency of strategy occurrence was calculated using the 30-s interval data from the 90 videotapes. Finally, to determine the extent to which there was a statistical correspondence between observed and reported use of coaching strategies during home visits, videotape data were analyzed at the individual provider level and the mean occurrence of each strategy was correlated with the mean of the reported data from the contact notes. These data generated continuous variables from which correlation coefficients could be derived.

Results

To what extent did these providers demonstrate implementation fidelity with the CEIP home visiting process?

Analysis of the fidelity of implementation data indicated that five of six providers used 77% of the process steps on the Fidelity Checklist (M = 77.33, SD = 15.54, range 71%-82%). The sixth provider averaged only 34% of the desired CEIP home visiting process steps, thus affecting aggregated program-level ratings.

How often did providers in this sample use coaching strategies expected by this program during their home visits?

Both binary and interval frequency analyses were used to address this question. The mean length of videotapes used for these analyses was 48.4 min (SD = 9.44, range 29-61). As is evident in Table 3, the binary analysis revealed that providers used each of the coaching strategies at least once in the 90 home visits. However, a frequency analysis of the contact note data revealed a consistent pattern of underreporting by providers. That is, discrepancies in the mean number of mentions were evident when contact note and observation data were compared. Because the binary analysis, necessary to compare video and contact notes, tended to render a global view of strategy use, a frequency analysis was undertaken using the 30-s-interval data. This analysis revealed that Conversation/Information Sharing and Joint Interaction were used most often during home visits. Furthermore, the analysis revealed that Child-Focused Interaction was observed in only 12.64% of the intervals. Problem Solving, as defined in the coding protocol, occurred in less than 1% of the intervals.

Table 3.

Observed and Reported Use of Coaching Strategies^a

Coaching strategy^b	Observed at least once	Reported	% Observed frequency^c
Conversation, information sharing	96.7	76.7	23.37
Observing	54.4	10.0	4.60
Problem solving	11.1	13.3	0.36
Joint interaction	95.6	NA	30.38
Demonstration	50.0	64.4	5.08
Caregiver practice with feedback	63.3	45.6	5.20
Guided practice with feedback	71.1	14.4	6.65
Direct teaching	51.1	42.2	2.69
Child-focused interaction	70.0	NA	12.64

Note: Contact notes reflected only those coaching behaviors that the provider was expected to use and did not include video-coding strategy items that either ran counter to promoting caregiver-as-leader behavior (child-focused interaction) or were too global for the provider to code (joint interaction).

Items reflect percentage of 90 videotapes in which the coaching strategy was observed at least once during the home visit session.

Strategy items are ordered from least to most directive per the coding protocol.

Percentage of 30-s intervals (n = 4,194) in which behavior was observed.

To what extent was there agreement between actual and reported use of coaching strategies?

Table 4 reflects the correlation between observed and reported use of coaching strategies. Given the initial professional development and ongoing support for use of these specific coaching strategies during home visits, we anticipated that there would be a moderate level of agreement between providers’ reported and actual practices. However, no significant relationship was detected for five of the seven strategies (Table 4). There was a weak, but significant, correlation between reported and observed use of Caregiver Practice with Feedback (r = .23, p < .05) and Direct Teaching (of the caregiver; r = .23, p < .05), indicating that as the frequency of use increased for each strategy so did its reported use. The reported use of Direct Teaching of the caregiver was weakly, though significantly, related to observed use of Demonstration/Modeling (r = .24, p < .05), Caregiver Practice with Feedback (r = .293, p < .01), and Guided Practice with Feedback (r = .237, p < .05), suggesting that providers used these three strategies when they reported doing Direct Teaching of the caregiver.

Table 4.

Summary of Correlations for Coaching Strategies Coded in Video and Reported in Case Notes

Strategy	1	2	3	4	5	6	7
1. CIS	.08	.07	−.03	−.28 **	.03	.00	.13
2. DT	.04	.23 *	.24 *	.293 **	−.19	.257 *	.20
3. DEM	−.01	.04	.00	.00	.13	−.19	−.07
4. CPF	−.20	−.06	−.03	.23 *	.16	.03	−.08
5. OB	.12	.15	−.01	.20	.02	−.08	−.04
6. GPF	.08	.03	.02	−.02	−.05	.07	−.13
7. PS	.16	.32 **	.013	.124	.003	−.051	.188

Abbreviations: CIS, conversation/information sharing; DT, direct teaching; DEM, modeling/demonstrating; CPF, caregiver practice with feedback; OB, observation/data collection; GPF, guided practice with feedback; PS, problem solving.

Note: Significant correlations are noted in bold type. Child-Focused Interaction and Joint Interaction are not reflected in this analysis because these behaviors were not included on the case note form.

p < .05. **p < .01.

Discussion

This study was undertaken to examine the implementation of coaching strategies used by six providers during early intervention home visits. As predicted, we found that providers in this case study used coaching strategies to a greater extent than had been reported previously in the literature. As a context for understanding the results, the analysis of fidelity of implementation data indicated that five of the six providers enacted most of the home visiting process steps on the Fidelity of Implementation Checklist. This suggests that initial and ongoing capacity-building efforts used by this Part C program were reasonably effective in producing some of the practices recommended in the field of early intervention (e.g., NECTAC, 2008). Our data did not, however, reveal the predicted level of agreement between actual and reported use of multiple coaching strategies. In this sample and setting, the level of agreement was less than anticipated, acknowledging what others have noted—that the shift from a child-focused intervention approach to the use of caregiver coaching strategies is difficult for providers to do and may require more intensive or different types of supports over a prolonged period of time relative to other types of programmatic changes (Campbell & Halbert, 2002).

Analyses of the videotape frequency data, at both the binary and more detailed interval level, indicated that these providers used a range of coaching strategies during their home visits. This finding is encouraging because it suggests that these providers used practices advocated by the field (e.g., Sandall et al., 2005) and did so in complex urban neighborhoods with families and children with diverse characteristics. However, in contrast to recent research (cf. Hagermoser-Sanetti et al., 2009), we found a consistent pattern of underreporting of specific coaching strategies on providers’ case notes. One interpretation is that these providers were not giving themselves “credit” for the range of practices they actually employed. It is not possible to determine from the data why the underreporting occurred. However, several possible explanations are plausible. It is possible that providers applied rigorous benchmarks to their use of coaching strategies such that they did not believe (record) that they had fully implemented the strategies during home visits (Armitage & Conner, 2001). It is equally possible that the strategy categories we adopted for coding may have presented pragmatically indiscernible gradations for providers to use in reporting what transpired during home visits (Friedman & Woods, 2010). That is, providers may have identified that they used Caregiver Practice with Feedback, which includes some element of observation, and not indicated they also used Observation as a strategy. Alternatively, they may have identified the use of Direct Teaching (DT) and omitted that they followed the DT episode with Demonstration. Evidence for this interpretation may lie in the rather even distribution of intervals related to coaching the caregiver to take the lead and implement intervention strategies (e.g., Observing, Caregiver Practice with Feedback, and Guided Practice with Feedback). It is likely that the coding protocol holds more value as a research tool than as a performance prompt for practitioners, and a more generic prompt would suffice as a mechanism for improving professional practice (Friedman, Woods, & Salisbury, 2010).

These providers’ actual practices represented, for the most part, strategies that were collaborative and family-centered. This finding is in contrast to prevailing data that suggest that home visits reflect child-focused, rather than collaborative, practices in Part C programs (Campbell & Sawyer, 2007; Hebbeler et al., 2008; Peterson et al., 2007). It is possible that the training, support, and supervision components of the CEIP approach contributed to this positive finding (Salisbury et al., 2010). Data from this study reflect both observed and perceived types of behavior related to the process of delivering collaboratively implemented home visits. Our findings suggest that despite professional development and support, these constructs and processes can be challenging for practitioners. Fidelity of implementation was acceptable for a majority of the providers, although not as high as the program’s administrators would have liked. Although it was possible to distinguish between high and low adopters, the extent to which implementation was affected by individual factors or by an interaction among provider, family, and contextual factors cannot be discerned from our data.

The fidelity measure helped identify areas for future professional development and pointed to the need to look at conceptual aggregates of several strategies. Combining the strategies of Direct Teaching (DT), Demonstration/Modeling (Dem), Guided Practice with Feedback (GPF), Caregiver Practice with Feedback (CPF), and Observing (OB) into a broader category of teaching revealed that more than 20% of home visit time was devoted to specific interaction between the provider and the caregiver–child dyad on embedded intervention within routines. Combined with Joint Interaction (JI) data, which is most easily described as practice time for the triad of participants, teaching episodes (broadly defined) would then reflect approximately 50% of home visit time. Precedent for a more global aggregate of strategies exists in recent research where coaching strategies were found to cluster into four major groupings (Observing/Information Sharing; Joint Interaction and Problem Solving; Practice with Feedback and Reflection; Direct Teaching and Guided Practice), suggesting that providers may think and act globally about their interactions with caregivers (Basu, Salisbury, & Thorkildsen, 2010). Practices such as problem solving that are linked to greater leadership by caregivers were not evidenced in either the global or specific measures. The limited use of problem solving by the providers could be related to the coding definitions, the function of the expert role assumed by the professional, limited time during the home visit, or the need for further professional development on strategies to engage family members in problem solving around their child’s intervention. These possibilities notwithstanding, findings from the current study suggest that these providers achieved a strong level of implementation that reflects many of the characteristics of quality early intervention programs.

Providers in this sample engaged with caregivers in ways that could be interpreted as supportive and collaborative. High rates of Conversation/Information Sharing may also be related to the isolated nature of these largely poor, urban households. For many of these families, the early interventionist represents someone they trust and who is willing to listen, as well as be someone they can learn from. The social nature of home visiting was extensively represented in our videotapes and we believe accurately reflects the supportive nature of interactions between these providers and their urban families. Service providers, as the most frequent and immediate visitor to the home, generally serve multiple functions beyond supporting caregiver-implemented intervention during home visiting. These functions include sharing information and resources, as well as discussion about services, the child’s progress, and family priorities. It would be inconsistent with the relationship-based aspect of family-centered practices for practitioners to only engage in caregiver and child interactions that relate to intervention. Conversations and information-sharing time are valuable and should be viewed as the foundation of the home visit. Unfortunately, at this time there are no guidelines for providers to help them determine how much or how little time spent in CIS builds a relationship that promotes a partnership with caregiver leadership.

Limitations

Several limitations in this study warrant specific mention. First, because data were collected to evaluate an aspect of a single program approach, broader generalization of the results to other Part C programs is not possible. Second, there may be a sampling bias related to the families who agreed to be videotaped and those who declined. Although 90 home visits were analyzed, there is still risk that the data are not representative of the larger pool of families receiving Part C services from this program. Third, although providers completed the check boxes of strategies used on the day of the home visit as a data-keeping and fidelity prompt, this process may have become unintentionally “rote” for them. The pattern of underreporting suggests that this might be the case. Fourth, as mentioned earlier in this report, comparing videotape and contact notes required the use of a binary analysis. Using this coding metric may have distorted the actual account of what happened during home visits. For this reason, frequency data (percentage of intervals) were also included in our analyses and we believe counters some of the limitations posed by the binary comparison. Finally, the sample size of only six providers is small and may not be representative of other providers in the field. With only six providers, we were unable to conduct any meaningful analysis at the individual provider level and hence have a limited ability to interpret the contributions of family, child, or provider variables to the patterns we observed. Despite this limitation, the providers and caregivers reflect cultural, ethnic, and linguistic diversity, adding an important dimension to our findings.

Implications for Research and Practice

With this sample of providers, we found limited agreement between the reported and actual use of coaching strategies. These providers reported using proximal coaching strategies (i.e., those where the flow of information came largely from the provider; e.g., Demonstration, Conversation/Information Sharing, Direct Teaching of caregiver) with greater accuracy than they did more distal coaching strategies that reflect behaviors designed to more intentionally turn leadership or control of the session over to the caregiver (e.g., Guided Practice with Feedback, Observing, Problem Solving). Given this pattern, it may be the case that narrowly defining coaching practices renders it more difficult or less meaningful for providers when completing their case notes. Future research is needed to evaluate what level of specificity is necessary for practitioners to report accurately what they know and do during home visits. Broader categories, such as Direct Teaching/Demonstration or Practice with Feedback, may be more useful to providers working in local Part C programs and may offer a preliminary framework for evaluating professional practice.

This case study’s data provide initial support for the use of contact notes as an index of fidelity. Contact notes may represent a cost-effective and informative method for evaluating the quality of home visits and the coaching/consultation strategies used by providers. Additional and larger studies are needed to examine implementation and reporting practices with diverse samples of providers. Recognizing that beliefs affect action, additional research is also needed to understand how best to address attitudinal barriers that affect provider willingness to implement coaching strategies. Although our augmented case note format appears to have bridged some of the do–say gap, it was not equally effective across all strategies. There was likely an interaction among provider beliefs, their actions within the home visit, and what they reported. Our case note prompts did not directly address the cognitive/attitudinal factors that can impact implementation. It may be that a think–do–say professional development approach would more effectively address implementation issues and reinforce the reflective component of the CEIP approach. Although both theoretical and empirical work exists that affirms the value of problem solving as a means of improving professional practice (Buysse & Wesley, 2005), this study found that practitioners reported and used problem solving on a very limited basis. Additional research is needed to explore how best to define the concept and teach the process to practitioners so that they can apply it as a support strategy when working with caregivers. With providers as partners in our evaluation and program improvement efforts, we envision changes to our contact note format and a renewed emphasis on the use of reflection for both supervision and collaborative consultation.

This study represents an initial effort toward understanding and addressing the disjuncture between expected and actual early intervention practices in natural environments. Insight about the relative difficulty of enacting and reporting coaching strategies can offer important information for professional development activities. To understand where the sources of difficulty lie and how best to approach their remediation, we believe it is essential to meaningfully involve providers as our partners in program improvement and research efforts. Adopting methods of inquiry that are more participatory and authentic may help address basic questions surrounding what works, what does not, and why.

Footnotes

Gratitude is expressed to the staff and families at the UIC Child & Family Development Center for their cooperation and support in this project. Special appreciation is extended to Courtney Golant for her assistance with data analysis.

Declaration of Conflicting Interests

The author(s) declared no conflicts of interests with respect to the authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This study as supported, in part, by a grant from the U.S. Department of Education, Office of Special Education Programs (H324M030128-05) to the University of Illinois–Chicago, Christine Salisbury, Principal Investigator. The opinions expressed are not those of the U.S. Department of Education and no official endorsement should be inferred.

References

Armitage

C. J.

Conner

(2001). Efficacy of the theory of planned behavior: A meta-analytic review. British Journal of Social Psychology, 40, 471–499.

Azjen

(1991). The theory of planned behavior. Organizational Behavior and Human Decision Process, 50, 179–211.

Basu

Salisbury

Thorkildsen

T. A.

(2010). Measuring collaborative consultation practices in natural environments. Journal of Early Intervention, 32, 127–150.

Brady

Peters

Gamel-McCormick

Venuto

(2004). Types and patterns of professional-family talk in home-based early intervention. Journal of Early Intervention, 26, 146–159.

Brorson

(2005). The culture of a home visit in early intervention. Journal of Early Childhood Research, 3, 51–76.

Brown

Palincsar

(1989). Guided, cooperative learning and individual knowledge acquisition. In Resnick

L. B.

(Ed.), Knowing, learning and instruction: Essays in honor of Robert Glaser (pp. 393–451). Hillsdale, NJ: Lawrence Erlbaum.

Bruder

(2000). Family-centered early intervention: Clarifying our values for the new millennium. Topics in Early Childhood Special Education, 20, 105.

Buysse

Wesley

P. W.

(2005). Consultation in early childhood settings. Baltimore, MD: Brookes.

Campbell

P. H.

Halbert

(2002). Between research and practice: Provider perspectives on early intervention. Topics in Early Childhood Special Education, 22, 213–226.

10.

Campbell

P. H.

Sawyer

Muhlenhaupt

(2009). The meaning of natural environments for parents and professionals. Infants and Young Children, 22, 264–278.

11.

Campbell

P. H.

Sawyer

L. B.

(2007). Supporting learning opportunities in natural settings through participation-based services. Journal of Early Intervention, 29, 287–305.

12.

Crais

E. R.

Roy

V. P.

Free

(2006). Parents’ and professionals’ perceptions of the implementation of family-centered practices in child assessments. American Journal of Speech-Language Pathology, 15, 365–377.

13.

Dunst

C. J.

Trivette

C. M.

(2009a). Using research evidence to inform and evaluate early childhood intervention practices. Topics in Early Childhood Special Education, 29, 40–52.

14.

Dunst

C. J.

Trivette

C. M.

(2009b). Let’s be PALS: An evidence-based approach to professional development. Infants & Young Children, 22, 164–176.

15.

Friedman

(2009). Caregiver coaching strategies in routines-based early intervention (Unpublished master’s thesis). Tallahassee: Florida State University.

16.

Friedman

Woods

(2010). Use of caregiver coaching strategies in routines-based intervention. Manuscript in preparation.

17.

Friedman

Woods

Salisbury

C. L.

Development and utilization of specific coaching strategies to facilitate parent implemented intervention. Manuscript in preparation.

18.

Hagermoser-Sanetti

Chafouleas

S. M.

Christ

T. J.

Gritter

K. L.

(2009). Extending use of direct behavior rating beyond student assessment. Assessment for Effective Intervention Online First, published April 8, 2009. Available from http://aci.sagepub.com

19.

Hall

Hord

S. M.

(2001). Implementing change: Patterns, principles, and potholes. Needham Heights, MA: Allyn & Bacon.

20.

Hanft

B. E.

Rush

D. D.

Shelden

M. L.

(2004). Coaching families and colleagues in early childhood. Baltimore, MD: Brookes.

21.

Hebbeler

Spiker

Morrison

Mallik

(2008). A national look at the characteristics of Part C early intervention services. In Peterson

Fox

Blasco

P. M.

(Eds.), Early intervention for infants and toddlers and their families: Practices and outcomes (pp. 1–18). Young Exceptional Children Monograph (10). Missoula, MT: Division for Early Childhood.

22.

Isreal

A. C.

(1978). Some thoughts on correspondence between saying and doing. Journal of Applied Behavior Analysis, 11, 271–276.

23.

Karlan

G. R.

Rusch

F. R.

(1982). Correspondence between saying and doing: Some thoughts on defining correspondence and future directions for application. Journal of Applied Behavior Analysis, 15, 151–162.

24.

McCollum

J. A.

Yates

T. J.

(1994). Dyad as focus, triad as means: A family-centered approach to supporting parent-child interactions. Infants & Young Children, 6, 54–63.

25.

McLaughlin

M. W.

(1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9, 171–178.

26.

National Early Childhood Technical Assistance Center. (2008). Workgroup on Principles and Practices in Natural Environments (2008, February). Agreed upon practices for providing early intervention services in natural environments. OSEP TA Community of Practice-Part C Settings. Available from http://www.nectac.org

27.

Odom

S. L.

(2009). The tie that binds: Evidence based practice, implementation science, and outcomes for children. Topics in Early Childhood Special Education, 29, 53–61.

28.

Peterson

C. A.

Luze

G. J.

Eshbaugh

E. M.

Jeon

Kantz

K. R.

(2007). Enhancing parent-child interactions through home visiting: Promising practice or unfulfilled promise? Journal of Early Intervention, 29, 119–140.

29.

Rogoff

(1990). Apprenticeship in thinking: Cognitive development in social context. New York, NY: Oxford University Press.

30.

Salisbury

Cambray-Engstrom

Woods

Friedman

(2008). Routine and Instructional Strategy Coding Protocol-IL. Unpublished manuscript. Child & Family Development Center, University of Illinois–Chicago.

31.

Salisbury

Woods

Copeland

(2010). Provider perspectives on adopting and using collaborative consultation in natural environments. Topics in Early Childhood Special Education, 30, 132–147.

32.

Sandall

Hemmeter

M. L.

Smith

McLean

(2005). DEC recommended practices: A comprehensive guide for practical application in early intervention/early childhood special education. Longmont, CO: Sopris West.

33.

Vygotsky

L. S.

(1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

34.

Woods

(2005). Family-guided, routines-based intervention project. Department of Communication Disorders, Florida State University, Tallahassee.

35.

Woods

Lindeman

D. P.

(2008). Gathering and giving information with families. Infants & Young Children, 21, 272–284.