Abstract
Various aspects of parent responsiveness are associated with child outcomes, such as play, language, and social development. However, behavioral coding methods used to measure parent responsiveness vary widely, making comparison of results across studies difficult. The purpose of this scoping review was to summarize current behavioral coding methods used in measuring parent responsiveness to children with autism or elevated likelihood of autism, synthesize the reported metrics used, and highlight the strengths and weaknesses in the reporting standards of available literature. A total of 101 articles met criteria for the review and were analyzed for metrics in demographics, coding system development and accessibility, characteristics of measured responsiveness, reliability, and validity. Results revealed variations in observational procedures, forms of measurement, and specific aspects of responsiveness measured. Details necessary for study replication or extension often were missing, such as parent demographics, clear definitions of parent responsiveness, and coder training procedures. The scoping review results reflect the wide variety of behavioral coding systems used and the inconsistent reporting in published literature on this topic. A case for a best practice model for behavioral coding metrics and reporting standards within parent responsiveness is presented in the discussion.
Lay abstract
The topic of how parents react (e.g., how they talk and act) to their child with autism or elevated likelihood of autism, often called parent responsiveness, has been studied by researchers for over 50 years. Many methods for measuring behaviors around parent responsiveness have been created depending on what researchers were interested in discovering. For example, some include only the behaviors that the parent does/says in reacting to something the child does/says. Other systems look at all behaviors in a period of time between child and parent (e.g., who talked/acted first, how much the child or parent said/did). The purpose of this article was to provide a summary of how and what researchers looked at around parent responsiveness, describe the strengths and barriers of these approaches, and suggest a “best practices” method of looking at parent responsiveness. The model suggested could make it more possible to look across studies to compare study methods and results. The model could be used in the future by researchers, clinicians, and policymakers to provide more effective services to children and their families.
Parent–child engagement is a complex interactive process, which makes measuring the qualities present in the dyadic interaction difficult to achieve (Fogel, 2009). A common method used to address this complexity is direct observation using behavioral coding measures (Aspland & Gardner, 2003). Behavioral coding measures involve the use of a coding system to identify specific behaviors based on the construct being measured (Bakeman & Gottman, 1997). Various coding tools have been developed to measure a variety of constructs within parent–child engagement, such as joint engagement (Adamson et al., 2014), parent responsiveness (Tamis-LeMonda et al., 2001), and infant responsiveness (Van Egeren et al., 2001). The primary focus of this scoping review is parent responsiveness.
Many derivatives of parent responsive constructs are found in literature, such as synchrony (Feldman, 2007), direct responsiveness (Landry et al., 2006; Mahoney & Perales, 2003), and mutuality (Deater-Deckerd & O’Connor, 2000). Similarly, varied treatment constructs, such as parent coaching (Kasari et al., 2014; Siller et al., 2018), family behavior management (McConachie et al., 2005; Poslawsky et al., 2015), and parent-mediated interventions (Shire et al., 2016; Siller et al., 2013; Watson et al., 2017), are also identified in parent responsiveness literature. Here, we define parent responsiveness as the act of a parent responding to a child’s behavior (e.g., maternal verbal responsiveness or turning to look at child) or the act of a parent’s overall behavior being attuned to their child in some aspect (e.g., parent’s responsiveness is attuned to their child’s overall developmental level; Mahoney & Perales, 2003). Examples of responsive behavior include follow-in directives or comments made by caregivers where their response corresponds to the child’s focus of attention. Follow-in directives refer to caregiver responses where there is the expectation of a response from the child, whereas follow-in comments refer to caregiver responses that describe the child’s focus of attention without an expectation for a child response (Haebig et al., 2013b).
Parent responsiveness has been shown to influence children’s development, suggesting the importance of fully understanding and appropriately measuring the construct. For example, parents’ responses to a child’s attentional focus and communication acts have been shown to be related to communication and language development of young children without clinical concerns. Adamson and Bakeman (1984) demonstrated that information presented in a responsive environment resulted in increased comprehension and vocabulary development for the children involved. References made within episodes where parents are attuned and responding to the child’s focus are shown to correlate with vocabulary growth compared to information presented outside of the shared focus (Tomasello & Farrar, 1986). Dunham et al. (1993) reported higher comprehension of novel words when parents followed the attentional focus of the child.
The importance of parent responsiveness is also observed in children with autism. More frequent parent responsiveness, both to child’s attentional focus and communication acts, is related to increased spoken vocabulary in this population (McDuffie & Yoder, 2010; Siller & Sigman, 2008), with both fathers and mothers (Flippin & Watson, 2015). The use of both follow-in directives and comments is shown to predict later comprehension and production abilities of children with autism (Haebig et al., 2013a, 2013b). In addition, Flippin and Watson (2011) noted higher parent responsiveness with more complex play.
Although parent responsiveness is found to be important for social-communicative development, children with autism may be delayed in developing skills that help to elicit parent responsiveness. For instance, a key area of difficulty for children with autism is joint attention (Adamson et al., 2009; Watson et al., 2013), which allows the child to indicate their focus of attention to the parent. Limited joint attention behaviors may make it more difficult for parents to recognize the child’s focus and respond accordingly (Kinard et al., 2017). Several interventions targeting responsiveness of parents of children with autism, or parents of infants and toddlers at elevated likelihood of an autism diagnosis, have shown efficacy in increasing parent responsiveness (Green et al., 2010; Ingersoll & Wainer, 2013; Siller et al., 2013; Venker et al., 2012; Watson et al., 2017). Examples of these interventions include Focused Playtime Intervention (Siller et al., 2013), Responsive Teaching (Mahoney & Perales, 2003), Adapted Responsive Teaching (Watson et al., 2017), and Project ImPACT (Ingersoll & Wainer, 2013).
Forms of measurement
Coding systems can generally be designed as discrete, global, or both (Aspland & Gardner, 2003; Bell & Bell, 1989; Gardner, 2000; Lotzin et al., 2015), where each quantifies different levels of information about an individual in the dyad and/or the dyad as a whole (Mesman, 2010). Measures of discrete variables are used to assess distinct behaviors present in the interaction (Gardner, 2000; Lotzin et al., 2015) and could include characteristics, such as eye contact (Vernon, 2014), physical touch (Chorney et al., 2010), and verbal responsiveness (Leezenbaum et al., 2014). These behaviors can be identified continuously (e.g., indicating each time a particular behavior occurs) or within intervals (e.g., coding whether the behavior of interest occurred at any time within each identified interval of an observation; Aspland & Gardner, 2003; Bell & Bell, 1989). Examples of discrete measures include the Dyadic Parent–Child Interaction Coding System (DPICS-III; Eyberg et al., 2005), Free Play Responsiveness Coding (Kinard et al., 2017), Infant and Caregiver Behavior Scale (ICBS; Muratori et al., 2011), and the Transactional Engagement Coding (Uzonyi et al., 2021).
Global measures are used to assess the general presence or absence of behavioral concepts or constructs in the observation as a whole (Bell & Bell, 1989; Lotzin et al., 2015). Global measures relating to parent–child engagement can be coded from the coordination of the behaviors between a parent and infant, such as synchrony (de Mendonça et al., 2011) and mutuality (Aksan et al., 2006), or the parent’s behavior relative to a construct, such as emotional availability (Biringen & Easterbrooks, 2012). Typically, an overall score, often reported on a rating scale, is provided that reflects the presence or quality of the behavior or construct measured throughout the total observed engagement time (Bell & Bell, 1989). These scales can be rated for each partner or the dyad as a unit. Examples of global measures include the Measure of Active Engagement and Transactional Support (MAETS; Wetherby et al., 2013), the Joint Engagement Rating Inventory (JERI; Adamson et al., 2016), and the Mutually Responsive Orientation scale (MRO; Aksan et al., 2006).
Reliability and validity
As for all measurement, the integrity of both discrete and global measures hinges on good reliability (i.e., behaviors measured the same way each time) and validity (i.e., the coding system measures what it claims to measure) to permit confidence in conclusions drawn from the data (Aspland & Gardner, 2003; Bell & Bell, 1989). We will first discuss reliability. Assessing reliability of a measurement system is important to identify the indicators of human error or bias in coding the behaviors present in the interaction (Aspland & Gardner, 2003). In measurement of parent responsiveness, the assessment of inter-observer reliability is of particular importance because it reveals the degree to which different observers’ ratings of parent behavior align (Aspland & Gardner, 2003; Bell & Bell, 1989). If different observers rate the same parent behavior in different ways, the trustworthiness of the data is compromised. Risks associated with low inter-observer reliability may be avoided by having clear operational definitions, thorough coding training, inter-observer reliability checks early in the coding process, and regular checks throughout the process. Furthermore, regular checks can prevent observer drift from the original coding system (Aspland & Gardner, 2003). Depending on the nature of the data (e.g., continuous vs categorical ratings), inter-observer reliability may be calculated using one of the several different mathematical procedures, such as intraclass correlation (ICC), Cohen’s kappa, or percent agreement (Aspland & Gardner, 2003; Bell & Bell, 1989). Additional reliability assessment may focus on internal consistency, test–retest, split half, and so on.
In summary, reliability assessment ensures that the codes generated are consistent across observers and over time. In addition to reliability, examination of the validity of a measure, including a behavioral coding system, is important to ensure that it adequately quantifies the target constructs (Bell & Bell, 1989). Specifically, content validity for coding measures is the relevance and representativeness of the examples and definitions to the stated object of measurement, whereas construct validity is the degree to which a measurement system yields variables that perform as expected by theory and logic (Yoder et al., 2018).
Inconsistencies in design and reporting standards
The parent responsiveness literature has wide inconsistencies in design and reporting standards within each type of responsiveness construct. For example, both Crandall et al. (2019) and McDuffie and Yoder (2010) examined parent verbal responsiveness in relation to later language development in children with autism. One study reported the specifics of the development of the behavioral coding system, detailed steps of the coding process, and provided charts of coding definitions (Crandall et al., 2019), while the other study provided a cursory overview of the coding system without further details. In addition, the length of coded behavior time varied between these two studies and the inter-observer reliability was reported in different statistical metrics. When parent responsiveness is used as an intervention mediation variable or intervention outcome, but full details of the coding system across studies are not provided, interpreting findings in the context of the larger literature is challenging. This is particularly the case when findings across similar studies appear to conflict with one another. In addition, providing limited details about the coding system hinders the ability of researchers to conduct replication studies to validate results.
Currently, there are no standards for conducting and reporting behavioral coding for studies investigating parent responsiveness. While seminal works, such as those by Bakeman and Gottman (1997) and Bell and Bell (1989), have described the intricacies of behavioral coding methods, neither addressed reporting standards that would support consistency in parent responsiveness measurement design and use. More recently, Yoder et al. (2018) highlighted the importance of designing or adapting coding manuals, selecting elements of the coding system, choosing methods of behavioral sampling, and reporting critical aspects, such as coder training, reliability, and validity. However, no reporting standards currently exist to ensure that important aspects of parent responsiveness coding methods, such as the development and availability of the coding system, and training methods for coders, among others, are communicated beyond the research team that completed the study.
Objectives
Scoping reviews are implemented to synthesize evidence in an area and determine the scope of literature on a topic (Tricco et al., 2018). A scoping review was implemented for this project to synthesize the availability of evidence on various behavior coding systems used to measure parent responsiveness in families with children with autism. Given the inconsistencies in parent responsiveness metrics and reporting standards in the extant literature, this scoping review aimed to (a) summarize current behavioral coding methods used in measuring parent responsiveness to children with autism or elevated likelihood of autism, (b) synthesize the reported metrics, (c) highlight strengths and gaps in the reporting standards in the available literature, and (d) make a case for a best practice model for metrics and reporting standards.
Methods
Scoping review
The current study followed guidelines outlined by PRISMA extension for scoping reviews (Tricco et al., 2018); however, the review could not be registered with PROSPERO because this entity does not currently accept scoping reviews, literature reviews, or mapping reviews. The PRISMA checklist (Tricco et al., 2018), detailing the information related to each aspect of the guidelines, is provided in Supplementary Material 1.
Identification of evidence
To be included in the scoping review, studies were required to include a definition of parent responsiveness. Responsive behavior of the parent was measured either in a specific area (e.g., verbal responsiveness) or through broader classifications of parent behaviors (e.g., sensitivity), as long as at least one observable responsive characteristic was identified in the broad classification system. For example, a parent sensitivity behavior might be measured when a parent makes a statement, gesture, or action in response to the child. In addition, studies had to include children with autism or elevated likelihood of autism; child mean age of 5 years or younger; observational measures that were global, discrete, continuous, interval, or some combination; and observational settings that were naturalistic or a laboratory environment. There was no limit on the date range of the articles, meaning the articles were searched from the inception of databases (e.g., first allowable) through December 2019, when the search was completed. Only studies originally written in English or translated into English were included. Both published and unpublished works were allowable. Both intervention and observational studies were included, and there was no limitation on study design within these categories. Articles with varying reporting of reliability and validity were also allowed.
The exclusion criteria included studies that focused on parent reports rather than direct observation; qualitative studies; case studies with fewer than 10 participants; studies focused only on coding child behaviors; and previous review studies. In addition, studies were excluded that coded behaviors based on transcription of language during parent–child engagement. Studies that did not focus on parent responsiveness or where the focus on other constructs impeded measurement of responsiveness were excluded. Studies focusing on behavior modification in a school-based, instructional, or applied behavior construct were not included.
A trained clinical health sciences librarian (S.T.W.) performed the comprehensive electronic search of publications using the following databases: PubMed, Cumulative Index to Nursing and Allied Health Literature via EBSCO, EMBASE.com, Web of Science Core Collection, Academic Search Premier via EBSCO, Education Full Text via EBSCO, Education Resources Information Center (ERIC) via EBSCO, and PsycInfo via EBSCO. Search terms were used to retrieve articles addressing the three main concepts of the search strategy: (1) autism; (2) parenting; and (3) responsiveness. The exact search strategy used in each of the electronic databases is reported in Supplementary Material 2. The search strategy was created for each database using multiple keyword terms and subject headings, if applicable (such as MeSH for PubMed). Results were downloaded to EndNote and duplicates were removed. All references were uploaded to Covidence Systematic Review software (https://www.covidence.org), a web-based tool designed to facilitate and track each step of the abstraction and review process (see Figure 1).

PRISMA flow diagram.
Three of the authors (T.E.U., H.L., and A.C.G.) screened articles to be included in the review. Prior to full screening implementation, all three reviewers underwent two to three sessions of training during which they reviewed articles together and discussed their criteria eligibility until agreement was 100% on preselected articles. Once all three reviewers completed training, full screening began for the study. Two reviewers (T.E.U. and H.L.) each screened all identified publications at the title and abstract level based on the inclusion and exclusion criteria. To maintain consistency among reviewers, disagreements were negotiated to reach consensus on 100% of articles. Two reviewers (T.E.U. and A.C.G.) independently assessed each remaining article at the full-text level to determine eligibility. Again, disagreements were discussed until a consensus was met for 100% agreement on all articles. The final list consisted of 101 studies.
Data extraction
A data extraction template was created in Covidence (2019) and updated by two reviewers (T.E.U. and A.C.G.). Data were extracted for demographics (e.g., age of children, age of caregivers, parental education, family income, percent male children, and diagnosis categories), coding measures (e.g., name of coding system used, observational setting, observation length, and aspects of responsiveness measured), and reliability and validity measures. Additional measures were included as quality indicators of the evidence including information on the blinding of coders and incomplete video data. The raw data extracted from the studies are included as a Supplementary Material 4. Twenty percent of the articles were coded by both reviewers to assess the reliability of data extraction. The proportion agreement from the inter-rater reliability system through Covidence was 0.79. However, Covidence marked differences in extraction that were actually the same content (e.g., Maternal Behavior Rating Scale (MBRS)). Therefore, the inter-rater reliability was likely higher than 0.79. Any disagreements were discussed between T.E.U. and A.C.G. until a consensus was reached. Data were extracted for 101 studies, and data charting was checked for missing data.
No autistic individuals or community members were involved in developing the research question, study design, measures, implementation, or interpretation and dissemination of the findings. However, one author is the parent of a youth with intellectual disabilities.
Results
In total, 11,814 articles were identified through database searches. After duplicates were removed, there were 5,117 remaining articles to be screened at the title and abstract level. Of those, 4,941 articles were excluded based on the inclusion and exclusion criteria. The remaining 176 articles were assessed at the full-text level. Following review, 101 articles were included in the final synthesis. Figure 1 outlines the PRISMA flowchart of the search strategy, screening, and full-text review process. Further information regarding the extracted data is provided in each of the broad categories investigated.
Demographic information
The average age of the children included in the study was 38.11 months (range 3–65 months). Six studies did not report the average age of the participants and one study included observations from recordings taken from within the first 18 months of life. It is important to note that some studies included a follow-up at later ages, but the ages extracted for the scoping review were at baseline. The mean age of the parents was reported in 43 studies (Mage = 35.49 years). An additional three studies reported separate averages for both mothers and fathers.
In 66 of the studies (65.3%), the majority of the parents were females. A handful of studies (10) included both male and female caregivers. The remaining studies did not report sex of the parent who participated in the observation. Most (94) reported sex of the children, which showed on average 74.6% children, were male. The proportion of minority children was reported in 53 studies, which indicated the average of 36% minority children included in the 101 studies. Many studies (42) included children with an autism or pervasive developmental disorder (PDD) diagnosis. However, 17 studies included children with autism or PDD and typically developing (TD) children. Several included combinations of children who had autism or PDD, were at elevated likelihood for being diagnosed with autism, and TD children. An additional 23 studies included a combination of other disorders or risk statuses.
The majority of studies (73) did not report the income level of the families. For the studies that included income level, the scoping review team made classifications of socioeconomic status (SES) based on categories from Pew Research Center (2018) where lower-class families made less than US$48,500, middle class made US$48,500–US$145,500, and upper class made greater than US$145,500. Of those reported, the SES classification included 12 studies with mostly middle/upper-class families, 9 studies with mostly middle class, 6 studies with mostly lower/middle class, and one study with mostly lower class. Studies were also classified by parent education status. Most studies had parents who earned degrees beyond high school (78). Three studies mainly had parents who did not complete high school. The remaining 20 studies did not report the parent education status. The demographic information for each study is included in Supplementary Material 3.
Coding systems
A detailed summary of the coding systems and data extraction across articles can be found in Tables 1 to 3. The majority of studies specifically reported the name of the coding system (68%). Similarly, most coding systems reported had been previously used in studies of parent responsiveness, but a small percentage (22%) adapted existing coding systems. The remaining studies (10) did not report whether the coding system employed was original or modified. However, 52% of studies used discrete coding methods and 42% used global coding methods. The remainder used a combination of discrete and global coding methods. Overall, 41% used rating systems, 27% employed continuous coding (i.e., recording every behavior observed during the length of the observation), and 18% used interval coding (i.e., observing if a behavior did or did not occur within a specified time period). Meanwhile, 15% of studies had varied sampling methods, which were combined interval/continuous, interval/rating, rate, and frequency counts. There was variability in the means in which coding systems could be accessed in the reviewed publications. Slightly more than half of the coding systems (54%) were available in previously published material, with 3% in supplemental materials, 6% accessed by contacting the author, and 37% of studies did not report where the system could be accessed. Very few studies reported if the coding system used in their study had been used in non-Western countries (10%).
Titled coding systems and responsiveness measures.
MBRS (Mahoney et al., 1986); PICCOLO (Roggman et al., 2013); DPIC (Eyberg, 2013); DCMA (Aldred et al., 2004; Green et al., 2010); EAS (Biringen et al., 2000); MEHRIT: Milton & Ethel Harris Research Initiative Treatment.
Previously addressed coding system.
Untitled coding systems and responsiveness measures.*
References for studies with adapted systems are included in Supplemental Material.
NR: not reported.
Coding data summary.
The observational setting of the recorded parent–child engagement was primarily in a laboratory setting (72%) or both laboratory and naturalistic setting (69%). Just 20% of recordings occurred in naturalistic settings only. Most studies (70%) used unstructured observational methods where specific instructions were not given to guide how parents behaved while engaging their child. However, 22% of studies provided parents with specific instructions for play interactions, and 8% used a combination of both (i.e., unstructured and structured play). The majority of studies used a standard set of toys (72%); 13% did not use standard toys; and the remaining 15% did not report this information.
The length of coded parent–child engagement samples ranged from 32 s to 51 min. Many studies coded video samples having a duration of 5–10 min (59%), followed by 11–15 min (17%) and 16–20 min (8%). However, 3% did not report a recorded time or reported times varied. In a handful of studies, duration of video recordings exceeded the duration of the segment coded (e.g., recorded for 20 min but the first 5 min were excluded, and the remaining 15 min were coded). Only 6% of studies reported the length of time to complete coding, which ranged from 4 min per observation to 4 months to complete coding for an entire observational dataset. The remaining studies did not report either the length of time required for observers to code behavior within the study-defined segment for a single parent–child engagement, or the time required to code an entire dataset.
Parent responsiveness
Most studies (85%) reported parent responsiveness as the primary aim of the project. When cited as a secondary aim, parent responsiveness was often a non-primary outcome variable in an intervention study. Over half (65%) of the studies measured parent responsiveness as part of an intervention outcome.
Within coding systems, parent responsiveness behaviors were 96% proximal (i.e., very closely related) and 4% distal (i.e., distantly related) to the purpose of the coding system. An example of a proximal relation was identified when parent verbal responses were measured with a verbal responsiveness coding system. An example of a distal relation was identified when parent smiling was identified with a secure attachment coding system. Across studies, the data extraction for specific aspects of parent responsiveness in the 101 articles revealed such a wide range that further clarity was needed to conceptualize the measured aspects in a more concrete manner. Therefore, measured aspects of parent responsiveness were classified into one of the three categories developed by the first three authors: (1) type of parent behavior in responsiveness (e.g., reaction to child vs overall parent behavior), (2) type of guidance in responsiveness (e.g., parents coached in responsiveness at any point of the study vs. uncoached natural responses during observation), and (3) responses associated with behavior management of children (e.g., positive, negative, or neutral). Most studies fell in the category of measuring the parent’s reaction to the child (83%). Many of the studies observed naturally occurring responses to the child (66%). In the remaining 34%, responsiveness was measured in parents who had been coached in responsive behaviors at some point during a study. The majority (79%) of studies observed responsiveness that was neutral, where the child’s behavior was not managed by parent responsiveness. The remaining focused on behavior management that involved a combination of positive–negative responses, neutral–positive responses, or negative–positive–neutral responses to manage a child’s behavior. Overall, across the three categories of responsiveness classification, parent responsiveness was most commonly conceptualized by verbal and non-verbal acts within play or caregiving circumstances.
Reliability and validity
Overall, 85% of articles reported information around the percentage of videos coded for inter-observer reliability. The most common percentage of videos coded was 20%–25%. However, 68% of the studies reporting number of videos used indicated that inter-observer reliability was assessed using 30% or fewer video recordings; only 4% of the studies used greater than 50% of the video recordings. A few studies reported an unconventional approach, such as using the first 3 min of all videos to assess reliability.
All studies were examined to determine the type of training coders received to perform reliability trials. Only 41% of studies reported information on how coders were trained, with a wide range in what was reported. For example, some studies provided a very detailed training protocol (Apicella et al., 2013; Zlomke et al., 2019) where the educational background of coders, number of training hours received, pre-established criteria for reliability, and criterion videos used for training were described. Other studies reported cursory details, such as “trained to 90% reliability” (Hirschler-Guttenberg, Feldman, et al., 2015). The remaining studies either did not provide any training details or mentioned “trained coders,” “certified coders,” or “acceptable level of agreement” with no indication of how coders were trained, certified, or what level of agreement was deemed acceptable.
The most common reliability statistics used across studies were ICC coefficient and Kappa. Other reported reliability statistics included inter-rater agreement, average agreement, Pearson’s correlation coefficient, multiple agreement, and percentage of exact agreement.
A low number of articles (16%) reported validity information. In total, 14 studies acknowledged validity in the form of measuring variables that perform as expected by theory and logic (Yoder et al., 2018) from previous literature (e.g., dyadic literature, social communication literature), or through statistical means, such as factor analysis (Mahoney et al., 1986) or bivariate correlations (Eyberg et al., 2005). Two articles acknowledged using measures that had not previously been validated and therefore validated the measures in the context of the study. Pertaining to reducing risk of measurement bias, nearly 60% of the studies indicated their coders were blind to the study purpose. A summary of reliability and validity is provided in Table 4.
Reliability and validity summary.
Discussion
This study focused on parent responsiveness and the many ways it has been assessed through behavioral coding measures. The aims of this scoping review were to (a) summarize current behavioral coding methods used in measuring parent responsiveness to children with autism or elevated likelihood of autism, (b) synthesize the reported metrics used, (c) highlight strengths and gaps in the reporting standards of behavioral coding, and (d) propose a best practice model for metrics and reporting standards.
Summary of behavioral coding methods and metrics
Our review showed that current behavioral coding methods in the 101 articles included global, discrete, and a combination of measures. Many of the studies were based on unstructured interactions between parent and child, with varying time duration and play contexts. Across the coding systems, frequently measured constructs included types of engagement, non-verbal versus verbal responses, and parental responses to the child or their behaviors independent of the child. The same constructs were incorporated into interventions designed to support parent responsive behavior with children with or at elevated likelihood of autism (Green et al., 2010; Ingersoll & Wainer, 2013; Watson et al., 2017).
The most common reliability metric reported was inter-rater reliability of coding, primarily analyzed using ICC and Kappa values. Validity metrics were reported infrequently. When validity was addressed, it was mostly by citing former literature to support construct validity. Approximately half of the studies used titled coding systems while the half using untitled systems adapted their coding systems from available literature. The reported literature referenced for adapted coding systems dated as far back as 1988, but most studies used literature within the past 20 years. Among the coding systems listed in Table 1, those used most frequently were as follows: MBRS (Mahoney et al., 1986); PICCOLO (Roggman et al., 2013); DPIC (Eyberg, 2013); DCMA (Aldred et al., 2004; Green et al., 2010); and EAS (Biringen et al., 2000).
Strengths and gaps in parent responsiveness coding in literature
The literature on behavior coding of parent responsiveness to young children diagnosed with or at elevated likelihood for autism featured several strengths. First, all studies reported at least some type of demographic information, particularly child characteristics. Second, inter-observer reliability statistics were reported, with many studies providing some level of basic details regarding how reliability standards were achieved between coders. Given that coding of parent responsiveness requires some subjective judgments in applying definitions to observed interactions (Aspland & Gardner, 2003; Bell & Bell, 1989; Yoder et al., 2018), reporting inter-observer reliability is necessary for determining if observational data accurately represent the behaviors that occurred. In addition, reporting the procedures used in a study to achieve reliability is critical for replication in implementing the coding system by other research teams.
Another relative strength is that studies explored many components of responsivity within the context of parent–child interactions; thus, the literature offers a rich consideration of parent responsivity as a construct (Green et al., 2015; Harrop et al., 2016; Parladé et al., 2020). In addition, there are numerous tools for capturing different components of responsivity (e.g., Manchester Assessment of Caregiver–Infant Child Interaction, Caregiver–Child Interaction, and Dyadic Parent–Child Interaction Coding System-IV, respectively), providing future researchers with a wealth of options to consider as they look for tools that will best support their specific research objectives.
Despite these strengths, noticeable gaps were identified in the parent responsiveness literature focused on autism. For example, relatively few studies reported demographic information for parents. With the complexities in examining the dyadic interaction within parent–child engagement (Fogel, 2009), missing parent demographics, such as age, sex, income, and education, reduces insights into potential mediators or moderators of parent responsiveness, especially within intervention studies that aim to increase parent responsiveness. Likewise, the frequent omission of specific details on how coders were trained impedes the evaluation of data fidelity and the ability of other researchers to use similar training protocols.
Another considerable gap is that validity (content and construct) was rarely addressed in the studies. Thus, the results of most of these studies cannot be fully interpreted in relation to how accurately parent responsiveness as coded represents the intended construct or dimension of parent responsiveness. This gap in turn limits our ability to interpret consistent and inconsistent findings across studies and to advance our theories regarding the role(s) of parent responsiveness in the development of children with autism.
Missing names of coding systems (mostly discrete measures) represent another gap in the existing literature. Current best practice recommendations for behavioral coding suggest that in the process of study development, well-vetted existing coding systems should be used when appropriate to the study aims, or a coding system should be adapted from an existing such system if needed to align more closely with the study aims (Yoder et al., 2018). Thus, having a named coding system, or an easily identified modification of a coding system, would improve researchers’ ability to locate existing coding systems that may measure aspects of parent responsiveness relevant to their own study aims. Furthermore, the previous studies that used those coding systems may include information on the reliability and validity of the coding system.
Although we acknowledged as a relative strength that the studies included in this review explored many components of parent responsiveness, this aspect of the literature also presented a logistical hurdle in the scoping review that will impact researchers seeking to study parent responsiveness. That is, across varied frameworks in which parent responsiveness has been examined (e.g., synchrony, de Mendonça et al., 2011; parenting, Landry et al., 2006; mutuality, Aksan et al., 2006; communication, Leezenbaum et al., 2014; emotional availability, Biringen & Easterbrooks, 2012), the definitions and conceptualized functions of responsivity vary in nuanced ways. Furthermore, we found that responsive behaviors were coded from various observable modalities, including verbal, gestural, tactile, and auditory cues. Such variations present challenges in determining whether researchers have measured the same construct of “parent responsiveness” across different studies or have measured different constructs that would be expected to have differential associations with aspects of child development. Thus, it is imperative for authors to make clear the specific concept and behavior(s) being studied, along with their operationalized parameters, to communicate their findings and the implications of their studies effectively.
Best practice recommendations for metrics and reporting standards
We propose an initial set of standards for adoption by autism researchers when focusing on parent responsiveness. First, a convention should be adopted to standardize the type of information reported within studies. These include clearly reporting parent demographic information, documentation of the reliability and validity of the coding system, and details regarding the training protocol for coders. In addition, operational definitions and parameters of all behavioral coding constructs should be reported (whether original or adapted) with indications of what modifications were made, and citing references for accessing such definitions. Naturally, inclusion of such information may increase article length to the point it exceeds a journal’s word or page count limit. Solutions to this dilemma include publisher creation of exemption criteria, a defined extension of page/word limit restrictions to permit such reporting, or requiring supplemental materials to report details about the coding system. Providing these details significantly increases the transparency of coding procedures and strengthens the science of behavioral coding. As a method that is used so widely to explore developmental behaviors and outcomes, behavioral coding measures should be reported with as much detail as statistical analysis.
Limitations and potential future directions
The goal of this review was to summarize behavioral coding measures used to evaluate parent responsiveness to children with autism or elevated risk of autism. While we employed a broad search criterion, we chose to emphasize behavioral coding systems that quantified parent responsiveness and therefore to exclude qualitative studies. Even though qualitative studies did not fit our framework for this scoping review, they could provide support in understanding broader concepts within the parent responsiveness literature and inform the description of responsive parent behaviors. Further review of this information could be helpful in creating more thorough and consistent standards for reporting in this area. As with any scoping review, our study did not systematically assess articles for quality and rigor. These components could be added to a future review to provide information about “levels of evidence” provided by the articles. However, we did indicate some components of quality, such as reliability, validity, and blinding of coders. A future systematic review could go beyond synthesis and aid in indicating the overall level of rigor of behavioral coding within the parent responsiveness literature. Furthermore, a systematic review of what components of available interventions are key or active ingredients that propel successful treatment outcomes could be extremely worthwhile to researchers and clinicians.
Conclusion
This report is a comprehensive summary of parent responsiveness measures used in studies focused on the child autism population. It can be used as a reference for researchers conducting future studies in the same topic area to identify existing coding systems that might be appropriate for their studies. However, our study suggests the need for increased standardization in the methods of measuring and reporting behavioral coding of parent responsiveness. Such standardization would allow for better comparison of findings across studies and inform future research. By knowing more details about the methodology used, and specifically the behaviors coded, the information may help clinicians know the kinds of behaviors that might be useful in their own interventions with autistic children.
Supplemental Material
sj-doc-1-aut-10.1177_13623613231152641 – Supplemental material for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism
Supplemental material, sj-doc-1-aut-10.1177_13623613231152641 for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism by Thelma E Uzonyi, Alaina C Grissom, Ranita V Anderson, Helen Lee, Sarah Towner-Wright, Elizabeth R Crais, Linda R Watson and Rebecca J Landa in Autism
Supplemental Material
sj-docx-2-aut-10.1177_13623613231152641 – Supplemental material for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism
Supplemental material, sj-docx-2-aut-10.1177_13623613231152641 for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism by Thelma E Uzonyi, Alaina C Grissom, Ranita V Anderson, Helen Lee, Sarah Towner-Wright, Elizabeth R Crais, Linda R Watson and Rebecca J Landa in Autism
Supplemental Material
sj-docx-3-aut-10.1177_13623613231152641 – Supplemental material for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism
Supplemental material, sj-docx-3-aut-10.1177_13623613231152641 for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism by Thelma E Uzonyi, Alaina C Grissom, Ranita V Anderson, Helen Lee, Sarah Towner-Wright, Elizabeth R Crais, Linda R Watson and Rebecca J Landa in Autism
Supplemental Material
sj-docx-4-aut-10.1177_13623613231152641 – Supplemental material for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism
Supplemental material, sj-docx-4-aut-10.1177_13623613231152641 for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism by Thelma E Uzonyi, Alaina C Grissom, Ranita V Anderson, Helen Lee, Sarah Towner-Wright, Elizabeth R Crais, Linda R Watson and Rebecca J Landa in Autism
Supplemental Material
sj-docx-6-aut-10.1177_13623613231152641 – Supplemental material for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism
Supplemental material, sj-docx-6-aut-10.1177_13623613231152641 for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism by Thelma E Uzonyi, Alaina C Grissom, Ranita V Anderson, Helen Lee, Sarah Towner-Wright, Elizabeth R Crais, Linda R Watson and Rebecca J Landa in Autism
Supplemental Material
sj-docx-7-aut-10.1177_13623613231152641 – Supplemental material for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism
Supplemental material, sj-docx-7-aut-10.1177_13623613231152641 for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism by Thelma E Uzonyi, Alaina C Grissom, Ranita V Anderson, Helen Lee, Sarah Towner-Wright, Elizabeth R Crais, Linda R Watson and Rebecca J Landa in Autism
Supplemental Material
sj-xlsx-5-aut-10.1177_13623613231152641 – Supplemental material for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism
Supplemental material, sj-xlsx-5-aut-10.1177_13623613231152641 for Scoping review of behavioral coding measures used to evaluate parent responsiveness of children with autism or elevated risk of autism by Thelma E Uzonyi, Alaina C Grissom, Ranita V Anderson, Helen Lee, Sarah Towner-Wright, Elizabeth R Crais, Linda R Watson and Rebecca J Landa in Autism
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding support for Thelma E. Uzonyi at the initial stages of this project included the Royster Society of Fellows at the University of North Carolina at Chapel Hill, the Institute of Education Sciences and the US Department of Education, Office of Special Education Programs Doctoral Leadership Grant #H325D160060. Funding support for Alaina Grissom included a graduate assistantship provided by the University of Tennessee Health Science Center provided to students in the Speech and Hearing Science doctoral program.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
