Individuals with autism spectrum disorder are differentially sensitive to interference from previous verbal feedback

Abstract

This study examined whether set-shifting ability for children with autism spectrum disorder without intellectual disability would be affected differentially by verbal or nonverbal feedback as the outcome of previous research tentatively suggests that verbal feedback may lead to slower set-shifting. Overall, 56 children participated (42 male; 14 female); 28 with a diagnosis of autism spectrum disorder (24 male) and 28 (21 male) typically developing children matched on cognitive and verbal abilities. Each group was exposed to a set-shifting task using cards varying in three dimensions. One group of typically developing children, and one group of autism spectrum disorder children, received verbal feedback on their performance, and one group received nonverbal feedback. Children with autism spectrum disorder learned an initial categorisation rule as fast as matched typically developing children. There was little difference in the impact of the type of feedback on acquisition. However, on shifting the classification rule, children with autism spectrum disorder showed slower rates of learning the new rule, relative to matched controls, which was worse when verbal feedback was used compared to nonverbal feedback. This finding has implications for the interpretations of set-shifting performance and for classroom use of feedback strategies.

Lay abstract

Individuals with autism spectrum disorder can find behavioural flexibility challenging, often exhibited in terms of repetitive behaviours or restricted ranges of interests and activities. An inability to shift efficiently from one situation to another is connected with problems in daily life, and identifying factors associated with this ability may help develop teaching strategies to improve behavioural flexibility. Some existing findings imply shifting performance for individuals with autism spectrum disorder is better with nonverbal, compared to verbal, feedback – even for those with strong verbal abilities. Unfortunately, there are few behavioural examinations that further explore these findings, which is the aim of this study. In this study, 28 children with a diagnosis of autism spectrum disorder and 28 typically developing children matched on cognitive and verbal abilities learned to sort cards according to one out of a possible three dimensions (colour, shape and number), and then had to relearn the sorting rule. One group of typically developing children, and one group of autism spectrum disorder children, received verbal feedback on their performance, and one group received nonverbal feedback. Children with autism spectrum disorder learned an initial categorisation rule as fast as matched typically developing children, and there was little difference in the impact of the type of feedback on acquisition. However, on shifting the classification rule, children with autism spectrum disorder showed slower rates of learning the new rule, which was worse when verbal feedback was used compared to nonverbal feedback. This finding has implications for the interpretations of set-shifting performance and for classroom use of feedback strategies.

Keywords

autism spectrum disorder nonverbal feedback set-shifting verbal feedback

Individuals with autism spectrum disorder (ASD) can find behavioural flexibility challenging; often exhibited in terms of the expression of repetitive behaviours or restricted ranges of interests and activities, and ongoing problems with social communications and interaction (APA, 2013). A single account of this wide range of issues may be unrealistic (Frith & Happé, 1994; Reed, 2016), and focus is typically given to exploring specific aspects of cognitive flexibility through set-shifting tasks (Gonzalez et al., 2019; Kenworthy et al., 2010; Reed, 2018; Yerys et al., 2009). An inability to shift efficiently from one situation to another is associated with repetitive or restricted behaviours and interests (Leekam et al., 2011; Reed et al., 2013; Yerys et al., 2009), and experiencing problems in daily life (Gonzalez et al., 2019; Granader et al., 2014). Thus, its theoretical understanding and clinical remediation are highly relevant for individuals with ASD (see Campbell et al., 2017; Faja et al., 2022; Kenworthy et al., 2014; Yerys et al., 2012).

Understanding factors enhancing set-shifting is important for the development of strategies helping individuals with ASD, especially school-aged children who are the target of several recent clinical interventions (Faja et al., 2022; Kenworthy et al., 2014; Tamm et al., 2021). Such clinical developments may benefit from integration with empirical and theoretical findings regarding factors promoting set-shifting, particularly concerning use of the verbal and nonverbal feedback, which often play roles in teaching programmes for ASD and set-shifting (Campbell et al., 2017; Demurie et al., 2011; Joseph et al., 2005; Yerys et al., 2012; see Reed, 2016, for a review). Improving feedback efficacy for people with ASD may enhance such teaching strategies for behavioural flexibility.

In fact, several theoretical and empirical models of flexibility for those with ASD imply whether feedback is delivered verbally or nonverbally may impact the efficacy of teaching interventions (e.g. Campbell et al., 2017; Russo et al., 2007; Yerys et al., 2012). However, the relative efficacy of verbal and nonverbal feedback in set-shift feedback has received scant direct investigation, and the few behavioural examinations reported have produced a mixed set of results (cf., Campbell et al., 2017; Yerys et al., 2012). Yerys et al. (2012) found higher verbal mental ages for those with ASD predicted better shifting performance. However, other studies have noted that, while verbal abilities support cognitive flexibility, individuals with ASD rely on nonverbal abilities in solving complex tasks (Campbell et al., 2017). Indeed, other studies of set-shifting for individuals with ASD suggest benefits of nonverbal over verbal feedback, even for those with strong verbal abilities (Joseph et al., 2005). However, this mixed pattern of data makes it difficult to interpret the impact of feedback on studies of set-shifting with individuals with ASD.

Comparison of verbal and nonverbal feedback in discrimination learning was the focus of a wide range of studies, conducted some years ago, with typically developing individuals (Barringer & Gholson, 1979; Spence & Segner, 1967). Generally, findings demonstrated that verbal feedback produced faster acquisition of discrimination learning than nonverbal tangible feedback (Penney & Lupton, 1961; Stevenson et al., 1959). In contrast, for some executive function (EF) tasks for individuals with ASD, verbal feedback appears less effective than nonverbal feedback (e.g. monetary reward; Demurie et al., 2011). Data from yet another training paradigm from Kuzmanovic et al. (2011) suggest reduced sensitivity to nonverbal stimuli for individuals with ASD in situations where there was conflicting information, which may be more relevant to a set-shifting experiment. Thus, the case for those with ASD remains completely unclear, and the impact of different forms of feedback on a set-shifting training procedure requires further investigation.

A task often used to examine set-shifting abilities is the Dimensional Change Card Sort (DCCS; Frye et al., 1995; Reed, 2018; Zelazo, 2006). In an initial training phase, individuals sort cards displaying images differing across multiple dimensions (e.g. shape, colour and number), according to one of those dimensions (e.g. colour). Following this training, individuals sort the same cards according to another dimension (e.g. number). Typically developing individuals usually learn the switch from one rule to another at a faster rate than those with ASD (Gonzalez et al., 2019; Reed, 2018; Yerys et al., 2009; Zelazo, 2006). A number of different measures can be used to assess this learning, such as the number of errors made until the new sorting rule is performed to a particular criterion, the number of times that those errors reflect sorting by the previous rule (perseverative errors) and the time take to perform the task (Reed, 2018; Reed et al., 2013; Zelazo, 2006).

Performance on DCCS tasks undoubtedly involves many cognitive and neural systems, including cognitive flexibility, inhibition of ongoing responses and working memory (Campbell et al., 2017; Yerys et al., 2012; see Russo et al., 2007 for a review). Individuals with ASD may recruit a variety of these systems for the task and employ a different range of systems to those driving performance for typically developing individuals. For example, Dirks et al. (2020) noted that children with ASD recruited memory-based systems (indexed by increased para-hippocampal activity) to a greater extent than a comparison group (see also Yerys et al., 2015 for similar conclusions on recruiting alternative systems). Recruitment of memory systems may rely on other skills; for example, it is suggested that children with ASD may not easily recruit verbal mediation strategies to maintain and monitor goal-related information in working memory, making this system less helpful in performing set-shifting tasks, and suggesting that verbal feedback may not be helpful (Campbell et al., 2017; Joseph et al., 2005). Such a suggestion has tentative support in the neuropsychological literature, as several studies have noted that different types of feedback in set-shifting tasks differentially activate areas of the midbrain involved in working memory (Albrecht et al., 2014; Henson, 2005). These findings (very tentatively) suggest that set-shifting performance for individual with ASD may be sensitive to the types of feedback they receive while learning the task; again, there are very few, if any, behavioural examinations of this suggestion. This lack of information may serve to impede the development of effective set-shifting training schedules for individuals with ASD that may ultimately aid increased behavioural flexibility.

To examine these effects, the current experiment compared children with ASD with a typically developing children matched for intellectual and verbal abilities. Each group was exposed to a set-shift task, using cards varying in three dimensions (Reed, 2018). One group of typically developing children, and one group of ASD children, received verbal feedback on their performance. One group of typically developing children, and one group of ASD children, received nonverbal feedback. The study aimed to explore whether either form of feedback leads to faster acquisition of the initial set, and then whether set-shifting would be faster, slower or unaffected, by the type of feedback given. Discovering the types of feedback most helpful for children with ASD when shifting attention may aid understanding of the processes important for behavioural flexibility in this population, development of clinically based strategies to facilitate behavioural flexibility, and this information may be important for everyday settings, such as the classroom.

Method

Participants

Primary schools were contacted to identify children with ASD, who had no other diagnosis, were linguistically able and whose native language was English. Overall, 56 children who were identified took part (42 male and 14 female); 28 with a diagnosis of ASD (24 male) and 28 (21 male) typically developing children matched on cognitive and verbal abilities. G-Power analysis suggested that, for an expected medium effect size (f = 0.25) on the basis of previous investigations of EF in people with ASD (e.g. Lai et al., 2017), and using a p < 0.05 rejection criteria, for 90% power, 46 participants would be required to detect a two-way interaction (group × feedback) using analysis of variance (ANOVA).

Participants in the ASD group were all identified as not having an intellectual disability, and as being linguistically competent, by their teacher. All had a diagnosis of ASD, but no comorbidities, made by a paediatrician (who assessed for a range of conditions during the diagnostic period) independent from the current study. The diagnosis was based on the Diagnostic and Statistical Manual of Mental Disorders (5th ed., DSM-5) criteria, an Autism Diagnostic Observation Schedule (ADOS) assessment made by the clinical team involved in the diagnosis (independent of this research) who were trained in its administration, and the paediatrician’s clinical judgement. The participants with ASD had a mean autism behaviour checklist (ABC; Krug et al., 1980) score of 76.46 (± 15.23; range = 53–112). Table 1 presents the group mean scores for the sample for age, nonverbal and verbal IQ, and reading age, which shows the two groups were well matched.

Table 1.

Group mean scores for the sample for age (months), nonverbal and verbal IQ (WASI-II), and reading age (BPVS).

	ASD	TD	Difference
N (gender)	26 (24 M:21 F)	26 (21 M:5 F)
Age (months)	101 (20.36; 59–128)	87.54 (22.64; 59–136)	2.38*, d = 0.635
Nonverbal IQ (WASI-II)	92.96 (13.14; 60–118)	97.75 (15.24; 72–143)	1.25, d = 0.336
Verbal IQ (WASI-II)	88.53 (27.72; 44–158)	90.89 (25.84; 58–153)	<1, d = 0.088
Reading age (months)	88.50 (27.35; 37–136)	90.43 (32.63; 47–147)	<1, d = 0.064

ASD: autism spectrum disorder; TD = typically developing; WASI-II: Wechsler Abbreviated Scale of Intelligence II.

p < .05.

Materials

ABC (Krug et al., 1980) is a 57-item scale assessing ASD symptoms in children who are 3 years of age and above. The items are each assessed on a five-point scale and provide scores in five sub-domains, and an overall score. The ABC is completed by a teacher who is familiar with the child. It has been found to discriminate between ASD and other disorders (Eaves et al., 2006), and the internal reliability of the overall scale for this study was 0.81.

Wechsler Abbreviated Scale of Intelligence II (WASI-II; Wechsler, 2011) measures intellectual ability across a wide age range. It comprises four subtests: two assessing language (vocabulary and similarities) and two performance measures (block design and matrix reasoning). Thus, the WASI-II generates two scores of abilities, verbal and performance scores, and a full score of intellectual functioning. Test reliability has been stated at 0.87–0.92.

British Picture Vocabulary Scale (BPVS; Dunn et al., 1982) measures receptive language ability for children aged 3–17 years. It asks the child to identify the picture that best represents a spoken target word. It provides a standard age score and receptive vocabulary age equivalent. It has an internal reliability (Cronbach’s α) of 0.93.

Set-shifting Materials: In total, 27 cards were used to test the ability to adapt to a change in performance-governing rules. Each card (15 × 11.5 cm²) was printed on white card, was laminated and displayed stimuli varying along three dimensions: shape (rectangle, circle or triangle); colour (red, blue or yellow) and number of shapes (the cards displayed, one, two or three). Each card displayed one, two or three pictures, of a particular shape, in a particular colour. Thus, these cards could be sorted by the shape, colour or number of stimuli on the card (see Berg, 1948; Reed et al., 2013).

Feedback: Participants in the verbal feedback conditioning were given a list of positive terms derived from those used in the study reported by Demurie et al. (2011): Good, Yes, Right, Well done, Good job, Completely good, Yes, What a champion, Wonderfully done, You’re a star and Great. Participants were asked to rate the value of each reinforcer from 1 (not helpful), through 4 (neutral), to 7 (very helpful, nice reward). Similarly, the participants in the nonverbal feedback condition were shown a range of stickers: animals, Thomas the Tank, football, rainbows and unicorns, weather symbols, numbers, and asked the same question. The most highly rated alternative was chosen for each participant. The median value for both the verbal and nonverbal feedback chosen was 6 (in fact, all chosen stimuli, except four with seven in each group, had this value).

Procedure

Consent for participation was first acquired by letter from the parents of the children, and from the teachers of the children. Following this, the children were asked if they wished to participate. All children were tested individually, with their teacher present in a quiet room separate from any distractions. Children were tested with the WASI-II and BPVS to establish their verbal and nonverbal IQ (WASI-II), and their receptive language mental age (BPVS). They then completed the test conditions. While the test occurred, the teacher completed the ABC for the child.

The test condition required children to take part in a card sorting task, using the cards, as described above. Feedback was given when the participant sorted a card and was either verbal or nonverbal. For half of the participants, feedback was verbal. The child was provided with positive feedback (depending on their preferred reinforcing phrase) if the card that they sorted was placed in the correct pile, but no feedback if it was not correctly sorted. For the other half of the participants, feedback was nonverbal. The child was presented with a sticker (which they had previously chosen) when the card was sorted correctly, or not presented with a sticker if the sorting was incorrect. Participants in the groups were informed of the type of feedback they would receive.

Phase 1 (Training): Three cards, each displaying three separate dimensions, were placed in front of a participant. All participants were then given the following instructions: ‘We are going to play a game. In this game you have to put these cards in to three piles. The cards that are like each other should be put together into the same pile’. Participants were told that they may take a break, or stop playing the game, at any time. They were then given a demonstration of how to sort the cards according to the training rule. First, the remaining cards were shuffled for 5–10 s and then placed face down in a single pile in the centre of the table. Participants were either shown how to sort the cards according to colour, shape or number of stimuli, displayed on the card, depending on the rule that they were first required to follow. This rule was determined randomly for each participant. The experimenter continued with this demonstration until all cards were exposed and placed into the correct pile.

After the demonstration, the experimenter shuffled the cards again for 5–10 s and placed them face down, in a single pile, in front of the participant. The participants were asked to sort the cards in the same way as they had just seen demonstrated. They were told that for each card, they would be given feedback as to whether they had placed the card correctly. The type of feedback given was predetermined, so half the participants received verbal feedback and the remaining participants received nonverbal feedback. The participant was required to turn over one card at a time from the top of the pile, and then place it on top of one of the three exposed cards according to the rule the experimenter has just demonstrated. Each time a participant placed a card in a pile, they were given feedback if their response was correct (but not if it was incorrect). If a card was placed incorrectly, participants were asked to continue to sort the rest of the cards into one of the three piles. Each trial lasted until all 27 cards had been placed. Between each trial, the cards were collected together and shuffled for 5–10 s. When the participant had sorted the cards according to the appropriate rule for three consecutive trials with no errors, the phase ended. This phase typically took the participant 5–8 min to complete.

Phase 2 : Immediately after the completion of Phase 1, Phase 2 began. This change of phase was not signalled to the participants, and no modelling of responses was performed. As in Phase 1, cards were shuffled for 5–10s and placed face down in front of the child, with one of each variant placed face up in front of the child. In this phase, a correct trial was when the participant had sorted the cards by a randomly predetermined alternative dimension to that trained in Phase 1 (e.g. if the cards had previously been sorted by shape, they were now required to be sorted by colour or number; those previously sorted by colour were now required to be sorted by number or shape, etc.). The type of feedback remained the same as in Phase 1. Each trial lasted until all 27 cards were placed into the piles. On completion of a trial, the cards were reshuffled, and another trial commenced. This lasted until the participant had sorted the cards according to the correct rule for three consecutive trials. A limit of eight trials was placed on each rule change. If participants were unable to adapt to the first rule change after eight consecutive trials, the experiment was stopped to avoid unnecessary distress for participants. Participants typically took 5–10 min to complete Phase 2.

Phase 3: Following Phase 2, the sorting rule was changed once more. The final rule required participants to sort the card according to the final dimension not previously tested. For example, those who had previously sorted by shape and colour were required to sort by the number. The same criteria were in place for Phase 3 as described for Phase 2. Participants typically took 5–10 min to complete Phase 3.

During all phases, the time taken to reach criterion was recorded. During Phase 1, the time was recorded from the first card the participant picked up until the phase was complete, and the modelling exercise was not included. The number of cards taken to reach criterion was recoded, and the number of errors made (cards placed incorrectly) during each trial. Both total errors, and initial perseverative errors made by sorting according to the rule in force during the previous phase, were recorded.

Community involvement

Not applicable.

Results

Table 2 shows the mean number of errors, and the time taken to reach criterion, for the verbal and nonverbal feedback conditions, for both groups, in the initial training (Phase 1). Inspection of these data reveals very small numbers of errors were made by either group in any feedback condition. The ASD group made slightly more errors in the nonverbal condition than the verbal condition, and this pattern was reversed for the control group. A two-way between-subject ANOVA, with group (ASD × control) and feedback (verbal vs nonverbal) as factors, was conducted on the total errors. This revealed no significant main effects, both Fs < 1, but there was a small- to medium-sized significant interaction between condition and feedback, F(1, 52) = 4.58, p = 0.037, $η_{p}^{2}$ = 0.081 [95% CI = 0.000–0.239]. However, simple effect analyses revealed no group differences for either verbal feedback, F(1, 52) = 3.31, p = 0.074, $η_{p}^{2}$ = 0.058 [0.000–0.208], or nonverbal feedback, F(1, 52) = 1.47, p = 0.231, $η_{p}^{2}$ = 0.027 [0.000–0.155]. Similarly, there was no feedback difference for the ASD, F(1, 52) = 1.47, p = 0.231, $η_{p}^{2}$ = 0.027 [0.000–0.155], or control, F(1, 52) = 1.18, p = 0.282, $η_{p}^{2}$ = 0.022 [0.000–0.144], groups.

Table 2.

Mean number of errors made during Phase 1, and the time taken to reach criterion, for the verbal and nonverbal feedback conditions, for both groups.

	Verbal	Nonverbal
ASD: Total errorsTime (s)	1.00 (1.24)192.43 (67.43)	3.29 (3.17)218.57 (9.709)
Control: Total errorsTime (s)	3.14 (4.13)223.29 (96.91)	1.86 (3.21)124.79 (125.52)

ASD: autism spectrum disorder.

The ASD group took slightly longer to complete the task in the nonverbal condition than in the verbal condition, and this pattern was reversed for the control group. However, a two-way ANOVA (condition × feedback) conducted on the time taken revealed no significant main effects, both Fs < 1, and no interaction, F(1, 52) = 1.04, p > 0.30, $η_{p}^{2}$ = 0.020 [0.000–0.139].

Figure 1 shows the mean total errors (top panel) and mean perseverative errors (bottom panel) made by each group, in each feedback condition, for both Phases 2 and 3. Inspection of the top panel shows that the ASD group made more total errors than the control group, made more total errors in Phase 3 than Phase 2 and made more total errors with verbal than nonverbal feedback. A three-factor mixed-model ANOVA (group × feedback × phase) revealed significant main effects of group, F(1, 52) = 77.68, p < 0.001, $η_{p}^{2}$ = 0.594 [0.412–0.701], feedback, F(1, 52) = 26.54, p < 0.001, $η_{p}^{2}$ = 0.333 [0.136–0.493], and phase, F(1, 52) = 8.01, p = 0.007, $η_{p}^{2}$ = 0.131 [0.011–0.300]. There were significant interactions between group and feedback, F(1, 52) = 24.43, p < 0.001, $η_{p}^{2}$ = 0.316 [0.121–0.478], and group and phase, F(1, 52) = 12.43, p < 0.001, $η_{p}^{2}$ = 0.190 [0.036–0.362], but no other significant interactions, both Fs < 1.

Figure 1.

Group-mean mean number of total errors (top panel) and perseverative errors (bottom panel) made by each group, in each feedback condition, for Phases 2 and 3.

Simple effect analysis conducted on the number of total errors made, averaged across Phases 2 and 3, revealed that the ASD group made many total errors than the control group with verbal feedback, F(1, 52) = 73.31, p < 0.001, $η_{p}^{2}$ = 0.580 [0.395–0.691], but there was only a much smaller difference between the groups for nonverbal feedback, F(1, 52) = 5.80, p = 0.019, $η_{p}^{2}$ = 0.098 [0.001–0.261]. With verbal feedback, the ASD group made more total errors than with nonverbal feedback, F(1, 52) = 39.41, p < 0.001, $η_{p}^{2}$ = 0.427 [0.222–0.571], but there was no difference between feedback type for the control group, F < 1, $η_{p}^{2}$ = 0.001 [0.000–0.007].

Inspection of the bottom panel of Figure 1 shows that the ASD group made more perseverative errors than the control group, made more errors in Phase 3 than Phase 2 and made more errors with verbal than nonverbal feedback. A three-factor mixed-model ANOVA (group × feedback × phase) revealed significant main effects of group, F(1, 52) = 54.99, p < 0.001, $η_{p}^{2}$ = 0.509 [0.311–0.636], feedback, F(1, 52) = 17.44, p < 0.001, $η_{p}^{2}$ = 0.247 [0.071–0.417], and phase, F(1, 52) = 24.33, p < 0.001, $η_{p}^{2}$ = 0.315 [0.120–0.477]. There were significant interactions between group and feedback, F(1, 52) = 16.81, p < 0.001, $η_{p}^{2}$ = 0.241 [0.066–0.411], group and phase, F(1, 52) = 29.88, p < 0.001, $η_{p}^{2}$ = 0.361 [0.159–0.5167], feedback and phase, F(1, 52) = 7.20, p = 0.010, $η_{p}^{2}$ = 0.121 [0.007–0.286], and between all three factors, F(1, 52) = 6.57, p = 0.013, $η_{p}^{2}$ = 0.110 [0.005–0.275].

To analyse the three-way interaction, two-way between-subject ANOVAs (group × feedback) were conducted on each phase, separately. For Phase 2, there were significant main effects of group, F(1, 52) = 11.35, p < 0.001, $η_{p}^{2}$ = 0.176 [0.029–0.347], and feedback, F(1, 52) = 4.87, p = 0.035, $η_{p}^{2}$ = 0.084 [0.000–0.242], and an interaction between group and feedback, F(1, 52) = 4.71, p = 0.035, $η_{p}^{2}$ = 0.082 [0.000–0.236]. Simple effects revealed that the ASD group made more perseverative errors than the control group with verbal feedback, F(1, 52) = 15.34, p < 0.001, $η_{p}^{2}$ = 0.225 [0.056–0.395], but there was no difference between the groups with nonverbal feedback, F < 1, $η_{p}^{2}$ = 0.013 [0.000–0.125]. With verbal feedback, the ASD group made more perseverative errors than with nonverbal feedback, F(1, 52) = 9.38, p = 0.003, $η_{p}^{2}$ = 0.150 [0.018–0.321], but there was no difference between feedback type for the control group, F < 1, $η_{p}^{2}$ = 0.001 [0.000–0.001].

For Phase 3, there were significant main effects of group, F(1, 52) = 67.54, p < 0.001, $η_{p}^{2}$ = 0.560 [0.370–0.676], and feedback, F(1, 52) = 19.59, p < 0.001, $η_{p}^{2}$ = 0.270 [0.086–0.438], and an interaction between group and feedback, F(1, 52) = 18.56, p < 0.001, $η_{p}^{2}$ = 0.249 [0.079–0.428]. Simple effects revealed the ASD group made more perseverative errors than the control group with verbal feedback, F(1, 52) = 78.47, p < 0.001, $η_{p}^{2}$ = 0.597 [0.415–0.703], and for nonverbal feedback, F(1, 52) = 7.65, p = 0.008, $η_{p}^{2}$ = 0.126 [0.009–0.293]. With verbal feedback, the ASD group made more perseverative errors than with nonverbal feedback, F(1, 52) = 38.14, p < 0.001, $η_{p}^{2}$ = 0.418 [0.214–0.564], but there was no difference between feedback type for the control group, F < 1, $η_{p}^{2}$ = 0.004 [0.000–0.095].

Figure 2 shows the time taken to reach criterion for each group, in each feedback condition, for both Phases 2 and 3. The ASD group took longer to reach criterion than the control group, took longer in Phase 3 than Phase 2 and took longer with verbal than nonverbal feedback. A three-factor mixed-model ANOVA (group × feedback × phase) revealed significant main effects of group, F(1, 52) = 52.71, p < 0.001, $η_{p}^{2}$ = 0.503 [0.302–0.633], feedback, F(1, 52) = 22.53, p < 0.001, $η_{p}^{2}$ = 0.302 [0.109–0.468], and phase, F(1, 52) = 19.66, p < 0.001, $η_{p}^{2}$ = 0.274 [0.088–0.443]. There were significant interactions between group and feedback, F(1, 52) = 13.63, p < 0.001, $η_{p}^{2}$ = 0.208 [0.045–0.381], group and phase, F(1, 52) = 26.91, p < 0.001, $η_{p}^{2}$ = 0.341 [0.140–0.501], not between feedback and phase, F(1, 52) = 2.04, p = 0.159, $η_{p}^{2}$ = 0.038 [0.000–0.176], but between all three factors, F(1, 52) = 6.86, p = 0.011, $η_{p}^{2}$ = 0.117 [0.006–0.284].

Figure 2.

Group-mean time taken to reach criterion for each group, in each feedback condition, for both Phase 2 and Phase 3.

To analyse the three-way interaction, two-way between-subject ANOVAs (group × feedback) were conducted on each phase, separately. For Phase 2, there were significant main effects of group, F(1, 52) = 18.53, p < 0.001, $η_{p}^{2}$ = 0.263 [0.079–0.432], and feedback, F(1, 52) = 14.53, p < 0.001, $η_{p}^{2}$ = 0.218 [0.051–0.391], and an interaction between group and feedback, F(1, 52) = 4.83, p = 0.032, $η_{p}^{2}$ = 0.085 [0.000–0.246]. Simple effects revealed that the ASD group took longer than the control group with verbal feedback, F(1, 52) = 21.15, p < 0.001, $η_{p}^{2}$ = 0.298 [0.099–0.456], but there was no difference for nonverbal feedback, F(1, 52) = 2.21, p = 0.143, $η_{p}^{2}$ = 0.041 [0.000–0.181]. With verbal feedback, the ASD group took longer than with nonverbal feedback, F(1, 52) = 9.38, p = 0.003, $η_{p}^{2}$ = 0.152 [0.018–0.325], but there was no difference between feedback type for the control group, F(1, 52) = 1.30, p = 0.259, $η_{p}^{2}$ = 0.024 [0.000–0.152].

For Phase 3, there were significant main effects of group, F(1, 52) = 68.21, p < 0.001, $η_{p}^{2}$ = 0.567 [0.377–0.682], and feedback, F(1, 52) = 20.61, p < 0.001, $η_{p}^{2}$ = 0.284 [0.095–0.452], and an interaction between group and feedback, F(1, 52) = 17.60, p < 0.001, $η_{p}^{2}$ = 0.253 [0.073–0.423]. Simple effects revealed that the ASD group took longer than the control group with verbal feedback, F(1, 52) = 77.52, p < 0.001, $η_{p}^{2}$ = 0.599 [0.415–0.705], and there was a smaller effect for nonverbal feedback, F(1, 52) = 8.26, p = 0.006, $η_{p}^{2}$ = 0.137 [0.012–0.308]. With verbal feedback, the ASD group took longer than with nonverbal feedback, F(1, 52) = 32.13, p < 0.001, $η_{p}^{2}$ = 0.382 [0.177–0.535], but there was no difference between feedback type for the control group, F(1, 52) = 0.06, p = 0.037, $η_{p}^{2}$ = 0.001 [0.000–0.046].

Discussion

This study examined whether set-shifting ability for individuals with ASD would be affected differentially by verbal and nonverbal feedback, to explore suggestions that set-shifting for individuals with ASD benefits from nonverbal rather than verbal feedback (Campbell et al., 2017; Joseph et al., 2005). The few previous studies that have examined this issue directly have produced mixed results (cf. Campbell et al., 2017; Yerys et al., 2012), and the literature derived from other tasks is inconclusive (cf. Demurie et al., 2011; Kuzmanovic et al., 2011). Beyond the empirical and theoretical implications, as verbal or nonverbal feedback often play roles in teaching programmes (Campbell et al., 2017; Joseph et al., 2005; see Reed, 2016, for a review), recent clinical interventions focusing on shifting for children with ASD may benefit from such information (Faja et al., 2022; Kenworthy et al., 2014; Tamm et al., 2021).

The main findings were that linguistically competent children with ASD learned an initial categorisation rule as fast as matched typically developing children. There was little difference in the impact of the type of feedback – with verbal feedback being slightly more effective for children with ASD. This finding is in line with limited literature available on the topic (Barringer & Gholson, 1979; Demurie et al., 2011; Reed et al., 2012; Spence & Segner, 1967). On shifting the classification rule, children with ASD showed slower learning of the new rule relative to typically matched controls. Moreover, this effect was more pronounced when verbal feedback was used compared to nonverbal feedback, which supports some previous studies (Campbell et al., 2017; Joseph et al., 2005). This finding has implications for interpretations of slower set-shifting observed in previous experiments (Gonzalez et al., 2019; Kenworthy et al., 2010; Reed, 2018; Yerys et al., 2009) and for the use of feedback strategies for shifting training (see Faja et al., 2022; Kenworthy et al., 2014; Kuzmanovic et al., 2011; Reed, 2016; Tamm et al., 2021).

That the initial learning of the rule was similar in both groups, with both types of feedback, implies no strong differences in the cognitive abilities of the sample nor in terms of the power of the feedback stimuli. This suggests that the matching procedures for the sample were adequate, and that the reinforcer titration produced equally effective stimuli for feedback. Previous studies have shown little difference between sensitivity to reward/feedback amount between individuals with and without ASD (Demurie et al., 2011; Reed et al., 2012; Scott et al., 2010). The social aspects of the reinforcement delivery in the current procedure did not seem to impact the learning of the children with ASD differentially, compared to the typically developing children (see also Yerys et al., 2012). Any slight differences reflected a very small numerical tendency for children with ASD to favour verbal feedback, in terms of rate of learning, and typically developing children to favour nonverbal feedback. A favouring of verbal feedback by those with ASD has been noted in some previous studies (Kuzmanovic et al., 2011; Yerys et al., 2012), but not all (see Campbell et al., 2017; Demurie et al., 2011). That typically developing children seemed to learn better with nonverbal cues stands at some odds with previous literature (Penney & Lupton, 1961; Stevenson et al., 1959). However, these effects were numerically very small, and not too much weight should be placed on them, except in that there was no strong difference between the groups or feedback in promoting initial learning.

The main finding was that the type of feedback given to individuals with ASD had a large effect on their ability to shift-set. Although individuals with ASD took longer than typically developing individuals to learn a new rule (Kenworthy et al., 2010; Reed, 2018; Yerys et al., 2009), verbal feedback resulted in a much longer time taken to learn to shift to the new rule, compared to nonverbal feedback (see also Joseph et al., 2005). This slower rate of set-shifting was seen irrespective of the index of learning examined. This finding relates to suggestions that observed slower set-shifting is as dependent on features of the task employed (Demurie et al., 2011; Reed, 2018; White, 2013; Williams & Jarrold, 2013), especially the nonverbal nature of the feedback (Campbell et al., 2017; Joseph et al., 2005).

Previous studies have noted that set-shifting correlates with verbal mental age (e.g. Yerys et al., 2012); whereas, the current study and others (e.g. Campbell et al., 2017; Joseph et al., 2005) have a stronger relationship with nonverbal cues and ability. The latter results do not preclude an association between verbal ability and shifting – but implying only this may not be as strong as that with nonverbal abilities and cues. Indeed, in the current study, the correlation between verbal IQ (WASI) and nonverbal IQ (WASI), and total shift errors in Phase 2 were −0.159 and −0.308, respectively. These scores for perseverative errors were −0.287 and −0.316, respectively, which corroborate suggestions about the importance of nonverbal ability made by Campbell et al. (2017), without contradicting claims by Yerys et al. (2012) regarding the importance of verbal ability. Unfortunately, it should be noted that the power of these correlations for the current study (based on 28 participants) is less than 50% (G-Power). This could be explored further in future studies.

In fact, verbal ability may well be important for shifting when this interacts with feedback type, as in the current study. It may be that the tendency to perseverate on a particular behaviour by those with ASD extends to a tendency to keep employing previous instructions, even when they are at odds with the experienced contingency. The stronger the verbal ability, the stronger this tendency, and the more errors are made. This would certainly corroborate the findings reported by Kuzmanovic et al. (2011) using a very different task, but this explanation remains to be explored. The fact that verbal feedback is less effective in producing good set-shifting performance on DCCS tasks gives some credence to suggestions that verbal mediation can be involved in recruitment of memory systems that aid set-shifting, but that those with ASD are less able to recruit such feedback to support their performance (Albrecht et al., 2014; Henson, 2005; Joseph et al., 2005). The weaker effect of verbal feedback on shifting performance for those with ASD may be important in the context of previous studies having employed verbal feedback as reward (Demurie et al., 2011; Reed et al., 2013).

Clinically, problems with shifting are associated with core problems for those with ASD (Gonzalez et al., 2019; Reed et al., 2013; Yerys et al., 2009). There is now, at least, one effective cognitive behavioural intervention for shifting for school-aged children (Kenworthy et al., 2014), and several novel interventions (Faja et al., 2022; Tamm et al., 2021). Delivering effective feedback is a key part of these programmes for ASD (Campbell et al., 2017; Demurie et al., 2011; Joseph et al., 2005; see Reed, 2016, for a review), and improving its efficacy may enhance such teaching strategies for behavioural flexibility. The use of nonverbal feedback strategies could, therefore, be considered in clinical and educational situations. For example, the use of nonverbal feedback may facilitate emergence of set-shifting and behavioural flexibility, which are important for everyday functioning (Gonzalez et al., 2019; Granader et al., 2014). Alternatively, training in the use of such verbal feedback may similarly promote set-shifting.

There are limits to the generalisations that can be made on the basis of these current data. For example, the sample employed was relatively high functioning and linguistically competent, and these findings may not generalise to other individuals with a different set of abilities. The task employed only reinforced correct choices, and it did not employ negative feedback for incorrect choices. Whether this alteration in the task has implications remains to be seen. Exploring a range of additional proactive interference tasks may be beneficial to see if the current findings would extend across all learning domains. Future research should aim to develop these findings, with a larger range of participants and tasks, to examine why the type of feedback given to children with ASD affects their behavioural flexibility.

Overall, this study supports and expands previous research on set-shifting ability in children with ASD. There was little difference of feedback type on initial set learning, but children with ASD exhibited difficulty in shifting this initial learning, which was worse when verbal feedback was used. This is a novel finding that has implications for slower set-shifting and for teaching strategies.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Phil Reed

References

Albrecht

Abeler

Weber

Falk

(2014). The brain correlates of the effects of monetary and verbal rewards on intrinsic motivation. Frontiers in Neuroscience, 8, Article 303.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). APA.

Barringer

Gholson

(1979). Effects of type and combination of feedback upon conceptual learning by children: Implications for research in academic learning. Review of Educational Research, 49(3), 459–478.

Berg

E. A.

(1948). A simple objective technique for measuring flexibility in thinking. The Journal of General Psychology, 39(1), 15–22.

Campbell

C. A.

Russo

Landry

Jankowska

A. M.

Stubbert

Jacques

Burack

J. A.

(2017). Nonverbal, rather than verbal, functioning may predict cognitive flexibility among persons with autism spectrum disorder: A preliminary study. Research in Autism Spectrum Disorders, 38, 19–25.

Demurie

Roeyers

Baeyens

Sonuga-Barke

(2011). Common alterations in sensitivity to type but not amount of reward in ADHD and autism spectrum disorders. Journal of Child Psychology and Psychiatry, 52(11), 1164–1173.

Dirks

Romero

Voorhies

Kupis

Nomi

J. S.

Dajani

D. R.

Odriozola

Burrows

C. A.

Beaumont

A. L.

Cardona

S. M.

Parlade

M. V.

Alessandri

Britton

J. C.

Uddin

L. Q.

(2020). Neural responses to a putative set-shifting task in children with autism spectrum disorder. Autism Research, 13(9), 1501–1515.

Dunn

L. M.

Dunn

L. M.

Whetton

Pintilie

(1982). British Picture Vocabulary Scale: Manual for long and short forms. NFER-NELSON.

Eaves

R. C.

Williams

T. O.

Woods-Groves

Fall

A. M.

(2006). Reliability and validity of the Pervasive Developmental Disorders Rating Scale and the Gilliam Autism Rating Scale. Education and Training in Developmental Disabilities, 41, 300–309.

10.

Faja

Clarkson

Gilbert

Vaidyanathan

Greco

Rueda

M. R.

Combita

L. M.

Driscoll

(2022). A preliminary randomized, controlled trial of executive function training for children with autism spectrum disorder. Autism, 26(2), 346–360.

11.

Frith

Happé

(1994). Autism: Beyond ‘theory of mind’. Cognition, 50(1-3), 115–132.

12.

Frye

Zelazo

P. D.

Palfai

(1995). Theory of mind and rule-based reasoning. Cognitive Development, 10(4), 483–527.

13.

Gonzalez-Barrero

A. M.

Nadig

A. S.

(2019). Can bilingualism mitigate set-shifting difficulties in children with autism spectrum disorders? Child Development, 90(4), 1043–1060.

14.

Granader

Wallace

G. L.

Hardy

K. K.

Yerys

B. E.

Lawson

R. A.

Rosenthal

Wills

M. C.

Dixon

Pandey

Penna

Schultz

R. T.

Schultz

R. T.

(2014). Characterizing the factor structure of parent reported executive function in autism spectrum disorders: The impact of cognitive inflexibility. Journal of Autism and Developmental Disorders, 44, 3056–3062.

15.

Henson

(2005). A mini-review of fMRI studies of human medial temporal lobe activity Associated with recognition memory. The Quarterly Journal of Experimental Psychology Section B, 58(3–4b), 340–360.

16.

Joseph

R. M.

Steele

S. D.

Meyer

Tager-Flusberg

(2005). Self-ordered pointing in children with autism: Failure to use verbal mediation in the service of working memory? Neuropsychologia, 43(10), 1400–1411.

17.

Kenworthy

Anthony

L. G.

Naiman

D. Q.

Cannon

Wills

M. C.

Luong-Tran

Werner

M. A.

Alexander

K. C.

Strang

Bal

Sokoloff

J. L.

Wallace

G. L.

(2014). Randomized controlled effectiveness trial of executive function intervention for children on the autism spectrum. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 55(4), 374–383.

18.

Kenworthy

Case

Harms

M. B.

Martin

Wallace

G. L.

(2010). Adaptive behavior ratings correlate with symptomatology and IQ among individuals with high functioning autism spectrum disorders. Journal of Autism and Developmental Disorders, 40(4), 416–423.

19.

Krug

D. A.

Arick

Almond

(1980). Behavior checklist for identifying severely handicapped individuals with high levels of autistic behavior. Child Psychology & Psychiatry & Allied Disciplines, 21, 221–229.

20.

Kuzmanovic

Schilbach

Lehnhardt

F. G.

Bente

Vogeley

(2011). A matter of words: Impact of verbal and nonverbal information on impression formation in high-functioning autism. Research in Autism Spectrum Disorders, 5(1), 604–613.

21.

Lai

C. L. E.

Lau

Lui

S. S. Y.

Lok

Tam

Chan

Cheng

K. M.

Cheng

K. M.

Siu Man Lam . (2017). Meta-analysis of neuropsychological measures of executive functioning in children and adolescents with high-functioning autism spectrum disorder. Autism Research, 10(5), 911–939.

22.

Leekam

S. R.

Prior

M. R.

Uljarevic

(2011). Restricted and repetitive behaviors in autism spectrum disorders: A review of research in the last decade. Psychological Bulletin, 137(4), 562–593.

23.

Penney

R. K.

Lupton

A. A.

(1961). Children’s discrimination learning as a function of reward and punishment. Journal of Comparative and Physiological Psychology, 54(4), 449–451.

24.

Reed

(2016). Interventions for autism: Evidence for educational and clinical practice. John Wiley & Sons.

25.

Reed

(2018). Behavioural flexibility of children with autism spectrum disorder on a card-sorting task with varying task difficulty. Heliyon, 4(10), Article e00842.

26.

Reed

Hawthorn

Bolger

Meredith

Bishop

(2012). Disrupted stimulus control but not reward sensitivity in individuals with autism spectrum disorders: A matching law analysis. Journal of Autism and Developmental Disorders, 42(11), 2393–2403.

27.

Reed

Watts

Truzoli

(2013). Flexibility in young people with autism spectrum disorders on a card sort task. Autism, 17(2), 162–171.

28.

Russo

Flanagan

Iarocci

Berringer

Zelazo

P. D.

Burack

J. A.

(2007). Deconstructing executive deficits among persons with autism: Implications for cognitive neuroscience. Brain and Cognition, 65(1), 77–86.

29.

Scott-Van Zeeland

A. A.

Dapretto

Ghahremani

D. G.

Poldrack

R. A.

Bookheimer

S. Y

. (2010). Reward processing in autism. Autism Research, 3(2), 53–67.

30.

Spence

J. T.

Segner

L. L.

(1967). Verbal versus nonverbal reinforcement combinations in the discrimination learning of middle-and lower-class children. Child Development, 29–38.

31.

Stevenson

H. W.

Weir

M. W.

Zigler

E. F.

(1959). Discrimination learning in children as a function of motive-incentive conditions. Psychological Reports, 5(1), 95–98.

32.

Tamm

Zoromski

A. K.

Kneeskern

E. E.

Patel

Lacey

H. M.

Vaughn

A. J.

Ciesielski

H. A.

Weadick

H. K.

Duncan

A. W.

(2021). Achieving independence and mastery in school: An open trial in the outpatient setting. Journal of Autism and Developmental Disorders, 51(5), 1705–1718.

33.

Wechsler

(2011). Wechsler Abbreviated Scale of Intelligence–Second Edition Manual. Pearson.

34.

White

S. J.

(2013). The triple I hypothesis: Taking another (‘s) perspective on executive dysfunction in autism. Journal of Autism and Developmental Disorders, 43(1), 114–121.

35.

Williams

Jarrold

(2013). Assessing planning and set-shifting abilities in autism: Are experimenter-administered and computerised versions of tasks equivalent? Autism Research, 6(6), 461–467.

36.

Yerys

B. E.

Antezana

Weinblatt

Jankowski

K. F.

Strang

Vaidya

C. J.

Schultz

R. T.

Gaillard

W. D.

Kenworthy

(2015). Neural correlates of set-shifting in children with autism. Autism Research, 8(4), 386–397.

37.

Yerys

B. E.

Wallace

G. L.

Harrison

Celano

M. J.

Giedd

J. N.

Kenworthy

L. E.

(2009). Set-shifting in children with autism spectrum disorders: Reversal shifting deficits on the Intradimensional/Extradimensional Shift Test correlate with repetitive behaviors. Autism, 13(5), 523–538.

38.

Yerys

B. E.

Wolff

B. C.

Moody

Pennington

B. F.

Hepburn

S. L.

(2012). Brief report: Impaired flexible item selection task (FIST) in school-age children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42(9), 2013–2020.

39.

Zelazo

P. D.

(2006). The Dimensional Change Card Sort (DCCS): A method of assessing executive function in children. Nature Protocols, 1(1), 297–301.