Abstract
Introduction
Clark, McEwan and Christie's systematic review 1 offers a timely examination of current literature assessing effects of a constraints-led approach (CLA) to training on ‘technical and cognitive skill in sport’, in comparison to traditional training methods. They concluded that, currently, there is strong evidence to advocate for the effects of training interventions that espouse benefits of constraints-led training on acquiring skill in interceptive actions. Clark, McEwan and Christie reported that 18 studies satisfied their proposed inclusion criteria and, of these studies, 77% provided evidence of the effectiveness of the CLA. Consequently, Clark, McEwan and Christie argued that a ‘the implementation of the constraints-led approach within interceptive sport can be advocated’ (p. 17). This is a revealing insight, which supports their claims that this finding ‘provides the opportunity for researchers to collect more compelling evidence to answer the question: “Does constraint-led training assist with the development of technical skills within interceptive sport?”’. While we endorse their call for more empirical evidence on the effectiveness of a CLA to practice and training design, we qualify it by highlighting some limitations of Clark, McEwan and Christie's systematic review.
In this commentary, we discuss key issues including lack of a complete assessment of methodological validity of the reviewed papers, inherent difficulties with the research methodologies employed in the studies, theoretical understanding to ensure that CLAs are distinguishable from other pedagogical approaches, veracity of the intervention in each study, sampling of the environment 2 and assessing performance, sample size (students versus elite populations), intervention length and further areas of research that need to be addressed.
Risk of bias and procedures for assessing methodological validity of studies
Abstract and keywords of the sample studies.
Manipulation of interacting constraints in an intervention or manipulation of independent variables in an experiment?
What does this initial statistical/methodological criticism imply practically? The implication is that the high risk of bias with regard to assessing study methodology in the systematic review by Clark, McEwan and Christie might have led to the de-selection of articles that did not investigate ‘technical development’ from a CLA embedded in the theoretical framework of ED. Essential to the evaluation of studies purporting to adopt the CLA is ensuring that each study assessed is not merely manipulating experimental variables but rather that the key principles of CLA underpin intervention design. For proper assessment with regard to a relevant theoretical rationale in ED, the term ‘constraints’ needs to be used in a theoretically specific way to refer to the boundaries which shape the emergence of coordinated behaviours 23 or to refer to information that drives the spontaneous self-organization tendencies in system dynamics.24–26
A CLA is a well-promoted framework for understanding how humans acquire and organize the necessary actions to successfully engage with sport and exercise contexts.6,27–29 The CLA articulates that, through the interaction of different constraints – task, environment, and organism – individuals will self-organize actions, perception and cognitions in an attempt to generate functional movement solutions. 27 ED is a theoretical framework that has evolved by interlacing the theories of dynamical systems and ecological psychology, which inform principles of a nonlinear pedagogy, in which the methodological nuances of a CLA are captured in learning design. 30 The fundamental relationship between theory and practice is not a trivial issue for philosophical reflection only, as James Gibson (the founder of ecological psychology) pointed out in drawing inspiration from the words of the Gestaltist, Kurt Lewin: ‘There is nothing so practical as a good theory’ 31 (p. 135). The ED framework illuminates the essential relationship between the learner and the environment as a key foundation of practice design and a theoretical tenet on which to consider skill acquisition processes. Adopting an ED approach drives practitioners to conceptualize learners as complex, adaptive dynamical systems, co-adapting with events, objects and significant others in an ever-changing performance environment. Through the practical articulation of key theoretical ideas of ED, guiding principles for the design of learning environments have been inferred, with relevant research still required to ‘frame’ the design of constraints-led practices. Philosophical and theoretical clarity has been provided by the extensive literature in the area of nonlinear pedagogy.6,29,30,32 Without a comprehensive assessment of the methodological quality of studies in adhering to a CLA, the conclusion remains that Clark, McEwan and Christie's systematic review is at high risk of bias. In practice, we are arguing that research should focus on the principled manipulation of candidate control parameters 33 (i.e., carefully chosen variables that drive changes in system order parameters), that will enhance understanding of interactive constraints in real world interventions. We argue that for high impact research and changes in competitive sport performance we must seek to develop and establish research methodologies that are true to the underpinning theoretical framework, such as representative design.
Comparison of effects of traditional pedagogies and constraints-led training approaches
Clark, McEwan and Christie proposed that a significant reason for their systematic review was, that ‘Currently, there is a lack of sufficient evidence to advocate whether the manipulation of specific task constraints benefit individuals more so than traditional training regimes’. Yet in their review, this comparison was not undertaken with respect to carrying out a separate systematic review of what could be defined as studies investigating the efficacy of traditional pedagogies. Indeed, in their review, no attempts were made to provide rigorous definitions of traditional skill learning practices, nor to compare these characteristics with constraints-based learning designs. This is an important challenge for future researchers in skill acquisition and sport pedagogy. First, clear definitions are needed to characterize different approaches and only then can rigorous assessment methodologies be undertaken to compare effectiveness of studies in different categories. For example, consider the work by Lee et al., 34 whose study comparing nonlinear and linear pedagogic approaches to motor skill acquisition provides a useful template for considering traditional vs. CLAs. The inherent intricacies in ‘measuring’ complex, emergent, adaptive, behaviours in skill performance should not serve as rationale for rejecting appropriate research methodologies, or indeed, serve as a rationale for rejecting an approach altogether. That we currently cannot effectively measure something, speaks more to the issue of methodological design and appropriate frameworks for representing a theoretical approach rather than it does for rejecting that approach as inappropriate for a field of study.
Representative learning design
Regardless of constraints manipulation, a major omission in the systematic review was the lack of evidence that studies were high in what Egon Brunswik 2 termed representative design. This is a major principle of ED for ensuring that task designs for learning and experiments contain relevant informational constraints to elicit the emergence of functional behaviours, as performers are drawn to exploit affordances available. 35 Representative task design 2 advocates the need to maintain action-fidelity. 36 Brunswik's 2 work has been adopted by ecological dynamicists,37,38 especially his request to sample performance contexts in the same way as researchers have traditionally considered the sampling of participants. Consequently, researchers and practitioners need to sample practice and experimental environments to ensure they have similar information flows to a performance environment, making them more representative and maintaining greater action fidelity. The concept of representative learning design (RLD) calls into question the value of practice task designs that are decontextualized through artificiality and reductionism (potentially breaking the coupling of perception and action systems) in a performance environment. To exemplify, in practical learning interventions, it is important not to design an environment that requires learners to dribble around cones or manikins – with the aim of creating realism. Without information from movements and locations of opposing defenders, spatial (line markings) or temporal (tempo of a ball feed) informational constraints (to exemplify), there will be little strengthening of the functional perception-action couplings required in skilled performance. While it is clear that further work is needed in developing clarity for practitioners in RLD, it is a key theoretical construct that should not be overlooked when considering criteria for reviewing effectiveness of interventions in research. The transfer of skill learning from training to performance environments is central to any coaching endeavour. We argue that RLD embedded in a CLA allows practitioners to develop experiences during practice that are more likely to positively influence learning (and therefore performance). This involves moving the emphasis away from the memorization of technique through isolated drills and onto the skilful execution of functionally relevant actions in a performance context.
Choice of interceptive actions only
An interesting question concerns the choice of interceptive actions by Clark, McEwan and Christie as the specific research domain for their systematic review, rather than sports in general such as sprinting, weightlifting, rowing and climbing. Key to this question is our assertion that this approach (choosing interceptive actions only) assumes that domain-specific actions can be studied in isolation, which is incompatible with the CLA. The issue of assessing the effectiveness of specific approaches to learning is important for the study of skill acquisition more generally, not just in interceptive actions, and not just when using constraints-based methodologies. The framing of the systematic review around studies of interceptive actions needs a comprehensive rationale for its selection. Additionally, interceptive actions include a much wider range of activities than those covered in the systematic review to include all sporting activities, as highlighted by the large number of studies (not intervention studies) that have ED as the underpinning theoretical model (e.g. athletics hurdling, 39 cricket bowling,40,41 diving, 42 rock climbing, 43 swimming). 44 Perhaps, Clark, McEwan and Christie have confused the CLA with other Games-Based pedagogies – which is a point we addressed in another paper, as a result of an uninformed reviewer's comment. 45 It is important to re-iterate our argument that studies purporting to use a CLA need verification of methodology by assessing that the theoretical principles of ED underpin the rationale in a study. Although the sample of only 18 papers in this systematic review is not a methodological issue, it does draw attention to the lack of literature available on interventions and it is, therefore, surprising that Clark, McEwan and Christie chose not to widen their range of analysis beyond interceptive actions. Regardless, it is worth reiterating a key finding of this paper that future research is required to determine the effectiveness of constraint-led training to include all areas of skill learning in a range of different sports and not just on performance of a limited range of interceptive actions. Significantly, this is not just of relevance for a CLA but is a major issue for developing our understanding of traditional approaches to learning sport skills, as well as frameworks like TGfU, schema theory, closed-loop control, variability of practice, contextual interference and the specificity of learning hypotheses, for which there have been no recent systematic or quantitative reviews conducted.
Further, it is not clear that quantitative reviews of experimental studies are the most appropriate way to engage with evidence on effectiveness of learning interventions, which is a major assumption behind the paper by Clark, McEwan and Christie. The inherent belief seems to be that a classical experimental design is best for examining skill acquisition in sport using constraints-based methodologies. We believe it is worth challenging this ideology. A positivist approach to experimental design is employed by all of the studies reviewed in their systematic review. With the aim of establishing reliable results, a reductionist approach to the control of methodological research design is common but not necessarily appropriate in all instances. By removing the inherent representative variability required in studies from the measurement of key dependent variables, researchers attempt to ensure that the experimental conditions are similar between the pre- and post-intervention trials. The key driver behind employing a constraints-based methodology is to create training environments that are representative of a specific performance environment in order to enhance the transfer of skill learning between practice simulations and performance. The enhanced ‘chaos’ and variability of a performance setting are important to replicate in practice design in order to enhance fidelity between training and performing contexts. This foundational idea in ED raises questions on measuring the impact of a constraints-led intervention in an environment designed for experimental control and may not be representative of a performance environment. The stable and predictable nature of such experimental designs (for the purposes of control) is a significant issue for practitioners as removing key affordances might diminish the purposefulness of the training intervention itself. In order to truly respond to the intrinsic variability of most sports performance environments and conduct real world, messy research, there is a need for a paradigm shift towards the use of methods that capture more qualitative information, in combination with quantitative methods. 46 Such a mixed-methods design might best serve the purposes of methodological evaluation in sport pedagogy.
Future studies should explore the use of rich and varied research methodologies such as single-case study analysis, 47 longitudinal designs and the analysis of action research interventions. 48 A mixed-method approach may be best suited for capturing skill acquisition processes over different timescales and the challenging issue of transfer fidelity from a range of learning environments to a competitive performance context. A good example here is the basketball study of Oudejans et al. 49 who adopted a group and single-case study design to study sports performers over a complete season.
Sample size and participants
Clark, McEwan and Christie questioned the low sample size in the studies in the systematic review with more than half of the studies having relatively small sample sizes of 6 to 10 participants per group and suggested that this may have had an impact on the reliability of the results. Sample size is a key challenge, especially if we want to work in messy, noisy, real-world, competitive sporting environments. However, when traditional experimental designs are seen as the gold-standard for research, there is an over emphasis on use of laboratory conditions for experiments and undergraduate students as participants. Research studies with elite athletes or developing experts, by virtue of their talent in adapting to challenging performance environments, are few and far between. Of course, they are worthy of study, despite obvious challenges in access to, or recruitment of this population.
The good news for skill acquisition scientists is that advances in technology, and also greater acceptance of evaluating differing types of knowledge, means we have moved beyond traditional research designs. We are now able to collect data in environments that best replicate settings that people normally learn, practice, perform, compete, act or participate in. The key concern then becomes what information to collect from the vast plethora of information that can be collected. 50 Collecting data in complex sporting environments often require a distinct approach to traditional hypothesis-based, experimental design, and the utilization of mixed methods may be entirely warranted. 46 This does not, however, mean that research of this type can abdicate itself from principled research design to test ideas. Instead, we argue that the approaches we are advocating must still be accountable to acceptable levels of scientific rigour. Rather, the paradigmatic assumptions of positivistic research methods are simply one way of developing knowledge, and there is a need for valid and reliable methods to access rich and varied forms of information available on pedagogical methodologies implemented in learning design.
Intervention length
The length of the training interventions employed in some of the reviewed studies is also questionable, but for some time this has been recognized as an inherent problem with the ubiquitous ‘6-week training study’ prevalent across all of the sport science disciplines. 51 Within skill acquisition research, intervention studies rarely use training periods longer than nine weeks. 49 In the sample of the systematic review, the studies by Masters et al., 10 Hagemann et al. 8 and Williams et al. 18 consisted of only a single session, while the longest study reviewed consisted of 45 sessions over a nine-week period table tennis. 13 Such short periods of training are unlikely to produce a change in performance, let alone a measurable one. Previous research within the ED realm has highlighted that learning can take place over different time scales. For example, attunement to a key informational constraint can lead to almost immediate improvements in performance (e.g. the cricket batter who is suddenly able to identify the wrist spin bowler's googly from the changes in his or her bowling action). 52 Alternately, changes can be more medium or long term, with varied learning trajectories. 53 In fact, the way athletes react to any intervention is likely to be specific to each individual and their developmental history. This point also highlights the limitations of traditional group designs where individual responses can be masked and emphasizes the need to move to research methodologies more in tune with the key ideas of ED and complex systems in general 50 for an extended discussion of how to capture expertise in real world settings.
The role of experiential knowledge in assessing effectiveness of learning designs
Elsewhere, it has been argued that many coaches and sport pedagogues implement a version of a constraints-based approach in their practice task design, which might be enhanced by a greater understanding of the theoretical concepts of ED.54,55 These studies have revealed the potential value of elite coaches’ experiential knowledge in understanding how to design training interventions, an often-overlooked source of knowledge. We have made the call for the experiential knowledge of coaches to be acknowledged, emphasizing the need for coaches and sport scientists to work together. 56 The value and role of experiential knowledge of coaches have often been neglected largely because of the inability to ‘collect’ data through classical experimental designs because of the inherent complexity of expertise or knowledge in coaching. A number of programmes of work are emerging that have meshed qualitative and quantitative research findings to enhance our understanding of expertise in sport. Pluijms et al. 56 in sailing and the PhD programmes of Sarah-Kate Millar 57 in rowing and Dan Greenwood 58 in sport run-ups. These innovative approaches need to be continued in further work. Future quantitative reviews need to also consider a range of different data sources, rather than simply sample experimental studies in the scientific literature. To exemplify, a blog by Brendan Nel 59 recently highlighted how the coach (Swys de Bruin) of the South African Super 16s franchise, the Lions, encouraged his players to enhance their adaptive variability and seek affordances from the opposition (‘what they offer’) to decide emergent game strategies.
Furthermore, what are we to make of the post on the blog, Connected Coaches, by Blake Richardson 60 outlining evidence behind Coach of the Year, Danny Kerry's, success at the 2016 Olympic Games in leading Team GB to an unexpected gold medal in field hockey? Important ‘watch words’ in the successful pedagogical practice included a CLA. Perhaps questions about understanding the efficacy of a CLA to skill acquisition need to also consider opinions in a football coaching blog by Richard Allen 61 asking: Do we really know how to utilize the CLA?
This criticism of Clark, McEwan and Christie's restricted approach in focusing on evidence of the CLA from scientific experiments only, and ignoring the voices of practitioners as stakeholders who regularly use the CLA in practice task design, does raise an interesting challenge for future research: How to assess the value of information from blogs, podcasts and media articles expounding the effectiveness of constraints-based coaching approaches used at an elite level of performance? Here, we have argued that sport science can no longer simply ignore this body of information because it has not been generated in controlled laboratory experiments.
Conclusion and recommendations for future learning studies utilizing a CLA
Clark, McEwan and Christie rightly, in our view, highlighted the need for more research examining the efficacy of constraints-based interventions in sport training and practice. However, this is an issue challenging skill acquisition theories and sport pedagogical frameworks across the board and is not just of relevance to practitioners and scientists interested in the CLA. A systematic review will only provide an analysis of the quality of findings from experimental studies of skill learning, some of which may contain reductionist methods less suited to providing the quality of evidence needed on intervention efficacy from a range of different sources. These include, but are not limited to, experiential knowledge of elite practitioners and athletes as well as information from action-based research in which researchers are embedded in sports training environments. A particular challenge here is to also ascertain the quality of information evident in digital media such as blogs, websites and podcasts – a constant source of coaching information. With regard to the specific systematic review undertaken by Clark, McEwan and Christie, there are a number of factors addressed in this commentary that highlight some of the potential limitations of the studies and the conclusions of their review. Key for us was the need to ensure that studies reviewed should evidence the theoretical framework of ED as the rationale underpinning the research.
Key principles for researchers interested in adopting CLA to inform future directions of research and practice, include the following:
CLA is applicable to all sports and physical activities not just those that contain interceptive actions. As CLA is based on an ED theoretical rationale, the methodologies of the reviewed studies also need to be consistent with that specific framework. Longitudinal Studies: Future research designs on the CLA need to track skill learning not only over days, weeks or months but also to include research questions that capture competitive cycles over seasons and years. Use of Individual or Multiple Baseline Methodologies: Traditional group-based designs with control groups are not necessarily the most appropriate when implementing a CLA in terms of the theoretical concepts or the ethics of impacting athlete's careers. This is especially the case when it comes to assessing impact of CLA interventions on elite and developing expert athletes. It is simply not feasible to undertake experimental manipulations with such groups. Representative ‘Testing’: Appropriate measurement of interventions should be developed that utilizes the knowledge of practitioners and scientists Participants and Sample Sizes: By definition, if we want to move away from ‘student’ populations and test the effectiveness of CLA in sports performance settings, interventions need to take place in the messy, noisy world of competitive sports performance. Robust Environment Design: Frameworks are required to bridge the gap between the theoretical understanding and its practical application. These will acts as a guidance tool for practitioners and researchers to ensure they are designing environments consistent with the underpinning principles of ED.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.