An Intervention-Related Comparison of Preschoolers’ Scores on the TGMD-2 and TGMD-3

Abstract

This study compared preschoolers’ fundamental motor skills (FMS) on the Test of Gross Motor Development 2nd and 3rd editions (TGMD-2 and TGMD-3) before (pretest) and after (posttest) a motor skill intervention. This study also compared FMS changes (TGMD raw and percentile scores for total score, and locomotor and object control/ball subscales) across the intervention when FMS were measured using the two editions. Sixty-four preschoolers (M_age = 4.4, SD = 0.44 years) completed the TGMD-2 and TGMD-3 before and after a motor skill intervention. We determined the level of agreement between assessments using intra-class correlations (ICC), Lin’s Concordance Correlation Coefficient, and Bland-Altman plots. Overall, our results support an acceptable relative agreement between the TGMD-2 and TGMD-3 for raw scores, but there were variable strengths of agreement (by subscales) for percentile scores. Both assessments supported the efficacy of the intervention. This consistent demonstration of efficacy but varied strength of agreement suggests that the TGMD-2 and TGMD-3 are similar but not interchangeable.

Keywords

motor development motor skills pediatrics intervention

Introduction

Children’s fundamental motor skills (FMS) develop between ages 3-6 years and serve as building blocks for more advanced movement (Clark & Metcalfe, 2002). FMS divide into three categories: locomotor (ability to propel the body through space), object control (ball skills; ability to propel or manipulate objects in space), and stability skills (ability to maintain posture; Gallahue et al., 2012). Past literature supports FMS as an essential component of developmental trajectories of human health (Robinson et al., 2015; Stodden et al., 2008). These skills are inversely related to weight status (D’Hondt et al., 2011) and positively related to physical activity (Jaakkola et al., 2016; Lubans et al., 2010), physical fitness (Cattuzzo et al., 2016), perceived motor competence (Robinson, 2011), and cognitive outcomes in young children (van der Fels et al., 2015). Their importance supports an ongoing need for valid and reliable FMS assessment tools for children.

The Test of Gross Motor Development (TGMD) is among the most widely used FMS measures (Ulrich, 2017). Designed to provide researchers and teachers with a valid, reliable, and well-normed assessment of gross motor skills in children aged 3-10 years (Ulrich, 2000, 2019), the TGMD uses a criterion-based scoring system to evaluate locomotor and object control (ball) skills. Each skill is broken down into 3-5 specific skill criteria, and skill performance trials are scored according to the presence or absence of each specific skill criterion. The test developer has regularly revised and standardized the TGMD, meeting best practice recommendations for educational and psychological testing (Ulrich, 2017). Accordingly, Ulrich released the first edition of the TGMD in 1985, the second edition in 2000, and the third edition in 2019 (Ulrich & Sanford, 1985; Ulrich, 2000, 2019). Adjustments to the TGMD-3 include modified skill criteria and skill changes based on feedback from TGMD researchers and practitioners (Ulrich, 2017, 2019).

As the field transitions from the TGMD-2 to the TGMD-3, test users need to understand how these two editions compare. Field et al. (2019) examined the comparability of the TGMD-2 and TGMD-3 among 3^rd -5^th graders (8-10-year-old children) and found these test versions to be similar. However, the TGMD-3 had a slightly lower percentage of maximum scores. Field et al. (2019) also reported that children’s scores on locomotor subtests of both versions progressed over time, whereas changes on the TGMD-2 object control subtests scores did not progress. Authors attributed the latter finding to the inclusion of the roll on the TGMD-2 object control subtest but not on the TGMD-3 ball skills subtest. This research concluded that the TGMD-2 and TGMD-3 are similar but not equal for 8-10-year-old children. Field et al. (2019) provided useful insight into the comparability of the TGMD-2 and TGMD-3 at a single time point and with children at the upper end of the TGMD age band. However, there remains a need for more research on the two versions’ comparability at different ages and in different contexts.

The TGMD has five primary purposes: (a) identification and screening of children with developmental disabilities, (b) inform motor skill instruction, (c) assess individual progress, (d) provide a research tool, and (e) evaluate program effectiveness (pp 5-6; Ulrich, 2019). The last purpose, to evaluate program effectiveness, serves an important role in the motor skill intervention literature. Meta-analyses and systematic reviews report that the TGMD-2 has been the primary assessment for measuring FMS changes following interventions (Logan et al., 2011; Palmer, Chinn, et al., 2017; Wick et al., 2017). Following the release of the TGMD-3, more recent research used this latest edition to evaluate motor skill intervention or programs (Brusseau et al., 2018; Chan et al., 2016; Hulteen et al., 2015; Palmer et al., 2020). As the field shifts to the TGMD-3, research on how these editions compare in different age groups and within different intervention contexts will help with the comparability of the results of newer and older intervention studies. For example, the Children’s Health Activity Motor Program (CHAMP) is a well-established intervention for improving children’s motor skills (Robinson & Goodway, 2009; Robinson et al., 2016, 2017; Veldman et al., 2016), perceived competence (Robinson et al., 2009), physical activity (Palmer, Matsuyama, et al., 2017; Robinson et al., 2018), and self-regulation (Robinson et al., 2016). Most of the supporting research for CHAMP was based on the TGMD-2 (Robinson & Goodway, 2009; Robinson et al., 2016, 2017; Veldman et al., 2016). While there is some newer research showing that CHAMP improved preschoolers’ motor skills when FMS were assessed with the TGMD-3 (Palmer et al., 2019), it is difficult to compare the earlier and more recent work without a clearer understanding of how the TGMD-2 and TGMD-3 compare with preschoolers in different intervention contexts.

In the present study, we compared preschoolers’ raw and percentile scores on both the TGMD-2 and TGMD-3 before (pretest) and after (posttest) a 5-week, 600-minute motor skill intervention, and we compared the children’s FMS change scores across the intervention. We hypothesized that preschoolers’ FMS raw and percentile scores on the TGMD-2 and TGMD-3 would be similar at both pretest and posttest and that the preschoolers’ FMS change scores on these two editions from pretest to posttest would be similar.

Method

Participants

A total of 67 preschoolers (24 girls; M_age = 4.4, SD = 0.44 years) served as participants. All participants were from a single Head Start center, the largest federally funded early childhood program serving low-income children, in a large Midwestern city in the United States. All preschoolers in the Head Start Center who were between 3.5-5 years of age were eligible to participate in the study. All preschoolers completed the motor skill intervention. An Institutional Review Board approved all experimental procedures, and both parental written consent through signed-letters and child verbal assent were obtained before the start of the research project.

FMS Assessments

We assessed FMS using the TGMD 2^nd and 3^rd editions (Ulrich, 2000, 2019). As noted, the TGMD is a criterion- and norm-referenced standardized assessment used to measure FMS in children aged 3-10 years. The TGMD-2 assesses six locomotor skills (run, jump, leap, hop, gallop, and slide) and six object control skills (overarm throw, strike off a tee, catch, kick, roll, and dribble). The TGMD-2 has shown high test-retest reliability for both the locomotor (0.88) and object control (0.93) subtests (Ulrich, 2000). The TGMD-3 is the revised version of the TGMD-2. The TGMD-3 assesses six locomotor skills (run, jump, skip, slide, hop, and gallop) and seven ball skills (strike off a tee, one-hand strike, dribble, catch, kick, overarm throw, and underhand throw). The TGMD-3 has also demonstrated high test-retest reliability for both the locomotor (0.93) and object control (0.87) subtests for preschoolers (Ulrich, 2019). Changes from the TGMD-2 to TGMD-3 include three new skills (skip, underhand throw, and one-hand strike), removal of two skills (leap and roll), and revision of skill criteria for six skills (gallop, hop, strike off a tee, kick, dribble, and overarm throw; see Figure 1).

Figure 1.

Skills Included on the TGMD-2 and TGMD-3.

We administered all TGMD skills per standardized protocols as detailed in the test manuals (Ulrich, 2000, 2019). Before testing, preschoolers received a demonstration of how to complete each skill. Children then completed a practice trial, and, if a child failed to demonstrate skill understanding during the practice trial, we provided a second demonstration. We provided all demonstrations using prerecorded videos to ensure identical verbal and visual demonstrations (Robinson et al., 2015). After either the practice trial or the second demonstration, the child completed two test trials. All test trials were video-recorded and then coded. Each skill had 3-5 specific performance criteria, and children received a sum score based on the number of correctly executed criteria in each skill performance (1 = correctly executed, 0 = missing/incorrectly executed). Scores from each subtest can be used as raw subtest scores or combined to produce a total raw score. Total raw scores on the TGMD-2 range from 0-96, and for the TGMD-3 range from 0-100. The raw subtest scores and the total raw scores for the TGMD-2 and TGMD-3 were converted to percentile scores using the standardization data for each version of the assessment (Ulrich, 2000, 2019). All motor skill assessments were digitally recorded and later coded by a blinded researcher. The coder was a motor development expert with graduate training in motor development and four years of prior experience with the TGMD. The coder underwent in-depth training sessions with other motor development experts biannually and had a previously established inter-rater reliability of 97% with three experts on both the TGMD-2 and TGMD-3. We coded skills that differed between the TGMD-2 and TGMD-3 on separate days to ensure we could objectively observe and accurately code any small changes in the criteria.

Motor Skill Intervention

The Children’s Health Activity Motor Program (CHAMP) served as the motor skill intervention for this study. CHAMP is a mastery-focused, evidence-based intervention that enhances motor skills (Robinson & Goodway, 2009; Robinson et al., 2016, 2017; Veldman et al., 2016), perceived competence (Robinson et al., 2009), physical activity (Palmer, Matsuyama, et al., 2017; Robinson et al., 2018), and self-regulation (Robinson et al., 2016) in young children. CHAMP is grounded in Achievement Goal Theory (Ames, 1992) and adheres to the TARGET structures (task, authority, recognition, grouping, evaluation, time; Epstein, 1989). CHAMP draws on effective instructional pedagogies from the motor development and physical education literature and focuses on critical elements and cue words, effective modeling and demonstration, continuous and appropriate feedback, and repetitive cycles of motor skills and tasks. The intervention consisted of 15, 40-minute CHAMP sessions over five weeks. Each session included a 2-minute warm-up; 3 - 4 minutes of motor skill introductions including skill demonstration, modeling and cue words; 20 - 25 minutes of autonomy-based motor skill engagement, 5 - 7 minutes of a large group activity, and 2 - 3 minutes of a closure activity that reinforced the critical elements and cue words for the motor tasks. The CHAMP intervention used in the current study included instruction on all fifteen skills across the TGMD-2 and TGMD-3. We allocated equal instructional and skill practice time to each skill across the five weeks. For a more detailed CHAMP description, see previously published work (Robinson & Goodway, 2009; Robinson et al., 2016, 2017; Robinson et al., 2020; Veldman et al., 2016).

Procedures

Preschoolers replaced their outdoor recess with CHAMP three days/week for five weeks, and they participated in the normal outdoor recess on the remaining two days/week. The CHAMP intervention’s total dose was 15, 40-minute sessions (600 total minutes with 300-375 minutes motor skill practice). All preschoolers completed the TGMD-2 and TGMD-3 before the start (pretest) and after (posttest) the intervention. We administered the TGMD to include the skills on both editions of the TGMD (15 skills total), and children completed all motor skills in a single session (see Figure 1). In comparing the administrations of the TGMD-2 and TGMD-3, the TGMD-3 now requires a small administration change in one skill (i.e., hop). To account for this change, all children performed the hop a second time at the end of the testing session. We counterbalanced the second administration of the hop so that half the children randomly completed the TGMD-2 hop protocol or the TGMD-3 hop protocol at the end of the session. We coded all other skills that vary between the TGMD-2 and TGMD-3 administrations twice to account for changes in skill criteria.

Data Analysis

For all inference tests, we set statistical significance at p < 0.05 a priori. To ensure effectiveness of the CHAMP intervention, we conducted paired-samples t-tests for pretest and posttest on locomotor, object control/ball skills, and total raw and percentile scores for each test edition (TGMD-2 and TGMD-3). We applied a Bonferroni correction to mitigate the risk of making a Type 1 error. We applied the correction for each type of score, so the final p-values were 0.008 (0.05/6) for all t-tests. We assessed relative agreement between the TGMD-2 and TGMD-3 using two-way mixed intra-class correlations (ICC; Ranganathan et al., 2017) and Lin’s Concordance Correlation Coefficients (CCC; Lawrence & Lin, 1989; Lin, 2000). An ICC value of > 0.70 was considered acceptable (Sim & Wright, 2000). Lin’s CCC were interpreted as > 0.80, “strong”, 0.60-0.80 “moderate”, 0.20 - 0.40 “weak” and <0.20 “poor” (Altman, 1991). We calculated ICCs in SPSS v 24 and Lin’s CCCs in Excel 2019. We completed all assessments for pretest, posttest, and change scores using both raw and percentile scores.

We examined agreement between the TGMD-2 and TGMD-3 raw and percentile scores using Bland-Altman plots. Bland-Altman plots are scatterplots created by plotting the mean difference scores (TGMD-2 – TGMD-3) on the y-axis and average scores (TGMD-2 + TGMD-3/2) on the x-axis. We created twelve plots in all: raw and percentile total, locomotor, and object control/ball skills before, after, and across the intervention (i.e., FMS change scores). We inspected plots for three elements to determine the agreement between assessments: (a) size of discrepancies between assessments as measured by the mean difference score, (b) size of agreement between assessments as measured by the size of the 95% confidence interval, and (c) systematic changes in variability in difference scores or mean scores (Giavarina, 2015). We considered plots with a mean difference score above or below zero, large 95% confidence intervals, or systematic changes in variability and outliers indicators of poor agreement between the TGMD-2 and TGMD-3. We selected these reliability metrics as these techniques are appropriate based on the research questions (e.g., comparability of two assessments) and type of data presented (e.g., continuous vs. categorical data; Giavarina, 2015; Lawrence & Lin, 1989; Lin, 2000; Ranganathan et al., 2017).

Results

Sample

Our final sample included 64 participants (24 girls; M _age = 4.4, SD = 0.44 years). Three preschoolers were excluded from the analysis because they withdrew from the center and did not complete the CHAMP program nor the posttest assessment.

Intervention Efficacy

See Tables 1 and 2 for the means and standard deviations of TGMD raw and percentile scores. Using the TGMD-2, preschoolers had higher posttest than pretest raw scores for total (M = 18.34, SD = 9.51 vs. M = 63.86, SD = 13.39; t(63) = 37.12, p < 0.001), locomotor (M = 9.05, SD = 5.30 vs. M = 30.84, SD = 7.49; t(63) = 30.11, p < 0.001), and object control (M = 9.30, SD = 5.47 vs. M = 33.02, SD = 6.86; t(63) = 31.72, p < 0.001). Similarly, preschoolers had higher TGMD-2 percentiles scores at posttest than pretest for total (M = 2.85, SD = 4.52 vs. M = 74.80, SD = 24.99; t(63) = 24.51, p < 0.001), locomotor (M = 4.27, SD = 5.36 vs. M = 63.26, SD = 24.54; t(63) = 21.17, p < 0.001), and object control (M = 7.43, SD = 8.96 vs. M = 79.12, SD = 20.95; t(63) = 29.91, p < 0.001). Using the TGMD-3, preschoolers had higher posttest than pretest scores for total (M = 18.44, SD = 9.82 vs. M = 63.83, SD =14.72; t(63) = 36.74, p < 0.001), locomotor (M = 9.09, SD = 5.50 vs. M = 29.89, SD = 7.35; t(63) = 30.63, p < 0.001), and object control (M = 9.34, SD = 5.98 vs. M = 33.94, SD = 8.96; t(63) = 27.56, p < 0.001). Similarly, preschoolers had higher TGMD-3 percentiles scores at posttest than pretest for total (M = 7.06, SD = 9.74 vs. M = 81.85, SD = 21.50; t(63) = 30.11, p < 0.001), locomotor (M = 9.25, SD = 10.12 vs. M = 77.09, SD = 21.89; t(63) = 27.54, p < 0.001), and object control (M = 11.56, SD =15.18 vs. M = 81.60, SD = 21.40; t(63) = 26.17, p < 0.001).

Table 1.

Descriptive Statistics and t-Scores (Pretest to Posttest) for Raw TGMD-2 and TGMD-2 Scores at All Time Points.

Version	Pretest		Posttest		Change
Version	M (raw)	SD	M (raw)	SD	M (raw)	SD	t ^a
LM
2	9.05	5.30	30.84	7.49	21.80	5.79	30.11***
3	9.09	5.50	29.89	7.35	20.55	7.00	30.63***
Diff	−0.05	0.44	0.95	2.67	1.25	6.58
Avg	9.07	5.23	30.67	7.30	21.17	5.52
OC
2	9.30	5.47	33.02	6.86	23.72	5.98	31.72***
3	9.34	5.98	33.94	8.96	24.59	7.14	27.56***
Diff	−0.05	4.45	−0.92	4.55	−0.88	6.29
Avg	9.32	5.28	33.48	7.65	24.16	5.79
Total
2	18.34	9.51	63.86	13.39	45.52	9.80	37.12***
3	18.44	9.82	63.83	14.72	45.39	9.88	36.74***
Diff	−0.09	5.06	0.03	5.50	0.13	7.36
Avg	18.39	9.26	63.84	13.80	45.45	9.13

LM= locomotor; OC= object control; ^a = pretest vs. posttest; *** denotes p < 0.001.

Table 2.

Descriptive Statistics and t-Scores (Pretest to Posttest) for Percentile TGMD-2 and TGMD-2 Scores at All Time Points.

Version	Pretest		Posttest		Change
Version	M (%)	SD	M (%)	SD	M (%)	SD	t ^a
LM
2	4.27	5.36	63.26	24.54	58.99	22.29	21.17***
3	9.25	10.12	77.09	21.89	67.84	19.70	27.54***
Diff	−4.98	7.85	−13.83	11.58	−8.84	14.79
Avg	6.75	7.08	70.17	22.52	63.41	19.70
OC
2	7.43	8.96	79.12	20.95	71.69	19.17	29.91***
3	11.56	15.18	81.60	21.40	70.04	21.41	26.17***
Diff	−4.13	12.46	−2.48	20.98	1.65	16.30
Avg	9.50	10.80	80.36	20.54	70.86	18.62
Total
2	2.85	4.52	74.80	24.99	71.95	23.49	24.51***
3	7.06	9.74	81.85	21.50	74.79	19.87	30.11***
Diff	−4.21	6.64	−7.05	11.04	−2.84	15.91
Avg	4.96	6.83	78.33	22.65	73.37	20.61

LM= locomotor; OC= object control; ^a = pretest vs. posttest; *** denotes p < 0.001.

Relative Agreement: ICCs and Lin’s CCC

Using raw scores, intra-class correlations revealed that the TGMD-2 and TGMD-3 had acceptable agreement (≥0.70) on all measured subtests except the locomotor change score (ICC = 0.64; see Table 3). Using percentile scores, intra-class correlations revealed that the TGMD-2 and TGMD-3 had acceptable agreement (≥0.70) on all measured subtests except the locomotor pretest score (ICC = 0.62) and the object control pretest score (ICC = 0.65; see Table 3).

Table 3.

ICC and Lin’s CCC Between the TGMD-2 and TGMD-3 Raw and Percentile Scores at Pretest, Posttest, and Across the Intervention.

	ICC		Lin’s CCC
	Raw scores	Percentile scores	Raw scores	Percentile scores
Pre
LM	0.94 (0.96–0.90)	0.62 (0.25–0.79)	0.88 (0.81–0.93)	0.45 (0.30–0.57)
OC	0.83 (0.89–0.71)	0.65 (0.42–0.79)	0.70 (0.55–0.80)	0.47 (0.31–0.61)
Total	0.93 (0.96–0.88)	0.70 (0.36–0.84)	0.86 (0.78–0.91)	0.53 (0.42–0.63)
Post
LM	0.96 (0.98–0.93)	0.85 (0.10–0.95)	0.93 (0.88–0.96)	0.74 (0.64–0.82)
OC	0.91 (0.95–0.85)	0.94 (0.89–0.96)	0.83 (0.75–0.89)	0.88 (0.80–0.92)
Total	0.96 (0.98–0.94)	0.92 (0.78–0.96)	0.92 (0.88–0.95)	0.85 (0.77–0.90)
Change
LM	0.64 (0.78–0.41)	0.82 (0.59–0.91)	0.75 (0.63–0.84)	0.69 (0.55–0.79)
OC	0.70 (0.82–0.52)	0.81 (0.69–0.88)	0.54 (0.35–0.69)	0.68 (0.52–0.79)
Total	0.84 (0.74–0.90)	0.88 (0.81–0.93)	0.72 (0.58–0.82)	0.79 (0.68–0.86)

Final score and 95% CI presented. LM= locomotor, OC= Object Control.

Using raw scores, Lin’s CCC revealed that the TGMD-2 and TGMD-3 had strong agreement (ρ_c ≥0.80) on all locomotor and object control subtests except the object control pretest score (ρ_c = 0.70). The raw change scores showed moderate agreement for locomotor (ρ_c = 0.75) and total (ρ_c = 0.72) scores but weak agreement for object control (ρ_c = 0.54). Using percentile scores, Lin’s CCC revealed that the TGMD-2 and TGMD-3 were in weak agreement for locomotor scores at pretest (ρ_c = 0.45), posttest (ρ_c = 0.47), and change scores (ρ_c = 0.53; see Table 3). There was moderate to strong agreement for object control percentile scores at pretest (ρ_c = 0.74), posttest (ρ_c = 0.88) and change (ρ_c = 0.85). Lastly, there was moderate agreement for total percentile scores at pretest (ρ_c = 0.69), posttest (ρ_c = 0.68), and change (ρ_c = 0.79). See Table three for all ρ_c values.

Bland-Altman Plots

See Figure 2 for Bland-Altman plots for raw TGMD scores. The Bland-Altman plots using the raw scores revealed good agreement between the TGMD-2 and TGMD-3 at pretest for total (M_diff = −0.09, 95% CI [−10.01, 9.82]), locomotor (M_diff = −0.05, 95% CI [−5.20, 5.10]), and object control/ball skills (M_diff = −0.05, 95% CI [−8.77, 8.68]). Similarly, the assessments had good agreement on total (M_diff = 0.03, 95% CI [−10.74, 10.80]), locomotor (M_diff = 0.95, 95% CI [−4.28, 6.18]), and object control/ball skills (M_diff = −0.95, 95% CI [−9.83, 7.99]) at posttest (see Figure 2). For change scores, there was overall fair agreement between assessments with a similar mean difference but a wide range on total (M_diff = 0.13, 95% CI [−14.31, 14.56]), locomotor (M_diff = 1.00, 95% CI [−6.52, 8.52]), and object control/ball skills (M_diff = −0.88, 95% CI [−13.20, 11.45]). A few outliers were also present in each plot (see Figure 2).

Figure 2.

Bland-Altman Plots for Raw TGMD-2 and TGMD-3 Scores Before, After, and Across the Intervention.

See Figure 3 for all Bland-Altman plots for percentile TGMD scores. The Bland-Altman plots using percentile scores revealed poor agreement between the TGMD-2 and TGMD-3 for total scores at pretest (M_diff = −4.21, 95% CI [−17.22, 8.80]) and posttest (M_diff = −7.05, 95% CI [−28.68, 14.60]), locomotor at pretest (M_{dif f} = −4.98, 95% CI [−20.36, 10.39]) and posttest (M_diff = −13.83, 95% CI [−36.53, 8.88]), and object control/ball skills at pretest (M_diff = −4.13, 95% CI [−28.56, 20.30]). There was fair agreement with large variability at posttest for object control/ball skills (M_diff = −2.48, 95% CI [−22.68, 17.71]). Change scores revealed fair agreement with large variability for total (M_diff = −2.84, 95% CI [−34.02, 28.35]) and object control/ball skills (M_diff = 1.65, 95% CI [−30.31, 33.60]; see Table 2), but poor agreement on locomotor (M_diff = −8.84, 95% CI [−37.83, 20.14]). Overall, the percentiles revealed large variability in the data, sizeable confidence intervals, and outliers (see Table 2 and Figure 3).

Figure 3.

Bland-Altman Plots for Percentile TGMD-2 and TGMD-3 Scores Before, After, and Across the Intervention.

Discussion

In 2019, a newer version of the TGMD was released, including the addition, removal, and modification of several skills, based on feedback from researchers and practitioners in the field (Ulrich, 2019). Despite the changes between the TGMD-2 and TGMD-3, limited research has compared these two assessments, especially with preschoolers. A recent study supported similarities between but not the interchangeability of the TGMD-2 and TGMD-3 when measuring FMS in older children (Field et al., 2019). To the best of our knowledge, this was the first study comparing preschoolers’ performance on raw and percentile scores on both editions of the TGMD in an intervention context. It is necessary to determine the comparability of these two editions in an intervention context as this is one of the most commonly used tools to measure FMS and evaluate children’s FMS changes across interventions (Logan et al., 2011; Palmer, Chinn, et al., 2017; Wick et al., 2017).

Our results show relative agreement in raw and percentile scores between the TGMD-2 and TGMD-3 both before and after the FMS intervention using ICCs. ICC values using raw scores at pretest and posttest were high, suggesting that the TGMD-2 and TGMD-3 raw scores are similar at either time point. Agreement between the assessments was lower when examining raw change scores; in particular, locomotor change scores assessed by the TGMD-2 and TGMD-3 did not achieve relative agreement. ICC values at posttest and change scores were high when using percentile scores. At pretest, ICC values for the percentile scores on both the locomotor and object control/ball skills subtests were below the acceptable threshold, but the agreement was acceptable for the total percentile score (ICC= 0.70). Interestingly, these results were only partially repeated when using Lin’s CCCs. The relative agreement from Lin’s CCC ranged from strong to weak, with the weakest values reported for locomotor percentile scores. There was a large difference between the ρ_c values reported for raw locomotor (ρ_c range = 0.70–0.88) as compared with the percentile locomotor (ρ_c range = 0.45 – 0.53). It is unclear why locomotor percentile scores showed such low agreement between assessments, but one possible explanation is the removal of the leap and addition of the skip may have implications on percentile but not raw scores for this population. In agreement with the ICC values, Lin’s CCC for locomotor and object control/ball skills were lower at pretest than other time points. These data may reflect a floor effect as percentile scores for both the TGMD-2 and TGMD-3 were low at this time point.

Similar to the Lin CCC values, the Bland-Altman plots did not always support agreement between the TGMD-2 and TGMD-3. The Bland-Altman plots examining raw scores demonstrated excellent agreement between the TGMD-2 and TGMD-3, as measured by mean difference scores (pretest, posttest, and change scores). All mean difference scores were between -1 and +1. A mean score around zero suggests that the TGMD-2 and TGMD-3 are reliable, and the variability in the difference of scores between each edition is not due to differences in the assessment tools (Giavarina, 2015). Conversely, Bland-Altman plots examining percentile scores reported lower agreement between the TGMD-2 and TGMD-3, as seen in negative mean difference scores. Therefore, children scored higher on the TGMD-3 percentiles compared with the TGMD-2. The mean difference percentile score between the TGMD-2 and TGMD-3 was relatively similar across the locomotor, object control/ball skill, and total at pretest (approx. −4.00), but this difference was more pronounced at posttest (range [−2.00 −13.00]), particularly for posttest locomotor skills (M_diff = 13.83). A possible explanation for large percentile differences and poor agreement may be due to individual skill changes among participants. We did not examine individual skills changes, and the percentile difference between the TGMD-2 and TGMD-3 at posttest may be due to changes in the skill/skill criterion on the different assessments and how children learned these skills across the intervention. For example, the skip is a more complex, continuous locomotor skill learned later in childhood. It is possible that preschoolers improved their performance of more complex skills (e.g., skip) but not their performance of more basic skills (e.g., leap), resulting in more points gained on the TGMD-3 at the posttest, thereby increasing posttest TGMD-3 locomotor percentiles. This rationale aligns with Field and colleagues (2019) who inferred that individual skills might under- or over-contribute to the final or subtest scores for the TGMD. Examining the contribution of individual skills fell outside the scope of the present investigation but maybe an area for future research. Lastly, Bland-Altman plots also revealed large variability in preschooler’s scores when using both raw and percentile scores. The variability in preschooler’s scores is unsurprising as the preschool years are a period of movement exploration and motor skill development, often accompanied by variability in FMS scores (Clark & Metcalfe, 2002; Gallahue et al., 2012).

We not did expect to see differences in agreement between the ICCs, Lin’s CCC, and Bland-Altman plots for the TGMD-2 and TGMD-3 percentile rankings. We provide two plausible explanations for this finding. First, differences in percentiles between the TGMD-2 and TGMD-3 may be due to differences in the normative samples. Both versions of the assessment are carefully normed to a sample that matches the U.S. national distribution of geographic region, race, educational level of parents, and household income level (Ulrich, 2000, 2019). However, it is possible that cohorts in the late 1990s (TGMD-2 normative sample) and mid-2010s (TGMD-3 normative sample) were different, and therefore, the distribution of normative data is different between these assessment editions. Previous research shows that preschoolers performed consistently on the standing long jump, a motor skill, between 1973 and 2007 (Roth et al., 2010); however, to our knowledge, differences between the normative samples on the TGMD have not been examined. Second, the TGMD-2 raw score is based on a maximum of 96 raw score points, whereas the TGMD-3 is based on a maximum of 100 raw score points. Our data support that preschoolers had similar raw score means on both the TGMD-2 and TGMD-3, but this mean raw score translates to different percentile ranks on the two editions. Interestingly, one might expect that these different ranges of possible raw score points would lead to greater score variability between the two editions on the object control/ball skills subtest (TGMD-3 range = 56 points vs. TGMD-2 range = 48 points) versus the locomotor subtest (TGMD-3 range = 44 points vs. TGMD-2 range = 48 points). Our results show an opposite pattern of a greater difference/variability in the locomotor percentile scores at posttest and change scores compared with the object control/ball skills subtest. Therefore, it appears that the new skills or changes to the object control/ball skills subtest in the TGMD-3 do not drive changes in percentile ranks in young children; rather, alterations to the locomotor subtest impact preschooler’s percentile ranks, particularly with respect to the effects of the CHAMP intervention. It is possible that differences in percentile ranks for the locomotor subtest after the intervention were driven by the exchange of the leap and skip skill on the TGMD-3. As aforementioned, skipping is a more complex skill. The intervention may have led to greater changes in this skill relative to the leap, which in turn translated to greater changes in percentile rankings for the locomotor subtest.

Overall, our results are consistent with and expand on Field et al.’s (2019) earlier findings with 8-10-year-olds in that the TGMD-2 and TGMD-3 were similar but not interchangeable. Despite differences between the two editions, it is important to note that both editions similarly evaluated intervention efficacy. The findings support some comparability between reports on intervention effects as measured by the TGMD-2 and TGMD-3. These findings are good news for researchers and practitioners as they transition to using the newer edition. Further, these data support that both versions of the TGMD would similarly screen young children for developmental delays despite differences in relative agreement reported by both the ICCs and Lin’s CCCs. Evidence from a recent multi-site study found that that 77% of children are at risk of motor delays (i.e., below the 25^th percentiles) and 30% of children had motor impairments (i.e., below the 5^th percentile) when motor skills were assessed by the TGMD (Brian et al., 2019). The high prevalence of gross motor delays supports the importance for measures to screen young children; therefore, the similarities between the TGMD-2 and TGMD-3 for evaluating motor delays in the current sample is meaningful for researchers and practitioners who use these measures to identify preschoolers with or at-risk for gross motor delays.

Limitation, Strengths, and Directions for Further Research

The present study is not without limitations. We collected all data from a single Head Start center in the Midwestern United States. Children from low-income families are known to have lower motor skills than children from higher-income families (Brian et al., 2019); therefore, future work should replicate the findings among children from different socioeconomic statuses and age groups in order to expand the generalizability of the results. Also, we did not calculate the sample size needed for the Bland-Altman plots a priori. Future work in this area should consider conducting an estimated sample size for Bland-Altman Plots (Lu et al., 2016). The preschoolers in the current study had low motor skills at the pretest evaluation, which may have created a floor effect. The relationship between the TGMD-2 and TGMD-3 might differ if children had better scores at the start of the study. Further, this study’s purpose was specific to the use of the TGMD within an intervention context; therefore, we did not include a control group, and could not examine the effects of development over time. Future work should include a control group to better understand how the TGMD-2 and TGMD-3 compare without the effects of an intervention in this population. Understanding how these two assessments align regarding longitudinal designs both with and without interventions is vital as current longitudinal designs use the TGMD-2 to evaluate motor skills across this period (Duncan et al., in 2021). As the field transitions to the new assessment, there is a need to understand how data from longitudinal design with the TGMD-2 align with the TGMD-3.

The present study also had several strengths. We used a previously established motor skill intervention, CHAMP, which included equal instruction for each of the fifteen skills scored by the TGMD-2 and TGMD-3. As such, changes in skills should not reflect differences in intervention instruction per skill. Also, our use of a 5-week, 600-minute intervention dosage. would be expected to be sufficient to enhance preschoolers’ motor skill competence (Robinson et al., 2016, 2017). All TGMD assessments were video recorded and later coded by a single, blinded expert coder. Work supports that inter-rater agreement between coders can range from acceptable but not ideal (i.e., kappa 0.67–0.77; Rintala et al., 2017) to acceptable and high (ICC 0.92–0.96; Maeng et al., 2017). Therefore, while it is possible to have high-reliability among expert coders, we eliminated all between-coder disagreements by using a single coder who had previously established sufficient inter-rater reliability.

Conclusion

FMS are important behaviors to measure in young children (Robinson et al., 2015; Stodden et al., 2008). These skills have been shown to predict health outcomes such as weight status, physical activity, and sedentary behavior (Duncan et al., 2021). One of the most commonly used assessments to measure FMS in young children is the TGMD. As researchers and practitioners transition from the 2^nd to the 3^rd edition of the TGMD, we attempted to help meet their need to understand how these two versions of the TGMD relate both at single time points and how these two editions evaluate FMS change across an intervention. Our results indicated that, while similar, the two measures differed in sensitivity to FMS performance changes elicited through a motor skill intervention with a pattern of results that aligned with previous research among older children (Field et al., 2019). Researchers should be cognizant of the differences between the TGMD-2 and TGMD-3 when using different editions to evaluate motor skill programming, and precautions are warranted when comparing current research using the TGMD-3 with past research using the TGMD-2, particularly at a single developmental time point. Future research is needed to examine how the TGMD-2 and TGMD-3 evaluate motor skills after interventions in different populations such as older children or children with disabilities.

Footnotes

Acknowledgments

The authors would like to thank Stephanie Palmer for her assistance with the preparation of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Kara K. Palmer

Supplemental material

Supplemental material for this article is available online.

Author Biographies

Kara K. Palmer: is a Clinical Assistant Professor and Research Scientist in the School of Kinesiology at the University of Michigan, Ann Arbor.

Katherine M. Chinn: was a Research Technician in the CMAH Laboratory when this project was completed. She is now completing her Physician Assistant degree at the University of Colorado.

Katherine Q. Scott-Andrews: is a PhD Candidate completing her PhD in Movement Science at the University of Michigan, Ann Arbor.

Leah E. Robinson: is a Professor in the School of Kinesiology at the University of Michigan, Ann Arbor and Director of the CMAH Laboratory.

References

Ames

(1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84(3), 261–271.

Brian, A., Pennell, A., Taunton, S., Starrett, A., Howard-Shaughnessy, C., Goodway, J. D., Wadsworth, D., Rudisill, M., & Stodden, D. (2019). Motor competence levels and developmental delay in early childhood: A multicenter cross-sectional study conducted in the USA. Sports Medicine, 49(10), 1609–1618.

Brusseau

T. A.

Hannon

J. C.

Fang

Nam

Goodrum

Burns

R. D.

(2018). Trends in physical activity, health-related fitness, and gross motor skills in children during a two-year comprehensive school physical activity program. Journal of Science and Medicine in Sport, 21(8), 828–832.

Cattuzzo, M. T., dos Santos Henrique, R., Ré, A. H. N., de Oliveira, I. S., Melo, B. M., de Sousa Moura, M., de Araújo, R. C., & Stodden, D. (2016). Motor competence and health related physical fitness in youth: A systematic review. Journal of Science and Medicine in Sport, 19(2), 123–129.

Chan

J. Y.

(2016). Improving fundamental movement skills in Hong Kong students through an assessment for learning intervention that emphasizes fun, mastery, and support: The a+ FMS randomized controlled trial study protocol. SpringerPlus, 5(1), 724–736.

Clark

J. E.

Metcalfe

J. S.

(2002). The Mountain of motor development: A metaphor. Motor Development: Research and Reviews, 2, 163–190.

D’Hondt

Deforche

Vaeyens

Vandorpe

Vandendriessche

Pion

, Philippaerts, R., De Bourdeaudhuij, I., & Lenoir

(2011). Gross motor coordination in relation to weight status and age in 5‐to 12‐year‐old boys and girls: A cross‐sectional study. Pediatric Obesity, 6, 1–9.

Duncan

Hall

Eyre

Barnett

James

(2021). Preschoolers fundamental movement skills predict BMI, physical activity and sedentary behaviour: A longitudinal study. Scandinavian Journal of Medicine & Science in Sports. 1, 8–14.

Epstein

J. L.

(1989). Family structures and student motivation: A developmental perspective. Research on Motivation in Education, 3, 259–295.

10.

Field

S. C.

Esposito Bosma

C. B.

Temple

V. A.

(2019). Comparability of the test of gross motor development—Second edition and the test of gross motor development—Third edition. Journal of Motor Learning and Development, 8(1), 107–125. https://doi.org/10.1123/jmld.2018-0058

11.

Gallahue

D. L.

Goodway

J. D.

Ozmun

J. C.

(2012). Understanding motor development: Infants, children, adolescents, adults (7th ed.). McGraw-Hill.

12.

Giavarina

(2015). Understanding Bland Altman analysis. Biochemia Medica, 25(2), 141–151.

13.

Hulteen

R. M.

Ridgers

N. D.

Johnson

T. M.

Mellecker

R. R.

Barnett

L. M.

(2015). Children’s movement skills when playing active video games. Perceptual and Motor Skills, 121(3), 767–790.

14.

Jaakkola

Yli‐Piipari

Huotari

Watt

Liukkonen

(2016). Fundamental movement skills and physical fitness as predictors of physical activity: A 6‐year follow‐up study. Scandinavian Journal of Medicine & Science in Sports, 26(1), 74–81.

15.

Lawrence, I., & Lin, K. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 1, 45, 255–268.

16.

Lin

(2000). A note on the concordance correlation coefficient. Biometrics 56, 324–325.

17.

Logan

S. W.

Robinson

L. E.

Wilson

A. E.

Lucas

W. A.

(2011). Getting the fundamentals of movement: A meta‐analysis of the effectiveness of motor skill interventions in children. Child: Care, Health and Development, 38(3), 305–315.

18.

M. J.

Zhong

W. H.

Liu

Y. X.

Miao

H. Z.

Y. C.

M. H.

(2016). Sample size for assessing agreement between two methods of measurement by Bland–Altman method. The International Journal of Biostatistics, 12(2), 1–8.

19.

Lubans

D. R.

Morgan

P. J.

Cliff

D. P.

Barnett

L. M.

Okely

A. D.

(2010). Fundamental movement skills in children and adolescents. Sports Medicine, 40(12), 1019–1035.

20.

Maeng

Webster

E. K.

Pitchford

E. A.

Ulrich

D. A.

(2017). Inter-and intrarater reliabilities of the test of gross motor development—Third edition among experienced TGMD-2 raters. Adapted Physical Activity Quarterly, 34(4), 442–455.

21.

Palmer

K. K.

Chinn

K. M.

Robinson

L. E.

(2017). Achievement goal theory in motor skill interventions: A systematic review. Sports Medicine, 47(12), 2569–2583.

22.

Palmer, K. K., Chinn, K. M., & Robinson, L. E. (2019). The effect of the CHAMP intervention on fundamental motor skills and outdoor physical activity in preschoolers. Journal of Sport and Health Science, 8(2), 98–105.

23.

Palmer

K. K.

Matsuyama

A. L.

Robinson

L. E.

(2017). Impact of structured movement time on preschoolers’ physical activity engagement. Early Childhood Education Journal, 45(2), 201–206.

24.

Palmer, K. K., Miller, A., Meehan, S. K., & Robinson, L. E. (2020). The Motor skills At Playtime intervention improves children’s locomotor motor skills: A feasibility study. Child: Care, Health, & Development, 46(5), 599-606.

25.

Ranganathan

Pramesh

C. S.

Aggarwal

(2017). Common pitfalls in statistical analysis: Measures of agreement. Perspectives in Clinical Research, 8(4), 187–191.

26.

Rintala

P. O.

Sääkslahti

A. K.

Iivonen

(2017). Reliability assessment of scores from video-recorded TGMD-3 performances. Journal of Motor Learning and Development, 5(1), 59–68.

27.

Robinson

L. E.

(2011). The relationship between perceived physical competence and fundamental motor skills in preschool children. Child: Care, Health and Development, 37(4), 589–596.

28.

Robinson

L. E.

Goodway

J. D.

(2009). Instructional climates in preschool children who are at-risk. Part I: Object-control skill development. Research Quarterly for Exercise and Sport, 80(3), 533–542.

29.

Robinson

L. E.

Palmer

K. K.

Bub

K. L.

(2016). Effect of the children’s health activity motor program on motor skills and self-regulation in head start preschoolers: An efficacy trial. Frontiers in Public Health, 4, 1–12.

30.

Robinson

L. E.

Palmer

K. K.

Irwin

J. M.

Webster

E. K.

Dennis

A. L.

Brock

S. J.

Rudisill

M. E.

(2015). The use of multimedia demonstration on the test of gross motor development–second edition: Performance and participant preference. Journal of Motor Learning and Development, 3(2), 110–122.

31.

Robinson

L. E.

Palmer

K. K.

Meehan

S. K.

(2017). Dose-response relationship: The effect of motor skill intervention duration on motor performance. Journal of Motor Learning and Development, 5(2), 280–18.

32.

Robinson

L. E.

Palmer

K. K.

Webster

E. K.

Logan

S. W.

Chinn

K. M.

(2018). The effect of CHAMP on physical activity and lesson context in preschoolers: A feasibility study. Research Quarterly for Exercise and Sport, 89(2), 265–271.

33.

Robinson

L. E.

Rudisill

M. E.

Goodway

J. D.

(2009). Instructional climates in preschool children who are at-risk. Part II: Perceived physical competence. Research Quarterly for Exercise and Sport, 80(3), 543–551.

34.

Robinson, L. E., Stodden, D. F., Barnett, L. M., Lopes, V. P., Logan, S. W., Rodrigues, L. P., & D’Hondt, E. (2015). Motor competence and its effect on positive developmental trajectories of health. Sports Medicine, 45(9), 1273–1284.

35.

Robinson

Wang

Colabianchi

Stodden

Ulrich

(2020). Protocol for a two-cohort randomized cluster clinical trial of a motor skills intervention: The promoting activity and trajectories of health (PATH) study. BMJ Open, 10(6), e037497.

36.

Roth

Ruf

Obinger

Mauer

Ahnert

Schneider

Graf

Hebestreit

, (2010). Is there a secular decline in motor skills in preschool children? Scandinavian Journal of Medicine & Science in Sports, 20(4), 670–678.

37.

Sim

Wright

(2000). Research in health care: Concepts, designs and methods. Nelson Thornes.

38.

Stodden

D. F.

Goodway

J. D.

Langendorfer

S. J.

Roberton

M. A.

Rudisill

M. E.

Garcia

L. E.

(2008). A developmental perspective on the role of motor skill competence in physical activity: An emergent relationship. Quest, 60(2), 290–306.

39.

Ulrich

(2000). The test of gross motor development-2. Pro-Ed.

40.

Ulrich

D. A.

(2017). Introduction to the special section: Evaluation of the psychometric properties of the TGMD-3. Journal of Motor Learning and Development, 5(1), 1–4.

41.

Ulrich

D. A.

(2019). Test of gross motor development (3rd ed.). Pro-Ed.

42.

Ulrich

Sanford

C. B.

(1985). Test of gross motor development. Pro-Ed.

43.

van der Fels

I. M. J.

Te Wierike

S. C. M.

Hartman

Elferink-Gemser

M. T.

Smith

Visscher

(2015). The relationship between motor skills and cognitive skills in 4–16 year old typically developing children: A systematic review. Journal of Science and Medicine in Sport, 18(6), 697–703.

44.

Veldman

S. L. C.

Jones

R. A.

Okely

A. D.

(2016). Efficacy of gross motor skill interventions in young children: An updated systematic review. BMJ Open Sport & Exercise Medicine, 2(1), e000067.

45.

Wick, K., Leeger-Aschmann, C. S., Monn, N. D., Radtke, T., Ott, L. V., Rebholz, C. E., Cruz, S., Gerber, N., Schmutz, E. A., Puder, J. J., Munsch, S., Kakebeeke, T. H., Jenni, O. G., Granacher, U., & Kriemler, S. (2017). Interventions to promote fundamental movement skills in childcare and kindergarten: A systematic review and meta-analysis. Sports Medicine, 47(10), 2045–2068.