An investigation into the measurement properties of the King-Devick Eye Tracking system

Abstract

Objectives

Eye tracking has been gaining increasing attention as a possible assessment and monitoring tool for concussion. The King-Devick test (K-DT) was expanded to include an infrared video-oculography-based eye tracker (K-D ET). Therefore, the aim was to provide evidence on the reliability of the K-D ET system under an exercise condition.

Methods

Participants (N = 61; 26 male, 35 female; age range 19-25) were allocated to an exercise or sedentary group. Both groups completed a baseline K-D ET measurement and then either two 10-min exercise or sedentary interventions with repeated K-D ET measurements between interventions.

Results

The test-retest reliability of the K-D ET ranged from good to excellent for the different variables measured. The mean ± SD of the differences for the total number of saccades was 1.04 ± 4.01 and there was an observable difference (p = 0.005) in the trial number. There were no observable differences for the intervention (p = 0.768), gender (p = 0.121) and trial (p = 0.777) for average saccade’s velocity. The mean ± SD of the difference of the total fixations before and after intervention across both trials was 1.04 ± 3.63 and there was an observable difference in the trial number (p = 0.025). The mean ± SD of the differences for the Inter-Saccadic Interval and the fixation polyarea before and after intervention across both trials were 1.86 ± 22.99 msec and 0.51 ± 59.11 mm2 and no observable differences for the intervention, gender and trial.

Conclusion

The results provide evidence on the reliability of the K-D ET, and the eye-tracking components and demonstrate the relationship between completion time and other variables of the K-D ET system. This is vital as the use of the K-DT may be increasing and the combination of the K-DT and eye tracking as one single package highlights the need to specifically measure the reliability of this combined unit.

Keywords

eye movements video-oculography King Devick exercise

Introduction

Recognizing sport-related concussion (SRC) is multifactorial and involves reviewing the mechanism of injury and kinematics, assessment of athlete signs and symptoms as well as a thorough clinical examination. The examination of an athlete suspected of sustaining a concussion has evolved and includes a variety of assessments inclusive of those that evaluate symptoms, clinical signs, more detailed ocular motor, vestibular function, and neurocognitive testing.^1,2 Many of these examinations are compiled within the Sport Concussion Assessment Tool-5 (SCAT5),^3–5 including the Standardized Assessment of Concussion (SAC).⁶ The SAC is reportedly the most utilised objective concussion test but a recent study identified that the SAC had a poor diagnostic accuracy and reliability.⁷

Another assessment instrument with increasing interest is the King-Devick test (K-DT), a rapid number naming task, that functions as a pseudo-reading test, broadly capturing aspects of afferent visual function, attention, language, visual fixation, and saccadic eye movements.^8,9 An advantage to the K-DT is that it can be utilized by attending clinicians with reasonable ease and has been validated as a sensitive sideline performance measure for concussion detection.^1,10–17 The K-DT has been reported to be a high performing objective sideline concussion screening test having the highest reliability, sensitivity, and specificity⁷ and is the only vision and ocular assessment tool which has had diagnostic accuracy investigated within an adult sporting population.¹ The summary sensitivity and specificity of the K-DT was 0.77 and 0.82, respectively.¹ However, in contrast, Fuller et al.¹⁸ showed a sensitivity of 59.6 and specificity of 39.2 in rugby players diagnosed with concussion and it is suggested that the K-DT should be one of several assessments used in concussion recognition and not a stand-alone test¹⁸

Outside its application for concussion, the K-DT exhibited a strong test-retest reliability in Eddy et al.¹⁹ study with an exercise or rest intervention, however it reported a high false-positive rate thereby the authors suggesting clinicians use caution when interpreting its results. The association between the K-DT and exertion or exercise is important as its application is intended for use with sporting activities which have exertion as an integral component and may play a role in its performance.^20–22 For example, Rist et al.²³ showed improved K-DT performance after a 15 min of high intensity (80% of predicted maximal hear rate) exercise but not moderate intensity (65% of predicted maximal heart rate) exercise. This is supported by a meta-analysis and systematic analysis²⁴ that showed improved K-DT scores of 1.4 s, in the pooled dataset, following vigorous exercise in the absence of concussion.

The K-DT was expanded to include supplementary hardware to its computer-based number-reading test in 2014; the K-DT with integrated infrared video-oculography-based eye tracker (K-D ET) leverages a portable light bar that directly mounts onto a laptop. The K-D ET test allows for the measurements of ocular motor control during the number-reading task. This is important as eye tracking, a technology for detecting eye movements and therefore visual information processing,²⁵ has been gaining increasing attention as a possible assessment and monitoring tool for concussion.²⁶ With studies showing eye movement aberrations with concussion^26–28 and the 2017 international consensus statement on SRC suggesting that ocular motor screening may add clinical utility in the recognition of SRC the importance of further research in this area is important.²⁹

Although, the K-D ET single-package unit has been available for a few years, there has been a paucity of literature reporting its validity against other similar or even research-grade high resolution eye-tracking units to establish inter-tracker validity, and reliability in research trials on concussion or with exercise or exertion. A few exceptions were the Onge et al.³⁰ study that utilized two identical K-D ETs with first-generation eye trackers along with participants alternating between the systems on successive trials. In a secondary phase of the Onge et al.³⁰ study, participants performed the testing on five successive days and twice on each system. The results showed that several of the systems’ ET measurements lacked face validity and they concluded the systems could not be used for scientific research. However, the K-D ET unit utilized in the Onge et al.³⁰ study has been discontinued. Hecimovich et al.³¹ utilized the current second-generation eye tracking unit, an nfrared-based, video-oculographic rig (120Hz VT3-Mini, EyeTech Digital Systems, Mesa, AZ) and a laptop, as part of their study on K-D ET on youth (ages 13-14) Australian footballers. Their study reported participants that had sustained a head impact recorded a slower mean total K-D ET time, fewer mean total saccades and had more mean blinks when compared with their baseline scores. The Hecimovich et al.³¹ study identified that the total mean blinks were the most sensitive measure for potential SRC.

In light of the increasing interest in the K-DT and therefore by extenssion its combined rapid number naming task and eye tracking unit, coupled with limited reliability evidence, for the K-D ET system to gain greater use in the recognition for SRC, it is vital to establish the test-retest reliability. Therefore, the aim was to provide evidence on the measurement properties of the K-D ET system. Specifically, the objectives were to measure the test-retest reliability of the King-Devick Eye Tracking system and determine if exercise influences the variables assessed by the King-Devick Eye Tracking unit.

Methods

Participants

Participants (N = 61; 26 male, 35 female; age range 19-25) were recruited on a university campus during the 2020-2021 academic year. Exclusion criteria included the use of bifocal, progressive, or other multi-focal corrective lenses at time of testing, or presence of an intraocular implant. Participants were required to have normal or corrected vision and ability to perform two cardiovascular activities at a vigorous rate. There were no exclusion criteria based on gender, ethnicity, race, or other demographics. Participants were allocated to the exercise or sedentary groups dependent upon the first contact with the primary investigator (i.e. first to exercise, second to sedentary etc.) until all participants were enrolled. Those in the exercise group (n = 31) were instructed to wear clothing and shoes for treadmill walking while those in the sedentary group (n = 30) were requested to bring a smartphone, with earphones, in order to listen to music. The project was approved by the Institutional Review Board (IRB 20-0131).

King-Devick Eye tracker test

Each participant sat in front of the K-D ET System, including an eye tracking unit, an infrared-based, video-oculographic rig (120Hz VT3-Mini, EyeTech Digital Systems, Mesa, AZ) and a laptop. Measurements from head to camera were made in order to ensure proper eye contact with the infrared camera. The K-D ET rig was positioned close to the bottom of the laptop screen while ensuring that it did not cover the display. A tape measure was used to position the subject 60 centimeters away from the K-D ET rig in a high back chair. Green lights on the K-D ET indicated correct positioning. For optimum conditions, the overhead lighting in the room was dimmed and outside window glare was minimized by darkened window shades. The participant was positioned in front of the screen and their eyes aligned to the center of the computer screen, in focus and clear. Participants were instructed to complete a calibration or pre-test by looking straight ahead and then at yellow targets in each corner of the screen. If the pupil, corneal reflections and/or crosshairs were not visible, out of focus, or unsteady in any of the viewing locations, the participant was cycled through the spatial location again to optimize the settings. At this point calibration commenced. With this, the participant followed a red target across the screen with their eyes.

Following calibration, a validation was conducted by verifying the eye position at 5 spatial ‘check’ points on the screen. Accurate viewing with a proper calibration changed the spatial check points blue. If successfully accomplished, the K-D ET test began. If the calibration was not accurate, calibration was repeated for best results. Corrective lenses were worn only if required for reading. Participants were instructed to read aloud the single digit number displayed from left to right, top to bottom, as quickly as he or she could without making any errors. Participants were instructed not use their hand or finger to help follow the numerical pattern. If the participant made an error and quickly corrected it, no error was recorded. An error was recorded for each omission, commission, and reversal. The K-D ET system provides post hoc eye movement and test analysis for the following measures: total saccades, average saccade velocity, peak saccadic velocity, total fixations, inter-saccadic interval (average fixation duration), average fixation polyarea, individual card, the number of blinks and overall completion time.

Intervention protocol

Participants were allocated into either an exercise group or sedentary group (see Figure 1). Prior to testing all participants had their vision assessed with a Snellen Eye Chart at 20 feet with corrected lenses, if applicable. Those who were assigned to the exercise group completed the 2020 PAR-Q + , an evidence-based pre-participation screening tool to determine if participants are able participate in physical activity or exercise.³²

Figure 1.

Participant allocation and exercise / rest protocol for the king-devick Eye tracking tests.

Participants in the exercise group completed two baseline K-D ET measurements and then moved into a separate room with a zero-grade horizontal treadmill. Once positioned onto the treadmill the participants were instructed to inform the investigator when they reached a level of 6 of 10 on a rate of perceived exertion scale (Borg CR-10 Scale)³³ indicating a level between strong (5) to very strong (7). This was achieved by increasing the speed of the treadmill in a relatively short period (< 1 min) until they reached the level of 6. Once they reached level 6, the participant walked for 10 min, without rest, with periodic adjustments to the speed to maintain the 6 level. This was classified as exercise session 1. Upon completion of exercise session 1, the participants were tested on the K-D ET system twice, with approximately 5 min between tests under identical conditions. Upon completion of exercise session 1 and the K-D ET re-tests, participants were moved back onto the treadmill for a second 10-min exercise session at an RPE of 6. This was exercise session 2. Upon completion of exercise session 2, they were retested twice on the K-D ET system, with approximately 5 min between tests under identical conditions. Altogether, each participant was tested six times on the K-D ET system.

Participants in the sedentary group completed two baseline K-D ET measurements and remained in the testing room and instructed to engage in listening to music on their own electronic device, with earphones, and not permitted to use the electronic device visually. During this session, the room lights were dimmed, and they remained in the room listening to music for 10 min. Upon completion of the sedentary session 1, they were tested on the K-D ET system and re-tested approximately 5 min later under identical conditions. Upon completion of sedentary session 1 and the K-D ET test, they were instructed to listen to their own music without electronic visual stimulation for 10 min. Upon completion of sedentary session 2, they were tested on the K-D ET system and re-tested approximately 5 min later under identical conditions. In all, each participant was tested six times on the K-DET system.

Statistical analysis

Data analysis were conducted using SPSS v27 (IBM Corp. released 2020). The following dependent variables were computed by the K-D ET test; time to complete (seconds), total saccades, average saccade velocity (°/s), total fixations, inter-saccadic interval (msec) (average fixation duration) and average fixation polyarea (mm²). The K-D ET test records the variables for each card within the test and the variables were subsequently transformed in different ways depending on what they represented: time to complete was the overall time to complete across all cards, total saccades were the sum of the values from all three cards, average saccade velocity were the average of the values from all three cards, peak saccadic velocity was the average peak saccadic velocity across all three cards, total fixations were the sum of the values from all three cards, inter-saccadic interval were the average of the values from all three cards, average fixation polyarea were the average of the values from all three cards and total blinks were the sum of the values from all three cards. An assessment of normality was performed using the Shapiro-Wilk test The mean (SD) for all dependent variables were determined.

The test-retest reliability of the K-D ET dependent variables were calculated from Trial 1 and Trial 2 (e.g. trials before introducing the intervention). Reliability was not assessed using the repeated measures (e.g. trials three through six) following the introduction of the intervention as the variables were deemed reliable from the pre-intervention trials and this method avoided potential confounders related to the intervention. The ICC and 95% confidence interval of two measurements ICC (3, 2) were calculated using a two-way, consistency, mixed-effects model. The ICC was considered to be poor, moderate, good and excellent if they were <0.50, 0.50-0.75, 0.75-0.90 and >0.90.³⁴

The correlation between completion time and the variables assessed by the K-D ET test (total saccades, average saccade velocity, peak saccadic velocity, total fixations, inter-saccadic interval, average fixation polyarea and total blinks) were assessed using a Pearsons r. The trial selected to perform the analysis of correlation was the final trial (trial six) due to the possible learning effect reported within this study and was therefore the trial most likely to be represent the relationship between time to complete and the K-D ET variable.

The baseline score for the time to complete the test was determined as the fastest time from T1 and T2. For all other dependent variables, the baseline score was calculated as the average of the two pre-intervention K-D ET tests. The difference following intervention trial one was calculated as the ‘baseline score minus the post intervention score (T3)’ and are presented as mean (SD). The difference following intervention trial two was calculated as the ‘post intervention score for trial one after rest (T4) minus the post intervention score (T5)’ and are presented as mean (SD).

The differences following intervention one and intervention two were included within a generalized estimating equation (GEE) as the dependent variable and the trial was included as a repeated measure with subjects included for within the GEE to allow for repeated measures. The intervention (either rest or exercise) and participant gender (female or male) were included as factors. Goodness of fit was examined using the Quasi likelihood under Independence Model Criterion (QIC) with a lower the value representing a better fit.

Results

A total of 61 participants (mean age 21.3 + 1.5 yrs., 35 females, 26 males) completed the assessments and these were used in the analysis. All participants had vision at either 20/20 or 20/30, with or without corrected vision. No participant was excluded due to the results of their 2020 Par-Q + .³² Data for three participants were not included in the analysis due to malfunctioning of the eye tracker unit. Mean ± standard deviation baseline and trial scores for both intervention groups are provided within Table 1.

Table 1.

King-Devick eyen-tracker test dependent variables for baseline and subsequent trials reported by mean and standard deviation.

	Baseline.Mean (SD)		Trial 3.Mean (SD)		Trial 4.Mean (SD)		Trial 5.Mean (SD)		Trial 6.Mean (SD)
	Exercise	Rest	Exercise	Rest	Exercise	Rest	Exercise	Rest	Exercise	Rest
Time to complete (s)	52.5 (7.0)	50.5 (8.1)	52.5 (6.75)	49.8 (8.3)	50.9 (7.2)	47.8 (7.8)	51.4 (7.4)	48.05 (8.0)	50.1 (7.2)	46.9 (8.4)
Total saccades (n)	165.6 (17.8)	166.9 (18.7)	162.9 (16.1)	159.2 (18.1)	159.1 (16.7)	157.0 (17.7)	158.8 (18.7)	157.2 (15.6)	155.5 (16.2)	157.8 (19.7)
Average saccade velocity (°/s)	134.9 (17.0)	134.2 (16.8)	134.0 (20.0)	133.4 (15.5)	136.7 (17.9)	134.6 (16.8)	135.8 (20.9)	134.4 (15.9)	139.0 (18.7)	133.3 (16.6)
Peak saccade velocity (°/s)	354.4 (35.3)	349.8 (41.2)	350.7 (40.1)	349.9 (39.9)	359.2 (41.3)	352.8 (43.7)	355.2 (40.5)	351.4 (43.7)	363.6 (42.9)	350.8 (42.0)
Total fixations (n)	159.2 (16.3)	161.3 (15.7)	156.9 (14.9)	155.1 (15.7)	153.6 (15.4)	153.2 (15.9)	152.8 (15.9)	152.3 (13.7)	149.5 (13.8)	153.3 (18.4)
Inter-saccadic interval (msec) (average fixation duration)	274.3 (49.1)	259.6 (50.9)	269.9 (49.4)	252.8 (51.6)	267.5 (51.6)	244.8 (46.0)	270.2 (53.5)	245.4 (45.4)	269.4 (47.7)	235.9 (47.0)
Average fixation polyarea (mm²)	236.3 (54.2)	273.1 (81.1)	234.4 (62.9)	267.5 (73.0)	230.5 (53.5)	289.9 (87.9)	238.8 (59.2)	286.9 (94.7)	244.6 (57.2)	276.9 (85.6)
Total Blinks (n)	4.4 (3.7)	3.9 (4.2)	4.5 (4.0)	3.3 (3.6)	3.5 (3.7)	3.1 (3.0)	4.3 (4.3)	3.9 (3.5)	4.3 (4.3)	3.4 (3.9)

SD = Standard Deviation; Baseline = fastest of trial 1 and 2; s = seconds; n = number; °/s = degrees per second; msec = milliseconds; mm² = millimeters squared.

There were significant correlations reported between the completion time and five variables of the K-D ET test: total saccades; average saccade velocity; peak saccadic velocity; total fixations; inter-saccadic interval (Table 2). Total saccades, total fixations and the inter-saccadic interval increased as the test duration increased whereas the average saccadic velocity and peak saccadic velocity decreased as the test duration increased.

Table 2.

Correlation of king-devick eye tracking variables to completion time.

Variable	Sample size	Pearson correlation (r)	p-value
Total Saccades.	61	0.466	<0.001*
Average saccade velocity.	61	−0.369	0.003*
Peak saccadic velocity.	61	−0.400	0.001*
Total fixations.	61	0.453	<0.001*
Inter-saccadic interval (average fixation duration).	61	0.759	<0.001*
Average fixation polyarea.	61	0.054	0.681
Total blinks.	61	0.200	0.121

*Statistically significant with p < 0.01.

Although the mean ± SD of the differences in the completion time both trials was −0.04 ± 2.75, there were no observable differences within the GEE for intervention (p = 0.530), gender (p = 0.072) and trial (p = 0.111) (see Table 3). The mean ± SD of the differences for the total saccades was 1.04 ± 4.01 and there was an observable difference detected by the GEE (p = 0.005) in the trial number with a reduction in two saccades for each subsequent intervention period (irrespective of whether the participants received rest or exercise). There were no observable differences for the intervention (p = 0.768), gender (p = 0.121) and trial (p = 0.777) for the average saccade’s velocity and the mean ± SD of the differences before and after the intervention across both trials was 0.46 ± 10.96 °/s. There were no observable differences for the intervention (p = 0.772), gender (p = 0.229) and trial (p = 0.952) for the peak saccade’s velocity and the mean ± SD of the differences before and after the intervention across both trials was 2.01 ± 25.78 °/s.

Table 3.

Generalised estimated equation for differences in the K-D ET by completion time, total saccades, average saccades velocity, fixation’s count, inter-saccadic interval, and fixation polyarea with quasi likelihood under independence model criterion (QIC) for intervention, gender and trial by beta distribution (β), standard error (SE) with 95% confidence interval and probability value (p = .).

Activity	Parameter	β	SE (95% CI)	p =
Completion time ^(QIC=876)
	Intervention (Exercise)^a	−0.33	0.52 (−1.34 to 0.69)	0.530
	Gender (Female)^b	0.95	0.53 (−0.09 to 1.99)	0.072
	Trial	−0.72	0.45 (−1.61 to −0.17)	0.111
Total Saccades ^(QIC=1832)
	Intervention (Exercise)^a	−0.33	0.70 (−1.70 to 1.04)	0.639
	Gender (Female)^b	0.36	0.69 (−0.99 to 1.71)	0.601
	Trial	−1.95	0.69 (−3.31 to −0.59)	0.005*
Peak Saccade Velocity ^{(QIC was 78330)}
	Intervention (Exercise)^a	−1.51	5.19 (−11.68 to 8.67)	0.772
	Gender (Female)^b	6.47	5.38 (−4.07 to 17.01)	0.229
	Trial	−0.23	3.83 (−7.74 to 7.27)	0.952
Average saccades velocity ^(QIC=14189)
	Intervention (Exercise)^a	−0.64	2.16 (−4.87 to 3.59)	0.768
	Gender (Female)^b	−3.49	2.25 (−7.90 to 0.92)	0.121
	Trial	−0.46	1.62 (−3.64 to 2.72)	0.777
Fixation’s count ^(QIC=1537)
	Intervention (Exercise)^a	−0.25	0.64 (−1.5 to 1.00)	0.695
	Gender (Female)^b	0.20	0.62 (−1.02 to 1.42)	0.745
	Trial	−1.39	0.62 (−2.61 to −0.18)	0.025*
Inter-Saccadic Interval ^(QIC=61148)
	Intervention (Exercise)^a	−1.48	4.19 (−9.70 to 6.73)	0.723
	Gender (Female)^b	5.79	4.14 (−2.33 to 13.91)	0.162
	Trial	−7.39	4.01 (−15.25 to 0.46)	0.065
Fixation polyarea ^(QIC=412063)
	Intervention (Exercise)^a	−12.21	11.14 (−34.05 to 9.63)	0.273
	Gender (Female)^b	−14.56	10.38 (−34.92 to 5.77)	0.160
	Trial	−7.63	9.92 (−27.07 to 11.80)	0.441
Total Blinks ^{(QIC was 787)}
	Intervention (Exercise)^a	0.16	0.50 (−0.81 to 1.13)	0.749
	Gender (Female)^b	−1.11	0.47 (−2.04 to −0.19)	0.018*
	Trial	−0.56	0.43 (−1.39 to 0.28)	0.191

(a) = Rest set to 0, (b) = Male set to 0. β = Beta distribution; SE = Standard Error; CI = Confidence Interval; *p < 0.05.

The mean ± SD of the difference of the total fixations before and after the intervention across both trials was 1.04 ± 3.63 and there was an observable difference detected by the GEE in the trial number (p = 0.025) with a reduction of 1.39msec for each subsequent intervention period (irrespective of whether the participants received rest or exercise). The mean ± SD of the differences for the Inter-Saccadic Interval and the fixation polyarea before and after the intervention across both trials were 1.86 ± 22.99 msec and 0.51 ± 59.11 mm². There were no observable differences for the intervention (p = 0.749) or trial (p = 0.191) for the total blinks. However, females blinked less than males (β = 1.11, p = 0.018). The mean ± SD total blinks of the differences before and after the intervention across both trials was 0.54 ± 2.6.

The reliability of the peak saccade velocity and the total blinks was poor to moderate. The average fixation polyarea and total saccades had good test-retest reliability (see Table 4). All other variables tested showed excellent test-retest reliability.

Table 4.

Test-retest reliability of components of the king-devick Eye tracking test.

	ICC (95% CI)	p =
Time to complete	0.917 (0.861 to 0.950)	<0.001
Total saccades	0.896 (0.826 to 0.938)	<0.001
Peak saccade velocity	0.489 (0.145 to 0.695)	0.005
Average saccade velocity	0.923 (0.870 to 0.954)	<0.001
Total fixations	0.907 (0.845 to 0.945)	<0.001
Inter-saccadic interval (average fixation duration)	0.917 (0.860 to 0.950)	<0.001
Average fixation polyarea	0.761 (0.601 to 0.858)	<0.001
Total blinks	0.294 (−0.178 to 0.416)	0.091

ICC = Inter-class correlation; CI = Confidence interval.

Discussion

The broad aim of this study was to provide evidence on the measurement properties of the K-D ET system, specifically to measure the test-retest reliability and determine if exercise influences the variables of the combined eye tracking unit.

The results of this study provide evidence on the reliability of the K-D ET, and in particular the eye-tracking components. Although previous research^19,23,35 on the number naming task (time only) has shown the K-DT to be a reliable tool under an exercise intervention, these studies did not directly measure eye tracking components. This is vital as the use of the K-DT may be increasing and the combination of the K-DT and eye tracking as one single package highlights the need to specifically measure the reliability of this combined unit.

The evidence of the reliability brought forth in this study needs to be accompanied with supporting validity in future research. It is important to note the K-D ET unit is not classified as a research-grade, or high-speed, system which can range from 250 Hz to 2000 Hz sampling rate.^36–40 There is support in the literature that sampling rates below 200 Hz are not able to provide as many measurements as a high-speed system, in particular peak saccade velocity,³⁸ as they are quick (900 + °/sec)^41,42 and brief (< 100 msec)⁴² movements of the fovea from one fixation point to another.⁴³ Thus, the capacity for accurate measurement is limited by the available technology. The current study recorded eye measurements with a 120 Hz sampling rate and an angular error of approx. 0.5° / Drift <0.3°; this was fixed given the infrared oculographic tracking unit from a third-party specifically utilized for the K-D ET. Therefore, the saccadic activity measured by the K-D ET unit may arguably be adequate.^39,44 However, it’s reasonable to report on these measurements for the use in future research comparing the combined unit to a research-grade system, keeping in mind that until more definitive evidence is available in both reliability and validity, caution needs to be used when interpreting the results for clinical decision-making.

There are several eye tracking units available on the market in the 30 Hz to 60 Hz range and these may be less than ideal for clinical use and this is supported in the literature. For instance, results from the Ooms et al.³⁸ study measuring the accuracy and precision of fixations revealed that an eye-tracking unit with a sampling rate at 30 Hz is less accurate in comparison to a 60 Hz system. Raynowski et al.³⁷ employed the use of the K-DT rapid number naming task with a research-grade eye tracking unit (EyeLink 1000 + ) against low-resolution units at 30 Hz and 60 Hz and reported the low-resolution units yielded significantly less detectable saccades and greater variance of ISI. Contrasting between a 60 Hz and 120 Hz sampling rate, Leube et al.⁴⁵ compared these two sampling rates during a reading task measuring saccade characteristics to a research-grade 1000 Hz eye-tracker and showed higher accuracy in the detection of fast eye movements and fixation durations with the 120 Hz system.

Although the aforementioned supports the use of higher sampling rates, there are studies that utilized eye-tracking units with lower sampling rates. For example, Lirani-Silva et al.⁴⁶ measured saccadic frequency, mean and peak velocity, and fixation frequency and duration during walking in people with mild traumatic injury using a Tobii Pro Glassess 2, 100 Hz eye tracking unit. They reported participants with mild traumatic brain injury showed reduced saccade frequency, duration and peak velocity compared with health controls. However, in their study, they used a custom-made validated velocity-based saccade detection algorithm which may have yielded more robust data. Recently, research using the K-D ET has been reported. Marchant et al.⁴⁷ used the K-D ET unit and reported on saccade velocity, and fixation time (ISI) in their study assessing ankle and finger somatosensation and lower limb muscle activity assessing visuomotor control in both conditions. Tejani et al.⁴⁸ utilized the K-D ET combined unit when assessing baseline differential eye movements and visual contrast acuity in competitive athletes and reported on total number and frequency of saccades and completion time, but not saccade velocities. As noted previously, Hecimovich et al.³¹ study with young Australian footballers measured number of saccades and blinks.

The use of video-oculographic eye tracking has been studied^28,49–52 in relation to concussion and clinicians may employ its use with other eye tracking devices, for example Tobii, SyncThink, and EyeLink. However, having the combined K-D ET unit may make for easier clinical utility, considering how effective the K-DT is for SRC evaluation.^1,7 Furthermore, eye movement abnormalities can persist in the absence of other post-concussion symptoms^53–55 therefore, monitoring ocular motor function may assist return-to-play decision making. Unfortunately, current evidence does not have sufficient strength to inform clinical decision-making, and research needs to establish guidelines for each specific unit as they differ in task.¹

The results from the current study indicated that there were no observable differences in all eye tracking components analysed when comparing participants gender and the intervention. However, when comparing saccadic changes, there were meaningful difference (saccades, p = 0.005) with subsequent intervention periods (irrespective of whether the intervention was exercise or rest) indicating a possible learning effect.¹¹ This needs to be considered when utilizing the K-D ET for concussion recognition or return to play, especially with multiple retests.^56–58

Although the results for total fixations revealed a meaningful difference in the trial number (p = 0.025), this did not occur for the ISI and the fixation polyarea. The impact of fixation count, ISI and polyarea are areas that may benefit from future head injury research and therefore accurate measurement is important. A fixation keeps the gaze stable on a selected stationary target⁵⁹ and ISI refers to the time between task-specific saccades fixation.^26,60,61 Changes in either of these measurements after a suspected concussive event may be an important indicator of SRC⁶² and further investigation is warranted. For example, a study by DiCesare et al.⁶² with older adolescents that had recently experienced a diagnosed concussion when compared with age-matched controls, reported that concussed individuals were notably less accurate while fixating a target between saccades and exhibited impaired performance in tracking a target along a predictable path when compared to the healthy controls. DiCesare et al.⁶² concluded that the clinical importance of fixation activity needs to be continued to be studied. Similarly, Rizzo et al.²⁶ utilized the K-DT with chronic concussion individuals and compared them to healthy controls and reported that the average ISI for task-specific saccades were significantly (p = 0.027) longer among concussed individuals compared to controls.

The K-D ET is specific to horizontal reading task and the thus comparisons to studies using other methods are difficult. The K-D ET unit provides measurement on duration (time to take to test), number of saccades, average saccade velocity (°/s), peak saccade velocity (°/s), number of fixations, ISI (ms), average fixation polyarea (mm²) and number of blinks. The average saccade velocity and average fixation polyarea are company-based metrics and were reported in the Onge et al.³⁰ and Marchant et al.⁴⁷ (average saccade velocity only) studies with both using the K-D ET, keeping in mind that the Onge et al.³⁰ study used the older, discontinued, model. Average saccade velocity was mentioned in Tad et al.⁶³ study, however, the usefulness of these two metrics needs to be established in the literature. There have been several studies^26,60,64,65 that have utilized the rapid number naming component of the K-DT with other eye tracking units. The results reported in these studies, such as the number of saccades and ISI, are similar to ones reported in the current study. For example, in Rizzo et al.⁶⁴ study investigating eye movements during sandbagging utilized the K-DT with an EyeLink 1000 Plus infrared-based video-oculographic camera (500 Hz). They reported median ISI in the 300 ms to 400 ms range, number of saccades in a 130-172 range, and average peak velocities of 299.5°/s. Gold et al.⁶⁵ assessed the relation between ISI prolongation and prolonged K-DT testing with results showing median ISI values of 379 + 199 ms which they contrasted to their previous research showing 235 + 119 ms⁶⁰ and 286 + 49 ms,²⁶ respectively. The Hecimovich et al.³¹ study with male youth Australian football participants reported mean saccadic measurements of 151 to 168 and 4 to 7 blinks per test Tejani et al.⁴⁸ study on baseline eye movements and visual contrast acuity noted an average number of saccades per card (over all three K-DT tests) between 56.9 + 14 and 101 + 41. Marchant et al.⁴⁷ with their study on somatosensation, muscle activity and visuomotor control and full body loading and unloading reported mean ISI of 257 ms and 287 ms, respectively, between groups. The results from the aforementioned studies are comparable with the current results (Table 1), for example total number of saccades per test ranged from the 150s to 160s, ISIs 230 ms to the 270 ms.

The limitations in this study need to be highlighted. For instance, initially participants in the exercise group were to perform the exercise activity at 60-70% HRmax, however due to the COVID-19 restrictions set forth from the institutional review board this was not permitted. As a result, the use of a perceived exertion scale (Borg CR-10 Scale)³³ was utilized instead and may not have been sufficient to replicate fatiguing exercise. Participants in this study were in a homogenous group comprised of university students and not representative of a wider population. This study appeared to be appropriately powered with acceptable Quasi likelihood under Independence Model Criterion for most variables within the GEE, however even with a larger sample the effect sizes did not seem clinically relevant. Although reading errors were documented, participants were not required to redo their test if they had two or less errors per test This decision was due to limiting the amount of time participants needed to complete all the requirements. The K-D ET unit utilized in the current study was a second-generation model whereas the Onge et al.³⁰ study utilized an older, and now discontinued version, highlighting the need for further research on this new version with a focus to validate this against other eye tracking units. We did not measure recovery following the rest period for the exercise group and are unable to determine if ten minutes was sufficient. The 10-min time frame for the exercise sessions may have impacted the results as Galetta et al.¹¹ reported that the K-DT test has been shown to have learning effects associated with repeat testing improvement of 3.4 s in median times. However, by developing a baseline, and assessing change based on the most recent test not always comparing the time to baseline we do not think this is likely to have impacted results from our generalized estimated equations. Finally, the baseline for the K-D ET duration was the fastest time recorded and replicates previous studies using the K-D test¹ and this may influence results (as opposed to using the mean of two tests or the slowest time) however, given the reliability of the test duration using these measures was reported as an ICC = 0.917 the influence on the results would likely be negligible.

Conclusion

The ICC results indicate the K-D ET system to be a reliable measure; however, due to the testing sessions being close to each other, learning effects may have led to these results. Nonetheless, the combination of the K-DT number reading task and eye tracking unit has potential as a post sideline-assessment tool (to be used after standard sideline tests such as the K-DT number reading task, and SACT5) to aid in detecting SRC and monitoring recovery. However, there lacks empirical evidence to guide clinical decision on the eye movement abnormalities. Furthermore, research now needs to focus on the combined units’ ability to detect changes that occur with SRC to provide confidence for its use clinically.

Future research needs to focus the reliability on the K-D ET combined unit and include a comparison against research-grade high-resolution eye tracking units to establish validity in the advent of widespread clinical use in the athletic setting for concussion recognition and monitoring.

Footnotes

Acknowledgements

We would like to thank Master of Athletic Training (MAT) students Katie Spellman, Mckinley Lillegraven and Grayson Benedict for their assistance in this project.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

M. Hecimovich

References

Harris

Dempsey

Mackie

, et al. Do sideline tests of vestibular and oculomotor function accurately diagnose sports-related concussion in adults? A systematic review and meta-analysis. Am J Sports Med 2021; In press. 10.1177/03635465211027946

Collins

Kontos

Reynolds

, et al. A comprehensive, targeted approach to the clinical care of athletes following sport-related concussion. Knee Surg Sports Traumatol Arthrosc 2014; 22: 235–246.

Echemendia

Meeuwisse

McCrory

, et al. The sport concussion assessment tool 5th edition (SCAT5): background and rationale. Brit J Sports Med 2017; 51: 848–850.

Echemendia

Broglio

Davis

, et al. What tests and measures should be added to the SCAT3 and related tests to improve their reliability, sensitivity and/or specificity in sideline diagnosis? A systematic review. Brit J Sports Med 2017; 51: 895–901.

Echemendia

Meeuwisse

McCrory

, et al. Sport concussion assessment tool - 5th edition. Brit J Sports Med 2017; 51: 851–858.

McCrea

Kelly

Randolph

, et al. Standardized assessment of concussion (SAC): on-site mental status evaluation of the athlete. J Head Trauma Rehabil 1998; 13: 27–35.

Harmon

Whelan

Aukerman

, et al. Diagnostic accuracy and reliability of sideline concussion evaluation: a prospective, case controlled study in college athletes comparing newer tools and established tests. Brit J Sports Med 2022; 56: 144–150. 10.1136/bjsports-2020-103840

Pillai

Gittinger

. Vision testing in the evaluation of concussion. Semin Opthalmol 2017; 32: 144–152.

Lawrence

Haider

Leddy

, et al. The king-devick test in an outpatient concussion clinic: assessing the diagnostic and prognostic value of a vision test in conjunction with exercise testing among acutely concussed adolescents. J Neurol Sci 2019; 15: 91–97.

10.

Galetta

Barrett

Allen

, et al. The king-devick test as a determinant of head trauma and concussion in boxers and MMA fighters. Neurology 2011; 76: 1456–1462.

11.

Galetta

Brandes

Maki

, et al. The king–devick test and sports-related concussion: study of a rapid visual screening tool in a collegiate cohort. J Neurol Sci 2011; 309: 34–39.

12.

Galetta

McCrossin

, et al. Saccades and memory: baseline associations of the king–devick and SCAT2 SAC tests in professional ice hockey players. J Neurol Sci 2013; 328: 28–31.

13.

Leong

Balcer

Galetta

, et al. The king–devick test as a concussion screening tool administered by sports parents. J Sports Med Phys Fit 2014; 54: 70–77.

14.

Galetta

Morganroth

Moehrnger

, et al. Adding vision to concussion testing: a prospective study of sideline testing in youth and collegiate athletes. J Neuroophthalmol 2015; 35: 235–241.

15.

King

Hume

Gissane

, et al. Use of the king-devick test for sideline concussion screening in junior rugby league. J Neurol Sci 2015; 357: 75–79.

16.

King

Gissane

Hume

, et al. The king-devick test was useful in management of concussion in amateur rugby union and rugby league in New Zealand. J Neurol Sci 2015; 351: 58–64.

17.

Hecimovich

King

Dempsey

, et al. The king-devick test is a valid and reliable tool for assessing sport-related concussion in Australian football: a prospective cohort study. J Sci Med Sport 2018; 21: 1004–1007.

18.

Fuller

Cross

Stokes

, et al. King-Devick concussion test performs poorly as a screening tool in elite rugby union players: a prospective cohort study of two screening tests versus a clinical reference standard. Br J Sports Med 2019; 53: 1526–1532.

19.

Eddy

Goetschius

Hertel

, et al.

Test-Retest reliability and the effects of exercise on the king-devick Test

Clin J Sport Med 2020; 30: 239–244.

20.

Hogan

Mata

Carstensen

. Exercise holds immediate benefits for affect and cognition in younger and older adults. Psychol Aging 2013; 28: 587–594.

21.

Llorens

Sanabria

Huertas

. The influence of acute intense exercise on exogenous spatial attention depends on physical fitness level. Exp Psychol 2015; 62(1): 20–29. 10.1027/1618-3169/a000270. PMID: 25270559.

22.

McMorris

Hale

Corbett

, et al. Does acute exercise affect the performance of whole-body, psychomotor skills in an inverted-U fashion? A meta-analytic investigation. Physiol & Behav 2015; 141: 180–189. ISSN 0031-9384.

23.

Rist

Cohen

Pearce

. King-Devick performance following moderate and high exercise intensity bouts. Int J Exerc Sci 2017; 10: 619–628.

24.

Galetta

Liu

Leong

, et al. The king-devick test of rapid number naming for concussion detection: meta-analysis and systematic review of the literature. Concussion 2015; 1: CNC8.

25.

Mele

Federici

. Gaze and eye-tracking solutions for psychological research. Cogn Process 2012; 13: 261–265.

26.

Rizzo

Hudson

Dai

, et al.

Rapid number naming in chronic concussion: eye movements in the king-devick test

Ann Clin Transl Neurol 2016; 3: 801–811.

27.

Cifu

Wares

Hoke

, et al. Differential eye movements in mild traumatic brain injury versus normal controls. J Head Trauma Rehabil 2015; 30: 21–28.

28.

Ventura

Balcer

Galetta

, et al. Ocular motor assessment in concussion: current status and future directions. J Neurol Sci 2016; 361: 79–86.

29.

McCrory

Meeuwisse

Dvořák

, et al. Consensus statement on concussion in sport-the 5^th international conference on concussion in sport held in Berlin, October 2016. Brit J Sports Med 2017; 51: 838–847.

30.

Onge

Temme

McAtee

, et al. Evaluation of the commercial, Off-the-shelf (COTS) king-devick Eye tracking system. Mil Med 2019; 184: 571–578.

31.

Hecimovich

King

Dempsey

, et al. In situ use of the king-devick Eye tracking and changes seen with sport related concussion: saccadic and blink counts. Phys Sportsmed 2019; 47: 78–84.

32.

Warburton

DER

Jamnik

Bredin

, et al. The physical activity readiness questionnaire (PAR-Q + ) and electronic physical activity readiness medical examination (ePARmed-X + ). Health Fitness J Can 2011; 4: 3–23.

33.

Borg

. Psychosocial basis of perceived exertion. Med Sci Sports Exerc 1982; 14: 377–381.

34.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiro Med 2016; 15: 155–163.

35.

Haider

Worts

Viera

, et al. Postexercise slowing on the king-devick test and longer recovery from sport-related concussion in adolescents: a validation study. J Athl Train 2020; 55: 482–487.

36.

Dowiasch

Wolf

Bremmer

. Quantitative comparison of a mobile and stationary video-based eye-tracker. Behav Res Methods 2020; 52: 667–680.

37.

Raynowska

Rizzo

Rucker

, et al. Validity of low-resolution eye-tracking to assess eye movements during a rapid number naming task: performance of the eye tribe eyetracker. Brain Inj 2018; 32: 200–208.

38.

Ooms

Dupont

Lapon

, et al. Accuracy and precision of fixation locations recorded with the low-cost Eye tribe tracker in different experimental set – ups. J Eye Move Res 2015; 8: 1–24.

39.

Wierts

Janssen

Kingma

. Measuring saccade peak velocity using a low-frequency sampling rate of 50 Hz. IEEE Trans Biomed Eng 2008; 55: 2840–2842.

40.

Holmqvist

Nyström

Andersson

, et al. Eye tracking: a comprehensive guide to methods and measures. Oxford, UK: Oxford University Press, 2011.

41.

Behrens

MacKeben

Schröder-Preikschat

. An improved algorithm for automatic detection of saccades in eye movement data and for calculating saccade parameters. Behav Res Methods 2010; 42: 701–708.

42.

Leigh

Zee

. The neurology of eye movements. 5th ed. New York: Oxford University Press, 2015.

43.

Mollenbach

Hanseb

Lillholm

. Eye movement in gaze interaction. J Eye Move Res 2013; 6: 1–15.

44.

DiScenna

Das

Zivotofsky

, et al. Evaluation of a video tracking device for measurement of horizontal and vertical eye rotations during locomotion. J Neurosci Methods 1995; 58: 89–94.

45.

Leube

Rifai

. Sampling rate influences saccade detection in mobile eye tracking of a reading task. J Eye Mov Res 2017; 10(3). Published 2017 Jun 7. 10.16910/jemr.10.3.3

46.

Lirani-Silva

Stuart

Parrington

, et al. Saccade and fixation Eye movements during walking in people With mild traumatic brain injury. Front Bioeng Biotechnol 2021; 9(701712). 10.3389/fbioe.2021.701712

47.

Marchant

Ball

Witchalls

, et al. The effect of acute body unloading on somatosensory performance, motor activation, and visuomotor tasks. Frontiers Phys 2020; 11: 318.

48.

Tejani

Vargas

Middleton

, et al. Differential eye movments and visual contrast acuity in national collegiate athletic association athletes. Int J Athl Ther Train 2020; 25: 270–275.

49.

Murray

Ambati

VNP

Contreras

, et al. Assessment of oculomotor control and balance post-concussion: a preliminary study for a novel approach to concussion management. Brain Inj 2014; 28: 496–503.

50.

Kelly

Kiderman

Akhavan

, et al. Oculomotor, vestibular, and reaction time effects of sports-related concussion: video-oculography in assessing sports-related concussion. J Head Trauma Rehabil 2019; 34: 176–188.

51.

Samadani

Ritlop

Reyes

, et al. Eye tracking detects disconjugate eye movements associated with structural traumatic brain injury and concussion. J Neurotrauma 2015; 32: 548–556.

52.

Ciuffreda

Ludlam

Thiagarajan

. Oculomotor diagnostic protocol for the mTBI population. Optometry 2011; 82: 61–63.

53.

Ciuffreda

Kapoor

Rutner

, et al. Occurrence of oculomotor dysfunctions in acquired brain injury: a retrospective analysis. Optometry 2007; 78: 155–161.

54.

Brahm

Wilgenburg

Kirby

, et al. Visual impairment and dysfunction in combat-injured service members with traumatic brain injury. Optom Vis Sci 2009; 86: 817–825.

55.

Goodrich

Flyg

Kirby

, et al. Mechanisms of TBI and visual consequences in military and veteran populations. Optom Vis Sci 2013; 90: 105–112.

56.

Elbin

Schatz

Mohler

, et al. Establishing test-retest reliability and reliable change for the king-devick test in high school athletes. Clin J Sport Med 2021; 31: e235–e239.

57.

Breedlove

Ortega

Kaminski

, et al. King-Devick test reliability in national collegiate athletic association athletes: a national collegiate athletic association–department of defense concussion assessment, research and education report. J Athl Train 2019; 54: 1241–1246.

58.

Oberlander

Olson

Weidauer

. Test-Retest reliability of the king-devick test in an adolescent population. J Athl Train 2017; 52: 439–445.

59.

Duchowski

. Eye tracking methodology: theory and practice. 2nd ed. London: Springer, 2007.

60.

Rizzo

Hudson

Dai

, et al.

Objectifying eye movements during rapid number naming: methodology for assessment of normative data for the king-devick test

J Neurol Sci 2016; 362: 232–239.

61.

Krauzlis

Goffart

Hafed

. Neuronal control of fixation and fixational eye movements. Phil. Trans. R. Soc. 2017; 372: B3722016020520160205. http://doi.org/10.1098/rstb.2016.0205

62.

DiCesare

Kiefer

Nalepka

, et al. Quantification and analysis of saccadic and smooth pursuit eye movements and fixations to detect oculomotor deficits. Behav Res Methods 2017; 49: 258–266.

63.

Brunyé

Gardony

. Eye tracking measures of uncertainty during perceptual decision making. Int J of Psychophysiology 2017; 120: 60–68.

64.

Rizzo

Hudson

Martone

, et al. How sandbag-able are concussion sideline assessments? A close look at eye movements to uncover strategies. Brain Inj 2021; 35: 426–435.

65.

Gold

Rizzo

Lee

YSC

, et al. King-Devick test performance and cognitive dysfunction after concussion: a pilot eye movement study. Brain Sci 2021; 11: 1571.