Eye Tracking During a Continuous Performance Test: Utility for Assessing ADHD Patients

Abstract

Objective:

The use of continuous performance tests (CPTs) for assessing ADHD related cognitive impairment is ubiquitous. Novel psychophysiological measures may enhance the data that is derived from CPTs and thereby improve clinical decision-making regarding diagnosis and treatment. As part of the current study, we integrated an eye tracker with the MOXO-dCPT and assessed the utility of eye movement measures to differentiate ADHD patients and healthy controls.

Method:

Adult ADHD patients and gender/age-matched healthy controls performed the MOXO-dCPT while their eye movements were monitored (n = 33 per group).

Results:

ADHD patients spent significantly more time gazing at irrelevant regions, both on the screen and outside of it, than healthy controls. The eye movement measures showed adequate ability to classify ADHD patients. Moreover, a scale that combined eye movement measures enhanced group prediction, compared to the sole use of conventional MOXO-dCPT indices.

Conclusions:

Integrating an eye tracker with CPTs is a feasible way of enhancing diagnostic precision and shows initial promise for clarifying the cognitive profile of ADHD patients. Pending replication, these findings point toward a promising path for the evolution of existing CPTs.

Keywords

ADHD continuous performance tests MOXO-dCPT eye movements

Introduction

ADHD is associated with impairment in various cognitive domains (Fuermaier et al., 2018). Continuous performance tests (CPTs) are commonly used to clarify the cognitive functioning of examinees with either suspected or confirmed ADHD (Albrecht et al., 2015; Hall et al., 2016). These tests include rapid presentation of visual and/or auditory stimuli over a relatively prolonged time, during which examinees are requested to respond to certain stimuli while ignoring others (i.e., targets and non-targets). Thereby, the clinician is provided with information (e.g., omission and commission errors) that may uncover attentional and executive dysfunctions such as impulsivity and response inhibition (see review; Marshall et al., 2019). However, the diagnostic precision of CPTs has been questioned. For example, Pettersson et al. (2018) found CPT indices to have poor discriminative ability and this finding aligns with the conclusions of a recent review by Marshall et al. (2019), according to which CPTs usually have good sensitivity but poor specificity for diagnosing ADHD (for an earlier review, see Hall et al., 2016). In other words, CPTs are sensitive to cognitive impairment, but have difficulty in establishing its source (i.e., ADHD vs. other neuro-psychiatric conditions). Overall, to date, CPTs have been found useful in clarifying the cognitive profile of examinees, while their ability to support clinical decision-making (i.e., diagnosis) has been deemed limited.

Psychophysiological measures have the potential to increase diagnostic precision and enhance treatment response monitoring in ADHD (Hall et al., 2016). This potential was explored in novel studies integrating psychophysiological data with conventional CPT indices (e.g., Hemoencephalography [nirHEG], quantified electroencephalography [Q-EEG] and event-related potentials [ERP]; Areces et al., 2018; Hall et al., 2016; Muller et al., 2020). Thereby, researchers have aimed to enhance the clinical utility of CPTs as well as validate biomarkers linked to the underlying pathology of ADHD. In this regard, eye movements are key biomarkers that offer to provide unique information regarding ADHD-related cognitive impairment. For example, eye tracking was useful in exploring response inhibition (Goto et al., 2010) and selective attention impairments in ADHD (Vakil et al., 2019). Surprisingly, we are aware of only one study in which eye movements were monitored while examinees performed a commercially available CPT (Fried et al., 2014). In this study, adult ADHD patients made more microsaccades (i.e., small, jerk-like eye movements during fixation) than controls while performing the Test of Variables of Attention (T.O.V.A.). However, obtaining reliable data of microsaccades requires expensive eye trackers with high precision and temporal resolution (Holmqvist et al., 2011, pp. 29−43). Therefore, their integration in commercially available CPTs is unlikely in the forthcoming years.

The current study aimed to assess the feasibility of integrating an eye tracker with the MOXO-dCPT. The MOXO-dCPT, a commercially available CPT, has several advantages for such a project. First, this CPT has attracted growing clinical and research attention, with data accumulating regarding children, adolescents and adult ADHD patients (for examples of recent studies, see Ben-Sheetrit et al., 2017; Dobrakowski & Lebecka, 2020; Jacoby & Lavidor, 2018; Shahaf et al., 2018; Slobodin et al., 2020; Slobodin & Davidovitch, 2019). This accumulated database is an advantage that is shared with other commercial CPTs and contrasts with those which are constructed specifically for an individual research project. In addition, commercial CPTs have financial backing, which may ease the integration of novel technologies such as eye tracking in daily clinical practice. They also lessen methodological challenges by providing a standard platform that researchers can utilize. The MOXO-dCPT has several additional advantages, including the use of an online platform that does not require special equipment as well as its high ecological validity. Regarding the latter, the MOXO-dCPT uses dynamic distractions, both visual and auditory, which simulate real-world settings (e.g., a barking dog, the opening of a bottle, and a crying baby). The use of these distractions joins the growing use of advanced technologies, such as virtual reality (Parsons & Rizzo, 2019), that aim to better simulate real life during the test. This, as opposed to other CPTs which do not use distractors (e.g., the T.O.V.A; Greenberg et al., 2018), was suggested to enhance the MOXO-dCPT ecological validity and, consequently, its utility (Berger & Cassuto, 2014; Cassuto et al., 2013). The MOXO-dCPT therefore serves as an environment conducive tool for exploring the cognitive impairment of ADHD patients. The current study capitalized on these properties of the MOXO-dCPT by dividing its visual environment into separate regions and assessing the allocation of attention of both ADHD patients and healthy controls while they performed the CPT. Thereby, we utilized the participants’ gaze direction, an intuitive and easily obtainable eye movement measure, to differentiate the groups. A secondary aim was to assess the potential of eye movement measures to enhance the discrimination capacity of conventional MOXO-dCPT indices. We hypothesized that while performing the CPT, ADHD patients would gaze for longer durations at regions which are irrelevant for success in the task (e.g., the AOI in which distractors are presented) compared to healthy controls. This hypothesis corresponds to a recent study in which ADHD patients gazed longer at distractors in a Stroop task than controls (Vakil et al., 2019). We further hypothesized that combining eye movement measures with the conventional MOXO-dCPT indices would enhance the discriminative capacity of the CPT, compared to that attained using only the conventional indices. Finally, we aimed to establish cutoffs for the eye movement measures and the combined scale with an adequate discrimination capacity (i.e., 80% sensitivity/specificity biomarker criterion, suggested by the World Federation of Societies of Biological Psychiatry, WFSBP; Thome et al., 2012)

Method

Participants

Thirty-five adult ADHD patients and matched healthy controls (±1 years of age/education), participated in the study. Participants were undergraduate students with normal or corrected-to-normal vision, who received academic credit for participating in the study. All ADHD patients had a previous diagnosis of ADHD, provided by a licensed clinician (314.0× according to the Diagnostic and Statistical Manual of Mental Disorders-5, DSM-5; American Psychiatric Association, 2013). They also met current diagnostic criteria, established by a trained clinician upon study entry, using the Structured Clinical Interview for DSM-5 (SCID-5-RV; First et al., 2015). Of these, 21 ADHD patients (60%) reported use of stimulant medications and were requested not to take the drug in the 24 hours prior to participating in the study. Exclusion criteria included past or present neuropsychiatric disorders (besides ADHD for participants in the ADHD patient group) and/or learning disabilities. Healthy controls were also excluded if they reported current attentional impairments, operationalized as total score ≥ 37 in the Adult ADHD Self-Report Scale, ASRS (criterion used, for example, by Lozano et al., 2020). Inspection of the data later revealed that two participants, one in each group, had abnormal eye movements values (i.e., duration of blinks and number of saccades). Their data, along with their matched participants, were discarded. Overall, data of 33 participants per group were included in the final analyses (N = 66).

The study received an institutional review board (IRB) committee approval and all participants provided written informed consent. The study sample partially overlapped with that of Berger et al. (2019).

Materials

MOXO-dCPT: The MOXO-dCPT consists of eight blocks and lasts 18.7 minutes. Each block consists of trials in which a stimulus card (target or non-target) is presented in the center of the screen, followed by a blank screen (59 trials; 34 targets and 25 non-targets). The target / non-target is presented for either 0.5 seconds, 1 seconds, or 4 seconds, followed by a “void” period of a similar duration. Participants are requested to press the spacebar once in response to targets and refrain from pressing it in response to non-targets. This is done while ignoring a series of distractors that appear above, below, or to the sides of the stimulus card. Distractors are presented for 8 seconds, with a fixed interval of 0.5 seconds between two distractors. The blocks include three types of distractors: (a) Visual distractors (blocks 2 and 3; e.g., an animation of a barking dog); (b) Auditory distractors (blocks 4 and 5; e.g., barking sound); and (c) Combination of visual and auditory distractors (blocks 6 and 7; e.g., an animation of a barking dog accompanied by a barking sound). Blocks 1 and 8 do not include any distractors. The blocks that include distractors differ in load; in the first block, only one distractor is presented at a time (i.e., blocks 2, 4, and 6), while in the second block, two distractors are presented simultaneously (i.e., blocks 3, 5, and 7). Sixteen distractors are presented in each block for an overall of 32 visual distractors, 32 auditory distractors and 32 combined distractors. Distractors are also presented during the void period and are not synchronized with target onset. Based on the participants’ performance in the MOXO-dCPT, four performance indices are calculated (see also Supplemental Appendix A in Berger et al., 2013): (a) Attention: Number of correct responses performed during stimulus presentation or the inter-stimulus interval (ISI) following it. (b) Timeliness: Number of correct responses performed only during the time in which the target stimulus is present on the screen. (c) Impulsivity: Number of commission responses performed during the time in which a non-target stimulus is presented. (d) Hyperactivity: Number of commission responses that not coded as impulsive responses (e.g., multiple keystrokes in response to a target stimulus). For each index, the participants’ performance was scored using the MOXO-dCPT’s normative database.

Adult ADHD Self-Repot Scale (ASRS; Kessler et al., 2007): An 18-item self-report questionnaire of adult ADHD symptoms, as detailed in the DSM-5 (American Psychiatric Association, 2013). Symptoms are rated on a 5-point Likert scale, ranging from 0 (never) to 4 (very often). The Hebrew version of the ASRS has an internal consistency of 0.87 (Cronbach’s alpha), sensitivity of 65%, and specificity of 68% (Zohar & Konfortes, 2010).

Wender Utah Rating Scale (WURS; Ward et al., 1993): A 61-item self-report questionnaire which provides a retrospective evaluation regarding the presence and severity of ADHD symptoms in childhood. It is based on the Utah criteria and is commonly used in research (e.g., Barbaresi et al., 2018). The items are rated on a 5-point Likert scale, with high scores indicating more severe symptoms. The 25-item WURS version was used in the current study. This version adequately classifies ADHD patients (Ward et al., 1993) and has an internal consistency score (i.e., Cronbach’s alpha) of 0.88 (Rossini & O’Connor, 1995).

Eye-tracking Apparatus and Eye Movement Measures

The stimuli were presented on an Alienware OptX 23” display screen with 1920 × 1080 resolution. Eye-to-screen viewing distance was approximately 60 cm. The eye tracking apparatus was based on the setting used by Elbaum et al. (2017). It includes a host PC which tracks and computes participants’ gaze position, as well as a display PC, which displays the stimuli (i.e., MOXO-dCPT). Binocular eye-movements were recorded by an EyeLink 1000 at a 250 Hz sampling rate with an accuracy of approximately 0.5° (SR Research Ltd., Mississauga, Ontario Canada). Participants were calibrated using an EyeLink 13-point calibration procedure and their heads were stabilized by a chin rest.

To assess the extent of visual attention directed at the task area compared to regions which are irrelevant for success in the MOXO-dCPT, the participants’ field of view (FOV) was divided into four areas of interest (AOIs; see Figure 1): (a) Task AOI: The only region relevant to the task, containing the target and non-target stimuli. (b) Distractors AOI: The screen region in which the distractors were presented. (c) Peripheral AOI: All other screen regions. (d) Outside the screen AOI: Regions beyond the computer screen.

Figure 1.

Participants’ FOV while performing the MOXO-dCPT, divided into the four AOIs. Presented is the target stimulus and examples of distractors.

Procedure

The participants filled out questionnaires (demographic/health, WURS 25-items, and ASRS), and underwent the SCID-5-RV. Next, they were moved to a sound-isolated room with black curtains behind the computer screen, to minimize external distractions. They were then calibrated to the eye-tracker and performed the MOXO-dCPT while their eye movements were monitored. Next, the ADHD patients filled out a brief questionnaire enquiring whether they feigned ADHD when originally diagnosed with the disorder. Participants performed the MOXO-dCPT approximately 40 to 50 minutes after beginning the experiment. As ADHD patients tire more easily (Rogers et al., 2017), a factor that may impact cognitive performance (Heaton et al., 2020), several measures was undertaken to limit the impact of possible fatigue; (a) The study was performed in the morning or early afternoon hours and participants were requested not to participate in the study following academic studies and/or work. (b) Before beginning the MOXO-dCPT, the research team member enquired regarding the participant’s fatigue and a short break was taken before proceeding to the MOXO-dCPT.

Data Analyses

Preliminary procedures and general remarks

For each AOI, the relative gaze duration was calculated (i.e., proportion of time the gaze was directed towards the AOI, relative to total gaze durations; Holmqvist et al., 2011, p. 118). All analyses were based on two-tailed hypotheses, with p < .05 marking statistical significance. Categorization of effect sizes (Cohen’s d) was based on Cohen (1992).

Primary analyses

A multivariate analysis of variance (MANOVA) with a between-subjects factor of group compared ADHD patients and healthy controls in the MOXO-dCPT indices. Independent-samples t-tests and chi-square analyses (for parametric and non-parametric variables, respectively) were used to compare the groups in all other measures. Welch’s unequal variance t-test was used when the assumption of group’s homogeneity of variance was not met (Ruxton, 2006). Group differences in relative AOI gaze duration were visualized using standard relative gaze duration heat maps, also termed density duration heat maps (Bojko, 2009, July). More specifically, a heat map was created per participant (i.e., accumulated time spent gazing at different regions on the screen, relative to the total duration of the task). Next, the heat maps of participants belonging to each group were aggregated. Color values were calculated using a standard EyeLink heat map generator, based on Gaussian density estimation (SR Research Ltd., Mississauga, Ontario Canada).

Pearson product-moment correlations between the MOXO-dCPT indices and the eye movement measures were performed as a preliminary step before performing the regression analyses, detailed later. The correlation matrix was inspected, and r > .8 criterion was utilized as a multicollinearity indicator. Multicollinearity was further assessed using the Variance Inflation Factor criterion (i.e., VIF < 4; O’Brien, 2007). Next, a simultaneous logistic regression assessed the ability of the eye movement measures to predict group membership. The regression model was then used to combine the eye movement measures into a continuous probability scale (termed combined eye movements scale or combined scale), the values of which represent the probability of being diagnosed with ADHD. Next, a two-stage hierarchal logistic regression was performed, aiming to determine whether the combined scale could enhance the predictive power of the MOXO-dCPT indices. The first stage model included the MOXO-dCPT indices in which the groups significantly differed as predictors, while the second stage included both the indices and the combined scale. In both regression analyses, a default probability cutoff of 0.5 was used to classify participants, based on the predicted values of the regression model.

Receiver operating characteristic (ROC) curve analyses assessed the classification accuracy of the eye movement measures and the combined eye movements scale. A ROC curve is a plot of true positive rates (sensitivity) against false positive rates (i.e., 1—specificity). The area under the curve (AUC) of these analyses was used as a measure of group discrimination capacity (range: 0.5–1.0; larger values indicating better discrimination capacity). Next, the AUC’s discrimination capacity was qualitatively interpreted in accordance with Hosmer et al. (2013, p. 177) and indices of classification accuracy (i.e., sensitivity and specificity) for various cutoffs were calculated.

Exploratory analyses

Differences in AOIs’ gaze durations could be due to fewer and longer gaze durations or to many short gazes. We therefore also analyzed group differences in the number of times the participants’ gaze exited or entered the task AOI (task AOI visit count measure).

Results

As expected, ADHD patients had significantly higher WURS 25-items and ASRS total scores than controls. Additionally, there were overall group differences in the MOXO-dCPT indices, F(4,61) = 3.28, p = .017, with follow-up analyses indicating that ADHD patients performed significantly worse than controls in the timeliness and impulsivity indices (p = .029, p = .012; respectively). See Table 1.

Table 1.

Descriptive Statistics and Statistical Analyses for ADHD Patients and Healthy Controls (n = 33 Per Group).

Measures		ADHD patients	Healthy controls	Statistical analyses	Effect size
Measures		Mean ± SD	Mean ± SD	Parametric analyses	Cohen’s d / η_p²
Age (years)		23.30 ± 2.13	23.33 ± 2.16	t(64) = .06, p = .954	0.01
Education level (years)		12.97 ± 1.10	12.88 ± 1.11	t(64) = .33, p = .740	0.08
WURS 25-items (no.)		28.27 ± 12.17	18.85 ± 9.74	t(64) = 3.7, p < .001	0.86
ASRS (no.)		30.88 ± 8.07	22.52 ± 5.84	t(64) = 4.82, p < .001	1.19
MOXO-d-CPT Indices	Attention (z-score)	−2.28 ± 3.31	−1.15 ± 3.06	F(1,64) = 2.06, p = .156	0.03
	Timelines (z-score)	−2.00 ± 2.04	−1.02 ± 1.48	F(1,64) = 5.00, p = .029	0.07
	Hyperactivity (z-score)	−2.08 ± 3.33	−0.76 ± 3.14	F(1,64) = 2.75, p = .102	0.04
	Impulsivity (z-score)	−1.01 ± 1.54	−0.14 ± 1.17	F(1,64) = 6.74, p = .012	0.10
Eye movements	Task AOI gaze duration (%)	80.48 ± 11.08	88.35 ± 6.63	t(52.3) = 3.50, p < .001	0.86
	Distractors AOI gaze duration (%)	18.18 ± 10.4	11.44 ± 6.44	t(53.4) = 3.17, p = .003	0.78
	Peripheral AOI gaze duration (%)	0.70 ± 0.79	0.14 ± 0.19	t(35.6) = 4.00, p < .001	0.97
	Outside the screen AOI gaze duration (%)	0.64 ± 1.18	0.07 ± 0.14	t(33) = 2.76, p < .001	0.68
	Task AOI visit count (no.)	249.76 ± 97.71	189.3 ± 89.55	t(64) = 2.62, p = .011	0.65
	Combined scale (probability)	0.6723 ± 0.2890	0.3276 ± 0.1627	t(50.4) = 5.97, p < .001	1.60
		Frequencies	Frequencies	Non-parametric analyses	OR
Gender (male / female)		11 / 22	9 / 24	χ²(1) = 0.29, p = .592	1.33
Birth country (Israel / other)		33 / 0	31 / 2	χ²(1) = 2.06, p = .151	___^a
Family status (single / married)		29 / 4	29 / 4	χ²(1) = 0.00, p = 1.00	1
Hand dominance (right / left)		32 / 1	27 / 5	χ²(1) = 3.08, p = .079	0.17
Corrected vision (yes / no)		6 / 27	4 / 29	χ²(1) = 0.47, p = .492	1.61

Note. AOI = area of interest; ASRS = adult ADHD self-report scale; WURS = Wender Utah rating scale; OR = odds ratio.

OR cannot be calculated due to division by zero.

Significant group differences were found in all eye movement measures. ADHD patients gazed less time at the task AOI (p < .001) and more time at the distractors / peripheral / outside the screen AOIs than controls (p = .002, p < .001, p = .008; respectively). They also had significantly higher combined scale scores than controls (p < .001). Effect sizes for these comparisons were medium (distractors AOIs gaze durations, outside the screen AOI gaze duration, task visit count) to large (task AOI gaze duration, peripheral AOI gaze duration, and combined scale). Importantly, all eye movement measures were above the recommended minimum effect size, thus representing a “practically” significant effect (Ferguson, 2009, p. 533). See Figure 2 for a visual presentation (i.e., heat maps) of the examinees’ eye movements while performing the MOXO-dCPT.

Figure 2.

Relative gaze duration heat maps, visualizing the proportion of accumulated gaze durations relative to the MOXO-dCPT total duration. The maps illustrate significant group differences in gaze behavior between the ADHD patients (A) and healthy controls (B).

Task AOI gaze duration and distractors AOI gaze duration measures were strongly negatively correlated (r= −0.99, p < .001), raising the possibility of multicollinearity between the two (see Supplemental Material 1). Since the latter measure had a lower effect size in the group comparisons (see Table 1), it was excluded from further analyses. All three remaining eye movement measures had VIF values < 4. A simultaneous logistic regression, therefore, included three measures as predicators (i.e., task AOI gaze duration, peripheral AOI gaze duration, and outside the screen AOI gaze duration). The model significantly predicted group membership (p < .001), correctly classifying 66.67% of ADHD patients and 90.91% of controls. See Table 2 for additional information, as well as the combined eye movements scale equation. The latter estimates the probability (range 0–1) of a participant being classified as an ADHD patient, based on their eye movement data. Next, a hierarchical logistic regression was conducted with timeliness and impulsivity MOXO-dCPT indices entered in the first stage, and both the indices and the combined scale entered in the second stage. The first stage model significantly predicted group membership, p = .003, correctly classifying 69.70% ADHD patients and 78.79% controls. The second stage model also significantly predicted group membership, p < .001, correctly classifying 75.76% of ADHD patients and 81.82% of controls. Out of the predictors, only the combined scale contributed significantly to the prediction of group membership (p < .001). Notably, the second stage model significantly enhanced the prediction of group membership, compared to the first stage model (p < .001). See Table 2.

Table 2.

Summary of Logistic Regression Analyses for Predicting ADHD Diagnosis (n = 33 per Group). (A) Simultaneous Logistic Regression with the MOXO-d-CPT Timeliness and Impulsivity Indices as Predictors. (B) Two-Stage Hierarchical Logistic Regression. The MOXO-d-CPT Timeliness and Impulsivity Indices Served as Predictors in Stage 1, While the Two Indices and the Combined Scale Served as Predictors in Stage 2.

Predictors	Model statistics		Coefficient statistics
A. Simultaneous logistic regression	χ²(df), p	Δχ²(df), p	B	SE	OR	OR 95% CI	Wald statistics
Stage 1	χ²(3) = 25.95, p < .001	____	____	____	____	____	____
1. Task AOI gaze duration (%)	____	____	0.0303	0.05	1.03	0.94–1.14	0.38, p = .539
2. Peripheral AOI gaze duration (%)	____	____	3.1063	1.35	22.34	1.59–314.44	5.30, p = .021
3. Outside the screen AOI gaze duration (%)	____	____	3.2209	2.05	25.05	0.45–1387.98	2.47, p = .116
Constant	____	____	−4.0899	4.59	0.02	____	0.80, p = .373
B. Two-stage hierarchical logistic regression	χ²(df), p	Δχ²(df), p	B	SE	OR	OR 95% CI	Wald statistics
Stage 1	χ²(2) = 11.80, p = .003	____	____	____	____	____	____
1. Timeliness index (z-score)	____	____	−0.3447	0.16	0.71	0.52–0.98	4.48, p = .034
2. Impulsivity index (z-score)	____	____	−0.5137	0.21	0.60	0.39–0.91	5.75, p = .016
Constant	____	____	−0.7872	0.38	0.46	____	4.28, p = .038
Stage 2	χ²(3) = 29.80, p < .001	χ²(1) = 18.00, p < .001	____	____	____	____	____
1. Timeliness index (z-score)	____	____	0.0850	0.206	1.089	0.73–1.63	0.17, p = 0.680
2. Impulsivity index (z-score)	____	____	−0.3520	0.240	0.703	0.44–1.13	2.157, p = .142
3. Combined eye movements scale (z-score)	____	____	5.8944	1.750	365.005	12.85–10225.76	11.956, p < .001
Constant	____	____	−2.8187	0.715	0.060	____	15.556, p < .001

Note. The combined eye movements scale equation:

P (A D H D) = \frac{1}{1 + e^{- (0.0303 * T a s k A O I G D + 3.1063 * P e r i p h e r a l A O I G D + 3.2209 * O u t s i d e t h e s c r e e n G D - 4.0899)}}

The equation’s outcome, P(ADHD), estimates the probability (range 0-1) of a participant being classified as an ADHD patient, based on their eye movement data.

AOI = area of interest; CI = confidence interval; OR = odds ratio; SE = standard error.

The ROC curve analyses indicated acceptable discriminative capacities of task AOI, peripheral AOI, and outside the screen AOI gaze durations (AUC = 0.731, AUC = 0.756, AUC = 0.775, respectively), while that of the combined scale was excellent (AUC = 0.826). The classification accuracy of cutoffs for these measures are presented in Supplemental Material 2. An inspection of the table reveals that perfect specificity could be reached using each of the measures. However, sensitivity was rather poor, ranging from 21.21% (outside the screen AOI gaze duration) to 30.30% for peripheral AOI gaze duration. Lowering specificity enables higher sensitivity levels to be reached, with the combined scale achieving the highest sensitivity, 75.76% (specificity = 81.82%). Note that specificity levels were capped at 80%.

Exploratory analyses

All MOXO-dCPT indices were inversely correlated with the combined scale, with effect size ranging between medium and strong (rs between –.29 and –.52). See Supplemental Material 1. Finally, ADHD patients made significantly more entries to the task AOI than controls (p = .011; See Table 1).

Discussion

The current study points toward the feasibility of integrating an eye tracker with a commercially available CPT (MOXO-dCPT). Importantly, four intuitive gaze direction measures, each associated with a different area of interest (AOI), significantly differentiated ADHD patients and healthy controls (effect sizes were large, except for outside the screen AOI gaze duration, which was associated with a moderate effect size). As visually represented in Figure 2, ADHD patients spent significantly more time than healthy controls gazing at screen regions in which distractors were presented, blank screen regions, and regions that were altogether outside of the screen. Only the center of the screen contains the stimuli that are relevant for success in the task. Such gaze behavior is therefore clearly counterproductive and can only impair performance on the CPT.

The above-mentioned group differences may stem from few, yet long, gazes at AOIs that are irrelevant for success in the CPT (i.e., wandering away from the task). Alternatively, they can be the product of a larger number of short gazes at the aforementioned AOIs (i.e., being drawn toward the distractors each time they appeared). Analyzing task AOI visits count indicated that ADHD patients made more entries to the task AOI, providing initial support for the latter possibility. Plainly put, ADHD patients were repeatedly drawn away from the task, perhaps by the MOXO unique “real-world” dynamic distractors. If so, the findings exemplify, in a face valid manner, the distractibility which may lie at the foundation of the higher-order deficits of ADHD patients (Butzbach et al., 2019). Surprisingly, although ADHD patients’ failure to ignore distractors is well-established (Barkley, 1997; Rubia et al., 1999; Tsal et al., 2005), only a single study has examined the effect of distractors on eye movements for the purpose of diagnosing ADHD (Vakil et al., 2019). Interestingly, the effect of the distractors on gaze direction measures best differentiated ADHD from controls in this study. Thus, this study reached the same conclusion as the current study, despite using a very different task (i.e., Stroop test). Such findings also correspond to those of studies using the anti-saccade task, in which participants are instructed to make a saccade (i.e., a rapid eye-movement to a specific location; Holmqvist et al., 2011, p. 23) away from a suddenly appearing stimulus. In these studies, ADHD patients fail to suppress saccades toward the stimulus (e.g., Coe & Munoz, 2017; O’Driscoll et al., 2005). This failure is related to the impulsivity aspect of ADHD (O’Driscoll et al., 2005; Roberts et al., 2011). All the above findings align with one another and point toward a failure by ADHD patients to inhibit responses toward distractors. Consequently, ADHD patients may pay less attention to stimuli that are relevant for success in daily tasks, thereby hampering their functioning. Corresponding to such a suggestion, the conventional MOXO-dCPT impulsivity index had the strongest effect size in differentiating the groups in the current study.

Beyond theoretical postulations, the current study aimed to assess the clinical utility of integrating the eye tracker with the MOXO-dCPT. The eye movement measures were, therefore, combined into a unified scale (termed combined eye movements scale or combined scale), which can be interpreted as an index of the participants’ distractibility (i.e., allocation of attention to irrelevant stimuli). The scale had the highest effect size out of the eye movement measures for differentiating the groups (Cohen’s d = 1.60). More importantly, however, it showed incremental utility in predicting an ADHD diagnosis, compared to the use of the conventional MOXO-dCPT indices. Correspondingly, the scale had excellent discrimination capacity (AUC = 0.83) and a 0.402 cutoff was associated with a 75.76% sensitivity and 81.82% specificity.¹ This is comparatively close to the 80% sensitivity/specificity biomarker criterion suggested by the WFSBP (Thome et al., 2012), and seems to align with other biomarkers of ADHD (e.g., EEG-TBR [theta/beta ratio] for classifying adults was associated with 83% sensitivity and 77% specificity; Muller et al., 2020). Overall, large effect sizes for differentiating the groups were found in the current study using the combined eye movements scale, tentatively suggesting that it may serve as a reliable neuropsychological diagnostic tool (Bezeau & Graves, 2001). However, several limitations of the study, detailed in the following paragraph, stress the preliminary nature of the findings, and call for further studies to validate and expand them.

By examining the utility of eye movements for the diagnosis of ADHD, the current study contributes to an under-explored area of research. Another asset is the study’s reliance on intuitive and face-valid gaze direction measures, capitalizing on the MOXO-dCPT’s distractors which are particularly suitable for exposing the distractibility of ADHD patients (Butzbach et al., 2019). Several limitations of the study should, however, be noted. First, the adult ADHD patients participating in the current study were all undergraduate students, and therefore likely represent highly functioning patients (Sedgwick, 2018). Second, participants with substantial comorbid psychopathology were excluded from the study. Since comorbidity is highly prevalent among ADHD patients (e.g., 55% of undergraduates with ADHD experience at least one comorbid condition; Anastopoulos et al., 2018), the generalizability of our findings is constrained. Third, the predictors of ADHD diagnosis were computed and tested in the same sample. This further emphasizes the need to validate the findings before reaching firm conclusions. In addition, future studies may address these limitations by assessing examinees undergoing real-life assessments, as well as assessing non-academic ADHD patients and those with comorbid psychopathology. Finally, comparing ADHD patients with those with other neuro-psychiatric disorder is in order (following Barkley, 2015), as well as investigating factors which may impact their performance in CPTs (e.g., fatigue; Rogers et al., 2017).

Summary

In this paper, we reported a feasibility study in which an eye tracker was integrated with a CPT, a neuropsychological task that is widespread in clinical practice. Using face-valid gaze direction measures, ADHD patients were found to spend more time gazing at irrelevant regions, either on the screen and outside of it, than healthy controls. Corresponding to the distractibility that is characteristic of ADHD, difficulty in suppressing spontaneous eye movements towards the MOXO-dCPT’s ecological distractors appears to drive the group differences. From a clinical perspective, a scale that combined eye movement measures and conventional indices showed excellent discrimination capacity. Moreover, initial evidence suggests that it may enhance the classification accuracy of conventional indices. Overall, the eye movement measures utilized in this study showed promise to increase our theoretical understanding of ADHD and improve clinical decision-making, aligning well with the rapidly expanding research of biomarkers in neuropsychiatry (Lozupone et al., 2019). However, further research is required to validate the findings while amending the study’s weaknesses.

Supplemental Material

sj-pdf-1-jad-10.1177_1087054720972786 – Supplemental material for Eye Tracking During a Continuous Performance Test: Utility for Assessing ADHD Patients

Supplemental material, sj-pdf-1-jad-10.1177_1087054720972786 for Eye Tracking During a Continuous Performance Test: Utility for Assessing ADHD Patients by Astar Lev, Yoram Braw, Tomer Elbaum, Michael Wagner and Yuri Rassovsky in Journal of Attention Disorders

Supplemental Material

sj-pdf-2-jad-10.1177_1087054720972786 – Supplemental material for Eye Tracking During a Continuous Performance Test: Utility for Assessing ADHD Patients

Supplemental material, sj-pdf-2-jad-10.1177_1087054720972786 for Eye Tracking During a Continuous Performance Test: Utility for Assessing ADHD Patients by Astar Lev, Yoram Braw, Tomer Elbaum, Michael Wagner and Yuri Rassovsky in Journal of Attention Disorders

Footnotes

Acknowledgements

The research reported in this paper was completed as part of the first author’s (Astar Lev) M.A. dissertation (Bar-Ilan University, Ramat Gan, Israel) and the undergraduate program requirements of Adi Narov, Adi Zafrir, Amit Yarkoni, Eden Katzav, Ela Drookman, Sean Shachar, and Tzoor Cohen. We thank them for their invaluable contribution to the study. Finally, we thank Corrine Berger who took part in the research project, as well as Estie Arram for proofreading and editing the final version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Yoram Braw

Yuri Rassovsky

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Astar Lev has an MA in Clinical-Rehabilitation Psychology (Bar-Ilan University) and is an intern at the Multiple Sclerosis Department at Sheba Medical Center, Israel. The current study was performed as part of her M.A. thesis.

Yoram Braw is a senior lecturer and the director of the Laboratory for Forensic and Clinical Neuropsychology, Department of Psychology, Ariel University, Israel. His research focuses on the integration of technology in neuropsychological tests and detection of non-credible performance.

Tomer Elbaum is a human factor engineer and a doctoral student at the Department of Industrial Engineering and Management at Ariel University, Israel. As part of his doctoral research, he uses advanced sensors as eye trackers and driver monitoring systems in the context of ADHD.

Michael Wagner is a senior lecturer and is the director of the Human-factors and Cognition Laboratory at the Department of Industrial Engineering and Management, Ariel, Israel. He is also the former head of the Israeli air-force human engineering branch and laboratory (lieutenant-colonel, retired). His research interests include eye movement analyses to the study of attention and use of other advanced technologies for the study of cognitive processes.

Yuri Rassovsky is an associate professor at the Department of Psychology and the Gonda multidisciplinary brain research center at Bar-Ilan University, Israel. He is also a research psychologist in the Department of Psychiatry and Biobehavioral Sciences at UCLA Semel institute for neuroscience and human behavior. His research focuses on innovative methods for cognitive enhancement and rehabilitation of neurocognitive deficits.

References

Albrecht

Uebel-von Sandersleben

Wiedmann

Rothenberger

(2015). ADHD history of the concept: The case of the continuous performance test. Current Developmental Disorders Reports, 2, 10−22.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Author.

Anastopoulos

A. D.

DuPaul

G. J.

Weyandt

L. L.

Morrissey-Kane

Sommer

J. L.

Rhoads

L. H.

Murphy

K. R.

Gormley

M. J.

Gudmundsdottir

B. G.

(2018). Rates and patterns of comorbidity among first-year college students with ADHD. Journal of Clinical Child and Adolescent Psychology, 47(2), 236−247. https://doi.org/10.1080/15374416.2015.1105137

Areces

Cueli

Garcia

Gonzalez-Castro

Rodriguez

(2018). Using brain activation (nir-HEG/Q-EEG) and execution measures (CPTs) in a ADHD assessment protocol. Journal of Visualized Experiments, 134, 56796. https://doi.org/10.3791/56796

Barbaresi

W. J.

Weaver

A. L.

Voigt

R. G.

Killian

J. M.

Katusic

S. K.

(2018). Comparing methods to determine persistence of childhood ADHD into adulthood: A prospective, population-based study. Journal of Attention Disorders, 22(6), 571−580. https://doi.org/10.1177/1087054715618791

Barkley

R. A.

(1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121(1), 65−94. http://www.ncbi.nlm.nih.gov/pubmed/9000892

Barkley

R. A.

(2015). Comorbid psychiatric disorders and psychological maladjustment in adults with ADHD. In Barkley

R. A.

(Ed.), Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (4th ed., pp. 343−355). Guilford Press.

Ben-Sheetrit

Tasker

Avnat

Golubchik

Weizman

Manor

(2017). Possible age-related progression of attentional impairment in ADHD and its attenuation by past diagnosis and treatment. Journal of Attention Disorders, 1−8. Advance online publication. https://doi.org/10.1177/1087054717743328

Berger

C. L. A.

Braw

Elbaum

Wagner

Rassovsky

(2019). Detection of feigned ADHD Using the MOXO-d-CPT. Journal of Attention Disorders, 1−16. Advance online publication. https://doi.org/10.1177/1087054719864656

10.

Berger

Cassuto

(2014). The effect of environmental distractors incorporation into a CPT on sustained attention and ADHD diagnosis among adolescents. Journal of Neuroscience Methods, 222, 62−68. https://doi.org/10.1016/j.jneumeth.2013.10.012

11.

Berger

Slobodin

Aboud

Melamed

Cassuto

(2013). Maturational delay in ADHD: Evidence from CPT. Frontiers in Human Neuroscience, 7, 1−11. https://doi.org/10.3389/fnhum.2013.00691

12.

Bezeau

Graves

(2001). Statistical power and effect sizes of clinical neuropsychology research. Journal of Clinical and Experimental Neuropsychology, 23(3), 399−406. https://doi.org/10.1076/jcen.23.3.399.1181

13.

Bojko

A. A.

(2009). Informative or misleading? Heatmaps deconstructed. In 13th International conference on human-computer interaction, San Diego, CA.

14.

Butzbach

Fuermaier

A. B. M.

Aschenbrenner

Weisbrod

Tucha

(2019). Basic processes as foundations of cognitive impairment in adult ADHD. Journal of Neural Transmission (Vienna, Austria: 1996), 126(10), 1347−1362. https://doi.org/10.1007/s00702-019-02049-1

15.

Cassuto

Ben-Simon

Berger

(2013). Using environmental distractors in the diagnosis of ADHD. Frontiers in Human Neuroscience, 7, 1−11. https://doi.org/10.3389/fnhum.2013.00805

16.

Coe

B. C.

Munoz

D. P.

(2017). Mechanisms of saccade suppression revealed in the anti-saccade task. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 372(1718), 20160198. https://doi.org/10.1098/rstb.2016.0192

17.

Cohen

(1992). A power primer. Psychological Bulletin, 112(1), 155−159.

18.

Dobrakowski

Lebecka

(2020). Individualized neurofeedback training may help achieve long-term improvement of working memory in children with ADHD. Clinical EEG and Neuroscience, 51(2), 94−101. https://doi.org/z10.1177/1550059419879020

19.

Elbaum

Wagner

Botzer

(2017). Cyclopean, dominant, and non-dominant gaze tracking for smooth pursuit gaze interaction. Journal of Eye Movement Research, 10(1), 1−14. https://doi.org/Artn210.16910/Jemr.10.1.2

20.

Ferguson

C. J.

(2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology-Research and Practice, 40(5), 532−538. https://doi.org/10.1037/a0015808

21.

First

M. B.

Williams

J. B. W.

Karg

R. S.

Spitzer

R. L.

(2015). Structured clinical interview for DSM-5- research version (SCID-5 for DSM-5, research version; SCID-5-RV). American Psychiatric Association.

22.

Fried

Tsitsiashvili

Bonneh

Y. S.

Sterkin

Wygnanski-Jaffe

Epstein

Polat

(2014). ADHD subjects fail to suppress eye blinks and microsaccades while anticipating visual stimuli but recover with medication. Vision Research, 101, 62−72. https://doi.org/10.1016/j.visres.2014.05.004

23.

Fuermaier

A. B. M.

Fricke

J. A.

de Vries

S. M.

Tucha

(2018). Neuropsychological assessment of adults with ADHD: A Delphi consensus study. Applied Neuropsychology: Adult, 26(5), 1−15. https://doi.org/10.1080/23279095.2018.1429441

24.

Goto

Hatakeyama

Kitama

Sato

Kanemura

Aoyagi

Sugita

Aihara

(2010). Saccade eye movements as a quantitative measure of frontostriatal network in children with ADHD. Brain & Development, 32(5), 347−355. https://doi.org/10.1016/j.braindev.2009.04.017

25.

Greenberg

L. M.

Holder

Kindschi

C. L.

Dupuy

T. R.

(2018). Test of variables of attention (T.O.V.A.): Clinical manual. The TOVA Company.

26.

Hall

C. L.

Valentine

A. Z.

Groom

M. J.

Walker

G. M.

Sayal

Daley

Hollis

(2016). The clinical utility of the continuous performance test and objective measures of activity for diagnosing and monitoring ADHD in children: A systematic review. European Child and Adolescent Psychiatry, 25(7), 677−699. https://doi.org/10.1007/s00787-015-0798-x

27.

Heaton

K. J.

Williamson

J. R.

Lammert

A. C.

Finkelstein

K. R.

Haven

C. C.

Sturim

Smalt

C. J.

Quatieri

T. F.

(2020). Predicting changes in performance due to cognitive fatigue: A multimodal approach based on speech motor coordination and electrodermal activity. The Clinical Neuropsychologist, 34(6), 1190−1214. https://doi.org/10.1080/13854046.2020.1787522

28.

Holmqvist

Nyström

Andersson

Dewhurst

Jarodzka

Van de Weijer

(2011). Eye tracking: A comprehensive guide to methods and measures. Oxford University Press.

29.

Hosmer

D. W.

Lemeshow

Sturdivant

R. X.

(2013). Applied logisitic regression (3rd ed.). John Wiley & Sons.

30.

Jacoby

Lavidor

(2018). Null tDCS effects in a sustained attention task: The modulating role of learning. Frontiers in Psychology, 9(476), 1−9. https://doi.org/10.3389/fpsyg.2018.00476

31.

Kessler

R. C.

Adler

L. A.

Gruber

M. J.

Sarawate

C. A.

Spencer

Van Brunt

D. L.

(2007). Validity of the world health organization adult ADHD self-report scale (ASRS) screener in a representative sample of health plan members. International Journal of Methods in Psychiatric Research, 16(2), 52−65. https://doi.org/10.1002/mpr.208

32.

Lozano

Ó. M.

Carmona

Muñoz-Silva

Fernández-Calderón

Díaz-Batanero

Sanchez-Garcia

. (2020). Adult ADHD self-report scale comprehensive psychometric study in a Spanish SUD sample. Journal of Attention Disorders, 24, 1674–1684. https://doi.org/10.1177/1087054716664410

33.

Lozupone

La Montagna

D’Urso

Daniele

Greco

Seripa

Logroscino

Bellomo

Panza

(2019). The role of biomarkers in psychiatry (ch. 7). In Guest

P. C.

(Ed.), Reviews on biomarker studies in psychiatric and neurodegenerative disorders (pp. 135−162). Springer.

34.

Marshall

Hoelzle

Nikolas

(2019). Diagnosing attention-deficit/hyperactivity disorder (ADHD) in young adults: A qualitative review of the utility of assessment measures and recommendations for improving the diagnostic process. The Clinical Neuropsychologist, 1−34. Advance online publication. https://doi.org/10.1080/13854046.2019.1696409

35.

Muller

Vetsch

Pershin

Candrian

Baschera

G. M.

Kropotov

J. D.

Kasper

Rehim

H. A.

Eich

(2020). EEG/ERP-based biomarker/neuroalgorithms in adults with ADHD: Development, reliability, and application in clinical practice. World Journal of Biological Psychiatry, 21(3), 172–182. https://doi.org/10.1080/15622975.2019.1605198

36.

O’Brien

R. M.

(2007). A caution regarding rules of thumb for variance inflation factors. Quality & Quantity, 41(5), 673−690. https://doi.org/10.1007/s11135-006-9018-6

37.

O’Driscoll

G. A.

Depatie

Holahan

A. L.

Savion-Lemieux

Barr

R. G.

Jolicoeur

Douglas

V. I.

(2005). Executive functions and methylphenidate response in subtypes of attention-deficit/hyperactivity disorder. Biological Psychiatry, 57(11), 1452−1460. https://doi.org/10.1016/j.biopsych.2005.02.029

38.

Parsons

T. D.

Rizzo

A. S.

(2019). A review of virtual classroom environments for neuropsychological assessment (ch. 11). In Rizzo

A. S.

Bouchard

(Eds.), Virtual reality for psychological and neurocognitive interventions (pp. 247−265). Springer.

39.

Pettersson

Soderstrom

Nilsson

K. W.

(2018). Diagnosing ADHD in adults: An examination of the discriminative validity of neuropsychological tests and diagnostic assessment instruments. Journal of Attention Disorders, 22(11), 1019−1031. https://doi.org/10.1177/1087054715618788

40.

Roberts

Fillmore

M. T.

Milich

(2011). Linking impulsivity and inhibitory control using manual and oculomotor response inhibition tasks. Acta Psychologica, 138(3), 419−428. https://doi.org/10.1016/j.actpsy.2011.09.002

41.

Rogers

D. C.

Dittner

A. J.

Rimes

K. A.

Chalder

(2017). Fatigue in an adult attention deficit hyperactivity disorder population: A trans-diagnostic approach. The British Journal of Clinical Psychology, 56(1), 33−52. https://doi.org/10.1111/bjc.12119

42.

Rossini

E. D.

O’Connor

M. A.

(1995). Retrospective self-reported symptoms of attention-deficit hyperactivity disorder: Reliability of the Wender Utah rating scale. Psychological Reports, 77(3 Pt 1), 751−754. https://doi.org/10.2466/pr0.1995.77.3.751

43.

Rubia

Overmeyer

Taylor

Brammer

Williams

S. C.

Simmons

Bullmore

E. T.

(1999). Hypofrontality in attention deficit hyperactivity disorder during higher-order motor control: A study with functional MRI. American Journal of Psychiatry, 156(6), 891−896. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=10360128

44.

Ruxton

G. D.

(2006). The unequal variance t-test is an underused alternative to Student’s t-test and the Mann-Whitney U test. Behavioral Ecology, 17(4), 688−690. https://doi.org/10.1093/beheco/ark016

45.

Sedgwick

J. A.

(2018). University students with attention deficit hyperactivity disorder (ADHD): A literature review. Irish Journal of Psychological Medicine, 35(3), 221−235. https://doi.org/10.1017/ipm.2017.20

46.

Shahaf

Nitzan

Erez

Mendelovic

Bloch

(2018). Monitoring attention in ADHD with an easy-to-use electrophysiological index. Frontiers in Human Neuroscience, 12, 1−9. https://doi.org/10.3389/fnhum.2018.00032

47.

Slobodin

Blankers

Kapitany-Foveny

Kaye

Berger

Johnson

Demetrovics

van den Brink

van de Glind

(2020). Differential diagnosis in patients with substance use disorder and/or attention-deficit/hyperactivity disorder using continuous performance test. European Addiction Research, 26(3), 151−162. https://doi.org/10.1159/000506334

48.

Slobodin

Davidovitch

(2019). Gender differences in objective and subjective measures of ADHD among clinic-referred children. Frontiers in Human Neuroscience, 13, 441. https://doi.org/10.3389/fnhum.2019.00441

49.

Thome

Ehlis

A. C.

Fallgatter

A. J.

Krauel

Lange

K. W.

Riederer

Romanos

Taurines

Tucha

Uzbekov

Gerlach

(2012). Biomarkers for attention-deficit/hyperactivity disorder (ADHD). A consensus report of the WFSBP task force on biological markers and the world federation of ADHD. World Journal of Biological Psychiatry, 13(5), 379−400. https://doi.org/10.3109/15622975.2012.690535

50.

Tsal

Shalev

Mevorach

(2005). The diversity of attention deficits in ADHD: The prevalence of four cognitive factors in ADHD versus controls. Journal of Learning Disabilities, 38(2), 142−157. http://www.ncbi.nlm.nih.gov/pubmed/15813596

51.

Vakil

Mass

Schiff

(2019). Eye movement performance on the stroop test in adults with ADHD. Journal of Attention Disorders, 23(10), 1160−1169. https://doi.org/10.1177/1087054716642904

52.

Ward

M. F.

Wender

P. H.

Reimherr

F. W.

(1993). The Wender Utah rating-scale - an aid in the retrospective diagnosis of childhood attention-deficit hyperactivity disorder. American Journal of Psychiatry, 150(6), 885−890.

53.

Zohar

A. H.

Konfortes

(2010). Diagnosing ADHD in Israeli adults: The psychometric properties of the adult ADHD self report scale (ASRS) in Hebrew. The Israel Journal of Psychiatry and Related Sciences, 47(4), 308−315. https://www.ncbi.nlm.nih.gov/pubmed/21270505

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.17 MB

0.23 MB