ImGo: A Novel Tool for Behavioral Impulsivity Assessment Based on Go/NoGo Tasks

Abstract

This manuscript aims to present a novel behavioral impulsivity test ImGo, which is suitable for impulsivity assessment in the general population. A series of three studies was conducted to validate its psychometric qualities. In Study 1 we describe the principles of ImGo and verify its test-retest and split-half reliability and its convergent validity with an impulsivity self-report scale and Stop Signal test. In Study 2 we re-analyze the convergent validity of ImGo with a Stop Signal test and examine the potential relationship between ImGo and oculomotor inhibition measured by an Anti-Saccades test. In Study 3 we present a robust research with a large sample size and investigate the discriminant validity of ImGo with tests of other related cognitive and executive processes. Backed by our findings from these studies we can safely claim ImGo is a powerful tool with a good level of reliability (both test-retest and split-half) and validity (convergent and discriminant). Its potential lies in its use in diagnostic and research practice of experts from various countries as the test has already been translated to 9 languages so far. The open-source Hypothesis platform, on which the ImGo test is running, provides the option of both individual and group testing in laboratory conditions as well as remotely through an internet browser.

Keywords

Behavioral impulsivity waiting impulsivity go/no-go test stop signal test validation anti-saccade task computer-based assessment group test open-source software Hypothesis

General introduction

Impulsivity is defined as a tendency to act prematurely without foresight (Dalley et al., 2011) and is considered a multidimensional construct. For example, Patton et al. (1995) proposed a three factor model consisting of attentional impulsivity, motor impulsivity and non-planning impulsivity; Nigg (2000) distinguished between four processes which generate impulsivity: interference control, cognitive inhibition, behavioral inhibition and oculomotor inhibition; Lynam et al. (2006) described five dimensions of impulsivity: negative urgency, positive urgency, lack of premeditation, lack of perseverance and sensation seeking; and Dalley et al. (2011) differentiated between stopping and waiting behavioral impulsivity.

The multitude of conceptual definitions for impulsivity is naturally reflected in a large number of methods used to measure it. Authors commonly use standardized self-report scales (e.g., Lynam et al., 2006; Patton et al., 1995) or performance tests based on variations of the Go/NoGo test, anti-saccades test and Stop Signal test. This diversity of measurement has naturally led to a general lack of intercorrelations between methods, which suggests that impulsivity manifests at the behavioral level in different ways (Dalley et al., 2011).

Consequently, impulsivity is associated with many other psychological and psychiatric phenomena, for example with attention disorders (Metin et al., 2012), drug addictions (Smith et al., 2014), personality disorders (Coffey et al., 2011), mental disorders (Enticott et al., 2008), interference control (Aichert et al., 2012), and intelligence (Lynam et al., 1993). Research into impulsivity has also a large potential in clinical psychology and psychiatry as an important predictor of other disorders within clinical populations (Moeller et al., 2001). The construct of impulsivity is also a core symptom domain in many various clinical diagnoses in the Diagnostic and Statistical Manual of Mental Disorders V (cf. Berlin & Hollander, 2014). Therefore, a large part of the research of impulsivity is performed on clinical and subclinical populations.

The present paper consists of three studies and its aim is to introduce a novel impulsivity test called ImGo, which is based on the Go/NoGo test and offers objective testing of behavioral impulsivity. For several reasons we believe that ImGo can be perceived as an improvement to the existing battery of similar objective methods for impulsivity assessment. First of all, our method is standardized on quite a large population, which is an issue with most of the available methods as reported by Matusiewicz (Matusiewicz et al., 2011). Even though there are other similar methods with relatively good psychometric properties such as Conners' Continuous Performance Test (CCPT II, Conners, 2000; Conners et al., 2003; Shaked et al., 2020) the ImGo has several properties that make it quite useful for especially large scale assessment. The test was implemented on the online testing platform Hypothesis (Šašinka, Morong, et al., 2017) that requires only an internet connection, mouse and keyboard and standard web browser, no special equipment or software is needed. The Hypothesis software solution thus allows quick online administration on multiple (up to several dozen) participants in a parallel fashion. Additionally, its module-based architecture allows for quick adaptation of the test (e. g. into other languages). Therefore, we believe that ImGo is suitable for testing both within a general population (research, traffic psychology, organizational psychology, etc.) and for clinical purposes (diagnosis of ADHD, differential diagnosis of dyslexia, pathological gambling, etc.).

In the first study, we describe the ImGo instrument and verify its test-retest and split-half reliability and its convergent validity with an impulsivity self-report scale and Stop Signal test. In the second study, we re-analyze the convergent validity of ImGo with a Stop Signal test and examine the potential relationship between ImGo and oculomotor inhibition measured by an Anti-Saccades test. The third study provides robust research with a large sample size and investigates the discriminant validity of ImGo with other related cognitive and executive processes, such as intelligence, attention and inference control.

Study 1

Introduction

The aim of Study 1 is to describe a newly developed ImGO test for measuring behavioral waiting impulsivity and gather evidence of its test-rest reliability, split-half reliability and convergent validity with other behavioral and self-report measures of impulsivity. Even though reliability estimation is relatively common in self-report questionnaire assessment, its thorough psychometric evaluation is not less crucial in the performance measures of cognitive processing, as high reliability reduces measurement error (Parsons et al., 2019). Nevertheless, even evidence of high reliability is not sufficient in psychological research if the method lacks validity, which, however, usually cannot be expressed – unlike reliability – by a certain number. We therefore also analyzed convergent validity (i.e., the relationship between two measures of a construct that should be theoretically related) of the ImGo with behavioral stopping impulsivity measured by a Stop-signal test and a self-report Impulsive Behavior Scale. Since we did not test concurrent validity (i.e., two instruments which measure the same construct), we did not expect strong correlations between methods. However, since all three tests were expected to measure different aspects of impulsivity, they should have been at least weakly associated (e.g., Cyders & Coskunpinar, 2011; Reynolds et al., 2006).

Methods

Tests

ImGo test. The ImGo test was designed by Řádová et al. (2018) as an instrument for behavioral measurement and analysis of waiting impulsivity (Dalley et al., 2011) on go/no-go trials. Stimuli are designed as geometric shapes, more specifically, square, cross, triangle, line, star and hexagon. Participants are instructed to press a spacebar as quickly as possible whenever they see a cross, triangle, line, star or hexagon (i.e., go trials), and to inhibit pressing the key when presented with a square (i.e., no-go trials; see Figure 1(a)). ImGo has the following general procedure: 1) fixation target presentation (500 ms), 2) stimuli presentation (200 ms or key press), and 3) blank screen (900 ms or key press; see Figure 1(b)).

Figure 1.

(a) NoGo and Go stimuli. (b) Procedure of one ImGo trial.

ImGo contains 28 training trials with feedback and 12 training trials without, plus 300 test trials (48 no-go and 252 go items; see Figure 2). The test is administered in a pseudo-random order, with a 30 second break in the middle of the test part. The whole test takes about 15 minutes.

Figure 2.

Overall ImGo procedure.

Three metrics are usually reported in the family of go/no-go methods (Meule, 2017, p. 1) commission errors (i.e., incorrectly pressed button in no-go trials; also called false alarms), 2) reaction times in go-trials, and 3) omission errors (i.e., incorrectly not pressing the button in go trials; also called misses). However, due to the asymmetric distribution of RTs, which is usually right-skewed (Van Zandt, 2002), scholars sometimes work with (besides the above-mentioned) coefficient of variation (i.e., relative standard deviation; calculated as $C V = σ / μ$ ) and ex-Gaussian parameters of reaction times (i.e., mu, sigma and tau; e.g., Epstein et al., 2011) or diffusion models (e.g., Ratcliff, et al., 2018).

Stop-signal test. The first method used to evaluate the convergent validity of ImGo is the stop-signal test (SST). SST is an instrument for stopping impulsivity assessment (see Dalley et al., 2011). The test was originally created by Fabianová (2019) with respect to the general recommendations for SST tasks (cf. Verbruggen et al., 2019). As with the Go/NoGo paradigm, the SST contains go and stop (no-go) trials. In the go trials, the participant must press the left or right button as quickly as possible depending on the direction of the arrow (Figure 3). In the stop trials, a stop symbol (violet rectangle around the arrow) appears to signal the participant not to press any button (Figure 3). The stop signal trials vary in interval between stimulus onset and the stop symbol appearance, a so-called stop-signal delay (SSD). In other words, the stop symbol in the stop signal trials appears with delays of various duration. Seven different SSDs are used (150 ms, 200 ms, 250 ms, 300 ms, 350 ms, 400 ms and 450 ms). One trial cycle is as follows: 1) fixation target presentation (800 ms) and 2) stimulus presentation (800 ms or key press; Figure 3).

Figure 3.

Procedure of the SST’s trial with go/stop symbol.

Figure 4.

Overall SST procedure.

In total, the test contains 720 trials (180 stop signal trials and 540 go trials) divided into six blocks (Figure 4). Each of the six blocks contains 30 stop signal and 90 go trials; the order of the trials in each block is pseudorandom. The whole test takes about 25 minutes.

Several metrics are commonly reported in the SST family of tests: a) go omission errors (i.e., no response) on go trials, b) choice errors on go trials, c) choice errors on unsuccessful stop trials, d) go RTs, and e) stop-signal RTs (SSRT; i.e., the time it takes to complete the inhibitory process after the appearance of the stop-signal; Verbruggen et al., 2019). However, because SSRT cannot be observed directly (unlike go RT), is it typically estimated within the independent horse-race model (Logan & Cowan, 1984), which offers various estimation procedures and possible behavioral metrics (for review, see Band et al., 2003; Verbruggen et al., 2019; Verbruggen & Logan, 2009). Since all SSDs were set a priori, we used the fixed-SSDs procedure (alternatively, a tracking procedure could have been used in the case of adaptive adjustments of SSDs based on the performance of the participants; cf. Verbruggen & Logan, 2009). The SSRT was estimated for each participant per each SSD block through the integration method, which in many ways outperforms the most commonly used mean method of estimation (Verbruggen et al., 2013), and subsequently averaged into a single score (Band et al., 2003).

UPPS-P & S-UPPS-P

The second method for ImGo convergent validity assessment is a Czech mutation of the self-report Impulsive Behavior Scale (UPPS-P), translated and validated by Linhartová et al. (2017). UPPS-P is a 59-item self-report questionnaire (sample item e.g., When I am upset, I often act without thinking; Lynam et al., 2006), which is composed of five subscales, namely: negative urgency, positive urgency, lack of premeditation, lack of perseverance and sensation seeking. Participants answer the items on a four-point ordinal scale (anchors: 1 – agree strongly to 4 – disagree strongly. A briefer version of the scale, S-UPPS-P (see Cyders et al., 2014), contains 20 items (four items per subscale).

Power analysis

An a priori power analysis was performed in G*Power (v3.1.9.7; Faul et al., 2009). Since previous research has reported rather small effect sizes for relationships between behavioral impulsivity and self-report questionnaires (e.g., Cyders & Coskunpinar, 2011) and relationships between two behavioral impulsivity instruments (e.g., Reynolds et al., 2006), we set the effect size to 0.30, significance level to .05 and power to .80 for two-tailed bivariate correlations. The power analysis results with these settings suggested that the total sample size should be at least 84 participants in order to achieve sufficient power.

Sample and procedure

We gathered data from 81 participants aged 19–45 years (M = 23.05, SD = 5.30). Most were women (n = 51, 63%) and had high school education (n = 60, 74%). Participants were tested twice in separate sessions. The time interval between the sessions was between 1 and 28 days (M = 8.09, SD = 5.21). Participants were contacted through student groups on social networking services. Participants received small rewards (e.g., USB flash drives) for their participation. Participants were asked not to consume alcohol and caffeine before testing. They were also examined for hand dominance and asked to use their dominant hand to press the button. Informed consent was obtained from participants before administration of the test battery.

The SST and ImGo tests were administered on SW Hypothesis (Šašinka, Morong, et al., 2017), the UPPS-P was administered via Google Forms. Data were collected under controlled conditions in a computer room at Masaryk university (common PC mouse, keyboard and LCD screen 22’ with resolution 1920 x 1080). All statistical analyses were performed in R (v3.6.2; R Core Team, 2020), especially with the packages lavaan (v0.6–6; Rosseel, 2012), semTools (v0.5–2; Jorgensen et al., 2018) and irr (v0.84.1; Gamer et al., 2019) and in JASP (v0.11.1.0).

Data inspection and cleaning

We set several criteria for data exclusion before data inspection and statistical analysis. Because the three methods used were of different character, we described the exclusion criteria for each separately. In the ImGo test, reaction times with anticipation reactions (i.e., participants reacted more quickly than he/she were able to even see the stimuli, because the stimulus did not appear yet – RTs less than 500 ms in case of ImGo) were not included in the computation of mean RTs. Mean RTs were calculated only from correctly answered go trials without errors (i.e., RTs were less than 1600 ms). Two participants were excluded because of a high number of ImGo omission errors (greater than 10%; based on Congdon et al., 2012).

In the SST test, individuals with an absolute response rate (i.e., probability of correctly responding on a stop trial was equal to 0% or 100%) were omitted from the SSRT estimation (this exclusion criterion is based on Congdon et al., 2012, who recommend excluding extreme values close to 100% or 0% of correctness in the fixed-SSDs procedure). RTs (both go and stop-signal) were calculated only from trials without errors. Four participants were excluded because of a high number of SST go errors (i.e., go commission errors + go choice error were greater than 15%; on the basis of a rule proposed by Congdon et al., 2012). As recommended, for example by Ratcliff (1993), we avoided any other type of rule-of-thumb removal of RT outliers (e.g., ±2SD) in both methods because of the already mentioned specific skewed data distribution typical for RTs.

In addition, five participants did not participate in the second testing and were excluded from the test-retest analysis for both the ImGo and SST.

Regarding the UPPS-P and S-UPPS-P scale, one item forming part of both versions of the questionnaire was discarded for technical problems which occurred during administration (item not recorded for most participants).

To choose a suitable statistical procedure, the assumptions of the correlation analyses were tested. We inspected data distributions, performed Shapiro–Wilk tests and analyzed the presence of outliers. We applied Spearman’s rank correlation (r_s) and Pearson’s product moment correlation (r_p) coefficients depending on the acquired results (de Winter et al., 2016). All p-values obtained from the correlation analyses were corrected with the Holm–Bonferroni method for multiple comparisons to reduce any potential type I error rate (Curtin & Schulz, 1998). Regarding the test-retest reliability of ImGo, intraclass correlation coefficients (ICC) with two-way random effects model were calculated (Weir, 2005).

Results

The aim of study 1 was to verify the psychometric properties of behavioral instruments for measuring impulsivity. In the first step, we examined the test-retest reliability of ImGo and SST which was satisfactory for both ImGo indicators (r = .418 – .717; ICC = .355 – .653) and the SST indicators (r = .424 – .732), suggesting that both instruments were stable in time (Table 1). In the next step, we performed split-half reliability estimation on two identical halves from the first wave of testing. The results also suggested satisfactory reliability for both ImGo (r = .504–.733) and SST (r = .696–.836; Table 1). All pairwise relationships showed medium to high correlation coefficients and were statistically significant. Their confidence intervals did not contain the value of zero effect (Table 1). Hence, both methods could be considered sufficiently reliable.

Table 1.

Descriptive statistics and reliability estimation of the ImGo and the SST.

Variable	M (SD)	Me (IQR)	S–W	Test-retest [95% CI]	Split-half [95% CI]
ImGo commissions (%)	46.5 (22.5)	43.7 (34.4)	p > .05	r_p = .559*** [.380, .697]ICC = .559*** [.381, .694]	r_p = .694*** [.561, .792]
ImGo omissions (%)	1.8 (1.9)	1.2 (2.2)	p < .05	r_s = .418*** [.211, .589]ICC = .355*** [.141, .538]	r_s = .504*** [.321, .651]
ImGo RT (ms)	355 (64.8)	332 (70.6)	p > .05	r_p = .486*** [.292, .642]ICC = .486*** [.292, .641]	r_p = .687*** [.552, .787]
ImGo CV	0.270 (.11)	0.255 (.10)	p < .05	r_s = .717*** [.586, .812]ICC = .653*** [.501, .766]	r_s = .733*** [.612, .820]
SST go omission errors (%)	4.7 (3.2)	4.1 (3.3)	p < .05	r_s = .424*** [.199, .607]	r_s = .753*** [.640, .834]
SST go choice errors (%)	1.4 (2.1)	0.5 (1.8)	p < .05	r_s = .503*** [.293, .666]	r_s = .696*** [.562, .792]
SST stop choice errors (%)	48.9 (23.1)	47.2 (35.0)	p > .05	r_p = .732*** [.592, .828]	r_p = .832*** [.752, .890]
SST go RT (ms)	526 (69.7)	535 (111)	p < .05	r_s = .646*** [.476, .770]	r_s = .836*** [.756, .892]
SST SSRT (ms)	250 (30.8)	244 (42.3)	p > .05	r_p = .552*** [.355, .703]	r_p = .746*** [.626, .827]

*p < .05 (two-tailed); **p < .01 (two-tailed); ***p < .001 (two-tailed); [ ]: 95% confidence intervals; M: mean; SD: standard deviation; Me: median; IQR: interquartile range; S-W: Shapiro–Wilk test; SST: stop-signal test; RT: reaction time; CV: coefficient of variance; SSRT: stop-signal reaction time; ICC: intraclass correlation coefficient.

The reliability and validity of the U-PSS-P questionnaire were subsequently verified. In the first step, we used confirmatory factor analysis to verify the five-factor UPPS-P configural model. Because of the small sample size, an MLR estimator was used instead of WLSMV since WLSMV yields moderate overestimation of the interfactor correlations at small sample sizes (Li, 2016). However, the 59-item version had unsatisfactory fit indices (χ²(1585) = 2674.425, p < .001, RMSEA [90% CI] = .091 [.085, .097], SRMR = .107, CFI = .611, TLI = .595). Consequently, the shortened version S-UPPS-P was also verified with much better fit indices (χ²(142) = 178.831, p = .020, RMSEA [90% CI] = .056 [.024, .080], SRMR = .092, CFI = .902, TLI = .882). Since the majority of fit indices are within commonly used rules-of-thumb (except values of SRMR and TLI), the configural model fit can be thus considered satisfactory for our purposes. Regarding internal consistency, the McDonald’s ω coefficients of the Czech S-UPPS-P varied between .623−.789 (total .841, M = .704), which is considered satisfactory in terms of a low number of items per scale (Table 2).

Table 2.

Descriptive statistics and reliability estimation for S-UPPS-P.

Subtest	M (SD)	S–W	McDonald's ω	Cronbach’s α [% CI]
S-UPPS-P NU	2.6 (0.60)	p > .05	.683	.674 [.540, .776]
S-UPPS-P LPR	2.8 (0.48)	p > .05	.718	.661 [.522, .767]
S-UPPS-P LPE	2.9 (0.62)	p > .05	.623	.597 [.417, .729]
S-UPPS-P SS	2.5 (0.72)	p > .05	.708	.680 [.549, .780]
S-UPPS-P PU	3.1 (0.59)	p > .05	.789	.785 [.696, .852]

[ ]: 95% confidence intervals; M: mean; SD: standard deviation; S–W: Shapiro–Wilk test; NU: negative urgency; PU: positive urgency; LPR: lack of premeditation; LPE: lack of perseverance; SS: sensation seeking.

Finally, the convergent validity of the ImGo test with the SST and the S-UPPS-P was verified. Only variables relevant for impulsivity measurement were included into the correlation analysis. Therefore, ImGo omissions, SST go omission errors, SST go choice errors and SST go RT were omitted. Results show that the SST stop choice errors correlate with ImGo commissions (r_p = .509). Correlations between the SST stop choice errors and ImGO RT (r_p = −.284) and ImGo CV (r_s = .294) were statistically significant only without correction for multiple comparisons (p < .05). Regarding convergent validity with the self-report questionnaire, the only significant correlation was observed between ImGo coefficient of variance and S-UPPS-P negative urgency subscale (r_s = −.394), meaning that participants with higher level of negative urgency also showed lower variability in their responses. Nevertheless, we did not find any significant correlations between coefficient of variation of the ImGo test and the stop-signal reaction time of the SST test, which can be considered the most important indicators of behavioral impulsivity (Table 3).

Table 3.

Convergent validity of the ImGo with SST and S-UPPS-P instruments.

	ImGo commissions	ImGo RT	ImGo CV
S-UPPS-P NU	r_p = –.204 [–.407, .018]	r_p = .180 [–.043, .386]	r_s = –.394** [–.566, –.189]
S-UPPS-P LPR	r_p = –.001 [–.222, .220]	r_p = .149 [–.075, .358]	r_s = –.048 [–.266, .175]
S-UPPS-P LPE	r_p = –.053 [–.271, .170]	r_p = –.015[–.236, .206]	r_s = –.037 [–.256, .186]
S-UPPS-P SS	r_p = .102 [–.122, .316]	r_p = .041[–.182, .260]	r_s = –.053 [–.271, .170]
S-UPPS-P PU	r_p = –.002 [–.223, .219]	r_p = .004 [–.217, .225]	r_s = –.092 [–.307, .132]
SST stop choice errors	r_p = .509*** [.319, .660]	r_p = –.284 [–.480, –.061]	r_s = .294 [.071, .488]
SST SSRT	r_p = .168 [–.061, .381]	r_p = .033 [–.195, .258]	r_s = .104 [–.126, .324]

*p_Holm < .05 (two-tailed); **p_Holm < .01 (two-tailed); *** p_Holm < .001 (two-tailed); [ ]: 95% confidence intervals; SST: stop-signal test; RT: reaction time; CV: coefficient of variance; SSRT: stop-signal reaction time; NU: negative urgency; PU: positive urgency; LPR: lack of premeditation; LPE: lack of perseverance; SS: sensation seeking.

Discussion

The aim of Study 1 was to verify the reliability and convergent validity of the ImGo instrument. Reliability was considered satisfactory for both the test-retest and split-half. Even though the findings suggested a lower ICC for ImGo omission errors, these two metrics are not necessary criteria for reliability since this indicator is primarily a control function. Moreover, similar test-retest results were observed by Hedge et al. (2018), who found that omission errors ICC were also lower than go RT and commission errors.

Regarding the convergent validity with self-report questionnaire, our results agree with previous studies. Meta-analysis of 28 studies of associations between the self-report and behavioral measures of impulsivity suggests that those associations are statistically significant yet very weak (r coefficients are lower than .30; Cyders & Coskunpinar, 2011). Other systematic review has even proposed that self-report and behavioral measures of impulsivity might reflect more distinct theoretical constructs (Newman & Meyer, 2014). More specifically, the vast majority of research has usually yielded a statistically significant correlation between the no/go-no method and only one subscale of UPPS-P or other self-report impulsivity questionnaires (particular subscales which demonstrate this association are inconsistent across studies, however) and shown that such correlations tend to be very weak (e.g., Aichert et al., 2012; Malesza & Ostaszewski, 2016; Perales et al., 2009; Reynolds et al., 2006; Spinella, 2004) or even non-significant (Hasegawa et al., 2019). In the terms of previous findings, a correlation found between ImGo and negative urgency (r_s = −.394) appears not only strong but also crucial from theoretical point of view since this subscale better explains the inter-individual variability in behavioral impulsivity instrument than other subscales (see Wilbertz et al., 2014), and thus, it is an important criterion of convergent validity. However, this finding is interesting also due to its valency, which is opposite than expected. Our result suggests that participants with a high level of negative urgency showed lower variability of their reaction times which means that they were more stable during answering. This result is probably caused by the fact that self-report methods measure a different facet of impulsivity than cognitive tests.

Similar results can be observed in previous studies regarding the association between two behavioral measures of impulsivity – go/nogo and SST. For example, Aichert et al. (2012) and Hasegawa et al. (2019) did not find a statistically significant association, Reynolds et al. (2006) found only weak correlation, and Hedge et al. (2018) found medium correlation. Previous findings suggest that the go/nogo tasks and stop-signal tasks might measure related yet slightly different cognitive processes (e.g., waiting and stopping impulsivity) and simultaneously saturate motoric impulsivity. These processes seem to recruit widely different neural dynamics (Dalley et al., 2011; Raud et al., 2020), which might manifest especially in cognitive deficits. For example, people with ADHD have demonstrated high levels of both types of impulsivity (e.g., Rubia et al., 2001; Schachar et al., 2007), whereas people with addictions (for review, see Smith et al., 2014) and people with borderline personality disorder (e.g., Barker et al., 2015; Cackowski et al., 2014; Wright et al., 2014 Wright et al., 2014) have exhibited differences in their levels of waiting and stopping impulsivity.

In summary, the obtained results (medium correlations between ImGo and other applied methods) correspond to the above-mentioned dual view on behavioral impulsivity as related yet slightly different cognitive processes. However, we did not identify any significant association between the coefficient of variation of the ImGo test and the stop-signal reaction time of the SST test, which is also crucial for ImGo validity assessment. The convergent validity of ImGo can therefore be considered only partially established.

Study 2

Introduction

The primary aim of Study 2 was to explore the potential relationships between oculomotor inhibition and behavioral impulsivity measured by an anti-saccade test. Previous research suggests that a lack of oculomotor inhibitory control is commonly viewed as a key characteristic of impulsivity, and simultaneously, as a key cause of lack of inhibitory control in oculomotor behavior (Nigg, 2000). In Study 1, we also investigated convergent validity between ImGo and SST, which are two methods used in the assessment of stopping and waiting impulsivity. Since convergent validity was only partially established, we decided to replicate the study using both methods to obtain more compelling evidence.

Methods

Tests

Anti-saccade task. The Anti-saccade task (AST) is an eye-tracking instrument for the measurement of oculomotor inhibition (Fabianová, 2019). It is based on a standardized protocol described by Antoniades et al. (2013). The AST contains two subtests: 1) pro-saccade and 2) anti-saccade. In the pro-saccade subtest, participants follow the presented stimuli with their eyes. In the anti-saccade subtest, participants move their eyes in the opposite direction to the presented stimuli (Figure 5). The procedure of trials is as follows: 1) target fixation at the center of the screen (1000–3500 ms, M = 1500 ms), and 2) stimulus presentation (1000 ms; randomly, half at 10° VA from the right of the target, the other half at 10° VA from the left of the target; Figure 5(a) shows a sequence of two pro-saccade trials, one left from the center, one right. Figure 5(b) shows a sequence of two anti-saccade trials).

Figure 5.

Procedure of the anti-saccade task: (a) Pro-saccade trials. (b) Anti-saccade trials.

The AST contains 22 training trials (11 pro saccade, 11 anti-saccade) and 260 test trials (80 pro-saccade, 180 anti-saccade) divided into five subtests: 1) 40 pro-saccade trials, 2) 60 anti-saccade trials, 3) 60 anti-saccade trials, 4) 60 anti-saccade trials, 5) 40 pro-saccade trials. A 10 second break follows each subtest (Figure 6 for the test procedure). The whole test takes about 15 minutes.

Figure 6.

Overall procedure of the anti-saccade task.

Since the pro-saccade subtest adopts only a control role, the main metrics are computed from the anti-saccade subtest, namely the number of anti-saccade errors, anti-saccade velocity and saccade latency (Antoniades et al., 2013). Even though Antoniades et al. (2013) proposed the absolute amount of false anti-saccades as a main metric, we believe that a ratio of anti-saccade errors/corrects is more suitable due to the very high number of excluded responses. Hence, we incorporated this indicator into the following analyses.

Power analysis

Power analysis was calculated using G*Power (v3.1.9.7; Faul et al., 2009) with the following settings: effect size: 0.37, significance level: .05, power: .80, analysis: one-tailed bivariate correlations. Effect size was set on the basis of results of Study 1, the strength of association between ImGo and SST, where effect sizes of significant correlations varied between 0.284 and 0.509 (M = 0.370). According to the power analysis results, assessment of 41 participants should be enough to achieve sufficient power.

Since previous research into the associations between eye-movements and go/no-go scores have shown ambiguous findings (see Taylor, 2016), we decided to use a rather exploratory approach in this case. These relationships were therefore not estimated in the power analysis. A one-tailed estimation setting was also used because we assumed the same directions of expected correlation as reported in Study 1.

Sample and procedure

In total, we gathered data from 37 participants. Six participants were removed from the analysis during the process of data cleaning due to problems with their eye-tracking data (missing data, faulty detection of fixations), which resulted in 31 participants included in the data analysis. Participants were aged 18 n of years (M = 24.29, SD = 5.73), most of them were women (n = 20, 64.5%) with mainly a high school education (n = 20, 64.5%). The participants were contacted through student groups on social networking services. Participants received small rewards (e.g., USB flash discs) for their participation and signed an informed consent.

The SST and ImGo tests were administered on SW Hypothesis (Šašinka, Morong, et al., 2017) under controlled conditions in HUME Lab at Masaryk university (common PC mouse, keyboard and LCD screen 22’ with resolution 1920 × 1080); the AST was administered via SMI Experiment Center (v3.7). Eye-movement data were collected with an SMI RED eye-tracker with 500 Hz sampling rate with integrated 22″ monitor (Dell P2213) with resolution of 1680 × 1050 px. The data were subsequently processed in SMI Begaze (v3.7) software. All other statistical analyses were performed in JASP (v0.11.1.0).

Data inspection and cleaning

The same data cleaning procedure as reported in Study 1 was used for ImGo and SST. Two participants were removed from ImGo due to high (i.e., >10%) omission errors, and six participants were removed from SST due to high (i.e., >15%) go commission and go choice errors.

Regarding the AST, three criteria had to be fulfilled to keep a trial in the dataset: 1) first saccade had to begin less than 40 px (equivalent of one visual angle) from the starting point (fixation target); 2) saccades had to begin between 60 ms and 500 ms (indicators of a quicker than physiologically possible reaction and very slow reaction); and 3) a blink could not appear before the start of the saccade. If all three conditions were fulfilled at the same time, a saccade was considered valid. 50.7% (n = 4087) of the maximum number (n = 8060; computed as 31 participants × 260 saccades) of saccades fulfilled all three criteria. Furthermore, only those saccades which were correct in the sense of their direction were included in the statistical analysis.

Since the data contained some extreme outliers (cf. min and max values in Table 4) and were considerably skewed, non-parametric correlation estimates were used. In addition, because of the limited sample size, we calculated Kendall’s Tau-b rank correlation coefficient (τ_b), which is (compared to Spearman correlations) considered less biased and does not overestimate significant associations in small samples (e.g., Arndt et al., 1999). Because of the known directionality of significant associations from Study 1 (ImGo – SST relationships) that we wanted to replicate, we applied a one-tailed ^(OT) method of null hypothesis significance testing. Furthermore, statistical corrections for multiple comparisons are used only in the confirmatory analyses of ImGo – SST relationships. For exploratory analyses of ImGo – AST relationships, we used a two-tailed ^(TT) method without correction for multiple comparisons.

Table 4.

Descriptive statistics of the ImGo, SST and AST instruments.

Variable	Min	Max	M (SD)	Me (IQR)
ImGo commissions (%)	37.5	87.5	62.8 (13.3)	58.3 (18.7)
ImGo omissions (%)	1.19	10	5.7 (2.2)	5.5 (1.9)
ImGo RT (ms)	243	537	355 (72.0)	328 (65.0)
ImGo CV	0.137	0.463	0.240 (.08)	0.226 (.08)
SST go omission errors (%)	0.19	12.45	4.3 (3.6)	3.1 (4.6)
SST go choice errors (%)	0	10	1.8 (2.8)	0.4 (2.4)
SST stop choice errors (%)	10	100	50.9 (26.6)	44.4 (41.6)
SST go RT (ms)	342	658	513 (82.1)	528 (159)
SST SSRT (ms)	183	312	240 (30.7)	235 (46.0)
AST correct anti-saccades (count)	16	141	68.6 (35.1)	65 (55.5)
AST false anti-saccades (count)	6	98	42.2 (24.6)	40 (35.5)
AST ratio C/F	0.163	14.1	2.6 (2.85)	1.5 (1.9)
AST latency anti-saccade (ms)	138	350	252.4 (42.91)	252.9 (54.1)
AST velocity anti-saccade (°/s)	120	431	262.5 (65.45)	264.6 (81.6)

M: mean; SD: standard deviation; Me: median; IQR: interquartile range; SST: stop-signal test; RT: reaction time; CV: coefficient of variance; SSRT: stop-signal reaction time; AST: anti-saccade task; C/F: correct/false.

Results

As mentioned above, the aims of Study 2 were: 1) to explore potential relationships between impulsivity and eye-movements, more specifically, between motor and oculomotor inhibition; and 2) to re-analyze the convergent validity of ImGo with the SST. The descriptive statistics of the basic metrics of all three methods used are summarized in Table 4.

In the correlation analysis between ImGo and AST metrics, we discovered that the number of AST false anti-saccades showed a statistically significant weak association with ImGo commissions errors (τ_b = .282) and ImGo reaction time (τ_b = −.291). ImGo RT also showed a weak significant association with the AST ratio of correct and false anti-saccades (τ_b = .282).

The replication of Study 1, an analysis of associations between ImGo and SST to ascertain the convergent validity of both methods, indicated two moderate correlations, more specifically, between the ImGO coefficient of variation and SST stop choice errors (τ_b = .333) and SSRT (τ_b = .431). However, only the latter was statistically significant after a Holm–Bonferroni correction for multiple comparisons (Table 5).

Table 5.

Convergent validity of the ImGo with SST and AST instruments.

	ImGo commissions	ImGo RT	ImGo CV
AST false anti-saccades	τ_b = .282* ^(TT) [.015, .549]	τ_b = –.291* ^(TT) [–.576 .007]	τ_b = .089 ^(TT) [–.180, .358]
AST ratio C/F	τ_b = –.141 ^(TT) [–.406, .125]	τ_b = .271* ^(TT) [.005, .536]	τ_b = –.143 ^(TT) [–.383, .097]
AST latency anti-saccade	τ_b = .040 ^(TT) [–.203, .284]	τ_b = .143 ^(TT) [–.115, .401]	τ_b = .044 ^(TT) [–.202, .291]
AST velocity anti-saccade	τ_b = –.045 ^(TT) [–.272, .181]	τ_b = –.049 ^(TT) [–.274, .176]	τ_b = –.217 ^(TT) [–.497, .063]
SST stop choice errors	τ_b = .234 ^(OT) [–.018, 1]	τ_b = –.051 ^(OT) [–1, .171]	τ_b = .333 ^(OT) [.104, 1]
SST SSRT	τ_b = –.053 ^(TT) [–.406, .301]	τ_b = .217 ^(TT) [–.107, .542]	τ_b = .431* ^(TT) [.146, .716]

*p/p_Holm < .05; [ ]: 95% confidence intervals; ^(OT): one-tailed; ^(TT): two-tailed; SST: stop-signal test; RT: reaction time; CV: coefficient of variance; SSRT: stop-signal reaction time; AST: anti-saccade task; C/F: correct/false.

Discussion

The first aim of Study 2 was to examine the relationship between motoric impulsivity measured by ImGo and oculomotor inhibition measured by the AST. Even though we found only a few weak and statistically significant relationships (despite the rigorous statistical estimation), especially between the ImGO RT and the AST false anti-saccades and AST ratio of correct/false anti-saccades, our results corresponded to previous studies which used similar methods. For example, Aichert et al. (2012) found weak associations between go/nogo tasks and the AST (r = .134) and Spinella (2002) found medium association (r = .470). Taylor (2016) did not find any significant relationship between the AST and self-report impulsivity questionnaire. Therefore, is it possible that the oculomotor inhibition shares some general underlying process with motor impulsivity, yet both are in fact distinct processes. This assumption is also supported by neurobiological findings (Nigg, 2000).

The secondary aim of this study was to re-analyze the convergent validity of the ImGo with the SST. Study 2 revealed medium association between the main indicators of motoric impulsivity, i.e., between the SSRT and the ImGo CV, which is crucial for establishing convergent validity (for detailed discussion, see Study 1). Hence, we can conclude that the ImGo instrument not only has satisfactory reliability (Study 1) but also satisfactory convergent validity with SST (Study 1 & Study 2) and is suitable for further use.

In summary, Study 2 provided evidence for convergent validity of the ImGo test, which can be used as a reliable and valid tool for motoric impulsivity assessment, and also revealed the relationship between motoric impulsivity and oculomotor inhibition.

Study 3

Introduction

Study 3 aimed to verify the discriminant validity of the ImGo test with other cognitive and executive processes. Based on the evidence of reliability and convergent validity reported in Study 1 and Study 2, further information about the psychometric properties of the instrument process could result in comprehensive evidence of the construct validity for the ImGo instrument. As already mentioned, impulsivity is a multidimensional construct stemming from multiple specific cognitive and executive processes and is therefore often related to a number of other constructs. Previous research has identified potential relationships of impulsivity, for example with personality disorders (Coffey et al., 2011), mental disorders (Enticott et al., 2008), attention disorders (Metin et al., 2012), interference control (Aichert et al., 2012) or intelligence (Lynam et al., 1993). If we want to be certain that ImGo measures impulsivity and not the above-mentioned constructs (evidence of discriminant validity), we would expect very weak (if any) relationships between ImGo and measurements of these constructs. In Study 3, we explore the relationship of ImGo impulsivity indicators with tests of other cognitive functions, namely: Positional Interference Test (selective attention), Flagtest (selectivity, concentration and stability of attention), Test of Verbal Reasoning (verbal reasoning and visuospatial skills), Viennese Matrices Test (non-verbal intelligence), The Personality Styles and Disorder Inventory (subclinical personality disorders), Inventory of Neurotic Symptoms (neurotic symptoms).

Methods

Tests

Positional inference test. The Positional Interference Test (PIT) is an instrument selective attention measurement based on the Stroop test (Stroop, 1935), more specifically, on its modification called the Spatial Stroop Test (e.g., Hilbert et al., 2014). The family of Stroop tests distinguishes between automatic and voluntary cognitive processes and is commonly used for the psychological assessment of cognitive inhibition (Swerdlow et al., 1995) closely related to the concept of behavioral impulsivity (Nigg, 2000). The PIT (Šašinka, Čeněk, Morong, Malatincová, et al., 2017) consists of three subtests with 40 trials per subtest (120 trials in total). The three subtests are (Figure 7): 1) reaction to position in congruent trials (i.e., psychomotor speed; PIT1); 2) reaction to semantic meaning in congruent trials (i.e., automatic process; PIT2); 3) reaction to semantic meaning in incongruent trials (i.e., voluntary process; PIT3).

Figure 7.

PIT subtests.

The main scores usually reported in similar tests are error rates, reaction times and the Stroop effect, computed as RT_incongruent ncRT_congruent (e.g., Bugg et al., 2008; higher score means higher Stroop effect). However, it is also possible to report, for example words per second (Hilbert et al., 2014). Regarding psychometric properties, the computerized Stroop tests show in general high validity and reliability (for review, see Din & Tat Meng, 2019). The PIT metrics explored in previous studies have shown satisfactory test-retest reliability (r_p varied between .743 and .813 for PIT RTs and r_p = .624 for the Stroop effect; Helísková, 2016). The whole test takes about 10 minutes.

Flagtest

The Flagtest is an instrument for measuring the selectivity, concentration and stability of attention (Šašinka, Čeněk, Morong, Urbánek, et al., 2017). The Flattest is a modified PC-administered adaptation of the traditional D2 test of attention (Brickenkamp & Zillmer, 1998). The main difference stems from the replacement of letters with flags in order to avoid the potential effect of reading skills on test performance (Figure 8). Participants identify the target flag (i.e., a hoisted flag with exactly two points – irrespective of their positions). The Flagtest consists of 20 sets of flags, each set being composed of 36 flags. Each trial is for 14 seconds and contains 8 or 9 target flags. The whole test takes about 10 minutes.

Figure 8.

Flagtest stimuli. (a) Examples. (b) Examples of correct answers and distractors.

The main score of the Flagtest is the average of correctly marked target flags across all subtests. The Flagtest has a high split-half reliability (r_sb between .908 and .950; Šašinka, Čeněk, Morong, Urbánek, et al., 2017) and concurrent validity in error rate with a D2 test of attention (r_s = .369; Tichá, 2017). Similar methods, for example the already mentioned D2 test of attention, show satisfactory reliability and validity in general (e.g., Bates & Lemay, 2004).

Test of verbal reasoning

The test of verbal reasoning (TVR) was created by Čeněk et al. (2017) on the basis of Baddeley’s (1968) grammatical transformation method, with special emphasis on sentence verification tasks (Clark & Chase, 1972). The test allows measurement of verbal reasoning and spatial skills (Kirschner et al., 2015). Moreover, a lack of verbal reasoning might also point to related deficits of selective attention (e.g., Nielsen et al., 2014) and work memory (e.g., Kane et al., 2005; Yuan et al., 2006).

Items consist of a sentence and combination of geometric shapes. Each item is a combination of two main dimensions: 1) negative and positive wording of sentence – argument (e.g., “The circle is not in a square.” or “The circle is in a square.”); and 2) congruence or incongruence of sentence and shape (e.g., “The circle is in a square” when the circle is indeed in a square) and incongruent (e.g., “The circle is not in a square,” when the circle is indeed in a square). Furthermore, two types of arguments and shapes are defined (in/out and above/below; see Figure 9). The TVR is composed of 40 pseudo-randomly presented items and takes around 7 minutes.

Figure 9.

Example of the TVR’s stimuli. (a) Inside/outside condition. (b) Above/below condition.

The main scores of TVR are average correctness and the sum of RTs (Čeněk et al., 2017). The TVR demonstrated satisfactory predictive validity with other instruments for measuring spatial skills and verbal reasoning (Potyková, 2017).

Viennese matrices test

The Viennese Matrices Test (VMT; Formann et al., 2011) is a nonverbal intelligence test which measures a one-dimensional general “g” factor of intelligence (Spearman, 1904). The VMT is basically an adapted PC-administered version of the Raven progressive matrices (Raven, 1958) and conforms to the Rasch model (Rasch, 1960). The VMT consists of 24 matrices (each matrix has 1 correct answer and 7 distractors), and the matrices are arranged with increasing difficulty. In this study, we use a Czech version of VMT validated by Klose et al. (2002).

The most important score of the VTM is the error rate (or simply the number of correct answers; Klose et al., 2002). The entire test has a 20-minute time limit. The original validation study reported satisfactory internal consistency (α = .76–.81), split-half reliability (r = .83) and concurrent validity with Raven progressive matrices (r = .74–.92; Formann & Piswanger, 1979; Formann et al., 2011). The Czech version also showed satisfactory test-retest reliability (r = .71) and concurrent validity with the Raven progressive matrices (r = .92) and the IST – Intelligence Structure Test (r = .82; Klose et al., 2002).

The personality styles and disorder inventory

The Personality Styles and Disorder Inventory (PSDI/PSSI, Kuhl & Kazén, 2009) is a self-report inventory for the assessment of personality styles which are defined as non-extreme and non-pathologic personality disorders (Švancara, 2002). It contains 14 subscales with 10 four-point Likert type items each (1 – does not apply at all, 4 – fully applies; 140 items in total). The test takes around 30 minutes. The following personality styles are included in the scale: paranoid, schizoid, schizotypal, borderline, histrionic, narcissistic, avoidant, dependent, obsessive-compulsive, negativistic, depressive, altruistic, rhapsodic and antisocial subscales. The original PSDI has satisfactory internal consistency (α = .73–.85), test-retest reliability (r = .68–.83) and concurrent validity with the Big Five questionnaire (r = −.57–.73; Kuhl & Kazén, 2009). The Czech adaptation of the scale (Švancara, 2002) was used in this study. The number of items can be reduced to 36, which results in a shorter version (with six subscales: schizotypal, borderline, narcissistic, avoidant, obsessive-compulsive and antisocial) called PSDI-6. Previous research suggested that this shorter version also has satisfactory confirmatory factor analysis results and reliability estimations in research (Hain et al., 2016).

For the purposes of our research, only the borderline subscale (e.g., my feelings often change abruptly and impulsively), which measures impulsive manifestation in behavior and feeling, and the obsessive-compulsive subscale (e.g., even under time pressure, I cannot stop being thorough), which measures thoroughness, diligence and accuracy, are the relevant scales. Only they were therefore included in further correlation analyses. Both scales are represented in full PSDI and the shorter PSDI-6.

Inventory of neurotic symptoms

The Inventory of Neurotic Symptoms (N-70) is a Czech questionnaire created with the purpose to detect individuals who may be too sensitive for military service (Vacíř, 1973). Even though its main purpose is a psychological diagnostic in military hospitals, it is also used for research purposes (e.g., Flegr et al., 2012; see Appendix for an English translation of N-70). It consists of 7 subscales with 10 four-point Likert type items per subscale (1 – never, 4 – often; 70 items in total). The subscales are anxiety, depression, obsession-phobia, hysteria, hypochondria, psychosomatic symptoms and psychasthenia subscales. The N-70 has satisfactory internal consistency in most of the subscales (α = .628–.867; Flegr et al., 2018). The test takes around 15 minutes.

For the purposes of our research, we used the subscales that measure symptoms potentially related to impulsive behavior: hysteria (e.g., do you feel like fainting when you are strongly keyed up?), obsession-phobia (e.g., do you get severely out of balance when your daily habits are disturbed?), and psychosomatics symptoms (e.g., does your heart flutter or start to race easily in demanding situations?; Vacíř, 1973).

Power analysis

Since a correlation analysis performed on a relatively small sample size with low statistical power might lead to both underestimated and spuriously large correlation coefficients compared to the real effect (Button et al., 2013), we also performed a robust study with a large sample size, because, for example, correlation analyses tend to stabilize in approximately 250 observations (see Schönbrodt & Perugini, 2013). Hence, no a priori power analysis was required in this case. Post-hoc power analysis of sensitivity (α = .05, 1−β = .80) revealed that the critical r value for statistical significance was only ± .037 (without correction for multiple testing), which suggests very high power.

Sample and procedure

Besides the methods described above, the entire battery also contained the ImGo instrument. We collected data from 2860 participants in total (2426 participants fulfilled all methods). Participants were between 18 dat years of age (M = 29.42, SD = 9.43), were either candidates to join the Czech Armed Forces or current members of Czech Armed Forces, and the majority being men (n = 2508, 87.7%). Participants agreed with the testing procedure as well as with the collection of personal data. However, for secondary data analysis only a fully anonymized dataset was used. Since the Czech Armed Forces are composed only of 12.96% women soldiers (Ministry of Defense and Armed Forces of Czech Republic, 2020), the gender proportion of the sample corresponds to the gender distribution in the Czech Armed Forces. In order to identify any potential selection and sampling bias caused by unbalanced gender proportion, we compared arithmetic and weighted means and medians in all variables, which showed only negligible changes. Furthermore, we performed an analysis of gender differences in all variables. These results were mostly non-significant (those, which were statistically significant, were practically insignificant, due to the small effect sizes). Hence, we can conclude that the mentioned gender disproportion should not be biasing the results in any way (see Appendix).

Data collection took place from October 2018 to March 2020 at the Military hospital at Brno. The whole testing procedure took approximately two hours. All methods were administered through the SW Hypothesis (Šašinka, Morong, et al., 2017). Participants were tested in groups (max. 20 per group), and the hardware (common PC mouse, keyboard and LCD 22’ screen size and resolution 1920 x 1080 px) used was identical for all tests and all participants. All statistical analyses were performed in R (v4.0.0; R Core Team, 2020), which included the packages lavaan (v 0.6–6; Rosseel, 2012), semTools (v0.5–2; Jorgensen et al., 2018), TOSTER (v0.3.4; Lakens, 2018) and robust (0.5–0; Wang et al., 2020), and in JASP (v0.12.2.0).

Data inspection and cleaning

As with Study 1 and Study 2, all exclusion criteria were set an a priori data analysis. When a participant fulfilled the exclusion criteria in a specific test, they were excluded, but only from the particular test. The scores from other tests remained in the data matrix. The ImGo data cleaning procedure is described in the corresponding section of Study 1. Based on the exclusion criteria, we discarded 140 participants (mainly due to the high rate of omission errors).

Regarding the PIT, all RTs less than 200 ms and greater than 3000 ms should be excluded from the analysis before computation of the mean RTs (Bugg et al., 2008). However, no participant fulfilled this criterion for exclusion. Four participants with error rates greater than 50% (Lupiáñez & Funes, 2005) were removed from further analysis.

The exclusion criteria of the error rate were set to 43% for the Flagtest and 50% for the TVR, based on the Czech validation studies and their population norms (Čeněk et al., 2017; Šašinka, Čeněk, Morong, Urbánek, et al., 2017), resulting in 48 excluded participants from the Flagtest and 31 excluded from the TVR. Regarding the exclusion criteria of the error rate for the VMT, participants with less than 8 correct answers (i.e., less than 33.3% of the VMT score) were excluded since this score could indicate misunderstanding of the test, temporary cognitive impairment or intellectual impairment or intellectual disability. This led to the exclusion of 141 participants.

The assumptions of parametric correlation and regression analyses were not met due to skewed non-normal data distributions, statistically significant Shapiro–Wilk tests (Table 6) and the presence of outliers (de Winter et al., 2016; Yu & Yao, 2017). Hence, we used non-parametric Spearman’s rank correlation coefficients (r_s) and robust multiple multivariate linear regressions (MM-estimation) with standardized robust beta coefficients (β_r) in all cases. As with Study 1 and Study 2, all p-values obtained from the correlation analyses were corrected for multiple comparisons with the Holm–Bonferroni method.

Table 6.

Descriptive statistics and reliability estimation of the ImGo, the PIT, the Flagtest, the TVR and the VMT and PSDI and the N-70 methods.

Variable	Min	Max	M (SD)	Me (IQR)	S–W
ImGo commissions (%)	2.083	100	37.3 (18.3)	35.4 (27.1)	p < .05
ImGo omissions (%)	0	10	2.9 (2.6)	1.9 (3.9)	p < .05
ImGo RT (ms)	216	647	384 (58.1)	359 (68.2)	p < .05
ImGo CV	0.116	0.734	.243 (.06)	.234 (.07)	p < .05
PIT score (%)	76.7	100	98.4 (1.92)	99.2 (2.5)	p < .05
PIT1 RT (ms)	306	1123	458 (107)	425 (114)	p < .05
PIT2 RT (ms)	478	1250	673 (99)	656 (128)	p < .05
PIT3 RT (ms)	598	2635	839 (130)	819 (156)	p < .05
PIT Stroop effect	–66.5	1816.6	166 (77.2)	155 (73.5)	p < .05
Flagtest score (%)	43.2	99.5	72.3 (10.8)	73.2 (15.8)	p < .05
TVR score (%)	50	100	85.5 (12.9)	90 (17.5)	p < .05
TVR RT sum (s)	93	838	241 (83.7)	223 (100)	p < .05
VMT score (%)	33.3	95.8	66.7 (14.9)	66.6 (25)	p < .05

[ ]: 95% confidence intervals; M: mean; SD: standard deviation; Me: median; IQR: interquartile range; S–W: Shapiro–Wilk test; RT: reaction time; CV: coefficient of variance; sum: suma; RT: reaction time; CV: coefficient of variance; PIT: positional inference test; TVR: test of verbal reasoning; VMT: Viennese matrices test.

Conditional equivalence testing (Campbell & Gustafson, 2018) can be performed if traditional null hypothesis significance testing fails to find statistically significant effects, because insignificant results do not always indicate the absence of effect, which is, however, important for evidence of discriminant validity. This procedure can be found in additional equivalence tests, namely two-one-sided t-tests (TOST; Lakens, 2017), which examine practical significance rather than statistical significance and provide evidence to support null hypotheses. We specified upper (ΔU) and lower (ΔL) equivalence bounds based on the smallest effect size of interest (SESOI; Lakens et al., 2018) to −.25 and .25 as equivalent to weak effect size. According to TOST, the values within the equivalence range lack practical significance and are therefore equivalent to the null hypothesis (i.e., no associations). If the correlation analyses supplemented by the TOST procedure resulted in these small or even non-significant coefficients within the mentioned equivalence range, we would consider them sufficient evidence of a lack of relationship, especially with respect to the large sample size and consequent high statistical power (Furr, 2017). Therefore, no other method of discriminant validity estimation, such as multi-trait multi-method (MTMM), is required.

Results

The aim of Study 3 was to verify the discriminant validity of the ImGo with other cognitive processes (e.g., intelligence, attention, verbal reasoning) and personality traits (e.g., obsession, borderline, hysteria). The following results section is divided into two subchapters: Discriminant validity of the ImGo and self-report questionnaires and Discriminant validity of the ImGo and performance methods. Again, only relevant variables were included in the following correlation and regression analyses (i.e., variables which would indicate some level of impulsivity or related psychological constructs). ImGo omissions, PIT score and PIT1 RT were therefore omitted. The descriptive statistics of all methods are summarized in Tables 7 and 8.

Table 7.

Descriptive statistics and reliability estimation of the PSDI and the N-70 methods.

Relevant subscales from questionnaire	McDonald’sω	Cronbach’s α	M (SD)	Me (IQR)	S-W
PSDI Borderline	.814	.807 [.796, .818]	1.41 (0.41)	1.33 (0.66)	p < .05
PSDI Obsessive-compulsive	.693	.684 [.665, .702]	2.95 (0.46)	3 (0.6)	p < .05
N-70 Hysteria	.683	.675 [.658, .692]	1.23 (0.22)	1.2 (0.3)	p < .05
N-70 Psychosomatics symptoms	.682	.668 [.650, .685]	1.26 (0.24)	1.2 (0.3)	p < .05
N-70 Obsession-phobia	.729	.707 [.692, .722]	1.28 (0.25)	1.2 (0.3)	p < .05

[ ]: 95% confidence intervals; M: mean; SD: standard deviation; Me: median; IQR: interquartile range; S–W: Shapiro–Wilk test.

Table 8.

Relationships between the ImGo and related questionnaires.

Variable	ImGo commissions	ImGo RT	ImGo CV
PSDI Borderline	r_s = .031^{ΔL ΔU} [–.006, .069]	r_s = –.025 ^{ΔL ΔU} [–.062, .013]	r_s = .114*** ^{ΔL ΔU} [.077, .151]
PSDI Obsessive-compulsive	r_s = –.030 ^{ΔL ΔU} [–.068, .008]	r_s = .041 ^{ΔL ΔU} [.004, .079]	r_s = –.057* ^{ΔL ΔU} [–.095, –.020]
N-70 Hysteria	r_s = .036 ^{ΔL ΔU} [–.001, .074]	r_s = –.088*** ^{ΔL ΔU} [–.125, –.050]	r_s = .059* ^{ΔL ΔU} [.021, .096]
N-70 Psychosomatics symptoms	r_s = –.015 ^{ΔL ΔU} [–.053, .022]	r_s = –.047 ^{ΔL ΔU} [–.084, –.009]	r_s = .059* ^{ΔL ΔU} [.021, .096]
N-70 Obsession-phobia	r_s = –.006 ^{ΔL ΔU} [–.044, .031]	r_s = –.041 ^{ΔL ΔU} [.004, .079]	r_s = .091*** ^{ΔL ΔU} [.054, .128]

*p_Holm < .05 (two-tailed); ***p_Holm < .001 (two-tailed); ^ΔU: upper equivalence bound significant at p < .05; ^ΔL: lower equivalence bound significant at p < .05; [ ]: 95% confidence intervals; RT: reaction time; CV: coefficient of variance.

Discriminant validity of the ImGo and self-report questionnaires

In the first step, we verified the factor structures of the questionnaires used in this study using CFA. The WLSMV estimator, which is suitable for non-Gaussian categorical data from Likert-type scales (Finney & DiStefano, 2006), was used in all cases. Since the standardized 14-dimensional version of the PSDI yielded unsatisfactory fit indices (χ²(9360) = 93844.088, p < .001, RMSEA [90% CI] = .056 [.055, .056], SRMR = .093, CFI = .670, TLI = .662), the shorter 6-dimensional version PSDI-6 was also verified. From the analysis of the modification indices, two more items were excluded from the measurement model (item 23 and item 107). This resulted in satisfactory fit indices and improvement of the model (χ²(512) = 5525.081, p < .001, RMSEA [90% CI] = .058 [.056, .059], SRMR = .060, CFI = .910, TLI = .901). PSDI-6 is therefore used in further statistical analyses. Regarding the N-70 inventory, the confirmatory factor analysis revealed excellent fit indices (χ²(2324) = 6345.151, p < .001, RMSEA [90% CI] = .023 [.023, .024], SRMR = .063, CFI = .956, TLI = .954), suggesting satisfactory construct validity. Hence, the N-70 can be used in further analyses without any changes. Regarding the internal consistency of relevant subscales of PSDI-6 and N-70, the McDonald’s ω coefficients varied in the range .682−.814, which we considered satisfactory (Table 6).

The correlation analysis of ImGO scores and scores of the relevant PSDI-6 and N-70 subscales suggested very weak or no relationships. Even though some associations were flagged as statistically significant, the correlation coefficients were very small (r_s varied between -.088 and .114; see Table 8). TOST analysis also revealed that all correlation coefficients between ImGo and the questionnaire subscales lacked any practical significance since their values were within the equivalence range (Table 8). In terms of the above-mentioned overpowered tests, the results of TOST procedure and with respect to the 95% confidence intervals (which are close to zero values), the results can be interpreted as evidence of no relationship between the ImGo and self-report impulsivity questionnaires.

We also performed robust multiple multivariate linear regressions to reveal the potential predictive power of the self-report questionnaires in the ImGo. The main indicators of impulsivity measured by ImGo were used as outcomes (i.e., ImGo commissions, ImGo RT and ImGo CV), and the relevant questionnaire subscales were used as predictors (Table 9). Even though some regression coefficients were statistically significant, their size was very close to zero. Moreover, the data from the questionnaires explained only 0.6–1.4% of the variability in the response data for indicators of impulsivity provided by the ImGo. We can therefore conclude that questionnaires do not have any notable predictive power on the ImGo under scrutiny.

Table 9.

Predictive power of self-report questionnaires in the ImGo for related subscales.

Variable	ImGo commissions	ImGo RT	ImGo CV
PSDI Borderline	β_r = .016	β_r = .013	β_r = .062***
PSDI Obsessive-compulsive	β_r = –.031	β_r = .044	β_r = –.030
N-70 Hysteria	β_r = .044**	β_r = –.058***	β_r = .007
N-70 Psychosomatics symptoms	β_r = –.029	β_r = –.010	β_r = –.025
N-70 Obsession-phobia	β_r = –.023	β_r = .009	β_r = .035*
Multiple R²	.0062	.0073	.0137

*p < .05 (two-tailed); **p < .01 (two-tailed); ***p < .001 (two-tailed); β_r: standardized robust regression coefficient; RT: reaction time; CV: coefficient of variance.

Discriminant validity of the ImGo and performance methods

The correlation analysis of ImGo, PIT, Flagtest, TVR and VMT showed relatively weak or no associations between the ImGo and related performance methods (r_s = −.156–.220; see Table 10). For reasons described above, we performed equivalence tests with the same SESOI. TOST suggests that all relationships (and their confidence intervals) were included in the equivalence ranges, which means that they were practically insignificant and equivalent to no relationship (Table 10). In conclusion, our results represent strong evidence for null effects and therefore also no relationship between the ImGo and methods for assessment of attention, verbal reasoning and visuospatial skills, or non-verbal intelligence.

Table 10.

Relationships between the ImGo and related performance methods.

Variable	ImGo commissions	ImGo RT	ImGo CV
PIT2 RT	r_s = .075* ^{ΔL ΔU} [.030, .119]	r_s = –.103*** ^{ΔL ΔU} [–.147, –.058]	r_s = .028 ^{ΔL ΔU} [–.016, .073]
PIT3 RT	r_s = .047 ^{ΔL ΔU} [.003, .092]	r_s = –.091*** ^{ΔL ΔU} [–.135, –.046]	r_s = .004 ^{ΔL ΔU} [–.040, .049]
PIT Stroop effect	r_s = –.059 ^{ΔL ΔU} [–.103, –.014]	r_s = .009 ^{ΔL ΔU} [–.036, .054]	r_s = –.060 ^{ΔL ΔU} [–.104, –.015]
Flagtest score	r_s = .189*** ^{ΔL ΔU} [.149, .228]	r_s = .220*** ^{ΔL ΔU} [.181, .228]	r_s = –.035 ^{ΔL ΔU} [–.076, .006]
TVR score	r_s = –.120*** ^{ΔL ΔU} [–.160, –.080]	r_s = .010 ^{ΔL ΔU} [–.030, .051]	r_s = –.139*** ^{ΔL ΔU} [–.179, –.099]
TVR RT sum	r_s = –.019 ^{ΔL ΔU} [–.059, .022]	r_s = .114*** ^{ΔL ΔU} [.074, .154]	r_s = .070*** ^{ΔL ΔU} [.029, .110]
VMT score	r_s = –.108*** ^{ΔL ΔU} [–.146, –.070]	r_s = –.042 ^{ΔL ΔU} [–.080, –.003]	r_s = –.156*** ^{ΔL ΔU} [–.194, –.119]

*p_Holm < .05 (two-tailed); **p_Holm < .01 (two-tailed); ***p_Holm < .001 (two-tailed); ^ΔU: upper equivalence bound significant at p < .05; ^ΔL: lower equivalence bound significant at p < .05; [ ]: 95% confidence intervals; RT: reaction time; CV: coefficient of variance; M: mean; sum: suma; PIT: Positional Inference Test; TVR: Test of Verbal Reasoning; VMT: Viennese Matrices Test.

To verify the predictive power of performance methods on the ImGo, robust multiple multivariate linear regressions were performed. The main indicators of impulsivity measured by ImGo were used as outcomes (i.e., ImGo commissions, ImGo RT and ImGo CV) and relevant indicators of performance methods were used as predictors in the regression model (Table 11). PIT 2 RT and PIT3 RT were not included in the regression model because of multicollinearity with the PIT Stroop effect. The results of the regression analyses suggested that the Flagtest score is weakly associated with ImGO RT (β_r = .299). The size of the rest of the regression coefficients is very small, sometimes non-significant (Table 10). The performance methods explained only 2.3–6.2% of the variability of indicators of impulsivity provided by the ImGo. The proportion of shared variance indicated by multiple R² is probably caused by psychomotor speed or attention qualities which are naturally saturated across performance methods in general. Since all the multiple R² and regression coefficients are very weak, we can conclude that the performance methods did not have any or had only negligible predictive power on the ImGo under scrutiny.

Table 11.

Predictive power of performance methods on the ImGo.

Variable	ImGo commissions	ImGo RT	ImGo CV
PIT Stroop effect	β_r = –.043*	β_r = .016	β_r = –.019
Flagtest score	β_r = –.157***	β_r = .299***	β_r = –.039
TVR score	β_r = –.040**	β_r = .027	β_r = –.036*
TVR RT sum	β_r = –.024	β_r = .023	β_r = .011
VMT score	β_r = –.112***	β_r = .003	β_r = –.169***
Multiple R²	.0401	.0619	.0231

*p < .05 (two-tailed); **p < .01 (two-tailed); ***p < .001 (two-tailed); β_r: standardized robust regression coefficient; RT: reaction time; CV: coefficient of variance; PIT: Positional Inference Test; TVR: Test of Verbal Reasoning; VMT: Viennese Matrices Test.

Discussion

The aim of Study 3 was to verify the differential validity of ImGo and the methods used to measure other potentially related psychological constructs (personality styles, neuroticism, visual and selective attention, verbal reasoning and visuospatial skills, and non-verbal intelligence) on a large sample of respondents. This approach leads to a decreased probability of type II error due to large power, and it can provide deeper insights into the potential motoric impulsivity correlates.

In the first step, we analyzed the relationships between the ImGo and the self-report questionnaires of personality styles and neuroticism. Correlation and regression analyses suggested that no practical significance existed among these constructs. We concluded that the motoric impulsivity measured by ImGo is not associated with either borderline and obsessive-compulsive personality styles (measured by PSDI), or neurotic characteristics such as hysteria, obsession-phobia and psychosomatics symptoms (measured by N-70), despite some studies reporting their relationship (especially in the case of borderline personality disorder) to various aspects of impulsivity (e.g., Barker et al., 2015; Cackowski et al., 2014; Coffey et al., 2011; Onur et al., 2016; Rentrop et al., 2008; Ruchsow et al., 2008; Wright et al., 2014). Our findings can be interpreted by the differences in our research design. First, we used a much larger sample size, and second, we gathered data from the general population rather than the clinical population with diagnosed personality disorders. Moreover, these findings correspond to the finding obtained in Study 1 on the lack of relationship between behavioral measures and self-report questionnaires (see discussion of Study 1).

In the second step, we examined the relationships between the ImGo and potentially related performance instruments. Although those associations were generally higher compared to the associations with the questionnaire methods, the TOST procedure suggested that no practical significant effects existed between behavioral impulsivity and other examined cognitive processes, such as intelligence, attention or verbal reasoning. Even though these processes share some common elementary cognitive aspects, for instance psychomotor speed, it is evident that ImGo measures a different construct. Since behavioral impulsivity should differ from other cognitive and executive processes, we consider these findings crucial evidence for the discriminant validity of ImGo.

Regarding the association between impulsivity and the interference control measured by the Stroop test, it is sometimes hypothesized that greater impulsivity should be associated with weaker interference control (Nigg, 2000). However, our results agree with other studies which have suggested that the association between impulsivity and selective attention measured by the Stroop family of tests is only small or non-significant (Aichert et al., 2012; Enticott et al., 2008; Morooka et al., 2012; Strasser et al., 2016).

As for the relationship between selective attention and impulsivity, the vast majority of research was conducted on samples with diagnosed attention-deficit/hyperactivity disorder (ADHD). Generally, it is assumed that lower attention should lead to higher error rates and slower reaction times in go/no-go trials (Metin et al., 2012). This type of research has indeed found that impulsivity is associated with attention (Bezdjian et al., 2009; Kulacaoglu et al., 2017) and distinguishes between clinical and general populations or between various types of attention deficit disorders (Lopez et al., 2015; Miller et al., 2009; Trommer et al., 1988). According to a recent meta-analysis, those results had stable effects (Metin et al., 2012) and were also observed within the general population (Chamorro et al., 2012). However, none of the studies mentioned applied behavioral measures of attention, for example a D2 test. Although the observed correlations between both constructs were highest in our analysis, they were not practically significant and therefore indicated a small or no relationship.

The last relationships we examined were between behavioral impulsivity and general and verbal intelligence measured by the Viennese Matrices Test and Test of Verbal Reasoning. We argue that impulsivity and intelligence are unrelated constructs and therefore the observed lack of relationship is essential for the discriminant validity of the ImGo instrument. Even though some studies suggest that impulsivity is related to both general and verbal intelligence, (e.g., Buchmann et al., 2011; Koolhof et al., 2007; Lozano et al., 2014; Lynam et al., 1993; Russo et al., 2008; Schweizer, 2002), other research has not discovered any direct relationship between these constructs or even labeled these associations as spurious (e.g., Lozano, 2015; Vigil-Coleṭ & Morales-Vives, 2005). Our research therefore supports the results of the second group of studies. It should be mentioned that the direct comparability of our results to the above-mentioned studies on the relationship of intelligence and behavioral impulsivity is limited since each study used a unique research design, was conducted on specific population (e.g., delinquent population), and used different behavioral and self-report measures.

Conclusion

The present paper provided a set of three separate studies with evidence of the validity and reliability of the ImGo instrument for studying the general population. In Study 1, moderate to high test-retest and split-half reliability of the ImGo were found. The results suggested that the indicators of behavioral impulsivity obtained by the ImGo are stable in time and show satisfactory internal consistency between two halves of the instrument, which is necessary for minimizing measurement errors and thus producing reliable results. Studies 1 and 2 together also provided sufficient evidence of convergent validity; the ImGo showed medium correlations with some related indicators of impulsivity measured by the self-report Impulsive Behavior Scale, i.e., negative urgency subscale, and by the Stop Signal test. Besides providing evidence of validity, Study 2 also used an exploratory eye-tracking method to identify whether associations existed between oculomotor inhibition and impulsivity. Nevertheless, these associations were rather weak or non-significant. The aim of Study 3 was to examine the differential validity of ImGo with personality disorders/styles, namely borderline personal style, hysteria, obsessive-compulsive behavior and psychosomatic symptoms measured by self-report scales and objective methods which measure general and verbal intelligence, attention and inference control. All showed that the associations tended to be very weak and lacked any practical significance. The discriminant validity is therefore considered highly satisfactory.

Our findings from Study 1 and Study 2 also showed that behavioral impulsivity differs to a certain extent from other impulsivity instruments, suggesting that impulsivity manifests differently at the behavioral level, yet the impulsivity instruments simultaneously share a general impulsive trait. Regarding the impulsivity correlates within the general population, which we examined in Study 3, our results showed that higher impulsivity is not associated (from the practical significance point of view) with lower verbal or general intelligence, higher cognition and inference inhibition and lower attention despite the general assumptions of previous clinical research.

In conclusion, all three studies combined offer a comprehensive view on behavioral impulsivity and its correlates within the general population. ImGo showed satisfactory reliability and construct validity, supported by the results of convergent and discriminant validity. ImGo is also easy to administer since it is possible to run online and has a free license for non-commercial use. ImGo has already been translated into several languages (English, German, Spanish, Portuguese, Slowak, Traditional Chinese, Simplified Chinese, Turkish), but further verification of its psychometric properties within other cultures and languages is still deemed desirable, for instance in cross-cultural studies. The Hypothesis software, which runs the ImGo instrument, is an open-source software (Apache License 2.0; https://github.com/poweredonhypothesis/hypothesis). Hence, the ImGo method is considered suitable for further scientific purposes and for practical use within the general population.

Supplemental Material

sj-pdf-1-prx-10.1177_00332941211040431 - Supplemental material for ImGo: A Novel Tool for Behavioral Impulsivity Assessment Based on Go/NoGo Tasks

Supplemental material, sj-pdf-1-prx-10.1177_00332941211040431 for ImGo: A Novel Tool for Behavioral Impulsivity Assessment Based on Go/NoGo Tasks by Č. Šašinka, D. Lacko, J. Čeněk, S. Popelka, P. Ugwitz, H. Řádová, M. Fabianová, A. Šašinková, J. Brančík and M. Jankovská in Psychological Reports

Footnotes

Acknowledgments

We would like to thank prof. Holmqvist for consultation of the eye tracking task and we would like to thank the HUME Lab–Experimental Humanities Laboratory, Masaryk University, for providing us with the necessary machine time and equipment.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This publication was supported by the Czech Science Foundation (GC19-09265J: The influence of socio-cultural factors and writing system on perception and cognition of complex visual stimuli).

ORCID iDs

Č. Šašinka

D. Lacko

S. Popelka

Author Biographies

Čeněk Šašinka, is a senior researcher at the Department of Information and Library Studies at Masaryk University. His area of specialization is research methodology, visualizations, and virtual reality.

David Lacko, is a Ph.D. candidate at the Department of Psychology atMasaryk University. His research interests include cross-cultural research of cognition and perception, psychometrics and advanced statistical methods, and videogames research.

Stanislav Popelka, works as the head of the eye-tracking laboratory at the Department of Geoinformatics, Palacký University in Olomouc. He is primarily interested in applying eye-tracking in the evaluation and optimization of maps and cartographic works.

Jiří Čeněk, holds a Ph.D. in Psychology from Masaryk University. He is a postdoc teacher and researcher. His area of specialization is cross-cultural differences in perception and cognition, psychological methodology, and data analysis.

Pavel Ugwitz, is a psychologist and computer scientist by training. He is finishing his Ph.D. degree at the Department of Geography and focuses on virtual reality.

Hana Řádová, is graduated from the Department of Psychology at Masaryk University.

Martina Fabianová, graduated from the Department of Psychology at Masaryk University.

Alžběta Šašinková, is a junior researcher at the Department of Information and Library Studies and the Department of Psychology. Her area of specialization is virtual reality and education.

Jiří Brančík, is a chief-psychologist in the Military Hospital in Brno, and he is responsible for developing psychological diagnostic tools.

Markéta Jankovská, is a project manager and applies psychological assessment into praxis.

References

Aichert

D. S.

Wöstmann

N. M.

Costa

Macare

Wenig

J. R.

Möller

H.-J.

Rubia

Ettinger

(2012). Associations between trait impulsivity and prepotent response inhibition. Journal of Clinical and Experimental Neuropsychology, 34(10), 1016–1032.

Antoniades

Ettinger

Gaymard

Gilchrist

Kristjánsson

Kennard

John Leigh

Noorani

Pouget

Smyrnis

Tarnowski

Zee

D. S.

Carpenter

R. H. S.

(2013). An internationally standardised antisaccade protocol. Vision Research, 84, 1–5.

Arndt

Turvey

Andreasen

N. C.

(1999). Correlating and predicting psychiatric symptom ratings: Spearman’s r versus Kendall’s tau correlation. Journal of Psychiatric Research, 33(2), 97–104.

Baddeley

A. D.

(1968). A 3 min reasoning test based on grammatical transformation. Psychonomic Science, 10(10), 341–342.

Band

G. P. H.

van der Molen

M. W.

Logan

G. D.

(2003). Horse-race model simulations of the stop-signal procedure. Acta Psychologica, 112(2), 105–142.

Barker

Romaniuk

Cardinal

R. N.

Pope

Nicol

Hall

(2015). Impulsivity in borderline personality disorder. Psychological Medicine, 45(9), 1955–1964.

Bates

M. E.

Lemay

E. P.

Jr. (2004). The d2 test of attention: Construct validity and extensions in scoring techniques. Journal of the International Neuropsychological Society, 10(3), 392–400.

Berlin

G. S.

Hollander

(2014). Compulsivity, impulsivity, and the DSM-5 process. CNS Spectrums, 19(1), 62–68. https://doi.org/10.1017/S1092852913000722.

Bezdjian

Baker

Lozano

Raine

(2009). Assessing inattention and impulsivity in children during the go/NoGo task. The British Journal of Developmental Psychology, 27(Pt 2), 365–383. https://doi.org/10.1348/026151008x314919.

10.

Brickenkamp

Zillmer

(1998). D2 – Test of attention. Hogrefe & Huber.

11.

Buchmann

Gierow

Reis

Haessler

(2011). Intelligence moderates impulsivity and attention in ADHD children: An ERP study using a go/nogo paradigm. The World Journal of Biological Psychiatry, 12(sup1), 35–39. https://doi.org/10.3109/15622975.2011.600354.

12.

Bugg

J. M.

Jacoby

L. L.

Toth

J. P.

(2008). Multiple levels of control in the stroop task. Memory & Cognition, 36(8), 1484–1494.

13.

Button

K. S.

Ioannidis

J. P. A.

Mokrysz

Nosek

B. A.

Flint

Robinson

E. S. J.

Munafò

M. R.

(2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews. Neuroscience, 14(5), 365–376.

14.

Cackowski

Reitz

A. C.

Ende

Kleindienst

Bohus

Schmahl

Krause-Utz

(2014). Impact of stress on different components of impulsivity in borderline personality disorder. Psychological Medicine, 44(15), 3329–3340.

15.

Campbell

Gustafson

(2018). Conditional equivalence testing: An alternative remedy for publication bias. PLoS One, 13(4), e0195145.

16.

Čeněk

Šašinka

Č.

Morong

Urbánek

Ugwitz

Potyková

Točík

Helísková

Strnadová

Sedlák

(2017). Test verbálního uvažování: Manuál psychodiagnostické metody TVU (verze 1.0) [Test of Verbal Reasoning: Manual of psychological assessment test (version 1.0)]. Masaryk University.

17.

Chamorro

Bernardi

Potenza

Grant

Marsh

Wang

Blanco

(2012). Impulsivity in the general population: A national study. Journal of Psychiatric Research, 46(8), 994–1001. https://doi.org/10.1016/j.jpsychires.2012.04.023.

18.

Clark

H. H.

Chase

W. G.

(1972). On the process of comparing sentences against pictures. Cognitive Psychology, 3(3), 472–517.

19.

Coffey

S. F.

Schumacher

J. A.

Baschnagel

J. S.

Hawk

L. W.

Holloman

(2011). Impulsivity and risk-taking in borderline personality disorder with and without substance use disorders. Personality Disorders, 2(2), 128–141. https://doi.org/10.1037/a0020574.

20.

Congdon

Mumford

J. A.

Cohen

J. R.

Galvan

Canli

Poldrack

R. A.

(2012). Measurement and reliability of response inhibition. Frontiers in Psychology, 3, 37.

21.

Conners

C. K.

(2000). Conners’ continuous performance test II. Multi-Health Systems.

22.

Conners

C. K.

Epstein

J. N.

Angold

Klaric

(2003). Continuous performance test performance in a normative epidemiological sample. Journal of Abnormal Child Psychology, 31(5), 555–562.

23.

Curtin

Schulz

(1998). Multiple correlations and Bonferroni’s correction. Biological Psychiatry, 44(8), 775–777.

24.

Cyders

M. A.

Coskunpinar

(2011). Measurement of constructs using self-report and behavioral lab tasks: Is there overlap in nomothetic span and construct 62 representation for impulsivity? Clinical Psychology Review, 31(6), 965–982.

25.

Cyders

M. A.

Littlefield

A. K.

Coffey

Karyadi

K. A.

(2014). Examination of a short English version of the UPPS-P impulsive behavior scale. Addictive Behaviors, 39(9), 1372–1376.

26.

Dalley

J. W.

Everitt

B. J.

Robbins

T. W.

(2011). Impulsivity, compulsivity, and top-down cognitive control. Neuron, 69(4), 680–694.

27.

de Winter

J. C. F.

Gosling

S. D.

Potter

(2016). Comparing the Pearson and Spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data. Psychological Methods, 21(3), 273–290.

28.

Din

N. C.

Tat Meng

E. C.

(2019). Computerized stroop tests: A review. Journal of Psychology & Psychotherapy, 9(1), 353–358.

29.

Enticott

Ogloff

Bradshaw

Fitzgerald

(2008). Cognitive inhibitory control and self-reported impulsivity among violent offenders with schizophrenia. Journal of Clinical and Experimental Neuropsychology, 30(2), 157–162. https://doi.org/10.1080/13803390701290055.

30.

Epstein

J. N.

Langberg

J. M.

Rosen

P. J.

Graham

Narad

M. E.

Antonini

T. N.

Brinkman

W. B.

Froehlich

Simon

J. O.

Altaye

(2011). Evidence for higher reaction time variability for children with ADHD on a range of cognitive tasks including reward and event rate manipulations. Neuropsychology, 25(4), 427–441.

31.

Fabianová

(2019). Souvislost mezi výkonem v Anti-saccade task a testech Go/No-go task a Stop-signal task [The relationship between performance on the Anti-saccade Task, Go/No-go task and the Stop-signal task] [Bachelor thesis]. Masaryk University.

32.

Faul

Erdfelder

Buchner

Lang

A.-G.

(2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.

33.

Finney

DiStefano

(2006). Non-normal and categorical data in structural equation modeling. Structural Equation Modeling: A Second Course, 10(6), 269–314.

34.

Flegr

Hampl

Černochová

Preiss

Bičíková

Sieger

Příplatová

Kaňková

Š.

Klose

(2012). The relation of cortisol and sex hormone levels to results of psychological, performance, IQ and memory tests in military men and women. Neuro Endocrinology Letters, 33(2), 224–235; 112.

35.

Flegr

Preiss

Balátová

(2018). Depressiveness and neuroticism in Bartonella seropositive and seronegative subjects – preregistered case-controls study. Frontiers in Psychiatry, 9, 314.

36.

Formann

A. K.

Piswanger

(1979). Wiener Matrizen-Test. Beltz.

37.

Formann

A. K.

Waldherr

Piswanger

(2011). Wiener Matrizen-Test 2: Ein rasch-skalierter sprachfreier kurztest zur erfassung der intelligenz. Hogrefe.

38.

Furr

(2017). Psychometrics: An introduction. SAGE Publications.

39.

Gamer

Lemon

Sings

(2019). irr: Various coefficients of interrater reliability and agreement. R package version 0.84.1. https://cran.r-project.org/package=irr

40.

Hain

Schermelleh-Engel

Freitag

Louwen

Oddo

(2016). Development of a short form of the personality styles and disorder inventory (PSDI-6): Initial validation in a sample of pregnant women. European Journal of Psychological Assessment, 32(4), 283–290.

41.

Hasegawa

Somatori

Nishimura

Hattori

Kunisato

(2019). Associations between self-reported impulsivity and a latent variable of impulsive action constructed from three laboratory tasks. Journal of Experimental Psychopathology, 10(3), 1–12. https://doi.org/10.1177/2043808719861894.

42.

Hedge

Powell

Sumner

(2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. https://doi.org/10.3758/s13428-017-0935-1

43.

Helísková

(2016). Možnosti a limity online počítačové diagnostiky: Evaluace adaptace Stroopova prostorového testu [The capabilities and the limits of online PC diagnostics: Adaptation of the Stroop test] [Diploma thesis]. Masaryk University.

44.

Hilbert

Nakagawa

T. T.

Bindl

Bühner

(2014). The spatial stroop effect: A comparison of color-word and position-word interference. Psychonomic Bulletin & Review, 21(6), 1509–1515.

45.

Jorgensen

T. D.

Pornprasertmanit

Schoemann

A. M.

Rosseel

(2018). semTools: Useful tools for structural equation modeling. R package version 0.5-2. https://CRAN.R-project.org/package=semTools

46.

Kane

Hambrick

Conway

(2005). Working memory capacity and fluid intelligence are strongly related constructs: Comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131(1), 66–71.

47.

Kirschner

Cruse

Chennu

Owen

Hampshire

(2015). A P300-based cognitive assessment battery. Brain and Behavior, 5(6), e00336.

48.

Klose

Černochová

Král

(2002). Vídeňský maticový test [viennese matrices test]. Testcentrum.

49.

Koolhof

Loeber

Wei

Pardini

D'escury

(2007). Inhibition deficits of serious delinquent boys of low intelligence. Criminal Behaviour and Mental Health : Cbmh, 17(5), 274–292. https://doi.org/10.1002/cbm.661.

50.

Kuhl

Kazén

(2009). Persönlichkeits-Stil- und Störungs-Inventar (PSSI) [Personality styles and disorder inventory (PSDI)]. Hogrefe.

51.

Kulacaoglu

Solmaz

Belli

Ardic

Akin

Kose

(2017). The relationship between impulsivity and attention-deficit/hyperactivity symptoms in female patients with borderline personality disorder. Psychiatry and Clinical Psychopharmacology, 27(3), 249–255. https://doi.org/10.1080/24750573.2017.1342317.

52.

Lakens

(2017). Equivalence tests: A practical primer for T tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355–362.

53.

Lakens

(2018). Two one-sided tests (TOST) equivalence testing. R package version 0.3.4. https://cran.r-project.org/web/packages/TOSTER/

54.

Lakens

Scheel

A. M.

Isager

P. M.

(2018). Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259–269.

55.

(2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949.

56.

Linhartová

Širůček

Barteček

Theiner

Jeřábková

Rudišinová

Kašpárek

(2017). České verze sebeposuzovacích modelů impulzivity barrattovy škály a škály UPPS-P a jejich psychometrické charakteristiky [Czech versions of impulsivity self-report scales the Barratt scale and the UPPS-P scale and their psychometric properties]. Česká a Slovenská Psychiatrie, 113(4), 149–157.

57.

Logan

G. D.

Cowan

W. B.

(1984). On the ability to inhibit thought and action: A theory of an act of control. Psychological Review, 91(3), 295–327.

58.

Lopez

Dauvilliers

Jaussent

Billieux

Bayard

(2015). A multidimensional approach of impulsivity in adult attention deficit hyperactivity disorder. Psychiatry Research, 227(2–3), 290–295. https://doi.org/10.1016/j.psychres.2015.03.023.

59.

Lozano

(2015). Are impulsivity and intelligence truly related constructs? Evidence based on the fixed-links model. Personality and Individual Differences, 85, 192–198. https://doi.org/10.1016/j.paid.2015.04.049.

60.

Lozano

Gordillo

Pérez

(2014). Impulsivity, intelligence, and academic performance: Testing the interaction hypothesis. Personality and Individual Differences, 61–62, 63–68. https://doi.org/10.1016/j.paid.2014.01.013.

61.

Lupiáñez

Funes

M. J.

(2005). Peripheral spatial cues modulate spatial congruency effects: Analysing the locus. Of the cueing modulation. European Journal of Cognitive Psychology, 17(5), 727–752.

62.

Lynam

Moffitt

Stouthamer-Loeber

(1993). Explaining the relation between IQ and delinquency: Class, race, test motivation, school failure or self-control? Journal of Abnormal Psychology, 2, 187–196.

63.

Lynam

D. R.

Whiteside

S. P.

Smith

G. T.

Cyders

M. A.

(2006). The UPPS-P: Assessing five personality pathways to impulsive behavior [Unpublished report]. Purdue University.

64.

Matusiewicz

A. K.

Reynolds

Lejuez

C. W.

(2011). Assessment instruments for impulsivity and impulse control disorders. In Grant

J. E.

Potenza

M. N.

(Eds.), The oxford handbook of impulse control disorders. Oxford Handbooks.

65.

Malesza

Ostaszewski

(2016). Dark side of impulsivity – associations between the dark triad, self-report and behavioral measures of impulsivity. Personality and Individual Differences, 88, 197–201.

66.

Metin

Roeyers

Wiersema

J. R.

van der Meere

Sonuga-Barke

(2012). A meta-analytic study of event rate effects on go/No-Go performance in attention-deficit/hyperactivity disorder. Biological Psychiatry, 72(12), 990–996. https://doi.org/(…)biopsych.2012.08.023.

67.

Meule

(2017). Reporting and interpreting task performance in go/No-Go affective shifting tasks. Frontiers in Psychology, 8, 701. https://doi.org/10.3389/fpsyg.2017.00701.

68.

Miller

Derefinko

Lynam

Milich

Fillmore

(2009). Impulsivity and attention deficit-hyperactivity disorder: Subtype classification using the UPPS impulsive behavior scale. Journal of Psychopathology and Behavioral Assessment, 32(3), 323–332. https://doi.org/10.1007/s10862-009-9155-z.

69.

Ministry of Defense and Armed Forces of Czech Republic. (2020). Kvantitativní genderová analýza k 1. lednu 2020 [Quantitative gender analysis – January 2020]. Ministry of Defense and Armed Forces of Czech Republic. https://www.mocr.army.cz/assets/informacni-servis/povinne-informace/1-rovne-prilezitosti/kvantitativni-genderova-analyza-k-1–1–2020.pdf.

70.

Moeller

F. G.

Barratt

E. S.

Dougherty

D. M.

Schmitz

J. M.

Swann

A. C.

(2001). Psychiatric aspects of impulsivity. The American Journal of Psychiatry, 158(11), 1783–1793. https://doi.org/10.1176/appi.ajp.158.11.1783

71.

Morooka

Ogino

Takeuchi

Hanafusa

Oka

Ohtsuka

(2012). Relationships between the color-word matching stroop task and the go/NoGo task: Toward multifaceted assessment of attention and inhibition abilities of children. Acta Medica Okyayma, 66(5), 337–386. http://ousar.lib.okayama-u.ac.jp/ja/49385.

72.

Newman

A. L.

Meyer

T. D.

(2014). Impulsivity: Present during euthymia in bipolar disorder? – A systematic review. International Journal of Bipolar Disorders, 2(2), 2.

73.

Nielsen

R. E.

Lindström

Telléus

G. K.

Levander

(2014). Is the PANSS cognitive scale measuring cognition? Nordic Journal of Psychiatry, 68(8), 573–578.

74.

Nigg

J. T.

(2000). On inhibition/disinhibition in developmental psychopathology: Views from cognitive and personality psychology and a working inhibition taxonomy. Psychological Bulletin, 126(2), 220–246.

75.

Onur

O. S.

Tabo

Aydin

Tuna

Maner

A. F.

Yildirim

E. A.

Çarpar

(2016). Relationship between impulsivity and obsession types in obsessive-compulsive disorder. International Journal of Psychiatry in Clinical Practice, 20(4), 218–223. https://doi.org/10.1080/13651501.2016.1220580.

76.

Parsons

Kruijt

A.-W.

Fox

(2019). Psychological science needs a standard practice of reporting the reliability of cognitive-behavioral measurements. Advances in Methods and Practices in Psychological Science, 2(4), 378–395. https://doi.org/10.1177/2515245919879695.

77.

Patton

J. H.

Stanford

M. S.

Barratt

E. S.

(1995). Factor structure of the Barratt impulsiveness scale. Journal of Clinical Psychology, 51(6), 768–774.

78.

Perales

J. C.

Verdejo-García

Moya

Lozano

Ó.

Pérez-García

(2009). Bright and dark sides of impulsivity: Performance of women with high and low trait impulsivity on neuropsychological tasks. Journal of Clinical and Experimental Neuropsychology, 31(8), 927–944.

79.

Potyková

(2017). Počítačová adaptace Testu verbálního uvažování v kontextu specifických faktorů inteligence [Computer adaptation of Verbal reasoning test in the context of specific intelligence factors] [Diploma thesis]. Masaryk University.

80.

Řádová

Čeněk

Jankovská

Morong

Šašinka

Č.

Ugwitz

Fabianová

Brančík

Sedlák

(2018). ImGO test impulzivity: Manuál psychodiagnostické metody ImGO (verze 1.0) [ImGO test of impulsivity: Manual of psychological assessment test (version 1.0)]. Masaryk University.

81.

R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

82.

Rasch

(1960). Probabilistic models for some intelligence and attainment tests. MESA Press.

83.

Ratcliff

(1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114(3), 510–532.

84.

Ratcliff

Huang-Pollock

McKoon

(2018). Modeling individual differences in the go/no-go task with a diffusion model. Decision, 5(1), 42–62.

85.

Raud

Westerhausen

Dooley

Huster

R. J.

(2020). Differences in unity: The go/no-go and stop signal tasks rely on different mechanisms. NeuroImage, 15, 116582.

86.

Raven

J. C.

(1958). Standard progressive matrices. Lewis.

87.

Rentrop

Backenstrass

Jaentsch

Kaiser

Roth

Unger

Weisbrod

Renneberg

(2008). Response inhibition in borderline personality disorder: Performance in a go/nogo task. Psychopathology, 41(1), 50–57. https://doi.org/10.1159/000110626.

88.

Reynolds

Ortengren

Richards

J. B.

de Wit

(2006). Dimensions of impulsive behavior: Personality and behavioral measures. Personality and Individual Differences, 40(2), 305–315.

89.

Rosseel

(2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. http://www.jstatsoft.org/v48/i02/.

90.

Rubia

Taylor

Smith

A. B.

Oksanen

Overmeyer

Newman

Oksannen

(2001). Neuropsychological analyses of impulsiveness in childhood hyperactivity. The British Journal of Psychiatry, 179(2), 138–143.

91.

Ruchsow

Groen

Kiefer

Buchheim

Walter

Martius

Reiter

Hermle

Spitzer

Ebert

Falkenstein

(2008). Response inhibition in borderline personality disorder: Event-related potentials in a go/nogo task. Journal of Neural Transmission, 115(1), 127–133. https://doi.org/10.1007/s00702-007-0819-0.

92.

Russo

De Pascalis

Varriale

Barratt

(2008). Impulsivity, intelligence and P300 wave: An empirical study. International Journal of Psychophysiology, 69(2), 112–118. https://doi.org/10.1016/j.ijpsycho.2008.03.008.

93.

Šašinka

Č.

Čeněk

Morong

Malatincová

Točík

Ugwitz

Urbánek

Helísková

Strnadová

(2017). Interfereční polohový test: Manuál diagnostické metody IPT (verze 1.0) [Positional Interference Test: Manual of psychological assessment test (version 1.0)]. Masaryk University.

94.

Šašinka

Č.

Čeněk

Morong

Urbánek

Ugwitz

Helísková

Strnadová

(2017). Flagtest: Manuál psychodiagnostické metody (verze 1.0) [Flagtest: Manual of psychological assessment test (version 1.0)]. Masaryk University.

95.

Šašinka

Č.

Morong

Stachoň

(2017). The hypothesis platform: An online tool for experimental research into work with maps and behavior in electronic environments. ISPRS International Journal of Geo-Information, 6(12), 407–422.

96.

Schachar

Logan

G. D.

Robaey

Chen

Ickowicz

Barr

(2007). Restraint and cancellation: Multiple inhibition deficits in attention deficit hyperactivity disorder. Journal of Abnormal Child Psychology, 35(2), 229–238.

97.

Schönbrodt

F. D.

Perugini

(2013). At what sample size do correlations stabilize? Journal of Research in Personality, 47(5), 609–612.

98.

Schweizer

(2002). Does impulsivity influence performance in reasoning? Personality and Individual Differences, 33(7), 1031–1043. https://doi.org/10.1016/s0191-8869(01)00209-4.

99.

Shaked

Faulkner

L. M.

Tolle

Wendell

C. R.

Waldstein

S. R.

Spencer

R. J.

(2020). Reliability and validity of the Conners’ continuous performance test. Applied Neuropsychology. Adult, 27(5), 478–487.

100.

Smith

J. L.

Mattick

R. P.

Jamadar

S. D.

Iredale

J. M.

(2014). Deficits in behavioural inhibition in substance abuse and addiction: A meta-analysis. Drug and Alcohol Dependence, 145, 1–33.

101.

Spearman

(1904). ‘General intelligence,’ objectively determined and measured. The American Journal of Psychology, 15(2), 201–293.

102.

Spinella

(2002). Correlations among behavioral measures of orbitofrontal function. The International Journal of Neuroscience, 112(11), 1359–1369.

103.

Spinella

(2004). Neurobehavioral correlates of impulsivity: Evidence of prefrontal involvement. The International Journal of Neuroscience, 114(1), 95–104.

104.

Strasser

Haffner

Fiebig

Quinlivan

Adli

Stamm

(2016). Behavioral measures and self-report of impulsivity in bipolar disorder: No association between stroop test and Barratt impulsiveness scale. International Journal of Bipolar Disorders, 4(1). https://doi.org/10.1186/s40345-16.0057-1.

105.

Stroop

J. R.

(1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662.

106.

Švancara

(2002). PSSI, inventář stylů osobnosti a poruch osobnosti [PSSI, inventory of personality styles and personality disorders]. Testcentrum.

107.

Swerdlow

N. R.

Paulsen

Braff

D. L.

Butters

Geyer

M. A.

Swenson

M. R.

(1995). Impaired prepulse inhibition of acoustic and tactile startle response in patients with Huntington’s disease. Journal of Neurology, Neurosurgery & Psychiatry, 58(2), 192–200.

108.

Taylor

A. J.

(2016). Trait impulsivity and oculomotor inhibition. Studia Psychologica, 58(2), 134–144.

109.

Tichá

(2017). Vztah mezi výkonem v Testu pozornosti d2, pracovním tempem a mírou chybovosti v performační zkoušce [The relationship between the d2 test of attention performance, work tempo and number of errors in performance test] [Diploma thesis]. Masaryk University.

110.

Trommer

Hoeppner

Lorber

Armstrong

(1988). The Go-No-Go paradigm in attention deficit disorder. Annals of Neurology, 24(5), 610–614. https://doi.org/10.1002/ana.410240504.

111.

Vacíř

(1973). Sledovánıí rozhodovacích procesů v časové tísni [Monitoring of decision making processes under time pressure] [Doctoral thesis]. Charles University.

112.

Van Zandt

(2002). Analysis of response time distributions. In Pashler

Wixted

(Eds.), Stevens' handbook of experimental psychology: Methodology in experimental psychology (pp. 461–516). John Wiley & Sons Inc.

113.

Verbruggen

Aron

A. R.

Band

G. P.

Beste

Bissett

P. G.

Brockett

A. T.

Brown

J. W.

Chamberlain

S. R.

Chambers

C. D.

Colonius

Colzato

L. S.

Corneil

B. D.

Coxon

J. P.

Dupuis

Eagle

D. M.

Garavan

Greenhouse

Heathcote

Huster

R. J.

, … Boehler

C. N.

(2019). A consensus guide to capturing the ability to inhibit actions and impulsive behaviors in the stop-signal task. eLife, 8, e46323.

114.

Verbruggen

Chambers

C. D.

Logan

G. D.

(2013). Fictitious inhibitory differences: How skewness and slowing distort the estimation of stopping latencies. Psychological Science, 24(3), 352–362.

115.

Verbruggen

Logan

G. D.

(2009). Models of response inhibition in the stop-signal and stop-change paradigms. Neuroscience and Biobehavioral Reviews, 33(5), 647–661.

116.

Vigil-Colet

Morales-Vives

(2005). How impulsivity is related to intelligence and academic achievement. The Spanish Journal of Psychology, 8(2), 199–204. https://doi.org/10.1017/s1138741600005072.

117.

Wang

Zamar

Marazzi

Yohai

Salibian-Barrera

Maronna

Zivot

Rocke

Martin

Maechler

Konis

(2020). robust: Port of the S+ “Robust Library”. R package version 0.5-0. https://CRAN.Rproject.org/package=robust

118.

Weir

J. P.

(2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. Journal of Strength and Conditioning Research, 19(1), 231–240. https://doi.org/10.1519/15184.1.

119.

Wilbertz

Deserno

Horstmann

Neumann

Villringer

Heinze

H.-J.

Boehler

C. N.

Schlagenhauf

(2014). Response inhibition and its relation to multidimensional impulsivity. NeuroImage, 103, 241–248.

120.

Wright

Lipszyc

Dupuis

Thayapararajah

S. W.

Schachar

(2014). Response inhibition and psychopathology: A Meta-analysis of go/no-go task performance. Journal of Abnormal Psychology , 123(2), 429–439.

121.

Yao

(2017). Robust linear regression: A review and comparison. Communications in Statistics – Simulation and Computation, 46(8), 6261–6282.

122.

Yuan

Steedle

Shavelson

Alonzo

Oppezzo

(2006). Working memory, fluid intelligence, and science learning. Educational Research Review, 1(2), 83–98.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB