A Robust Test for Checking the Homogeneity of Variability Measures and Its Application to the Analysis of Implicit Attitudes

Abstract

A new two-sample test for comparing variability measures is proposed. To make the test robust and powerful, a new modified structural zero removal method is applied to the Brown–Forsythe transformation. The t-test-based statistic allows results to be expressed as the ratio of mean absolute deviations from median. Extensive simulation study demonstrates that the proposed test is robust to small or unequal sample sizes across many distributions. Moreover, careful exploratory analysis provides a new method for calculating the implicit association test scores for reaction time data with multiplicative treatment effects. Using this, a possible difference between variability of men and women’s implicit attitudes toward gay men is analyzed.

Keywords

Brown-Forsythe transformation effect size implicit association test

1. Introduction

Comparing means to characterize group differences is common, but sometimes variability differences between groups are more meaningful. For instance, age groups differ by variability in sleep spindles and rapid eye movements (Peters, Ray, Fogel, Smith, & Smith, 2014). Likewise, certain mental disorders are indicated by variability differences; larger intravariability in reaction times (RTs) is observed among individuals with schizophrenia and attention deficit hyperactivity disorder (Smyrnis et al., 2009; Tamm et al., 2012; Theleritis, Evdokimidis, & Smyrnis, 2014). Additional applications in archaeology, environmental sciences, economics, medical research, and legal studies are also documented by Gastwirth, Gel, and Miao (2009). The variety of potential applications motivates the main goal of our article, which is to develop a reliable test for checking the equality of variances or other robust measures of variability.

The popular variance ratio test (also known as the F test) is highly sensitive to nonnormal distributions (Box, 1953; Shoemaker, 2003). Therefore, many attempts have been made to develop a hypothesis test for comparing variability measures that is robust to nonnormal distributions, outliers, and small or unequal sample sizes. One of the most popular tests is developed by Levene (1960), which compares variability measures among samples based on the absolute deviation from the sample mean. His transformation, referred to as Levene’s transformation (Neuhäuser, 2007), makes the resulting hypothesis test nonrobust when the underlying distribution is skewed.

Brown and Forsythe (1974) suggested a more useful modification that employs the absolute deviation from the sample median, which we refer to as the Brown–Forsythe transformation. Through simulations, they showed that their transformation makes the test more robust to skewed distributions. Their results were confirmed by a large simulation study in Conover, Johnson, and Johnson (1981) and theoretically by Carroll and Schneider (1985).

The Brown–Forsythe transformation, however, makes the test very conservative when the sample sizes are small and odd. This weakness was demonstrated in various simulation studies (Carroll & Schneider, 1985; Conover, Johnson, & Johnson, 1981; Lim & Loh, 1996; Martin, 1976). Conover et al. (1981) and Hines and O’Hara Hines (2000) identified that the weakness is largely due to the presence of a “structural zero” (the transformed observation which is equal to zero, as one observation is exactly equal to the sample median). Removing the structural zero makes the test less conservative and more powerful.

Another line of research examines the bias of the sample mean of the Levene- or Brown–Forsythe-transformed normal random variables to the standard deviation as functions of sample sizes. To obtain a more reliable hypothesis test or confidence interval in the unequal sample size cases, several correction factors have been suggested (Bonett & Seier, 2003; Keyes & Levy, 1997; O’Brien, 1978; O’Neill & Mathews, 2000). Moreover, Noguchi and Gel (2010) considered the zero-correction method that combines the structural zero removal method of Hines and O’Hara Hines (2000) and the correction factor suggested by O’Brien (1978) for the Brown–Forsythe transformation. A modified version of the zero-correction method, which we explain in detail in the following section, tends to make the test both robust and powerful under unequal or odd sample sizes and skewed or heavy-tailed distributions.

Considering the independent two-sample case in particular, Marozzi (2011) compared a number of two-sample tests and recommended the Brown–Forsythe test as well as several resampling-based methods including the bootstrap-based test of Hall and Padmanabhan (1997) and Pan (1999). However, Bonett and Seier (2003) claimed that recent modifications of the classical F test suggested by Shoemaker (2003), the bootstrap-based test of Hall and Padmanabhan (1997), and the delta method–based test of Pan (1999) are not robust with respect to violations of distributional assumptions or unequal sample sizes. Specifically, they have pointed out the nonrobustness of the abovementioned tests when the samples are drawn from nonidentical distributions. To address these issues, they have developed their own method in the form of a confidence interval and showed that theirs is much more robust with respect to such violations.

Therefore, we aim at developing a robust independent two-sample test for comparing variability measures, which is at least as competitive as that of Bonett and Seier (2003), using the idea of Pan (1999). To do so, we incorporate ideas of the Brown–Forsythe transformation, structural zero removal, new correction factor, and improved effective degrees of freedom based on Noguchi and Marmolejo-Ramos (2016) in our test. Our novel correction factor, based on the results of O’Neill and Mathews (2000), is more suitable for the Brown–Forsythe transformation than the previously suggested correction factors. Moreover, we modify the zero-correction method of Noguchi and Gel (2010) by taking into account the simulation results of Loh (1987). Then, we show through carefully planned sets of simulations that our test is at least as robust and powerful as that of Bonett and Seier (2003), outperforming in a number of situations especially when the sample sizes are small or odd.

To demonstrate an application of the proposed test, we conduct a thorough reanalysis of the data set used in Lemm (2006). The reanalysis offers new insights into characterization of gender differences based on variability in implicit attitudes toward gay men (ATGM). To obtain a reliable conclusion, we also develop a robust method for calculating the implicit association test (IAT) scores for the RT data with multiplicative treatment effects and errors. Using the results from the proposed test based on the robust IAT scores, we show how the differences in variability may imply a gender difference in mean ATGM conditional on the levels of close contact by considering the contact hypothesis of Allport (1954).

This article is organized as follows. In Section 2, we suggest a modified zero-correction method based on a new correction factor and structural zero removal method. In Section 3, we discuss the new test statistic and its Welch–Satterthwaite type small-sample approximation. In Section 4, we discuss the performance of the proposed test in terms of the empirical size and power for various sample size and distribution combinations, which are plausible for comparing RT data. Then, in Section 5, we propose a robust IAT score calculation method, called log ratio analysis, for the multiplicative treatment effect. Based on the RT data used in Lemm (2006), we examine the differences between men and women in terms of mean and variability of the implicit ATGM scores. Lastly, in Section 6, we offer concluding remarks and mention possible future extensions.

2. Modified Zero-Correction Method

We start by explaining a modified zero-correction method, which is essentially a combination of the structural zero removal method of Hines and O’Hara Hines (2000) and an appropriate novel correction factor in the unequal sample size case. By devising the modified zero-correction method, the proposed test that will be described later becomes less conservative and more powerful for any sample size combinations.

Let $x_{i j} \sim F_{i}$ , $i = 1, 2$ , $j = 1, 2, . . ., n_{i}$ , be independent random variables with $E [x_{i j}] = μ_{i}$ and $Var (x_{i j}) = σ_{i}^{2} > 0$ . Let ${\bar{x}}_{i} = median (x_{i 1}, . . ., x_{i n_{1}})$ and ${\bar{x}}_{i} = \sum_{j = 1}^{n_{i}} x_{i j} / n_{i}$ , $i = 1, 2$ be the sample median and sample mean, respectively. Then, as shown in Brown and Forsythe (1974), Conover et al. (1981), and Lim and Loh (1996), it is well known that the Brown–Forsythe transformation, $d_{i j} = | x_{i j} - {\tilde{x}}_{i} |$ , makes the test for comparing variability measures robust to nonnormal distributions. Therefore, the test explored in this article is based on $d_{i j}$ , $i = 1, 2$ , $j = 1, 2, . . ., n_{i}$ .

To understand how $d_{i j}$ are related to the standard deviations $σ_{i}$ , O’Neill and Mathews (2000) showed that $E [d_{i j}] = κ_{n_{i}} σ_{i}$ for some constant $κ_{n_{i}} > 0$ that depends on n_i, where $κ_{n_{i}} \to \sqrt{2 / π}$ as $n_{i} \to \infty$ , assuming that F_i are normal. Although the actual calculation of $κ_{n_{i}}$ requires numerical integrations, values of $κ_{n_{i}}$ for some selected n_i can be obtained from Table 5 of O’Neill and Mathews (2000). A similar result based on the absolute deviation from the sample mean, $d_{i j}^{'} = | x_{i j} - {\bar{x}}_{i} |$ , shows that $E [d_{i j}^{'}] = \sqrt{2 / π} \sqrt{(n_{i} - 1) / n_{i}} σ_{i}$ , assuming that F_i are normal (Keyes & Levy, 1997; O’Neill & Mathews, 2000; Peters, 1856). Based on that, O’Brien (1978) and Noguchi and Gel (2010) suggested to use $κ_{n_{i}} \approx \sqrt{2 / π} \sqrt{(n_{i} - 1) / n_{i}}$ as an approximation. On the other hand, Bonett and Seier (2003) have suggested yet another approximation that $κ_{n_{i}}$ is approximately proportional to $(n_{i} - 1) / n_{i}$ based on their empirical observations. As a remark, the approximation only needs to be determined up to a constant multiple because such a constant multiple gets canceled out in the test statistic.

Even though several approximations for $κ_{n_{i}}$ have been suggested previously, we suggest an improved novel approximation by exploiting the fact that $κ_{n_{i}} = κ_{n_{i} + 1}$ when n_i is even (O’Neill & Mathews, 2000). Specifically, let us define $r_{i} = n_{i}$ if n_i is even, and $r_{i} = n_{i} - 1$ if n_i is odd. Then, assuming that $κ_{n_{i}} \approx \sqrt{2 / π} {[r_{i} / (r_{i} + 1)]}^{β}$ for some suitable β, a linear regression of $log (κ_{n_{i}})$ on $log (\sqrt{2 / π} [r_{i} / (r_{i} + 1)])$ using $κ_{n_{i}}$ values from Table 5 of O’Neill and Mathews (2000) provides the least square estimate of $β \approx 5 / 6$ . Indeed, $κ_{n_{i}} \approx \sqrt{2 / π} {[r_{i} / (r_{i} + 1)]}^{5 / 6}$ provides an excellent approximation for any sample size as shown in Table 1.

Table 1.

Comparison of the Three Approximations of $κ_{n_{i}}$

n_i	$κ_{n_{i}}$	$\sqrt{\frac{2}{π}} {(\frac{r_{i}}{r_{i} + 1})}^{5 / 6 a}$	$\sqrt{\frac{2}{π}} {(\frac{n_{i} - 1}{n_{i}})}^{1 / 2 b}$	$\sqrt{\frac{2}{π}} (\frac{n_{i} - 1}{n_{i}})$ ^c
2	.564	.569	.564	.399
3	.564	.569	.651	.532
4	.663	.662	.691	.598
5	.663	.662	.714	.638
6	.704	.702	.728	.665
7	.704	.702	.739	.684
8	.725	.723	.746	.698
9	.725	.723	.752	.709
10	.739	.737	.757	.718
20	.768	.766	.778	.758
30	.777	.776	.784	.771
40	.782	.782	.788	.778
50	.786	.785	.790	.782
60	.788	.787	.791	.785
70	.789	.789	.792	.786
80	.790	.790	.793	.788
90	.791	.791	.793	.789
100	.792	.791	.794	.790
∞	.798	.798	.798	.798

Note. ^aModified zero-correction in this article; ^bMethod of O’Brien (1978); ^cMethod of Bonett and Seier (2003).

Another improvement that we discuss in this article is a modification of the structural zero removal method briefly suggested in Conover et al. (1981) and discussed in detail by Hines and O’Hara Hines (2000). When n_i is odd, there is an observation $x_{i j}$ for some j that is exactly equal to ${\tilde{x}}_{i}$ , resulting in $d_{i j} = 0$ , which is called a “structural zero” (Hines & O’Hara Hines, 2000). The structural zero reduces the power of the Brown–Forsythe transformation-based test, making the test very conservative especially in the case of small odd sample sizes (Conover et al., 1981). A simple remedy is to remove the structural zero in the case of odd sample sizes. For the case of even sample sizes, Hines and O’Hara Hines (2000) have suggested data transformation that creates a structural zero to be removed. However, a simulation study by Loh (1987) suggests that the Brown–Forsythe transformation-based test does not become conservative even when the even sample size is as small as 4. Therefore, we do not consider a structural zero removal method for even sample sizes.

Based on the observations above, we suggest an approach of combining the structural zero removal method and the new correction factor ${[(r_{i} + 1) / r_{i}]}^{5 / 6}$ , similar to the method suggested in Noguchi and Gel (2010). Firstly, note that $E [{\bar{d}}_{i}] = κ_{n_{i}} σ_{i}$ , where ${\bar{d}}_{i} = \sum_{j = 1}^{n_{i}} d_{i j} / n_{i}$ is the $i th$ sample mean of $d_{i j}$ , $i = 1, 2$ , when F_i are normal. Then, let us consider a new sample where the structural zero is removed when n_i is odd. The new sample without the structural zero, denoted by $\{d_{i 1}^{(0)}, d_{i 2}^{(0)}, . . ., d_{i (n_{i} - 1)}^{(0)}\}$ , has a sample size of $n_{i} - 1$ and thus ${\bar{d}}_{i}^{(0)} = \sum_{j = 1}^{n_{i} - 1} d_{i j}^{(0)} / (n_{i} - 1) = \sum_{j = 1}^{n_{i}} d_{i j} / (n_{i} - 1) = [n_{i} / (n_{i} - 1)] {\bar{d}}_{i}$ . In other words, for the sample mean ${\bar{d}}_{i}^{(0)}$ to be unbiased for $σ_{i}$ under the assumption of F_i being normal, it is necessary to consider the transformation $[(n_{i} - 1) / (n_{i} κ_{n_{i}})] d_{i j} \approx \sqrt{π / 2} [(n_{i} - 1) / n_{i}] {[(r_{i} + 1) / r_{i}]}^{5 / 6} d_{i j}$ to each $d_{i j}$ for which $d_{i j} \neq 0$ after the structural zero removal. A summary of the modified zero-correction method is given below.

Modified Zero-Correction Method:

When n_i is odd:

Remove the structural zero ( $d_{i j} = 0$ ).

Let $r_{i} = n_{i} - 1$ . Apply the transformation $d_{i j}^{⋆} = [(n_{i} - 1) / n_{i}] {[(r_{i} + 1) / r_{i}]}^{5 / 6} d_{i j} = {[(r_{i} + 1) / r_{i}]}^{−1 / 6} d_{i j}$ to the remaining observations.

When n_i is even:

Let $r_{i} = n_{i}$ . Apply the transformation $d_{i j}^{⋆} = {[(r_{i} + 1) / r_{i}]}^{5 / 6} d_{i j}$ to all the observations.

As a remark, the constant multiple $\sqrt{π / 2}$ is dropped in the summary as it does not affect the calculation of the test statistic.

3. Proposed Test Statistic and Its Small-Sample Approximation

The test statistic we consider in this article is based on the t statistic with the delta method applied to $g ({\bar{d}}_{i}^{⋆}) = log ({\bar{d}}_{i}^{⋆})$ , where ${\bar{d}}_{i}^{⋆} = \sum_{j = 1}^{r_{i}} d_{i j}^{⋆} / r_{i}$ , which is similar to the one suggested in Pan (1999, 2002). To understand the motivation for the logarithmic transformation, let $τ_{i} = E [| x_{i j} - {\tilde{μ}}_{i} |] > 0$ be the population mean absolute deviation from median ( ${\tilde{μ}}_{i}$ ), which is estimated by ${\bar{d}}_{i}^{⋆}$ . Then, because $τ_{i} > 0$ , a common way of comparing two positive quantities of interest is to take their ratio, similar to the F test that utilizes the variance ratio.

To come up with a hypothesis testing procedure for checking the equality of $τ_{1}$ and $τ_{2}$ using their ratio, note that testing the hypothesis $H_{0} : τ_{1} / τ_{2} = 1$ is equivalent to testing $H_{0} : log (τ_{1}) = log (τ_{2})$ . Moreover, we may expect an increase in power as the distribution of $g ({\bar{d}}_{i}^{⋆})$ may be more symmetric than that of ${\bar{d}}_{i}^{⋆}$ itself. An appropriate standardization can be achieved by a direct application of the bivariate delta method (Lehmann, 2011) with Theorem A.1 of Gastwirth (1982), which gives that

\begin{matrix} T_{g} = \frac{g ({\bar{d}}_{1}^{⋆}) - g ({\bar{d}}_{2}^{⋆})}{\sqrt{{\frac{[s_{d_{1}^{⋆}} g^{'} ({\bar{d}}_{1}^{⋆})]}{r_{1}}}^{2} + \frac{{[s_{d_{2}^{⋆}} g^{'} ({\bar{d}}_{2}^{⋆})]}^{2}}{r_{2}}}}, \end{matrix}

where $s_{d_{i}^{⋆}}^{2} = \sum_{j = 1}^{r_{i}} {(d_{i j}^{⋆} - {\bar{d}}_{i}^{⋆})}^{2} / (r_{i} - 1)$ . The test statistic T_g converges in distribution to $Z \sim N (0, 1)$ under $H_{0}$ as $min \{r_{1}, r_{2}\} \to \infty$ with $r_{i} / (r_{1} + r_{2}) \to λ_{i} \in (0, 1)$ for any continuously differentiable function g such that $g^{'} (τ_{i}) \neq 0$ , $i = 1, 2$ . For a small-sample approximation, a slight modification of Noguchi and Marmolejo-Ramos (2016) gives that T_g∻ $t_{ν_{g}}$ under $H_{0}$ , where

\begin{matrix} ν_{g} = \frac{{(\frac{{[s_{d_{1}^{⋆}} g^{'} ({\bar{d}}_{1}^{⋆})]}^{2}}{r_{1}} + \frac{{[s_{d_{2}^{⋆}} g^{'} ({\bar{d}}_{2}^{⋆})]}^{2}}{r_{2}})}^{2}}{\frac{{[s_{d_{1}^{⋆}} g^{'} ({\bar{d}}_{1}^{⋆})]}^{4}}{[r_{1}^{2} (r_{1} - 1)]} + \frac{{[s_{d_{2}^{⋆}} g^{'} ({\bar{d}}_{2}^{⋆})]}^{4}}{[r_{2}^{2} (r_{2} - 1)]}}, \end{matrix}

and that “∻” denotes “approximately distributed as.” As shall be seen in Section 4, Equation 2 is useful to well approximate the distribution of Equation 1 under $H_{0}$ in the cases of small or unequal sample sizes and nonnormality. In our case, $g ({\bar{d}}_{i}^{⋆}) = log ({\bar{d}}_{i}^{⋆})$ gives $g^{'} ({\bar{d}}_{i}^{⋆}) = 1 / {\bar{d}}_{i}^{⋆}$ . That is,

\begin{matrix} T_{log} = \frac{log ({\bar{d}}_{1}^{⋆}) - log ({\bar{d}}_{2}^{⋆})}{\sqrt{\frac{s_{d_{1}^{⋆}}^{2}}{(r_{1} {\bar{d}}_{1}^{⋆ 2})} + \frac{s_{d_{2}^{⋆}}^{2}}{(r_{2} {\bar{d}}_{2}^{⋆ 2})}}} . \end{matrix}

For a small-sample approximation, although Pan (2002) suggested a Student’s t distribution with $r_{1} + r_{2} - 2$ degrees of freedom, such an approximation is valid only when equality of variances of $d_{1 j}^{⋆}$ and $d_{2 j}^{⋆}$ is assumed. However, because this assumption is often violated, a more robust approximation is suggested using Equation 2. In particular, t∻ $t_{ν_{log}}$ under $H_{0}$ , where

\begin{matrix} ν_{log} = \frac{{(\frac{s_{d_{1}^{⋆}}^{2}}{(r_{1} {\bar{d}}_{1}^{⋆ 2})} + \frac{s_{d_{2}^{⋆}}^{2}}{(r_{2} {\bar{d}}_{2}^{⋆ 2})})}^{2}}{\frac{s_{d_{1}^{⋆}}^{4}}{[r_{1}^{2} (r_{1} - 1) {\bar{d}}_{1}^{⋆ 4}]} + \frac{s_{d_{2}^{⋆}}^{4}}{[r_{2}^{2} (r_{2} - 1) {\bar{d}}_{2}^{⋆ 4}]}} . \end{matrix}

Based on the idea above, an approximate $100 (1 - α) %$ confidence interval for $τ_{1} / τ_{2}$ can be constructed. Specifically, $T_{log}$ ∻ $t_{ν_{log}}$ implies that

\begin{matrix} Pr (|log ({\bar{d}}_{1}^{⋆}) - log ({\bar{d}}_{2}^{⋆})| \leq t_{α / 2, ν_{log}} \sqrt{\frac{s_{d_{1}^{⋆}}^{2}}{(r_{1} {\bar{d}}_{1}^{⋆ 2})} + \frac{s_{d_{2}^{⋆}}^{2}}{(r_{2} {\bar{d}}_{2}^{⋆ 2})}} | |H_{0}) \approx 1 - α, \end{matrix}

where $t_{α / 2, ν_{log}}$ denotes the $100 (1 - α / 2) th$ percentile of Student’s t distribution with $ν_{log}$ degrees of freedom. Then, an approximate $100 (1 - α) %$ confidence interval for $log (τ_{1}) - log (τ_{2})$ is given by

\begin{matrix} log ({\bar{d}}_{1}^{⋆}) - log ({\bar{d}}_{2}^{⋆}) \pm t_{α / 2, ν_{log}} \sqrt{\frac{s_{d_{1}^{⋆}}^{2}}{(r_{1} {\bar{d}}_{1}^{⋆ 2})} + \frac{s_{d_{2}^{⋆}}^{2}}{(r_{2} {\bar{d}}_{2}^{⋆ 2})}} . \end{matrix}

After exponentiation, the confidence interval in Equation 6 becomes

\begin{matrix} [\frac{{\bar{d}}_{1}^{⋆}}{{\bar{d}}_{2}^{⋆}} exp (- t_{α / 2, ν_{log}} \sqrt{\frac{s_{d_{1}^{⋆}}^{2}}{r_{1} {\bar{d}}_{1}^{⋆ 2}} + \frac{s_{d_{2}^{⋆}}^{2}}{r_{2} {\bar{d}}_{2}^{⋆ 2}}}), \frac{{\bar{d}}_{1}^{⋆}}{{\bar{d}}_{2}^{⋆}} exp (t_{α / 2, ν_{log}} \sqrt{\frac{s_{d_{1}^{⋆}}^{2}}{r_{1} {\bar{d}}_{1}^{⋆ 2}} + \frac{s_{d_{2}^{⋆}}^{2}}{r_{2} {\bar{d}}_{2}^{⋆ 2}}})], \end{matrix}

which is a $100 (1 - α) %$ confidence interval for $τ_{1} / τ_{2}$ . A significant difference between $τ_{1}$ and $τ_{2}$ at level α with $H_{a} : τ_{1} / τ_{2} \neq 1$ may be declared when 1 is not included in Equation 7.

Simulation Study

We have conducted a simulation study to check the robustness and power of the proposed test. Both the simulated size and power of the tests are calculated by generating 10,000 data sets from various distributions. In particular, the normal, Student’s t, gamma, Weibull, and ex-Gaussian distributions are used. The four nonnormal distributions are chosen with three different input parameters, representing a wide variety of sampling distributions with skewness and kurtosis in the ranges of [0.00, 2.00] and [2.50, 9.00], respectively. The parameters are chosen such that the resulting distributions are suitable for representing real world data, such as RTs, which are typically modeled by one of the abovementioned distributions. Table 2 lists the chosen parameters and resulting distribution details. For the convenience of presenting the results, these 13 distributions are divided into three groups based on similarity (Group 1: symmetric, Group 2: moderately skewed, Group 3: highly skewed and heavy tailed). Because each sample may come from one of the 13 distributions, in total, we have considered $13 \times 13 = 169$ distinct distribution combinations.

Table 2.

Distribution Details for the Simulation Study

Distribution Group	Skewness	Kurtosis
1
N (0, 1)	0.00	3.00
t ₅	0.00	9.00
t ₁₀	0.00	4.00
t ₁₅	0.00	3.55
2
Gamma (4, 1)	1.00	4.50
Gamma (6, 1)	0.82	4.00
Gamma (8, 1)	0.71	3.75
Weibull (2.5, 1)	0.36	2.86
Weibull (3, 1)	0.17	2.73
Weibull (3.5, 1)	0.03	2.71
3
Ex-Gaussian (0, 1, 1.5)	1.15	5.88
Ex-Gaussian (0, 1, 2)	1.43	6.84
Ex-Gaussian (0, 1, 2.5)	1.60	7.46

For a comprehensive evaluation, we have selected 16 different combinations of sample sizes $(n_{1}, n_{2})$ . Specifically, we have $n_{i} = 5, 20, 35, 50$ , $i = 1, 2$ , covering the small, moderate, and large even and odd sample sizes with both the balanced and unbalanced designs. The cases involving small odd sample size(s) (i.e., $n_{i} = 5$ ) have been of particular interest because the tests based on the Brown–Forsythe transformation tend to be very conservative without the structural zero removal (Hines & O’Hara Hines, 2000). Similar to the distributions, for convenience, we classify the sample sizes as Group 1 (if $n_{i} = 5$ ), Group 2 (if $n_{i} = 20$ ), and Group 3 (if $n_{i} \geq 35$ ). Combining the cases with the 13 distributions for each sample, in total, we have 169 × 16 = 2,704 cases.

The performance of the proposed test is compared to the test suggested by Bonett and Seier (2003), originally presented in terms of confidence intervals. The Bonett–Seier (B-S) test statistic is a suitable benchmark test as they have favorable robustness properties compared to other popular tests for comparing variability measures (Bonett & Seier, 2003). The B-S test statistic is given by

T_{B-S} = \frac{log [\frac{n_{1} {\bar{d}}_{1}}{n_{1} - 1}] - log [\frac{n_{2} {\bar{d}}_{2}}{n_{2} - 1}]}{\sqrt{\frac{({\hat{δ}}_{1}^{2} + {\hat{γ}}_{1} - 1)}{n_{1}} + \frac{({\hat{δ}}_{2}^{2} + {\hat{γ}}_{2} - 1)}{n_{2}}}},

where ${\hat{δ}}_{i} = ({\bar{x}}_{i} - {\tilde{x}}_{i}) / {\bar{d}}_{i}$ , ${\hat{γ}}_{i} = s_{i}^{2} / {\bar{d}}_{i}^{2}$ and $s_{i}^{2} = \sum_{j = 1}^{n_{i}} {(x_{i j} - {\bar{x}}_{i})}^{2} / (n_{i} - 1)$ . The distribution of $T_{B-S}$ under $H_{0}$ is assumed to be $N (0, 1)$ , and the p value is given by $Pr (| T_{B-S} | \geq | t | | H_{0})$ , where t is the observed value of the test statistic.

For both the size and power calculations, we let $τ_{1} = 1.0$ , while $τ_{2}$ is allowed to be 1.0, 1.2, 1.4, 1.6, 1.8, or 2.0 with proper scaling of the generated data. For the size calculations (when $τ_{1} = τ_{2} = 1$ ), the nominal level of $α = .05$ is used. The results are summarized by the distributions and sample sizes using the box plots. For the power calculations (when $τ_{2} > 1$ ), the results are summarized using the bar plots, representing the proportion of each test that has performed considerably better than the other.

Figures 1 and 2 show that the proposed test maintains the nominal size of the test very well across different distribution and sample size combinations. However, the proposed test seems slightly liberal when there is a large imbalance in the sample sizes. On the other hand, the B-S test tends to be somewhat conservative, especially when at least one of the sample sizes is equal to 5. That is, at least one of the sample size groups is 1 in Figure 2. The odd and small sample size is less problematic for the proposed test as it benefits from the modified zero-correction method.

Figure 1.

Sizes of the test by different distribution group combinations.

Figure 2.

Sizes of the test by different sample size group combinations.

Figures 3 and 4 summarize the power comparisons between the proposed test and the B-S test. To make sure that the size of the test is very close to the nominal level of $α = .05$ , only those cases with $n_{1} \geq 35$ and $n_{2} \geq 35$ are considered. The proportions of the results for which the proposed test is at least $1 %$ more powerful than the B-S test are indicated by dark gray color, whereas those for which the proposed test is within $1 %$ of the B-S test are indicated by gray color. The x-axis represents the ratio $τ_{2} {/τ}_{1}$ , where $τ_{1} = 1$ . As expected, when $τ_{2} {/τ}_{1} = 1$ , all the sizes of the proposed test are within $1 %$ of the B-S test. However, as the ratio increases, up to about $50 %$ of the time, the proposed test is at least $1 %$ more powerful than the B-S test when the distribution is symmetric or moderately skewed (see Figure 3). On the other hand, the advantage of the proposed test somewhat diminishes when at least one of the distributions is highly skewed and heavy tailed (see Figure 4). In all cases, when $τ_{2} / τ_{1} = 2$ , the powers of the proposed test are within $1 %$ of that of the B-S test as the powers approach near 1. In addition, there is no case where the B-S test is at least $1 %$ more powerful than the proposed test.

Figure 3.

Power comparisons of the tests by different distribution group combinations for n₁ ≥ 35 and n₂ ≥ 35. Dark gray: The proposed test is at least 1% more powerful than the Bonnet–Seier test.

Figure 4.

Power comparisons of the tests by different distribution group combinations for n₁ ≥ 35 and n₂ ≥ 35. Dark gray: The proposed test is at least 1% more powerful than the Bonnet–Seier test.

Based on these observations, the proposed test displays robustness to unequal or small sample sizes and a wide range of skewness and kurtosis. Moreover, the proposed test appears to be more robust and powerful than the B-S test, especially when the underlying distributions are symmetric or moderately skewed. Therefore, our test seems applicable in various situations where the differences in variability measures are the parameters of interest to researchers.

5. Real-Life Application

5.1. The IAT

To demonstrate an application of the proposed test, we investigate data from a study by Lemm (2006) on examining relationships between implicit and explicit ATGM using IAT. Implicit associations are representative of automatic cognitive processes taking place outside of one’s awareness. As such, they must be measured indirectly, as opposed to explicit attitudes that are frequently measured through tools like self-report questionnaires. The IAT has become a standard in measuring implicit associations through RTs since its initial introduction by Greenwald, McGhee, and Schwartz (1998). It has been heavily employed in the field of social psychology to understand attitudes and biases.

To measure implicit associations, the IAT requires subjects to categorize sets of stimuli into one of four categories. These stimuli represent dichotomies from two dimensions. One dimension is the target concept, which highlights the phenomenon under investigation. For example, studies examining smoking behavior use stimuli representing exercise and smoking for the target concept (Perugini, 2005), while studies examining racial bias may use stimuli representing people who are Black or White (Wittenbrink, Judd, & Park, 2001). The second dimension represents the evaluative attribute. The dichotomy of pleasant/unpleasant is commonly used for the attribute domain.

The IAT compares RTs between trials with different pairings of these dimensions, where the two pairings are referred to as critical blocks. For example, in Perugini (2005), one critical block paired responses for “smoking” and “pleasant” to the same response key, while “exercise” and “unpleasant” were paired to another key. The second critical block swaps this pairing. Subjects who associate a given pairing more strongly (i.e., a congruent pairing) tend to make faster and more accurate categorizations of stimuli compared to incongruent pairings. Larger differences in critical block RTs indicate a greater difference in congruence between these pairings, resulting in a larger IAT score, whose calculation will be explained later. Interpretation of the IAT scores typically indicates the preference or bias of a subject by noting which target concept has lower RTs when paired with the pleasant attribute.

5.2. Lemm’s RT Data on ATGM

The study by Lemm (2006), which we reexamine in this section, analyzes RT data taken from $n_{1} = 40$ male and $n_{2} = 43$ female heterosexual undergraduate students at Western Washington University, Bellingham, Washington. One practice IAT, one word-based ATGM IAT, and one picture-based ATGM IAT were administered to each student. Each subject responded to 240 items (20 items for each practice IAT critical block, 40 items for each word ATGM IAT critical block, and 60 items for each picture ATGM IAT critical block).

5.3. The IAT Score Calculation

To better understand the traditional IAT score calculation of Greenwald et al. (1998) used in Lemm (2006), let $r_{g, i, j, k, b}$ be RTs recorded in milliseconds, where $g = 1, 2$ , represent a male or female subject, respectively, and $i = 1, . . ., n_{g}$ , is the subject number within each group. Let $j = 1, 2$ denote the word and picture version of the IAT test, respectively. Within each critical block, let the order of appearance of the stimulus be $k = 1, . . ., n_{j}$ . Moreover, the pairing assigned to a block is denoted by $b = 1, 2$ , for gay/pleasant (straight/unpleasant) or straight/pleasant (gay/unpleasant) pairings, respectively. Then, after setting $w_{g, i, j, k, b} = r_{g, i, j, k, b}$ if $r_{g, i, j, k, b} \in [300, 3000]$ , $w_{g, i, j, k, b} = 300$ if $r_{g, i, j, k, b} < 300$ , and $w_{g, i, j, k, b} = 3,000$ if $r_{g, i, j, k, b} > 3,000$ , the difference in log mean corrected RTs between critical blocks, $d_{g, i} = \sum_{j = 1}^{2} \sum_{k = 1}^{n_{j}} [log (w_{g, i, j, k, 1}) - log (w_{g, i, j, k, 2})] / \sum_{j = 1}^{2} n_{j}$ , was used as the IAT score for each subject.

As a remark, Greenwald, Nosek, and Banaji (2003) suggested steps for calculating an improved IAT score, commonly referred to as the D score. The D score is analogous to the popular Cohen’s d effect size measure, which attempts to generate more reliable results by standardizing the mean difference. Although the interested reader is referred to the second and third columns of Table 4 in their paper for the details, notable differences from the traditional calculation include removing RTs below 400 milliseconds and measuring the standardized mean difference between the two types of parings without log transformation.

Among the findings based on the traditional IAT score calculation, Lemm (2006) noted that internal motivation to respond without prejudice (Plant & Devine, 1998) and having positive contact with gay men had the most significant relationship to lower IAT scores, indicating less prejudice toward gay men. Although no significant difference in the mean IAT scores between men and women was observed, Lemm (2006) interpreted the result with caution because of the possible influence of having a sample of students from the liberal Pacific Northwest. Nevertheless, it is possible that other gender differences in IAT scores may exist, which may be better described by a more suitable IAT score calculation method. Specifically, we consider taking the possible multiplicative nature of RTs into account.

5.4. The IAT Score Calculation for the Multiplicative RT Pattern

The D score is well established within the research community as a standard IAT score calculation method. In particular, if each subject’s RTs follow an additive pattern (i.e., RTs from one critical block tend to differ from the other by an added constant) and additive error, the D score, which compares the difference between block types, is well justified mathematically. However, this approach may be less appropriate for nonlinear RT patterns. Also, note that the study conducted by Greenwald et al. (2003) to show the superiority of the D score is based on a very large sample size ( $\sim 10,000$ ). In contrast, for studies with small to medium sample sizes, using sample mean as a central tendency of each critical block may be misleading as it is easily affected by a small number of extreme RT values.

To address the assumption regarding the RT response pattern, we may consider the multiplicative RT pattern mentioned in Salthouse and Hedden (2002) instead. Under the multiplicative RT pattern, latencies between treatments differ by a positive constant multiple rather than an additive constant. If we consider the multiple IAT tasks and responses to stimuli from the target concept and evaluative attribute as distinct, then the data set from Lemm’s (2006) study may also reveal a multiplicative response pattern. Indeed, RTs of subjects revealed some features suggestive of this pattern (see Figure 5). That is, delayed responses are typically amplified by some constant factor for incongruent pairings compared to the congruent ones. In addition, for the data showing a multiplicative pattern, it is natural to assume a multiplicative error (cf. Noguchi, Aue, & Burman, 2016). Because the raw data on the left-hand side are contaminated with noise, they are smoothed using the moving average filter and displayed on the right-hand side. To accommodate a multiplicative RT pattern with a multiplicative error, calculation of the IAT scores using the ratio of RTs between blocks instead of taking their differences may be more desirable.

Figure 5.

Subjects suggesting a multiplicative RT pattern. Gray: Incongruent pairings. Black: Congruent pairings.

To construct an IAT score calculation method for the multiplicative RT pattern, as before, let $r_{g, i, j, k, b}$ be the RTs recorded in milliseconds, $g = 1, 2$ , $i = 1, . . ., n_{g}$ , $j = 1, 2$ , $k = 1, . . ., n_{j}$ , $b = 1, 2$ . Due to the delay of the nervous system propagating visual stimuli to the brain for processing, a lower threshold for visual RT is removed to provide a more accurate ratio of RTs between blocks. Using a minimum visual RT of 200 milliseconds suggested in McMorris (2005), we let $r_{g, i, j, k, b}^{⋆} = max \{0, r_{g, i, j, k, b} - 200\}$ . As described in Section 4, it is typical for the distribution of RTs to be positively skewed, as is true for Lemm’s (2006) data set. Moreover, each subject’s RTs typically varied widely and included a number of large outliers. In calculating the IAT scores, very small RTs are also at risk of distorting an accurate picture of a subject’s RT because they usually represent premature responses. Thus, the sample median may serve as a better measure of central tendency than the sample mean. Therefore, we let the representative IAT score for the $i th$ individual of the $g th$ gender under the multiplicative RT pattern be given by $y_{g, i} : = log [{median}_{j, k} (r_{g, i, j, k, 1}^{⋆} / r_{g, i, j, k, 2}^{⋆})] = {median}_{j, k} [log, (r_{g, i, j, k, 1}^{⋆} / r_{g, i, j, k, 2}^{⋆})]$ . The natural log of the ratio is taken to eliminate a possible artificial effect due to the choice of the critical block in the numerator or denominator of the ratio. The novel multiplicative IAT score calculation, which we refer to as the log ratio analysis method, gives two groups of IAT scores to be compared, namely, $y_{1} = (y_{1, 1}, . . ., y_{1, n_{1}})$ for the male group and $y_{2} = (y_{2, 1}, . . ., y_{2, n_{2}})$ for the female group.

To answer the question of whether the log ratio analysis method is reliable compared to the D score method, we have conducted a simulation study assuming a multiplicative error structure without any presence of an effect (for details, see the Supplemental Material available in the online version of this article). The results suggest that the log ratio analysis method appears to be more reliable, ensuring that the simulated effect sizes are kept reasonably close to zero when the effect is absent. On the other hand, the results from the D score method might not be reliable as there seems to be a high probability of producing unacceptably high effect sizes even when the effect is absent.

5.5. Mean and Variability Comparisons Using the IAT Scores

As mentioned before, the goal of this analysis is to investigate a possible gender difference in the variability of the IAT scores. Let $x_{g, i, 1}$ and $x_{g, i, 2} : = y_{g, i}$ be the IAT scores for the $i th$ individual of the $g th$ gender calculated using the D score and the log ratio analysis, respectively. Moreover, let ${\tilde{μ}}_{g, ℓ}$ be the population median of $x_{g, i, ℓ}$ , $i = 1, . . ., n_{g}$ . Then, we test $H_{0} : τ_{1, ℓ} / τ_{2, ℓ} = 1$ , where $τ_{g, ℓ} = E [| x_{g, i, ℓ} - {\tilde{μ}}_{g, ℓ} |]$ , against its two-sided alternative hypothesis. To do so, we have applied our proposed test (Equation 3) and the B-S test (Equation 8). In addition, we have tested equality of their means using the Welch two-sample t test.

Table 3 summarizes the results of mean and variability comparisons using the two types of the IAT score calculations. Consistent with Lemm’s (2006) original findings, no significant difference in the mean IAT scores is observed between men and women. Of primary interest, however, is the possibly significant difference found in IAT variability between men and women. All three tests indicate some evidence for the difference in the IAT score variability between men and women using D scores. However, as we discussed previously, for the data having a number of extreme observations and showing a possibly multiplicative RT pattern, it may make more sense to use the log ratio analysis method. Based on the log ratio analysis method, the statistical significance has reduced compared to the D score method, implying that the statistically significant result for the D score method may be affected by extreme observations. Nevertheless, the p values of .092 and .098 for our test and the B-S test, respectively, may offer some evidence for a possible gender difference explained by the multiplicative effects.

In addition, we have calculated Cohen’s d effect size of the sample means (for the mean comparisons) and sample mean absolute deviations from median (for the variability comparisons). Using the common effect size interpretation of .2 (small), .5 (medium), and .8 (large) for Cohen’s d, we note that the effect is small for the mean comparisons and small to medium for the variability comparisons by looking at the log ratio analysis (see Table 3). That lends support to our finding that the difference in the IAT score variability may be more significant than the difference in the mean IAT scores.

Table 3.

Summary of the p Values and Effect Sizes Comparing IAT Scores for Men and Women

Test	RT Treatment
	Difference	Log Ratio
	(D score)	(Our Approach)
Mean comparisons
t-test	.405	.194
Cohen’s d	−.190	.285
Variability comparisons
Bonnet–Seier	.026	.098
Proposed	.030	.092
Cohen’s d	.252	.378

5.6. Interpretation of the Results

The results about the possible differences in variability may be interpreted in terms of the contact hypothesis of Allport (1954) and the gender difference in the network of supportive or close friends. The contact hypothesis essentially states that interaction with stigmatized group members can lead to reduced prejudice and stereotyping among dominant group members, under certain circumstances (Lemm, 2006). Also, studies have shown that female heterosexuals are more likely than male heterosexuals to know a gay person (Herek & Capitanio, 1996) and that late adolescent girls have both a significantly greater number of supportive or close friends than boys (Colarossi, 2001; Feiring, 1999). The studies by Colarossi (2001) and Feiring (1999) also showed an increase in variability in the number of supportive or close friends for late adolescent girls than boys, suggesting a positive mean–variance relationship in the close contact variable. Thus, we postulate that females are more likely to have a greater variability in having close relationships with gay people than males, possibly contributing to the greater variability in the implicit ATGM.

However, it should also be examined that the results in Table 3 showed no statistically significant difference regarding the mean implicit ATGM between men and women, similar to the previous research on the explicit ATGM by Herek and Capitanio (1996) based on a national sample. For now, let us assume that both males and females with the same level of contact with gay people have the same mean level of implicit ATGM. Then, assuming a positive mean–variance relationship in the close contact variable with gay people, the implicit ATGM should be lower for the female group than the male group given the contact hypothesis, on the contrary to what the results suggest.

That is, it seems more natural to postulate that males and females may have different mean levels of implicit ATGM, given that they have the same level of contact with gay people. Further investigations on the relationship between the level of contact with gay people and implicit ATGM may shed light on a deeper understanding of the gender difference. Additional considerations of the results can be found in the online Supplemental Material.

6. Conclusion

We have discussed a new test for comparing the variability measures in the independent two-sample case. Specifically, the test is made robust and powerful by considering the modified structural zero removal method with a new correction factor on the Brown–Forsythe transformation. Moreover, noting that the Brown–Forsythe-transformed variables are positive, the test statistic utilizes the ratio of their means by devising the delta method with the log transformation. To ensure that the test performs reliably even when the sample sizes are small, odd, or unequal, a modification of the Welch–Satterthwaite type small-sample approximation is presented, properly accounting for the log transformation in the test statistic. The comprehensive simulation results covering many RT data scenarios suggest that the proposed test seems to be more robust and powerful under various sample size and distribution configurations.

Another contribution we have made in this article is the log ratio analysis method, which could be highly useful for the IAT score calculations when the data show a multiplicative treatment effect and a multiplicative error with a number of extreme observations. Through a careful reexamination of the data set analyzed by Lemm (2006), we have proposed that such RT data may exist, and more reliable IAT scores may be calculated by considering the median of the log ratio of the RTs in the two different blocks for each individual. The statistical analysis using the proposed test indicates some possibility of the difference in the IAT score variability between men and women. Based on that, we postulated that this finding may be related to the difference in mean implicit ATGM between men and women conditional on the same level of contact with gay people. Further investigations on the relationship between the level of contact with gay people and implicit ATGM are worthwhile as they may contribute to a deeper understanding of the gender difference. All the suggested procedures in this article and data are implemented in an R package “intervcomp,” and examples are provided in the online Supplemental Material.

Although the main purpose of the article was to develop a robust test for comparing variability measures of two independent samples, it is possible to extend the approach to the multisample cases. Specifically, noting that researchers often have a set of specific comparisons they wish to make, a modification of the multiple contrast test suggested in Hothorn, Libiger, and Gerhard (2012) can be considered for future work.

Supplemental Material

Supplementary_Material - A Robust Test for Checking the Homogeneity of Variability Measures and Its Application to the Analysis of Implicit Attitudes

Supplementary_Material for A Robust Test for Checking the Homogeneity of Variability Measures and Its Application to the Analysis of Implicit Attitudes by Ryan C. Erps and Kimihiro Noguchi in Journal of Educational and Behavioral Statistics

Footnotes

Acknowledgments

The authors are grateful to two anonymous reviewers and the Editor for their helpful comments that have led to substantial improvement in the quality of the article. The authors are also grateful to Kristi Lemm for providing the RT data and to Fernando Marmolejo-Ramos for helpful comments.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Kimihiro Noguchi

References

Allport

G. W.

(1954). The nature of prejudice. Cambridge, MA: Addison-Wesley.

Bonett

D. G.

Seier

(2003). Confidence intervals for mean absolute deviations. The American Statistician, 57, 233–236.

Box

G. E. P.

(1953). Non-normality and tests on variances. Biometrika, 40, 318–335.

Brown

M. B.

Forsythe

A. B.

(1974). Robust tests for the equality of variances. Journal of the American Statistical Association, 69, 364–367.

Carroll

R. J.

Schneider

(1985). A note on Levene’s tests for equality of variances. Statistics & Probability Letters, 3, 191–194.

Colarossi

L. G.

(2001). Adolescent gender differences in social support: Structure, function, and provider type. Social Work Research, 25, 233–241.

Conover

W. J.

Johnson

M. E.

Johnson

M. M.

(1981). A comparative study of tests for homogeneity of variances, with applications to the outer continental shelf bidding data. Technometrics, 23, 351–361.

Feiring

(1999). Other-sex friendship networks and the development of romantic relationships in adolescence. Journal of Youth and Adolescence, 28, 495–512.

Gastwirth

J. L.

(1982). Statistical properties of a measure of tax assessment uniformity. Journal of Statistical Planning and Inference, 6, 1–12.

10.

Gastwirth

J. L.

Gel

Y. R.

Miao

(2009). The impact of Levene’s test of equality of variances on statistical theory and practice. Statistical Science, 24, 343–360.

11.

Greenwald

A. G.

McGhee

D. E.

Schwartz

J. L. K.

(1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464–1480.

12.

Greenwald

A. G.

Nosek

B. A.

Banaji

M. R.

(2003). Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197–216.

13.

Hall

Padmanabhan

A. R.

(1997). Adaptive inference for the two-sample scale problem. Technometrics, 39, 412–422.

14.

Herek

G. M.

Capitanio

J. P.

(1996). “Some of my best friends”: Intergroup contact, concealable stigma, and heterosexuals’ attitudes toward gay men and lesbians. Personality and Social Psychology Bulletin, 22, 412–424.

15.

Hines

W. G. S.

O’Hara Hines

R. J.

(2000). Increased power with modified forms of the Levene (med) test for heterogeneity of variance. Biometrics, 56, 451–454.

16.

Hothorn

L. A

Libiger

Gerhard

(2012). Model-specific tests on variance heterogeneity for detection of potentially interacting genetic loci. BMC Genetics, 13, 1–6.

17.

Keyes

T. K.

Levy

M. S.

(1997). Analysis of Levene’s test under design imbalance. Journal of Educational and Behavioral Statistics, 22, 227–236.

18.

Lehmann

E. L

. (2011). Fisher, Neyman, and the creation of classical statistics. New York: Springer Science & Business Media.

19.

Lemm

K. M.

(2006). Positive associations among interpersonal contact, motivation, and implicit and explicit attitudes toward gay men. Journal of Homosexuality, 51, 79–99.

20.

Levene

(1960). Robust tests for equality of variances. In Olkin

(Ed.), Contributions to probability and statistics (pp. 278–292). Stanford, CA: Stanford University Press.

21.

Lim

T.-S.

Loh

W.-Y.

(1996). A comparison of tests of equality of variances. Computational Statistics & Data Analysis, 22, 287–301.

22.

Loh

W.-Y.

(1987). Some modifications of Levene’s test of variance homogeneity. Journal of Statistical Computation and Simulation, 28, 213–226.

23.

Marozzi

(2011). Levene type tests for the ratio of two scales. Journal of Statistical Computation and Simulation, 81, 815–826.

24.

Martin

C. G.

(1976). Comment on Levy’s “an empirical comparison of the Z-variance and Box-Scheffé tests for homogeneity of variance.” Psychometrika, 41, 551–556.

25.

McMorris

(2005). Acquisition and performance of sports skills. Hoboken, NJ: John Wiley.

26.

Neuhäuser

(2007). A comparative study of nonparametric two-sample tests after Levene’s transformation. Journal of Statistical Computation and Simulation, 77, 517–526.

27.

Noguchi

Aue

Burman

(2016). Exploratory analysis and modeling of stock returns. Journal of Computational and Graphical Statistics, 25, 363–381.

28.

Noguchi

Gel

Y. R.

(2010). Combination of Levene-type tests and a finite-intersection method for testing equality of variances against ordered alternatives. Journal of Nonparametric Statistics, 22, 897–913.

29.

Noguchi

Marmolejo-Ramos

(2016). Assessing equality of means using the overlap of range-preserving confidence intervals. The American Statistician, 70, 325–334.

30.

O’Brien

R. G.

(1978). Robust techniques for testing heterogeneity of variance effects in factorial designs. Psychometrika, 43, 327–342.

31.

O’Neill

M. E.

Mathews

(2000). A weighted least squares approach to Levene’s test of homogeneity of variance. Australian & New Zealand Journal of Statistics, 42, 81–100.

32.

Pan

(1999). On a Levene type test for equality of two variances. Journal of Statistical Computation and Simulation, 63, 59–71.

33.

Pan

(2002). Confidence intervals for comparing two scale parameters based on Levene statistics. Journal of Nonparametric Statistics, 14, 459–476.

34.

Perugini

(2005). Predictive models of implicit and explicit attitudes. British Journal of Social Psychology, 44, 29–45.

35.

Peters

C. A. F.

(1856). Über die Bestimmung des wahrscheinlichen Fehlers einer Beobachtung aus den Abweichungen der Beobachtungen von ihrem arithmetischen Mittel [On the determination of the probable error of observation from the deviations of the observations from their arithmetic mean]. Astronomische Nachrichen, 44, 30–31.

36.

Peters

K. R.

Ray

L. B.

Fogel

Smith

C. T.

(2014). Age differences in the variability and distribution of sleep spindle and rapid eye movement densities. PLoS One, 9, e91047.

37.

Plant

E. A.

Devine

P. G.

(1998). Internal and external motivation to respond without prejudice. Journal of Personality and Social Psychology, 75, 811–832.

38.

Salthouse

T. A.

Hedden

(2002). Interpreting reaction time measures in between-group comparisons. Journal of Clinical and Experimental Neuropsychology, 24, 858–872.

39.

Shoemaker

L. H.

(2003). Fixing the F test for equal variances. The American Statistician, 57, 105–114.

40.

Smyrnis

Karantinos

Malogiannis

Theleritis

Mantas

Stefanis

N. C.

… Evdokimidis

(2009). Larger variability of saccadic reaction times in schizophrenia patients. Psychiatry Research, 168, 129–136.

41.

Tamm

Narad

M. E.

Antonini

T. N.

O’Brien

K. M.

Hawk

L. W.

Epstein

J. N.

(2012). Reaction time variability in ADHD: A review. Neurotherapeutics, 9, 500–508.

42.

Theleritis

Evdokimidis

Smyrnis

(2014). Variability in the decision process leading to saccades: A specific marker for schizophrenia? Psychophysiology, 51, 327–336.

43.

Wittenbrink

Judd

C. M.

Park

(2001). Spontaneous prejudice in context: Variability in automatically activated attitudes. Journal of Personality and Social Psychology, 81, 815–827.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.66 MB