MIMIC Models for Uniform and Nonuniform DIF as Moderated Mediation Models

Abstract

In this article, the authors describe how multiple indicators multiple cause (MIMIC) models for studying uniform and nonuniform differential item functioning (DIF) can be conceptualized as mediation and moderated mediation models. Conceptualizing DIF within the context of a moderated mediation model helps to understand DIF as the effect of some variable on measurements that is not accounted for by the latent variable of interest. In addition, useful concepts and ideas from the mediation and moderation literature can be applied to DIF analysis: (a) improving the understanding of uniform and nonuniform DIF as direct effects and interactions, (b) understanding the implication of indirect effects in DIF analysis, (c) clarifying the interpretation of the “uniform DIF parameter” in the presence of nonuniform DIF, and (d) probing interactions and using the concept of “conditional effects” to better understand the patterns of DIF across the range of the latent variable.

Keywords

differential item functioning item response theory MIMIC models mediation moderated mediation

Introduction

One of the primary aims of measurement research is to develop and identify valid sets of items that measure a specific latent variable. Much research in measurement, particularly within item response theory (IRT), focuses on differential item functioning (DIF) as it can be a major threat to the validity of a scale or set of items. DIF occurs when the probability of a response on a specific item is dependent on some external factor even after conditioning on the latent trait. For instance, it is crucial in educational assessment that a mathematics test is equally valid for all students taking the test. This means that controlling for a student’s math ability, there should be no other factors, which increase or decrease the probability of getting an item correct.

Zumbo (2007) deemed the “Third Generation” of DIF research to be an era where the authors investigate and aim to understand why DIF is occurring rather than just detect or correcting for it. One of the primary aims of this article is to provide a more intuitive understanding of uniform and nonuniform DIF using a multiple indicators multiple cause (MIMIC) model and the concepts of indirect effects, direct effects, and interactions commonly used in mediation and moderation. The authors believe that this type of framework may help researchers to understand DIF in a way that would facilitate thinking about why or how DIF occurs, by focusing on DIF as a process rather than a nuisance.

Uniform and Nonuniform DIF

DIF can occur in two primary ways: uniform and nonuniform DIF (Mellenbergh, 1982). Figure 1 gives four examples of DIF. Early definitions of uniform and nonuniform DIF emphasized the mutually exclusive nature of these two types of DIF (Ackerman, 1992; Mellenbergh, 1982; Millsap & Everson, 1993; Narayanan & Swaminathan, 1996). For example, Ackerman (1992) defined uniform DIF as when the “ICCs for the different groups differ by only a horizontal translation (i.e., they are parallel but not coincident).” Alternatively, nonuniform DIF is when the ICCs are nonparallel. Using these definitions of uniform and nonuniform DIF, Panel a in Figure 1 is the only item, which would fit the definition of uniform DIF, and all other panels would have nonuniform DIF, because the ICCs are not parallel. The authors rely on definitions of uniform and nonuniform DIF such that they are mutually exclusive. In addition, the authors describe how to interpret the parameter which in models without nonuniform DIF is used to describe uniform DIF, in the presence of nonuniform DIF. This parameter is what differentiates Panels b, c, and d in Figure 1.

Figure 1.

Item characteristic curves when (a) $γ_{i}^{β} = 1.5$ and $γ_{i}^{α} = 0$ , (b) $γ_{i}^{β} = 0$ and $γ_{i}^{α} = 1$ , (c) $γ_{i}^{β} = 1.5$ and $γ_{i}^{α} = 1$ , and (d) $γ_{i}^{β} = 4.5$ and $γ_{i}^{α} = 1$ .

MIMIC Models for DIF Testing

A number of methods have been developed for detecting and investigating DIF including Mantel–Haenszel tests (Holland & Thayer, 1988), multidimensional approaches (e.g., SIBTESTs; Shealy & Stout, 1993), area methods (Raju, 1988, 1990), logistic regression (Swaminathan & Rogers, 1990), multiple group IRT and structural equation models (Jöreskog, 1971), and MIMIC models (B. O. Muthèn, 1985, 1989; B. O. Muthèn, Kao, & Burstein, 1991). For reviews of these methods, see Magis, Bèland, Tuerlinckx, and De Boeck (2010) and Zumbo (2007). In this article, the focus will be on the method of using MIMIC models for studying DIF. MIMIC models are characterized by the inclusion of independent variables (or causes, $X$ ) of a latent variable and its observed indicators ( $Y$ ; Hauser & Goldberger, 1971; Joreskog & Goldberger, 1975). Figure 2 displays an example MIMIC model with a case of three observed indicators and three observed causes.

Figure 2.

An example of MIMIC model with three observed indicators ( $Y_{1}$ , $Y_{2}$ , $Y_{3}$ ) and three observed causes ( $X_{1}$ , $X_{2}$ , $X_{3}$ ) for the latent variable $θ_{t}$ (with $ξ_{t}$ being a residual).

MIMIC models were first used to study DIF by B. O. Muthèn (1985), who showed that a typical two-parameter IRT model assumes no direct path between an independent variable (cause) and an individual item response (indicator). A path between an independent variable and an item can be included in a MIMIC model. If this direct path is found to be statistically significant, this suggests that for two people matched on the latent variable, the item is easier for one of those people. This is an indication of uniform DIF.

The accuracy of MIMIC models for detecting DIF has been investigated by Woods (2009) and Woods and Grimm (2011) and others. Researchers have also compared MIMIC methods to other DIF detection methods such as the Mantel–Haenszel test, SIBTEST, and likelihood-ratio tests based on multiple group IRT models (Finch, 2005; Woods, 2009; Woods and Grimm, 2011). Simulation studies have found that the MIMIC method works similarly to other tests, but requires smaller sample sizes than other methods to detect uniform DIF (Woods, 2009).

The MIMIC approach is not immune to many of the issues common to most DIF testing techniques. Scale purification is a procedure for detecting which items among a set of items have DIF (Lord, 1980). This process is incredibly important, as in scale development researchers often have a set of items and they want to detect which items within the set have DIF, rather than focusing on a single item. Wang, Shih, and Yang (2009) studied the performance of an iterative purification method with MIMIC models and found this method worked well for identifying items with DIF. However, like many other scale purification methods, when there are too many items with DIF, the scale purification methods have Type I error rates that are too high to be acceptable (Shih & Wang, 2009; Wang et al., 2009). Shih and Wang (2009) proposed a method for identifying a short anchor (a small set of DIF free items). This method seems to perform better than iterative purification methods when the proportion of items with DIF is large. Indeed only one DIF free item was needed to control Type I error; however, longer anchors increased power to detect DIF. In addition, Shih and Wang (2009) showed how a slight alteration of the iterative methods proposed by Wang et al. (2009) could be used to identify anchor items (rather than identifying DIF items). In comparison with other DIF detection methods, MIMIC methods are less susceptible to DIF in anchor items (Finch, 2005).

Both MIMIC models and multidimentional IRT models can be used to relax some of the more stringent assumptions of traditional IRT models, such as unidimensionality (Lee, Bulut, & Suh, 2016; Wang et al., 2009; Zumbo, 2007). In a MIMIC model, by allowing individual items to load on multiple latent variables, multidimensionality is allowed, while testing for DIF. There are many instances where researchers attribute DIF to multidimensionality. Cheng, Shao, and Lathrop (2016) proposed a method for understanding DIF using multidimensional MIMIC models. If there are indicators of a latent construct, which the researchers believe to be causing DIF, a MIMIC model can be estimated including that latent construct and testing whether there is DIF above and beyond this additional latent construct. Alternatively, if there is no remaining DIF this would support the claim that the additional latent construct explains all (or most of) the DIF (Cheng et al., 2016). The Mantel–Haenszel test and SIBTEST are not appropriate for multidimensional modeling because they are designed for only one latent trait (Bulut & Suh, 2017); however, a multidimensional SIBTEST has been developed (Stout, Li, Nandakumar, & Bolt, 1997).

MIMIC models are can be used for continuous, categorical, or mixes of continuous and categorical outcomes. Throughout this article, the authors provide examples and equations for dichotomous outcomes only. Because MIMIC models are estimated in a structural equation modeling framework, model fit indices are available for these models. However, many model fit statistics are only valid for models with continuous outcomes (e.g., root mean square error approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), Tucker–Lewis index (TLI); Yun, 2002). Other measures such as information criteria can be used to compare models with dichotomous outcomes (Kang & Cohen, 2007).

Aims and Structure

In DIF studies, researchers are often concerned about what characteristics of an individual (e.g., gender, race, socioeconomic status) might lead to uniform or nonuniform DIF. However, applied researchers seem to pay little attention to models of how independent variables affect measurement and do not include DIF as part of their model. By viewing MIMIC models as mediation models, it becomes clear that there are multiple ways an independent variable can affect an item response, not all of which undesirable.

The purpose of this article is to show that MIMIC models for uniform and nonuniform DIF analysis can be conceptualized as mediation and moderated mediation models. Mediation and moderation models provide a framework for understanding the process through which some effect occurs and for modeling contingencies in those processes. An advantage of understanding DIF in the context of mediation and moderation analysis is that it provides us with an opportunity to investigate the mechanisms through which independent variables influence measurements. What the authors find is that DIF is one of these processes. In addition, conceptualizing within a mediation and moderation framework allows us to apply useful concepts and ideas from the mediation and moderation literature to improve the understanding of DIF. This includes (a) appreciating uniform and nonuniform DIF as direct effects and interactions, (b) understanding the implication of indirect effects in DIF analysis, (c) revising a conventional interpretation of the uniform DIF parameter in the presence of nonuniform DIF, and (d) using the concept of probing interactions to better understand the patterns of DIF.

The remainder of this article is structured as follows: In “MIMIC DIF Models as Mediation and Moderated Mediation Models”, it is shown that how MIMIC models for uniform and nonuniform DIF analysis can be understood within a mediation and moderation framework. In the subsequent section (“Applying Ideas of Moderation and Mediation to DIF”), how some important concepts and ideas from the mediation and moderation literature can be applied to improve understanding of DIF will be discussed in detail. Throughout “MIMIC DIF Models as Mediation and Moderated Mediation Models” and “Applying Ideas of Moderation and Mediation to DIF” sections, the analysis of a data set to show how MIMIC DIF analysis can be done and to provide a concrete example for the advantages of using mediation and moderation ideas to interpret DIF are discussed. The article will then end in “Discussion” section with some concluding remarks.

MIMIC DIF Models as Mediation and Moderated Mediation Models

To illustrate how MIMIC DIF models can be understood within a mediation and moderation framework, the authors use a data set, which explored cohort differences in intelligence testing on samples of children, aged 12 years to 14 years, from Estonia (Must & Must, 2014).¹ The first cohort was collected during 1933 to 1936 ( $n = 890$ ) and the second cohort was collected in 2006 ( $n = 913$ ). The study focused on differences in intelligence across cohorts, as measured by 10 subtests, which cross a variety of domains. For simplicity, only one subtest will be examined: arithmetic, with 16 items. Each student responded with an open response, these responses were given a binary coding: 1 (correct) or 0 (incorrect). The 16 items were administered in Estonian (Haggerty, Terman, Thorndike, Whipple, & Yerkes, 1921). Approximate English translations for each item are in Table 1 along with proportions of correct answers for each cohort.

Table 1.

Estonian National Intelligence Test: Arithmetic.

Item	Item text English translation^a	Proportion	Correct
Item	Item text English translation^a	1933/1936	2006
1	How much is half of 8 minutes?	0.9273	0.9651
2	How many dimes in 6 nickels?	0.7700	0.8651
3	Mari buys an apple for 4 cents and a cake for 5 cents. How many cents does he have to pay?	0.9400	0.9759
4	Vilu had eight chickpeas and sold three. How many are left?	0.9831	0.9857
5	How much longer is 12 yards than a meter?	0.7526	0.6625
6	How many chairs are in nine rooms if there are 40 chairs in each room?	0.8467	0.8698
7	A gift costs 96 cents between four girls. How much should each girl pay?	0.8795	0.8929
8	How much do 12 sweets cost if three sweets costs 1 cent?	0.5393	0.5886
9	How many square centimeters is a card with a length of 5 cm and a width of 3 cm?	0.9510	0.8482
10	A man bought a plot of land for 100 kroons and sold it for 120 kroons. He profited 5 kroons per acre. How many acres was the plot of land?	0.5258	0.5051
11	How many times does $1 \frac{1}{2}$ need to be added to 6 to get 15?	0.5161	0.4981
12	Half a kilo of seeds costs 8 kroons. How many seeds can you buy with 50 kroons?	0.1787	0.0721
13	It is necessary to carry 56 kilograms of equipment to the camp. A, B, and C distribute the equipment among themselves so that three parts are to A, two parts B, and two parts C. How many kilos did A need to wear?	0.4491	0.4194
14	How many bulls does a hunter have to shoot to hit the mark 100 times when it hits on 40% of the shots?	0.1859	0.2425
15	How many times heavier is $\frac{1}{2}$ of a load weighing one and a half tons than a half-ton load?	0.1140	0.1266
16	The pocket watch was set correctly at 12 noon on Wednesday. At 6 o’clock the next night, it was 15 seconds ahead. How much does the clock go for half an hour?	0.0426	0.0692

Original items are from Haggerty, Terman, Thorndike, Whipple, and Yerkes (1921) Scale A, Form 2, Edition 2. Items were translated to English for ease of understanding in this article. Items were administered in Estonian.

For 10 of the 16 items, the 2006 cohort performed better than the 1933/1936 cohort. The exceptions where the 1933/1936 cohort outperformed the 2006 cohort include Items 5, 9, 10, 11, 12, and 13. These data, here, are used to explore cohort differences in latent arithmetic ability as well as the potential for DIF. There are a variety of counterexplanations for why the later cohort outperformed the earlier cohort, and these are discussed in Must and Must (2013). These analyses are not to be taken as novel theoretical findings, but rather an example used for showing how to conduct DIF analysis. Two items were selected to demonstrate how to test for DIF with MIMIC models: Items 5 and 10. These items were selected because they show interesting patterns of DIF, as will be seen in later sections. The focus of this article is on the estimation of DIF within a single item; however, when these analyses are done in empirical data, the authors recommend researchers should use the methods proposed by Shih and Wang (2009) or Wang et al. (2009) as described in “MIMIC Models for DIF Testing” section. Alternatively, researchers can use substantive knowledge to inform which items to explore for DIF. The authors did not conduct any scale purification for these analyses. Data analysis was done using Mplus Version 8.1 (L. K. Muthèn & Muthèn, 1998-2011). For this analysis, maximum likelihood estimation with robust standard error estimates was used. The model was unidimensional, assuming that all items loaded onto a single latent variable, which the authors call arithmetic ability. Select input files are included in the appendices to aid implementation.

Mediation Model for Uniform DIF

In the example the authors are concerned with whether there are cohort differences on the probability of correctly answering an item. One reason that there may be differences in the probability of responding is differences in “true” latent ability. However, there may be concern that for a specific item there are cohort differences in the probability of responding that are not attributable to “true” differences in latent ability. For this reason, the authors add a direct path between the cohort variable and the item response, as represented in Figure 3a.

Figure 3.

(a) A MIMIC model for uniform DIF as a mediation model and (b) a MIMIC model for nonuniform DIF as a moderated mediation model.

For notation, the authors use $Y_{it}$ to denote a binary response to item $i$ ( $i = 1, . . ., I$ ) for person $t$ ( $t = 1, . . ., N$ ), $X_{t}$ to indicate the independent variable that indicates 1933/1936 cohort (coded as 0) and 2006 cohort (coded as 1), and $θ_{t}$ to represent a latent trait (“true” arithemtic ability) that need to be measured using the $I$ items (in the example $I = 16$ ). This MIMIC model with a direct path can be viewed as a mediation model that allows the effect of $X_{t}$ on $P (Y_{it})$ ² to be mediated by $θ_{t}$ . Two regression models will be specified that include the three paths in Figure 3a (Paths a, b, and c).

First, the model for Path a is specified with the latent trait $θ_{t}$ as the dependent variable.

θ_{t} = δ X_{t} + ξ_{t},

(1)

where the regression coefficient $δ$ represents the influence of the variable $X_{t}$ on the $θ_{t}$ and it is assumed that the residual $ξ_{t}$ follows a normal distribution with $ξ_{t} ~ N (0, σ^{2})$ . In the arithmetic example, the $δ$ parameter indicates the mean difference between the 1933/1936 cohort and the 2006 cohort on “true” arithmetic ability ( $θ_{t}$ ), which is called impact in the measurement literature (Ackerman, 1992; Camilli, 1993).

Estimating Equation 1 with the arithmetic data and allowing Item 5 to have uniform DIF provides an estimate of $δ$ , 0.149. This means that the 2006 cohort is on average 0.149 units higher than the 1933/1936 cohort on latent arithmetic ability ( $p = . 051$ ). This effect can be interpreted in standard deviation units, as the latent variable is standardized with a variance of 1. The effect is relatively small and only nears statistical significance when using a relatively generous level of 0.05.

Second, a model for Paths b and c,

P (Y_{it} | θ_{t}, X_{t}) = g^{- 1} (β_{i} + γ_{i}^{β} X_{t} + α_{i} θ_{t}),

(2)

where $g^{- 1}$ indicates the inverse of a logit link function.³ The coefficient $α_{i}$ denotes the influence of the latent variable $θ_{t}$ on $P (Y_{it})$ , controlling for $X_{t}$ . The coefficient $γ_{i}^{β}$ denotes the influence of the independent variable $X_{t}$ on $P (Y_{it})$ , while controlling for the latent variable. In the mediation literature, this is called the direct effect, because it is the effect of the variable $X_{t}$ on the outcome, not through the mediator. Here the mediator $θ_{t}$ is a continuous latent variable with a distributional assumption, $θ_{t} ~ N (0, 1)$ . Note that Equation 2 corresponds to the MIMIC model for uniform DIF (Woods, 2009) where $γ_{i}^{β}$ corresponds to the uniform DIF parameter for item $i$

Estimating Equation 2 for Item 5,

P (Y_{5 t} | θ_{t}, X_{t}) = g^{- 1} (1.28 - 0.68 X_{t} + 1.03 θ_{t}) .

The influence of $θ_{t}$ on the probability of correct response on Item 5 is positive ( ${\hat{α}}_{5} = 1.03$ , $SE = 0.121$ , $p < . 001$ ). This means that those who have a higher latent ability are more likely to get this item correct. However, the estimate of the effect of group on Item 5 is negative, ( ${\hat{γ}}_{5}^{β} = - 0.68$ , $SE = 0.15$ , $p < . 001$ ). This suggests controlling for latent ability, individuals in the 2006 cohort are less likely to answer this item correctly than individuals in the 1933/1936 cohort. This is uniform DIF or, in the mediation framework, a direct effect. Item 5 was one of the items where the 2006 cohort performed worse than the 1933/1936 cohort. An approximate translation of the item is “How much longer is 12 yards than a meter?” It is difficult to stipulate why Item 5 performs this way. Must and Must (2014) noted that the 1930s cohort has more students who were rural than the 2006 cohort. Perhaps rural children were more familiar with distances such as yards and meters.

A similar analysis can be conducted on Item 10, which is another item where the 1933/1936 cohort performed better than the 2006 cohort. The results showed that $\hat{δ} = 0.09$ , $SE = 0.079$ , $p = 0.26$ .⁴ In this analysis, there is less convincing evidence that the two cohorts significantly differ on latent arithmetic ability. Based on the estimated model for Equation 2, the estimated effect of $θ_{t}$ on the probability of a correct response on Item 10 is 0.94, $SE = 0.13$ , $p < . 001$ . Individuals with higher latent arithmetic ability are more likely to answer Item 10 correctly. Controlling for latent ability, there is not a significant difference between cohorts on the probability of answering Item 10 correctly, ${\hat{γ}}_{10}^{β} = - 0.16$ , $SE = 0.15$ , $p = . 29$ . There is not sufficient evidence of a direct effect (i.e., uniform DIF) for Item 10.

By plugging in Equation 1 as $θ_{t}$ in Equation 2, additional information about indirect and direct effects becomes clear.

\begin{array}{l} P (Y_{i t} | θ_{t}, X_{t}) & = g^{- 1} (β_{i} + γ_{i}^{β} X_{t} + α_{i} (δ X_{t} + ξ_{t})), \\ = g^{- 1} (β_{i} + γ_{i}^{β} X_{t} + α_{i} (δ X_{t} + ξ_{t})), \\ = g^{- 1} (α_{i} δ X_{t} + γ_{i}^{β} X_{t} + α_{i} ξ_{t} + β_{i}), \\ = g^{- 1} ((\underset{indirect effect}{\underset{︸}{{δα}_{i}}} + \underset{direct effect}{\underset{︸}{γ_{i}^{β}}}) X_{t} + α_{i} ξ_{t} + β_{i}) . \end{array}

(3)

Equation 3 shows how the effect of $X_{t}$ on $P (Y_{it})$ is parsed into two effects: a direct effect ( $γ_{i}^{β}$ ) and an indirect effect ( $δ α_{i}$ ). Specifically, the indirect effect quantifies the effect of $X_{t}$ on $P (Y_{it})$ through the mediator $θ_{t}$ (represented by Paths a and b), whereas the direct effect quantifies the remaining effect of $X_{t}$ on $P (Y_{it})$ (through Path c). From the above exercise, it is seen that specifying a MIMIC model with a direct path (for uniform DIF) can be viewed as a mediation model, while this perspective provides an opportunity to discuss two potential mechanisms (direct and indirect routes) that generates group differences on $P (Y_{it})$ .

Uniform DIF examples for Item 5 and Item 10 can be used to generate estimates of indirect effects. The estimated indirect effect of cohort on Item 5 through latent arithmetic ability is $0.149 \times 1.03 = 0.153$ . The estimated indirect effect of cohort on Item 10 through latent arithmetic ability is $0.09 \times 0.94 = 0.084$ . How these estimates can be interpreted is in the “Indirect Effects” section.

Moderated Mediation Model for Nonuniform DIF

It may be possible that some items on the questionnaire provide more information about the latent abilities of the individuals in one of the cohorts compared with the other. This means that the items’ ability to discriminate among people of different latent abilities may depend on which cohort those people come from. For instance, as a latent ability increases, a child’s probability of correctly answering the item might increase faster if that child is in the 1933/1936 cohort compared with the 2006 cohort. This type of effect is described as nonuniform DIF. To test this hypothesis, the authors allow the path between $θ_{t}$ and $P (Y_{it})$ , as specified in “Mediation Model for Uniform DIF” section to be a function of $X_{t}$ in the MIMIC model. This modification is displayed in Figure 3b.

This revised MIMIC model is also a moderated mediation model. Modifying the regression model for Paths b and c by including the interaction between $X_{t}$ and $θ_{t}$ , provides an additional parameter represented by Path d as follows:

P (Y_{it} | θ_{t}, X_{t}) = g^{- 1} (β_{i} + γ_{i}^{β} X_{t} + α_{i} θ_{t} + γ_{i}^{α} X_{t} θ_{t}),

(4)

= g^{- 1} (β_{i} + γ_{i}^{β} X_{t} + (α_{i} + γ_{i}^{α} X_{t}) θ_{t}) .

(5)

Note that Equation 4 includes the interaction between $X_{t}$ and $θ_{t}$ . Equation 5 shows how the relationship between $θ_{t}$ and $P (Y_{it})$ now depends on $X_{t}$ . If the parameter $γ_{i}^{α}$ is zero, then the relationship between $θ_{t}$ and $P (Y_{it})$ does not depend on $X_{t}$ . Hence, the coefficient $γ_{i}^{α}$ indicates the moderation of the effect of $θ_{t}$ on $P (Y_{it})$ by $X_{t}$ . Note that Equation 4 corresponds to the MIMIC model for studying nonuniform DIF (Woods & Grimm, 2011), where $γ_{i}^{α}$ corresponds to the nonuniform DIF parameter for item $i$ and $θ_{t} = δ X_{t} + ξ_{t}$ with $ξ_{t} ~ N (0, σ^{2})$ . In the arithmetic example, $γ_{i}^{α}$ can be interpreted as a difference in the relationship between “true” arithmetic ability and the probability of answering correctly on item $i$ for the 2006 cohort compared with the 1933/1936 cohort. So if this parameter is positive, the item is more discriminating for the 2006 cohort. If it is negative, it is more discriminating for the 1933/1936 cohort. Similarly, it is thought that estimating Equation 2 with just the 1933/1936 cohort and again with just the 2006 cohort, $γ_{i}^{α}$ would indicate the difference in the $α_{i}$ parameters between the two cohorts.

Estimating Equations 1 and 4 for Item 5, an estimate of $δ$ (0.147) and an estimated equation is obtained.

P (Y_{5 t} | θ_{t}, X_{t}) = g^{- 1} (1.20 - 0.57 X_{t} + 0.75 θ_{t} + 0.68 X_{t} θ_{t}) .

As will be discussed more in depth in “Probing Conditional Effects” section, the coefficient for $θ_{t}$ is no longer the overall effect of the latent variable on the probability of correct response. Now it is the effect of the latent variable on the probability of responding correctly when $X_{t} = 0$ (i.e., for individuals in the 1933/1936 cohort). So for individuals in the 1933/1936 cohort, $θ_{t}$ positively predicts the probability of a correct response on Item 5 ( ${\hat{α}}_{5} = 0.75, SE = 0.14, p < . 001$ ). As mentioned previously the $γ_{5}^{α}$ parameter denotes the differences in the item discrimination parameter across the cohorts. Alternatively, this parameter can be interpretted as the degree to which $X_{t}$ moderates the relationship between the latent variable, $θ_{t}$ , and the probability of correct response, $P (Y_{it})$ . So for individuals in the 2006 cohort the item discrimination parameter is $0.75 + 0.68 = 1.43$ . The test on the difference suggests that Item 5 has a greater item discrimination parameter for the 2006 cohort compared with the 1933/1936 cohort ( ${\hat{γ}}_{5}^{α} = 0.68$ , $SE = 0.27$ $p = . 012$ ). The coefficient for $X_{t}$ represents the effect of cohort on the probability of correct response for individuals average on the latent arithmetic trait. This suggests for individuals who are average on the latent arithmetic trait, those in the 1933/1936 cohort are more likely to answer the item correctly ( ${\hat{γ}}_{5}^{β} = - 0.57$ , $SE = 0.16$ , $p < . 001$ ).

The authors conducted the same analysis, allowing for nonuniform DIF for Item 10 only. The estimated cohort difference on the latent trait is 0.09 ( $SE = 0.08$ , $p = . 270$ ). The effect of the latent trait on the probability of answering Item 10 correctly for individuals in the 1933/1936 cohort is 1.601 ( $SE = . 30$ , $p < . 001$ ). So for this cohort, increased latent ability predicts a higher probability of correctly answering the item. But for the 2006 cohort, the item discrimination parameter is much lower: $1.601 - 0.987 = 0.614$ . The delta method can be used to test if the item discrimination parameter is significantly different from zero.⁵ The results show that the item discrimination parameter for the 2006 cohort is still significantly different from zero ( $SE = 0.146$ , $p < . 001$ ). The test on ${\hat{γ}}_{10}^{α}$ suggests that the item discrimination parameter is significantly lower for the 2006 cohort compared with the 1933/1936 cohort ( $SE = 0.335$ , $p = . 003$ ). This suggests that the strength of the relationship between latent ability and probability of correct response is moderated by cohort. For individuals at the mean of the latent arithmetic trait, there was no sufficient evidence that there were cohort differences in the probability of answering Item 10 correctly ( ${\hat{γ}}_{i}^{β} = - 0.036$ , $SE = 0.163$ , $p = . 827$ ).

Just as the authors did with the models for uniform DIF, Equation 5 can be combined with Equation 1 to get information about indirect and direct effects,

\begin{array}{l} P (Y_{i t} | θ_{t}, X_{t}) & = g^{- 1} ((α_{i} + γ_{i}^{α} X_{t}) θ_{t} + γ_{i}^{β} X_{t} + β_{i}), \\ = g^{- 1} ((α_{i} + γ_{i}^{α} X_{t}) (δ X_{t} + ξ_{t}) + γ_{i}^{β} X_{t} + β_{i}), \end{array}

(6)

= g^{- 1} ((\underset{= \underset{indirect effect}{\underset{︸}{(α_{i} + γ_{i}^{α} X_{t}) δ}}}{\underset{︸}{({δ α}_{i} + {δ γ}_{i}^{α} X_{t})}} + \underset{direct effect}{\underset{︸}{(γ_{i}^{α} ξ_{t} + γ_{i}^{β})}}) X_{t} + α_{i} ξ_{t} + β_{i}) .

(7)

Equation 7 is equivalent to a moderated mediation model that shows that the indirect effect of $X_{t}$ on $P (Y_{it})$ through $θ_{t}$ is a function of $X_{t}$ . To understand this, see the first term in Equation 7, which represents the indirect effect, can be rewritten as $(δ α_{i} + {δ γ}_{i}^{α} X_{t}) = (α_{i} + γ_{i}^{α} X_{t}) δ$ . The fact that the indirect effect depends on a categorical grouping variable $X_{t}$ means that the indirect effect may differ across groups. Note that if ${δ γ}_{i}^{α} = 0$ in Equation 7, the indirect effect no longer depends on $X_{t}$ , meaning that the indirect effect is the same across groups. This means that if ${δ γ}_{i}^{α} \neq 0$ there is no single indirect effect that applies to all groups of people and group-specific indirect effects should be estimated.

For instance, in the arithmetic example with Item 5, the indirect effect is $δ α_{i} = 1.47 \times 0.75 = 1.102$ for the 1933/1936 cohort ( $X_{t} = 0$ ) and $δ α_{i} + {δ γ}_{i}^{α} = 1.47 \times . 75 + 1.47 \times 0.68 = 2.102$ for the 2006 cohort ( $X_{t} = 1$ ). In the mediation literature ${δ γ}_{i}^{α}$ is called the index of moderated mediation, which quantifies the change in the indirect effect with a one-unit change in the moderator, in this case $X_{t}$ (Hayes, 2015).

Applying Ideas of Moderation and Mediation to DIF

In “MIMIC DIF Models as Mediation and Moderated Mediation Models” section, the authors discussed that MIMIC models for studying uniform or nonuniform DIF can be viewed as mediation or moderated mediation models, respectively. Specifically, the uniform DIF parameter $γ_{i}^{β}$ from Equation 2 corresponds to a direct effect in a mediation model. This means that uniform DIF indicates the degree to which the independent variable $X_{t}$ has an effect on the outcome $P (Y_{it})$ not through the mediator $θ_{t}$ . In addition, the nonuniform DIF parameter $γ_{i}^{α}$ from Equation 6 corresponds to an interaction effect. This implies that nonuniform DIF indicates the degree to which the effect of the mediator $θ_{t}$ on the outcome variable $P (Y_{it})$ is moderated by the independent variable $X_{t}$ .

In this section, how some of the additional ideas from the mediation and moderation literature can improve the understanding of DIF will be discussed.

Null Direct Effects

In mediation analysis, a null direct effect implies that the cumulative effects of the other ways (not through the mediator) in which an independent variable ( $X$ ) affects the outcome ( $Y$ ) average out to be zero. It does not mean that the independent variable has no impact on the outcome through any other factors (Hayes, 2018; Rucker, Preacher, & Tormala, 2011). Suppose the effect of an independent variable on an outcome is mediated through three variables ( $M$ ) where each indirect effect amounts to 2, –2, and 2, respectively. If a researcher specifies a model with only one mediator (with an indirect effect of 2), the effects of the other two mediators (–2 and 2) will average out to be zero. Accordingly, the direct effect of $X$ on $Y$ is zero in this case, even though there are additional ways that $X$ affects $Y$ .

Similarly, a zero uniform DIF parameter value ( $γ_{i}^{β} = 0$ ) implies that the effects of all other factors, which may mediate the effect of $X_{t}$ on $P (Y_{it})$ are canceled out. For instance, in the arithmetic example with Item 10, it was originally found that the uniform DIF parameter was not significantly different from zero. Suppose farming experience was considered as an additional mediator in studying uniform DIF for Item 10. With such an inclusion of a relevant mediator, the direct effect of $X_{t}$ on $P (Y_{it})$ (or uniform DIF) may no longer be zero. This means that Item $10$ may be equally difficult across the cohorts among those matched on “true” arithmetic ability. But the item may not be equally difficult among individuals in the two cohorts matched on “true” arithmetic ability as well as farming experience. Just as mediation experts do not recommend interpreting a null direct effect as implying that all mediators have been accounted for, similar cautions should be taken when interpreting a lack of uniform DIF.

Indirect Effects

An indirect effect in Figure 3a (Paths a and b) represents the effect of the independent variable ( $X_{t}$ ) on the outcome ( $P (Y_{it})$ ) through the mediator ( $θ_{t}$ ). That is, the indirect effect measures the degree to which a group difference on the latent trait $θ_{t}$ contributes to a group difference on the probability of endorsing item $i$ . The indirect effect can be computed as the product of the two coefficients for Paths a and b, that is, $δ α_{i}$ .

It is discussed in “Mediation Model for Uniform DIF” section that the effect of $X_{t}$ on $P (Y_{it})$ is additively decomposed into a direct effect (represented by the uniform DIF parameter $γ_{i}^{β}$ ) and an indirect effect (represented by the product of the two parameters $δ α_{i}$ ). This means that the indirect effect ( $δ α_{i}$ ) carries information about group differences on $P (Y_{it})$ , which can be explained only by group differences in $θ_{t}$ and the relationship between $θ_{t}$ and $P (Y_{it})$ .

Both in the presence and absence of a direct effect (or uniform DIF), it may be worthwhile to examine and understand the indirect effect. Consider the uniform DIF MIMIC model. For the sake of simplification, $α_{i}$ was set to 1. In this example, by setting $α_{i} = 1$ , it is meant that there is always a relationship between $θ_{t}$ and $P (Y_{it})$ controlling for $X_{t}$ . Four cases will be considered: where the direct and indirect effects are (a) of the same sign, (b) of differing signs, (c) the case when there is no indirect effect, and (d) the case when there is no direct effect. By examining the difference between these cases, it is clear that it is beneficial to consider both the indirect effect and direct effect. The marginal differences on item responses across groups are a poor substitute for understanding the underlying pattern of effects.

Suppose $δ α_{i} = 3$ meaning that there is a cohort difference in $θ_{t}$ of 3. In addition, suppose there is a positive direct effect, $γ_{i}^{β} = 3$ . Just examining the effect of cohort on the item (as is done in Table 1), not considering the latent variable, would reveal that there is a large effect of cohort on the item. Some researchers may see this and assume that there is DIF based on this alone or this difference relative to other items. However, half of the difference can be attributed to the indirect effect of cohort on the item response through the latent variable. This means that there is a cohort difference in $θ_{t}$ and this contributes to an observed cohort difference in $P (Y_{it})$ . The portion of the difference, $δ α_{i}$ , which is attributable to the latent variable is still of interest to the researcher and could be examined even in the presence of DIF.

Consider now if $δ α_{i} = - 3$ meaning that the 2006 cohort has a lower arithmetic ability than the 1933/1936 cohort. In addition, suppose that $γ_{i}^{β} = 3$ , which means that for item $i$ , individuals in the 2006 cohort show a higher probability of endorsing the item compared with individuals in the 1933/1936 cohort with the same “true” arithmetic ability. Note that in this case, marginal cohort differences could not be seen in $P (Y_{it})$ . Some researchers may interpret this lack of marginal cohort differences as indicating that there is no DIF. However, because the direct and indirect effects cancel out, there are no marginal differences across groups on the item responses, but the magnitude of DIF is still the same as the previous example. It is the indirect effect, which has changed, showing the indirect effect and direct effect must both be taken into account to understand the marginal patterns across groups on item responses.

If $δ α_{i} = 3$ and $γ_{i}^{β} = 0$ then the 2006 cohort has a higher “true” arithmetic ability, and individuals from each cohort with the same latent ability have the same probability of getting the item correct. This illustrates a situation where there is an impact, but no DIF (Ackerman, 1992; Camilli, 1993). Examining the marginal cohort differences on item responses, a researcher might see that the two cohorts respond differently to this item. The researcher may conclude that this is attributable to DIF, but that would be a mistake. Once the latent variable is taken into account, it is clear that cohort differences come into play because there are cohort differences on the latent variable. When cohort, or the independent variable which causes DIF, is of primary interest, cases like this will likely be correctly identified as not attributable to DIF. However, if the effect of cohort on the latent variable were secondary, for example, if the researcher were studying differences in arithmetic ability across different ages, cohort’s effect on the latent variable could be overlooked, and the researcher may attribute cohort differences on item responses to DIF rather than to cohort differences on the latent variable.

Finally, consider the case where $δ α_{i} = 0$ and $γ_{i}^{β} = 3$ . This means that there is no overall cohort difference in the “true” arithmetic ability, but for item $i$ individuals in the 2006 cohort show a higher probability of endorsing the item compared with individuals in the 1933/1936 cohort with the same “true” arithmetic ability. Note that in this case, it will be observed that a marginal cohort difference in $P (Y_{it})$ , which is just the same as the third example. But the reason for this marginal group difference is attributable to DIF rather than the indirect effect. These are cases, which would need to be correctly identified as DIF, and that is only done by examining both the direct and indirect effects. The marginal differences on item responses across groups are not informative enough to make a decision about DIF.

Consider the examples of DIF analysis for Items 5 and 10. Each item had marginal group differences on the item response probabilities that deviated from the rest of the items. However, it is seen that Item 5 suffered from uniform DIF, but Item 10 did not. These items seemed similar when examining marginal group differences on item response probabilities. However, the mechanism by which those marginal group differences arose are quite different.

The above exercise clearly explains why it is important to take into account potential group differences in the latent trait $θ_{t}$ in investigating uniform DIF. Importantly, the indirect effect and direct effect are equally scaled and thus are directly comparable. This is what makes this exercise useful. Alternatively, researchers might be temped to compare $α_{i}$ to $γ_{i}^{β}$ . This comparison does not take into account the size of the effect of the independent variable on the latent variable, $δ$ and thus these two effects would not be comparable as they are not on the same scale.

Symmetry of Moderation

The model expressed in Equation 6 allows the effect of $θ_{t}$ on $P (Y_{it})$ to be moderated by $X_{t}$ . Equation 6 can be rewritten in an alternative way as follows:

P (Y_{it} | θ_{t}, X_{t}) = g^{- 1} ((α_{i} + γ_{i}^{α} X_{t}) θ_{t} + γ_{i}^{β} X_{t} + β_{i}),

(8)

= g^{- 1} (α_{i} θ_{t} + \underset{direct effect = γ_{i}^{β *}}{\underset{︸}{(γ_{i}^{β} + γ_{i}^{α} θ_{t})}} X_{t} + β_{i}) .

(9)

Note that Equation 8 shows that the effect of $θ_{t}$ on $P (Y_{it})$ is moderated by $X_{t}$ (see the first term), whereas Equation 9 specifies that the effect of $X_{t}$ on $P (Y_{it})$ is moderated by $θ_{t}$ (see the second term). Equations 8 and 9 are mathematically equivalent, thus statistical evidence that $γ_{i}^{α} \neq 0$ suggests that $X_{t}$ moderates the effect of $θ_{t}$ on $P (Y_{it})$ or that $θ_{t}$ moderates the effect of $X_{t}$ on $P (Y_{it})$ . There is no statistical way to distinguish between these two types of moderation. This is the symmetry property of moderation (Hayes, 2018; Hayes & Matthes, 2009).

Equation 9 can be useful for conceptualizing nonuniform DIF. The coefficient for $X_{t}$ , is a function of $θ_{t}$ . Recall that testing the coefficient of $X_{t}$ in Equation 2 (i.e., the direct effect of $X_{t}$ on $P (Y_{it})$ ), corresponded to testing a uniform DIF (discussed in “Mediation Model for Uniform DIF” section). If the parameter $γ_{i}^{α}$ takes a nonzero value (i.e., if nonuniform DIF exists for item $i$ ), this implies that the direct effect depends on the value of $θ_{t}$ . In other words, group differences on $P (Y_{it})$ are not constant on the $θ_{t}$ continuum.

An important implication is that in the presence of nonuniform DIF (or with a significant $γ_{i}^{α}$ parameter value), the coefficient $γ_{i}^{β}$ loses its interpretation as “uniform DIF” because it is no longer true that “group difference in the endorsement probability is constant over the latent continuum” (Woods & Grimm, 2011). As $γ_{i}^{β *} = γ_{i}^{β}$ holds only when $θ_{t} = 0$ (if $γ_{i}^{α} \neq 0$ ), the coefficient $γ_{i}^{β}$ indicates the group difference on $P (Y_{it})$ when $θ_{t} = 0$ . In the arithmetic example, a significant $γ_{i}^{β} > 0$ means that individuals in the 2006 cohort with a “true” arithmetic ability of zero (population mean) have a higher probability of endorsing item $i$ compared with individuals from the 1933/1936 cohort with the same arithmetic ability. It is important to interpret the $γ_{i}^{β}$ coefficient correctly in the presence of nonuniform DIF.

Probing Conditional Effects

The authors have discussed that nonuniform DIF can be understood as an interaction between the latent variable of interest and an external grouping factor in predicting the probability of response on a given item indicated by the coefficient $γ_{i}^{α}$ . A nonzero $γ_{i}^{α}$ value indicates that the group difference on $P (Y_{it})$ (or the direct effect) is not constant across the mediator $θ_{t}$ . The interaction effect in DIF studies implies that the item characteristic curves for different groups cross at a point of the $θ_{t}$ continuum. See Figure 1 for an illustration of different types of nonuniform DIF (Panels b, c, and d) in comparison with a zero nonuniform DIF case (Panel a).

In moderation analysis, once a significant moderation effect is found, researchers often apply probing methods (e.g., Hayes & Matthes, 2009; Spiller, Fitzimons, Lynch, & McClelland, 2013). These methods can be used to understand where the group-specific item characteristic curves cross. Specifically, it can be solved for the point along $θ_{t}^{*}$ where cohort differences on $P (Y_{ip})$ are estimated to be zero for item $i$ by setting $γ_{i}^{β *}$ to zero as follows:

\begin{matrix} γ_{i}^{β *} = γ_{i}^{β} + γ_{i}^{α} θ_{t}^{*} = 0, \\ γ_{i}^{α} θ_{t}^{*} = - γ_{i}^{β}, \\ θ_{t}^{*} = - \frac{γ_{i}^{β}}{γ_{i}^{α}} . \end{matrix}

It is useful to know the point of $θ_{t}^{*}$ where the group-specific item characteristic curves cross because this point informs us that one group has a higher probability of endorsing the item of interest, compared with the other groups whose latent trait value is $θ_{t} < θ_{t}^{*}$ ; however, the opposite is true for those people with $θ_{t} \geq θ_{t}^{*}$ . See Figures 1b and 1c for the nonuniform DIF cases with different $θ_{t}^{*}$ locations. If the value of $θ_{t}^{*}$ is very small (e.g., smaller than –5) or very large (e.g., larger than 5), the group-specific item characteristic curves may look parallel (no crossing) within the $θ_{t}$ range from –5 to 5. See Figure 1d for such an example. It is also possible to make an inference about the point $θ_{t}^{*}$ after computing its standard error using the delta method.

Discussion

In this article, the authors have discussed how the MIMIC models for studying uniform and nonuniform DIF can be conceptualized within the mediation and moderation framework. Specifically, it is shown how estimating and testing a direct effect in a mediation model aligns with a test of uniform DIF and how a test of an interaction effect in a moderated mediation model aligns with a test of nonuniform DIF. A benefit of conceptualizing DIF within the mediation and moderation framework is that useful ideas and methods from mediation and moderation literature can be applied to improve the understanding of DIF. Typical DIF studies are often exploratory and researchers who wish to study DIF tend to act as if they have no prior information about what type of DIF they might expect. However, understanding DIF in the mediation and moderation context may help researchers apply their substantive knowledge in such a way as to develop more directed hypotheses about DIF for particular items. By developing specific hypotheses about DIF, researchers can transition into a more confirmatory study of DIF.

An additional benefit of conceptualizing DIF within the mediation and moderation framework is that it is clarified that the coefficient $γ_{i}^{β}$ loses its original interpretation in the presence of nonuniform DIF (or with nonzero $γ_{i}^{α}$ ). Furthermore, potential usefulness of examining indirect effects in DIF research is discussed.

Throughout this article, authors have discussed the MIMIC models for exploring DIF, however, the ideas in this article generalize to other models as well. The MIMIC model is equivalent to the two-parameter logistic model (MacIntosh & Hashim, 2003; B. O. Muthèn et al., 1991). When a one parameter logistic model is desired, the MIMIC model can be adjusted so the relationship between $θ_{t}$ and $P (Y_{it})$ is constrained to 1 for all items. Finch (2005) found that MIMIC models perform comparably with other uniform DIF detection methods for data generated from a 3PL model, except when the scale is short (20 items or less). Shih and Wang (2009) showed that data generated using a 3PL model can be effectively analyzed for uniform and nonuniform DIF using MIMIC models. However, there is lower power to detect DIF for data generated from a 3PL compared with a 2PL. Constructing a MIMIC model, which incorporates the guessing parameter is not straightforward. However, it is still useful to discuss uniform and nonuniform DIF with a 3PL model using the concepts of mediation and moderated mediation.

Throughout this article, the analyses were described using a general link function, but for the data example a logistic link function was used. When a probit link is used, a normal-ogive MIMIC model can be constructed, which is equivalent to a normal-ogive logistic model. B. O. Muthèn (1985) showed how to derive the parameters for the normal-ogive IRT model from a normal-ogive MIMIC model. MacIntosh and Hashim (2003) followed up on this work deriving the standard errors for the parameters for the normal-ogive IRT model from the standard errors in the normal-ogive MIMIC model. The MIMIC model can be used for either logistic or normal-ogive link functions.

In “MIMIC Models for DIF Testing” section, authors discussed previous research on selecting anchor items and scale purification using the MIMIC model. These methods, described by Wang et al. (2009) and Shih and Wang (2009) have only investigated detection methods for uniform DIF. The nonuniform DIF MIMIC model is slightly more recent (Woods & Grimm, 2011), and no research has yet to explore how best to go about selecting anchor items or conducting scale purification using MIMIC models for nonuniform DIF. Future research should examine how best to do this type of analysis, particularly when some items have uniform DIF and others have nonuniform DIF.

The authors are not the first to discuss some of the connections between mediation analysis and DIF. Cheng et al. (2016) proposed applying mediation analysis to understand how uniform DIF occurs. Specifically, Cheng et al. (2016) proposed to introduce additional mediators in a MIMIC model to explain the process of uniform DIF; however, the authors did not acknowledge that a MIMIC model could already be seen as a mediation model as discussed in this article. Adding extra mediators in a MIMIC model for uniform DIF as suggested by Cheng et al. (2016) would result in extending a single mediator model to a multiple mediator model in the framework. The additional mediators can be used to test whether the uniform DIF (or a direct effect of $X_{t}$ on $P (Y_{it})$ ) may be accounted for by the additional mediators. The authors have discussed the possibility of including additional mediators in “Null Direct Effects” section in the context of interpreting null direct effects.

An additional benefit of MIMIC models is that multiple independent variables can be included in the model, as was briefly discussed in “Null Direct Effects” section. Including multiple indepedent variables would allow the researcher to estimate indirect and direct effects for each independent variable. However, it is important to remember that indirect and direct effects are scaled by the independent variable (i.e., they are interpreting with respect to a one unit change in the covariate). This means that direct effects through different independent variables may not be directly comparable in terms of magnitude.

Finally, the authors would like to mention that though conceptualizing uniform and nonuniform DIF MIMIC models within a mediation and moderation framework can be very useful, they caution against using this framework to make causal inferences without thoroughly investigating the assumptions needed to do so. There is a growing literature on how to make valid causal inferences, which may be additionally complicated by the use of a latent variable. Researchers interested in making causal inferences should consult the literature on causal mediation analysis (e.g., Coffman, MacKinnon, Zhu, & Ghosh, 2016; Imai, Kelle, & Tingley, 2010; Robins & Greenland, 1992).

Footnotes

Appendix A

Appendix B

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant DGE-1343012.

ORCID iD

Amanda K. Montoya

Notes

References

Ackerman

T. A.

(1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91.

Bulut

Suh

(2017). Detecting multidimensional differential item functioning with the multiple indicators multiple causes model, the item response theory likelihood ratio test, and logistic regression. Frontiers in Education, 2, Article 51.

Camilli

(1993). The case against item bias detection techniques based on internal criteria: Do item bias procedures obscure test fairness issues? The use of differential item functioning statistics: A discussion of current practice and future implications. In Holland

P. W.

Wainer

(Eds.), Differential item functioning (pp. 397-413). Hillsdale, NJ: Lawrence Erlbaum.

Cheng

Shao

Lathrop

Q. N.

(2016). The mediated MIMIC model for understanding the underlying mechanism of DIF. Educational and Psychological Measurement, 76(1), 43-63.

Coffman

D. L.

MacKinnon

D. P.

Zhu

Ghosh

(2016). A comparison of potential outcome approaches for assessing causal mediation. In He

Chen

D.-G.

(Eds.), Statistical causal inferences and their applications in public health research (pp. 263-293). New York, NY: Springer.

Finch

(2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29, 278-295.

Haggerty

M. E.

Terman

L. M.

Thorndike

E. L.

Whipple

G. M.

Yerkes

R. M.

(1921). National intelligence tests: Manual of directions. New York, NY: World Book.

Hauser

R. M.

Goldberger

A. S.

(1971). The treatment of unobservable variables in path analysis. Sociological Methodology, 3, 81-117.

Hayes

A. F.

(2015). An index and test of linear moderated mediation. Multivariate Behavioral Research, 50, 1-22.

10.

Hayes

A. F.

(2018). Introduction to mediation, moderation, and conditional process analysis (2nd ed.). New York, NY: Guilford Press.

11.

Hayes

A. F.

Matthes

(2009). Computational procedures for probing interactions in OLS and logistic regression: SPSS and SAS implementations. Behavior Research Methods, 41, 924-936.

12.

Holland

P. W.

Thayer

D. T.

(1988). Differential item performance and the Mantel-Haenszel procedure. In Wainer

Brown

H. I.

(Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.

13.

Imai

Kelle

Tingley

(2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309-334.

14.

Jöreskog

K. G.

(1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409-426.

15.

Jöreskog

K. G.

Goldberger

A. S.

(1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of American Statistical Association, 70(351), 631-639.

16.

Kang

Cohen

A. S.

(2007). IRT model selection methods for dichotomous items. Applied Psychological Measurement, 31, 331-358.

17.

Lee

Bulut

Suh

(2016). Multidimensional extension of multiple indicator multiple cases models to detect DIF. Educational and Psychological Measurement, 77, 545-569.

18.

Lord

F. M.

(1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

19.

MacIntosh

Hashim

(2003). Variance estimation for converting MIMIC model parameters to IRT parameters in DIF analysis. Applied Psychological Measurement, 27, 372-379.

20.

Magis

Bèland

Tuerlinckx

De Boeck

(2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847-863.

21.

Mellenbergh

G. J.

(1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105-118.

22.

Millsap

R. E.

Everson

H. T.

(1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297-334.

23.

Must

(2013). Changes in test-taking patterns over time. Intelligence, 41, 780-790.

24.

Must

(2014). Data from “changes in test-taking patterns over time” concerning the Flynn effect in Estonia. Journal of Open Psychology Data, 2(1), e2.

25.

Muthèn

B. O.

(1985). A method for studying the homogeneity of test items with respect to other relevant variables. Journal of Educational Statistics, 10, 121-132.

26.

Muthèn

B. O.

(1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557-585.

27.

Muthèn

B.O.

Kao

C.-F.

Burstein

(1991). Instructionally sensitive psychometrics: Application of a new IRT-based detection technique to mathematics achievement test items. Journal of Educational Measurement, 28, 1-22.

28.

Muthèn

L. K.

Muthèn

B. O.

(1998-2011). Mplus user’s guide (6th ed.). Los Angeles, CA: Author.

29.

Narayanan

Swaminathan

(1996). Identiﬁcation of items that show nonuniform DIF. Applied Psychological Measurement, 20, 257-274.

30.

Raju

N. S.

(1988). The area between two item characteristic curves. Psychometrika, 53, 495-502.

31.

Raju

N. S.

(1990). Determining the signiﬁcance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14, 197-207.

32.

Robins

J. M.

Greenland

(1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology, 3, 143-155.

33.

Rucker

D. D.

Preacher

K. J.

Tormala

Z. L.

(2011). Mediation analysis in social psychology: Current practice and new recommendations. Personality and Social Psychology Compass, 5, 359-371.

34.

Shealy

Stout

W. F.

(1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTT as well as item bias/DIF. Psychometrika, 58, 159-194.

35.

Shih

C.-L.

Wang

W.-C.

(2009). Differential item functioning detection using the multiple indicators, multiple causes method with a pure short anchor. Applied Psychological Measurement, 33, 184-199.

36.

Spiller

S. A.

Fitzimons

G. J.

Lynch

J. G.

Jr. McClelland

G. H.

(2013). Spotlights, floodlights, and the magic number zero: Simple effects tests in moderated regression. Journal of Marketing Research, 50, 277-288.

37.

Stout

Nandakumar

Bolt

(1997). MULTISIB—A procedure to investigate DIF when a test is intentionally multidimensional. Applied Psychological Measurement, 21, 195-213.

38.

Swaminathan

Rogers

H. J.

(1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.

39.

Wang

W.-C.

Shih

C.-L.

Yang

C.-C.

(2009). The MIMIC method with scale purification for detecting differential item functioning. Educational and Psychological Measurement, 69, 713-731.

40.

Woods

C. M.

(2009). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44, 1-27.

41.

Woods

C. M.

Grimm

K. J.

(2011). Testing for nonuniform differential item functioning with multiple indicator multiple cause models. Applied Psychological Measurement, 35, 339-361.

42.

Yun

C.-Y.

(2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes (Unpublished doctoral dissertation). University of California, Los Angeles.

43.

Zumbo

B. D.

(2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.