Meta-Analytical SEM: Equivalence Between Maximum Likelihood and Generalized Least Squares

Abstract

Meta-analysis plays a key role in combining studies to obtain more reliable results. In social, behavioral, and health sciences, measurement units are typically not well defined. More meaningful results can be obtained by standardizing the variables and via the analysis of the correlation matrix. Structural equation modeling (SEM) with the combined correlations, called meta-analytical SEM (MASEM), is a powerful tool for examining the relationship among latent constructs as well as those between the latent constructs and the manifest variables. Three classes of methods have been proposed for MASEM: (1) generalized least squares (GLS) in combining correlations and in estimating the structural model, (2) normal-distribution-based maximum likelihood (ML) in combining the correlations and then GLS in estimating the structural model (ML-GLS), and (3) ML in combining correlations and in estimating the structural model (ML). The current article shows that these three methods are equivalent. In particular, (a) the GLS method for combining correlation matrices in meta-analysis is asymptotically equivalent to ML, (b) the three methods (GLS, ML-GLS, ML) for MASEM with correlation matrices are asymptotically equivalent, (c) they also perform equally well empirically, and (d) the GLS method for SEM with the sample correlation matrix in a single study is asymptotically equivalent to ML, which has being discussed extensively in the SEM literature regarding whether the analysis of a correlation matrix yields consistent standard errors and asymptotically valid test statistics. The results and analysis suggest that a sample-size weighted GLS method is preferred for combining correlations and for MASEM.

Keywords

combining correlations generalized least squares maximum likelihood Monte Carlo random-effect model

Introduction

In the literature of meta-analysis, combining correlation matrices across studies is of great interest, since measurements across studies may not be in the same scale. Meaningful results can be obtained by performing the analysis on the combined correlation matrix. In particular, combining meta-analysis and structural equation modeling (SEM) allows researchers to validate or test a substantive theory involving latent constructs that would not be accomplished within a single study (see, e.g., Bamberg & Möser, 2007; Gaertner, 2000; Hom, Caranikas-Walker, Prussia, & Griffeth, 1992; Viswesvaran & Ones, 1995). A well-known development in combining correlations is Becker (1992; see also Becker & Schram, 1994) who proposed a generalized least squares (GLS) method to combine correlations. Another method for combining correlations is the sample-size weighted average, which is a solution to the GLS function with a common weight matrix but proportional to the sample size in each study (see, e.g., Hafdahl, 2007). Following each GLS method, meta-analytical SEM (MASEM) can be performed by fitting the structural model to the combined correlation matrix, using GLS. Under the assumption of multivariate normality for the population distribution in each individual study, a theoretically more sound method is maximum likelihood (ML). One can combine correlations by ML and conduct SEM analysis on the resulting correlation matrix using GLS (ML-GLS; Cheung & Chan, 2005) or using ML both in combining correlations and in the SEM stage (Oort & Jak, 2016). While the ML method is optimal in theory, the estimation process involves the standard deviations of all the variables in each study, and the total number of standard deviations can be huge when either the number of variables or the number of studies is large.

In this article, we show that (a) the GLS methods for combining correlation matrices in meta-analysis are asymptotically equivalent to ML; (b) the methods GLS, ML-GLS, and ML for MASEM with correlation matrices are asymptotically equivalent; (c) the three methods of MASEM also perform equally well according to Monte Carlo results; and (d) the GLS method for SEM with the sample correlation matrix in a single study is asymptotically equivalent to ML, which has been discussed extensively in the SEM literature regarding whether the analysis of a correlation matrix yields consistent standard errors (SEs) and asymptotically valid test statistics (Bentler, 2007; Cudeck, 1989; Krane & McDonald, 1978). Although the focus of the article is on MASEM under a homogeneity condition, we will also discuss equivalence of methods when the population correlations across individual studies are not equal.

We will give an overview of the methods for combining correlations and for MASEM in the second section, aiming to provide the background information for the study in the current article. Then, we establish the asymptotic equivalence between ML and GLS in steps. In the third section, we will give a matrix expression of the asymptotic covariances (Acovs) of the ML estimates of correlations for a single study. In the fourth section, we establish the equivalence between ML and GLS in combining correlations across multiple studies, using the results obtained in the third section. In the fifth section, we obtain a matrix expression for the Acovs of the ML estimators of the parameters of a structured correlation matrix in a single study. Asymptotic equivalence between ML and GLS for a correlation structure with m studies will be proved in the sixth section, using the result obtained in the fifth section. A simulation study checking the empirical performances of the ML and GLS estimators at finite samples is given in the seventh section. Methods under a random-effect model when the population correlations across individual studies are not equal will be examined in the eighth section. We will summarize the results and discuss the implications of asymptotic equivalence and empirical advantages of different methods in the concluding section.

Overview of Existing Methods of MASEM

Suppose we have m studies, and each study contains p observed variables and thus $p_{*} = p (p - 1) / 2$ correlations. Let $r_{t}$ be the vector of correlations from the tth study, with sample size n_t, $t = 1, 2, . . ., m$ . The population counterpart of $r_{t}$ is $ρ_{t}$ . The condition that the $ρ_{t}$ are equal across the m studies is called homogeneity. Methods for combining $r_{t}$ and MASEM have been developed under the homogeneity condition and when the homogeneity condition does not hold. This section provides an overview of methods developed under the homogeneity condition. In the eighth section of the article, we will study methods when the homogeneity condition is violated.

The Acov matrix of $r_{t}$ is an important component in all methods of MASEM. Let $r_{t i j}$ be the sample correlation between the ith and jth variables in the tth study, and $ω_{t i j, t k l}$ be the Acov between $\sqrt{n_{t}} r_{t i j}$ and $\sqrt{n_{t}} r_{t k l}$ , then $ω_{t i j, t k l}$ does not depend on n_t, and we can ignore the t from the subscript for simplicity. In particular, under the normality assumption, the form of $Ω = Acov (\sqrt{n} r) = (ω_{i j, k l})$ has been given by Olkin and Siotani (1976, p. 238; see also Hsu, 1949; Pearson & Filon, 1898) as

ω_{i j, i j} = (1 - ρ_{i j}^{2})^{2}

when $i = k$ and $j = l$ , and

ω_{i j, k l} = ρ_{i j} ρ_{k l} (ρ_{i k}^{2} + ρ_{i l}^{2} + ρ_{j k}^{2} + ρ_{j l}^{2}) / 2 + ρ_{i k} ρ_{j l} + ρ_{i l} ρ_{j k} - (ρ_{i j} ρ_{i k} ρ_{i l} + ρ_{j i} ρ_{j k} ρ_{j l} + ρ_{k i} ρ_{k j} ρ_{k l} + ρ_{l i} ρ_{l j} ρ_{l k})

otherwise, where $ρ_{i j}$ is the population counterpart of $r_{t i j}$ .

With the elements of Ω being given in Equations 1 and 2, letting

ρ = (ρ_{21}, ρ_{31}, \dots, ρ_{p 1}, ρ_{32}, ρ_{42}, \dots, ρ_{p 2}, \dots, ρ_{p (p - 1)})^{'}

be the vector of the $p_{*}$ population correlations. Under the homogeneity condition, the GLS method for combining $r_{t}$ given by Becker (1992) is to minimize

Q (ρ) = \sum_{t = 1}^{m} (r_{t} - ρ)^{'} {\hat{W}}_{t} (r_{t} - ρ),

for $\hat{ρ}$ , where ${\hat{W}}_{t} = n_{t} {\hat{Ω}}_{t}^{- 1}$ with ${\hat{Ω}}_{t}$ being estimated at $r_{t}$ according to Equations 1 and 2. As described in Becker (1992), the Acov matrix of the estimated $\hat{ρ}$ is consistently estimated by ${(\sum_{t = 1}^{m} {\hat{W}}_{t})}^{- 1}$ , which can be further used to formulate a GLS function for estimating a structural model $ρ(γ)$ defined over the $p_{*}$ correlations. Commonly used structural models include factor analysis and structural equation models. Regression model can also be regarded as a structural model and was studied by Becker (1992). Let us use ${GLS}_{1}$ to denote the GLS method of MASEM of fitting $ρ(γ)$ to $\hat{ρ}$ and weighted by $(\sum_{t = 1}^{m} {\hat{W}}_{t})$ , to distinguish it from other GLS methods.

Alternatively, parallel to Equation 3, a GLS function can also be defined via a common Ω matrix:

Q_{c} (ρ) = \sum_{t = 1}^{m} (r_{t} - ρ)^{'} {\hat{W}}_{c t} (r_{t} - ρ),

where ${\hat{W}}_{c t} = n_{t} {\hat{Ω}}^{- 1}$ with $\hat{Ω}$ being evaluated according to Equations 1 and 2 at the sample-size weighted average:

\bar{r} = \sum_{t = 1}^{m} n_{t} r_{t} / N,

where $N = \sum_{t = 1}^{m} n_{t}$ . Actually, the sample-size weighted average $\bar{r}$ in Equation 5 also minimizes the GLS function $Q_{c} (ρ)$ in Equation 4, as has been noted by Hafdahl (2007). Under the homogeneity assumption of a common ρ across the m studies, for the sample-size weighted average $\bar{r}$ , we have

Acov (\sqrt{N} \bar{r}) = Ω .

Thus, MASEM can also be performed via the GLS function of fitting $\bar{r}$ by $ρ(γ)$ and weighted by ${\hat{Ω}}^{- 1}$ , where $\hat{Ω}$ is evaluated at $\bar{r}$ . We will refer to this GLS method as ${GLS}_{2}$ .

For MASEM, Cheung and Chan (2005) proposed a two-stage procedure. Let $S_{t}$ be the sample covariance or correlation matrix from the tth study and $Σ_{t} = D_{t} P D_{t}$ , where $P = (ρ_{i j})$ is a saturated but common correlation matrix across the m studies and $D_{t} = diag (d_{t 1}, d_{t 2}, \dots, d_{t p})$ is a diagonal matrix of standard deviations or scale parameters that vary across studies. The first stage in Cheung and Chan (2005) is to estimate the common correlations in $P$ using a multigroup approach of SEM, which is to minimize

F_{m l}^{(m)} (θ^{(m)}) = \sum_{t = 1}^{m} (n_{t} / N) F_{m l t} (θ_{t}),

to estimate $θ^{(m)}$ , where:

\begin{matrix} F_{m l t} (θ_{t}) = tr (S_{t} Σ_{t}^{- 1} (θ_{t})) - log | S_{t} Σ_{t}^{- 1} (θ_{t}) | - p, \\ θ_{t} = (ρ', d_{t 1}, d_{t 2}, \dots, d_{t p})^{'} and θ^{(m)} = (ρ^{'}, d_{11}, d_{12}, \dots, d_{1 p}, \dots, d_{m 1}, d_{m 2}, \dots, d_{m p})^{'} . \end{matrix}

Let $\tilde{ρ}$ be the vector of correlations obtained by minimizing Equation 7. In the second stage of Cheung and Chan (2005), $\tilde{ρ}$ is fitted by a structural model $ρ(γ)$ using GLS in which the weight matrix is the inverse of the covariance matrix of $\tilde{ρ}$ obtained from the information matrix associated with minimizing Equation 7. Let us call the resulting estimate the ML-GLS estimator and denote it as $\tilde{γ}$ . According to Cheung and Chan, the ML-GLS estimator $\tilde{γ}$ is much more efficient than the estimator of $γ$ by ${GLS}_{1}$ , and the resulting statistic for testing the homogeneity of correlations across the studies by the likelihood ratio statistic associated with Equation 7 is also more reliable. Currently, the two-stage approach ML-GLS is the recommended procedure for MASEM (Cheung, 2015; Cheung & Hafdahl, 2016).

Recently, Oort and Jak (2016) proposed to estimate the structural parameter $γ$ by directly minimizing the function $F_{m l}^{(m)} (θ^{(m)})$ in Equation 7 in which each $Σ_{t} = D_{t} P (γ) D_{t}$ contains a restricted structure on the common correlation matrix. The method of Oort and Jak (2016) is essentially the normal-distribution-based ML for multigroup analysis in SEM (Jöreskog, 1971). Thus, there are four methods of MASEM in estimating the structural parameter γ, two are GLS, one is ML-GLS, and one is ML.

We are going to show that the ML estimator $\tilde{ρ}$ in Cheung and Chan (2005) is asymptotically equivalent to the sample-size weighted average $\bar{r}$ in Equation 5. Consequently, parameter estimates of γ by ${GLS}_{2}$ and ML-GLS are asymptotically equivalent. We will also show that the ML estimator of γ in Oort and Jak (2016) is asymptotically equivalent to those by ${GLS}_{2}$ and ML-GLS. Because the sample-size weighted average $\bar{r}$ is much simpler to compute than the ML estimator $\tilde{ρ}$ , our results imply that ${GLS}_{2}$ is the preferred method for MASEM, especially with balanced data. We are going to discuss alternative procedures when not all sample correlation matrices have the same number of elements, and thus, the formula for computing $\bar{r}$ does not apply. The estimator of γ by ${GLS}_{1}$ is also asymptotically equivalent to those by the other three methods. But the conditions are different, and we are going to discuss the conditions as well.

Since $\bar{r}$ by ${GLS}_{2}$ and $\tilde{ρ}$ by ML-GLS are consistent estimators of ρ and are asymptotically normally distributed, we only need to show that they have the same Acov matrix for the two methods to be asymptotically equivalent. Similarly, we are going to show that the Acov matrices of the estimates of γ by ${GLS}_{2}$ and ML-GLS equal to that by ML. For such a purpose, we will first show that the inverse of the information matrix corresponding to $\tilde{ρ}$ by minimizing the $F_{m l}^{(m)} (θ^{(m)})$ in Equation 7 equals the Ω matrix defined via Equations 1 and 2. We do this by arguing that the inverse of the information matrix corresponding to $Acov (\sqrt{n} \tilde{ρ})$ for a single study equals the Ω matrix, and then we show that the inverse of the information matrix with m studies equals that for a single study.

In addition to directly combining correlations, one can work with the normalized scores following Fisher z transformation on individual correlations in each study (see Hedges & Olkin, 1985). However, according to Hunter and Schmidt (2004) “Fisher z transformation produces an estimate of the mean correlation that is upwardly biased and less accurate than an analysis using untransformed correlations” (p. 82). Thus, we will not study the asymptotic properties of meta-analysis of correlations following Fisher z transformation.

There are also various developments for estimating and testing correlation structures outside the literature of MASEM. Jennrich (1970) developed a χ² statistic for testing the equality of two correlation matrices and pointed out errors in earlier literature. For a single study, Browne (1977, equation 3.7) considered the analysis of patterned correlation matrices and defined a GLS function that involves jointly modeling the sample standard deviations and correlations. Steiger (1980, p. 339) defined an asymptotic ML function by assuming that the vector $r$ of sample correlations follows a multivariate normal distribution. The objective function of asymptotic ML is essentially the same as the GLS function for a single study within ${GLS}_{1}$ and ${GLS}_{2}$ defined via Equations 3 and 4, respectively. Steiger also defined a GLS method, which differs from asymptotic ML only in the weight matrices, where the weight matrix in the GLS is obtained from initial least squares estimates of the structured model while the weight matrix in the asymptotic ML is formulated using the population covariance matrix of $r$ . Thus, the asymptotic ML and GLS as defined in Steiger are asymptotically equivalent because the least squares estimates are consistent. However, Steiger did not show that GLS is equivalent to ML. Also, like in Browne (1977), Steiger (1980) only considered patterned correlation structures in which the model $ρ(γ)$ can be represented as a vector of linear functions of γ. While patterned correlation structures may include many interesting models, most models in SEM or latent variable modeling cannot be represented as patterned correlation matrices. Since our interest is to establish the equivalence of different methods of MASEM, we will not further discuss estimation methods that have not been used in MASEM.

ML Estimate $\tilde{ρ}$ With a Single Study

In this section, we obtain a matrix form of the Acovs for the vector $r$ of sample correlations with a single study and argue that the matrix equals the Ω defined by Equations 1 and 2. The obtained matrix form will be used to show the asymptotic equivalence between ML and GLS for combining correlations with m studies in the following section.

Let $S = (s_{i j})$ be the sample covariance matrix and $Σ (θ)$ be a structural model for $E (S) = Σ$ . In SEM, the normal-distribution-based ML method is typically performed by minimizing the discrepancy function:

F_{m l} (θ) = tr (S Σ^{- 1} (θ)) - log | S Σ^{- 1} (θ) | - p .

When $S$ in Equation 8 is replaced by the ML estimator of $Σ$ (i.e., the biased one), then parameter estimate by minimizing $F_{m l} (θ)$ becomes identical to the ML estimate. With an unbiased $S$ in Equation 8, the estimate of $θ$ by minimizing $F_{m l} (θ)$ is asymptotically equivalent to the ML estimate.

Consider a saturated model with $Σ (θ) = D P D$ , where $D = diag (d_{1}, d_{2}, \dots, d_{p})$ is a diagonal matrix of scale parameters or standard deviations; $P = (ρ_{i j})$ is a correlation matrix, that is, $ρ_{i i} = 1$ , $i = 1, 2, . . ., p$ ; and the vector of parameter is given by $θ = (ρ', d')^{'}$ with ρ containing $p_{*} = p (p - 1) / 2$ elements and $d = (d_{1}, d_{2}, \dots, d_{p})^{'}$ . It is easy to see that minimizing Equation 8 yields ${\tilde{d}}_{i} = s_{i} = s_{i i}^{1 / 2}$ and ${\tilde{ρ}}_{i j} = r_{i j}$ , where $r_{i j}$ is the sample correlation between the ith and jth variables. Thus, the Acov matrix of $\sqrt{n} \tilde{θ}$ is consistently estimated by the inverse of the corresponding information matrix, where $\tilde{θ} = (r', s')^{'}$ with $r' = (r_{21}, r_{31}, \dots, r_{p 1}, r_{32}, r_{42}, \dots, r_{p 2}, \dots, r_{p (p - 1)})$ and $s' = (s_{1}, s_{2}, \dots, s_{p})$ . In particular, the form of the information matrix is given by $A = (a_{i j})$ , where

a_{i j} {= 2}^{- 1} E (\frac{\partial^{2} F_{m l} (θ)}{\partial θ_{i} \partial θ_{j}} {) = 2}^{- 1} tr (Σ^{- 1} {\dot{Σ}}_{i} Σ^{- 1} {\dot{Σ}}_{j}),

with ${\dot{Σ}}_{i}$ being the partial derivative of $Σ$ with respect to the ith element of $θ = (ρ', d^{'})^{'}$ .

We need to obtain an analytical form of the information matrix. With $Σ (θ) = D P D$ , the differential of $Σ$ is given by

d Σ = (d D) P D + D (d P) D + D P (d D) .

The partial derivatives of $Σ (θ)$ with respect to $ρ_{i j}$ is given by

{\dot{Σ}}_{ρ_{i j}} = D (e_{i} {e^{'}}_{j} + e_{j} {e^{'}}_{i}) D = d_{i} d_{j} (e_{i} {e^{'}}_{j} + e_{j} {e^{'}}_{i}),

where $e_{i}$ is a p-dimensional vector whose ith element is 1 and others are 0; and the partial derivative of $Σ (θ)$ with respect to d_k is

{\dot{Σ}}_{d_{k}} = (e_{k} {e^{'}}_{k}) P D + D P (e_{k} {e^{'}}_{k}) .

Let $P^{- 1} = (ρ^{i j})$ . It follows from Equations 9 and 10 that the element of the information matrix $A$ corresponding to $ρ_{i j}$ and $ρ_{k l}$ is given by

\begin{array}{l} a_{i j, k l} {= 2}^{- 1} tr [D^{- 1} P^{- 1} D^{- 1} {D (e_{i} e_{j^{'}} + e_{j} e_{i^{'}}) D} D^{- 1} P^{- 1} D^{- 1} {D (e_{k} e_{l^{'}} + e_{l} e_{k^{'}}) D}] \\ {= 2}^{- 1} tr [P^{- 1} (e_{i} e_{j^{'}} + e_{j} e_{i^{'}}) P^{- 1} (e_{k} e_{l^{'}} + e_{l} e_{k^{'}})] \\ {= 2}^{- 1} (ρ^{l i} ρ^{j k} + ρ^{k i} ρ^{j l} + ρ^{l j} ρ^{i k} + ρ^{k j} ρ^{i l}) \\ = (ρ^{i l} ρ^{j k} + ρ^{i k} ρ^{j l}) . \end{array}

We denote the information matrix corresponding to $\tilde{ρ}$ , with elements given by Equation 12, as $A_{ρρ}$ , which is a $p_{*} \times p_{*}$ matrix.

It follows from Equations 9, 10, and 11 that the element of $A$ corresponding to $ρ_{i j}$ and d_k is given by:

\begin{array}{l} a_{i j, k} {= 2}^{- 1} tr [D^{- 1} P^{- 1} D^{- 1} {D (e_{i} {e^{'}}_{j} + e_{j} {e^{'}}_{i}) D} D^{- 1} P^{- 1} D^{- 1} {(e_{k} {e^{'}}_{k}) P D + D P (e_{k} {e^{'}}_{k})}] \\ {= 2}^{- 1} tr [D^{- 1} P^{- 1} (e_{i} {e^{'}}_{j} + e_{j} {e^{'}}_{i}) P^{- 1} D^{- 1} {(e_{k} {e^{'}}_{k}) P D + D P (e_{k} {e^{'}}_{k})}] \\ {= 2}^{- 1} (δ_{i k} ρ^{j k} + δ_{j k} ρ^{k i} + δ_{j k} ρ^{i k} + δ_{i k} ρ^{k j}) / d_{k} \\ = (δ_{i k} ρ^{j k} + δ_{j k} ρ^{i k}) / d_{k}, \end{array}

where $δ_{i k} = 1$ when $i = k$ and 0 otherwise. Let $A_{ρ d}$ be the $p_{*} \times p$ matrix with elements being given by Equation 13. Then

A_{ρ d} = {A^{'}}_{d ρ} = B_{ρ} D^{- 1},

where

B_{ρ} = (\begin{matrix} ρ^{21} & ρ^{21} & 0 & 0 & \dots & 0 & 0 \\ ρ^{31} & 0 & ρ^{31} & 0 & \dots & 0 & 0 \\ ρ^{41} & 0 & 0 & ρ^{41} & \dots & 0 & 0 \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ \\ ρ^{p 1} & 0 & 0 & 0 & \dots & 0 & ρ^{p 1} \\ 0 & ρ^{32} & ρ^{32} & 0 & \dots & 0 & 0 \\ 0 & ρ^{42} & 0 & ρ^{42} & \dots & 0 & 0 \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ \\ 0 & ρ^{p 2} & 0 & 0 & \dots & 0 & ρ^{p 2} \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ \\ 0 & 0 & 0 & 0 & \dots & ρ^{p (p - 1)} & ρ^{p (p - 1)} \end{matrix}),

and each row of $B_{ρ}$ , corresponding to $ρ_{i j}$ , has nonzero elements $ρ^{i j}$ in the ith and jth columns, and the other $p - 2$ elements are zero. It follows from Equations 9 and 11 that the element of the information matrix $A$ corresponding to d_j and d_k is given by

\begin{array}{l} a_{j k} {= 2}^{- 1} tr [D^{- 1} P^{- 1} D^{- 1} {(e_{j} {e^{'}}_{j}) P D + D P (e_{j} {e^{'}}_{j})} D^{- 1} P^{- 1} D^{- 1} {(e_{k} {e^{'}}_{k}) P D + D P (e_{k} {e^{'}}_{k})}] \\ {= 2}^{- 1} tr [D^{- 1} (e_{j} {e^{'}}_{j}) D^{- 1} (e_{k} {e^{'}}_{k}) + D^{- 1} P^{- 1} D^{- 1} (e_{j} {e^{'}}_{j}) P (e_{k} {e^{'}}_{k}) \\ + P (e_{j} {e^{'}}_{j}) D^{- 1} P^{- 1} D^{- 1} (e_{k} {e^{'}}_{k}) + D^{- 1} (e_{j} {e^{'}}_{j}) D^{- 1} (e_{k} {e^{'}}_{k})] \\ = (δ_{j k} + ρ_{j k} ρ^{j k}) / (d_{j} d_{k}) . \end{array}

Let $A_{d d}$ be the $p \times p$ matrix whose elements are given by Equation 16, and $H = (h_{j k})$ with $h_{j k} = ρ_{j k} ρ^{j k}$ . Then

A_{d d} = D^{- 1} (I + H) D^{- 1} .

Thus, the information matrix in estimating $θ = (ρ', d')^{'}$ with a single study is given by

A = (\begin{matrix} A_{ρ ρ} & A_{ρ d} \\ A_{d ρ} & A_{d d} \end{matrix}) .

Let

A^{- 1} = (\begin{matrix} A^{ρ ρ} & A^{ρ d} \\ A^{d ρ} & A^{a d} \end{matrix}) .

Then, it follows from the formula of matrix inverse for a block matrix that

A^{ρ ρ} = (A_{ρ ρ} - A_{ρ d} A_{d d}^{- 1} A_{d ρ})^{- 1} = [A_{ρ ρ} - B_{ρ} {(I + H)}^{- 1} {B^{'}}_{ρ}]^{- 1},

which does not depend on the elements of $D = diag (d_{1}, d_{2}, \dots, d_{p})$ . It follows from the theory of ML that the Acov matrix of $\sqrt{n} r$ is

Acov (\sqrt{n} r) = A^{ρρ} .

Recall that the elements of the Acov matrix Ω of $\sqrt{n} r$ was given by Olkin and Siotani (1976, p. 238) as defined in Equations 1 and 2. Now we have two formulas for the Acov matrix of $\sqrt{n} r$ . Since both of them are mathematically correct, they must be equal. That is,

Acov (\sqrt{n} r) = A^{ρ ρ} = Ω,

and both $A^{ρρ}$ and Ω are obtained based on a normally distributed population.

The matrix $A^{ρρ}$ in Equation 18 is essentially a matrix expression of Equations 1 and 2. An alternative matrix expression of Equations 1 and 2 was given by Nel (1985) who obtained the formula by using an asymptotic expansion of the correlation matrix directly. Parallel to formulas in Equations 1 and 2, Steiger and Hakstian (1982) obtained the asymptotic variance and covariance of sample correlations when the population is nonnormally distributed, and a matrix form of these asymptotic variances and covariances was obtained by Browne and Shapiro (1986).

Example 1. Let $p = 3$ and the three correlations are $ρ_{12} = .30$ , $ρ_{13} = .40$ , and $ρ_{23} = .50$ . Then,

A^{ρ ρ} = Ω = (\begin{matrix} .8281 & .3450 & .2265 \\ .3450 & .7056 & .1270 \\ .2265 & .1270 & .5625 \end{matrix})

without rounding errors. This can be verified by computing the Ω matrix according to Equations 1 and 2, and $A^{ρρ}$ according to Equation 18, respectively, or by evaluating and inverting the information matrix $A$ directly.

While $A^{ρρ}$ and $A_{ρρ}$ do not depend on the values of the scales or standard deviations, the rest submatrices of $A$ and $A^{- 1}$ do depend on the values of the scales.

ML Estimate of ρ With m Studies

In this section, we study the ML estimator $\tilde{ρ}$ by combining m studies. In particular, we will show that the Acov matrix of $\sqrt{N} \tilde{ρ}$ is also given by the matrix $A^{ρρ}$ in Equation 18.

We will need to deal with the function $F_{m l}^{(m)} (θ^{(m)})$ in Equation 7 to get the information matrix for $\tilde{ρ}$ , where the vector $θ^{(m)}$ contains $p_{*} + p m$ elements, as defined following Equation 7. While there are much more parameters in m studies than in a single study, each function $F_{m l t} (θ_{t})$ in Equation 7 has its own scale parameters $d_{t} = (d_{t 1}, d_{t 2}, \dots, d_{t p})^{'}$ , and we will call $d_{t}$ the d parameters for the tth study. Parallel to Equation 9, let

a_{i j}^{(m)} {= 2}^{- 1} E (\frac{\partial^{2} F_{m l}^{(m)} (θ^{(m)})}{\partial θ_{i} \partial θ_{j}}),

be the element in the ith row and jth column of the information matrix $A^{(m)}$ corresponding to minimizing the $F_{m l}^{(m)} (θ^{(m)})$ in Equation 7. Then $a_{i j}^{(m)} = 0$ if the ith and jth parameters are the d parameters corresponding to different studies. Notice that, except for the order of the d parameters being arranged according to the studies labeled $1$ to m, the structure of each $F_{m l t} (θ_{t})$ is the same as that of the function $F_{m l} (θ)$ for a single study in Equation 8 that was examined in the previous section. In particular, we have

A^{(m)} = \sum_{t = 1}^{m} \frac{n_{t}}{N} A_{t},

with

A_{t} = (\begin{matrix} A_{ρ ρ} & 0_{12} & A_{ρ t} & 0_{14} \\ 0_{21} & 0_{22} & 0_{23} & 0_{24} \\ A_{t ρ} & 0_{32} & A_{t t} & 0_{34} \\ 0_{41} & 0_{42} & 0_{43} & 0_{44} \end{matrix}),

where $A_{ρρ}$ is the same as for a single study with elements defined in Equation 12,

A_{ρ t} = {A^{'}}_{t ρ} = B_{ρ} D_{t}^{- 1}, A_{t t} = D_{t}^{- 1} (I + H) D_{t}^{- 1}

with $B_{ρ}$ and $H$ being defined the same as in Equations 15 and 17, respectively, $0_{12} = {0^{'}}_{21}$ , $0_{14} = {0^{'}}_{41}$ , $0_{22}$ , $0_{23} = {0^{'}}_{32}$ , $0_{24} = {0^{'}}_{42}$ , $0_{34} = {0^{'}}_{43}$ , and $0_{44}$ are zero matrices with subscript 1 corresponding to $p_{*}$ rows or columns, 2 corresponding to $(t - 1) p$ , 3 corresponding to p, and 4 corresponding to $(m - t) p$ rows or columns.

With the positions of the 0s in mind and $h_{t} = n_{t} / N$ , it follows from Equations 21 and 22 that the information matrix for m studies corresponding to minimizing the $F_{m l}^{(m)} (θ^{(m)})$ in Equation 7 is

A^{(m)} = (\begin{matrix} A_{ρ ρ} & h_{1} A_{ρ 1} & h_{2} A_{ρ 2} & \dots & h_{m} A_{ρ m} \\ h_{1} A_{1 ρ} & h_{1} A_{11} & 0 & \dots & 0 \\ h_{2} A_{2 ρ} & 0 & h_{2} A_{22} & \dots & 0 \\ ⋮ & ⋮ & ⋮ & ⋱ & ⋮ \\ h_{m} A_{m ρ} & 0 & 0 & \dots & h_{m} A_{m m} \end{matrix}) .

We next use the formula of matrix inverse for block matrix by focusing on the block corresponding to $A_{ρρ}$ . Let

M_{ρ} = (\begin{matrix} h_{1} A_{11} & 0 & \dots & 0 \\ 0 & h_{2} A_{22} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & \dots & h_{m} A_{m m} \end{matrix}) and C_{ρ} = (\begin{matrix} h_{1} A_{ρ 1} & h_{2} A_{ρ 2} & \dots & h_{m} A_{ρ m} \end{matrix}) .

Then the top-left submatrix of ${(A^{(m)})}^{- 1}$ , corresponding to $A_{ρρ}$ , is given by

\begin{array}{l} A^{ρ ρ} = (A_{ρ ρ} - C_{ρ} M_{ρ}^{- 1} {C^{'}}_{ρ})^{- 1} \\ = (A_{ρ ρ} - \sum_{t = 1}^{m} h_{t} A_{ρ t} A_{t t}^{- 1} A_{t ρ})^{- 1} \\ = [A_{ρ ρ} - B_{ρ} {(I + H)}^{- 1} {B^{'}}_{ρ}]^{- 1}, \end{array}

which is identical to Equation 18. Thus, according to Equation 6, the Acov matrix of $\sqrt{N} \tilde{ρ}$ from combining m studies by ML is identical to that of $\sqrt{N} \bar{r}$ . A direct consequence of this result is that the method ${GLS}_{2}$ for MASEM is asymptotically equivalent to the method ML-GLS. In particular, let $ρ=ρ(γ)$ be the structural model, the Acov matrix of $\hat{γ}$ by ${GLS}_{2}$ or $\tilde{γ}$ by ML-GLS is given by

A c o v (\sqrt{N} \hat{γ}) = {({\dot{ρ}}^{'} Ω^{- 1} \dot{ρ})}^{- 1} = A c o v (\sqrt{N} \tilde{γ}),

where $\dot{ρ} = \dot{ρ} (γ) = \partial ρ (γ) / \partial γ^{'}$ is the Hessian matrix of $ρ(γ)$ , and $Ω = A^{ρρ}$ .

In discussing the GLS method of Becker (1992), we noted that the Acov matrix of $\hat{ρ}$ by ${GLS}_{1}$ is consistently estimated by ${(\sum_{t = 1}^{m} {\hat{W}}_{t})}^{- 1}$ , where ${\hat{W}}_{t} = n_{t} {\hat{Ω}}_{t}^{- 1}$ . Under the assumption of a common ρ across the m studies, ${\hat{Ω}}_{t}$ is consistent for Ω. Thus, for a given m, if $n_{t} / N \to h_{t} > 0$ as all the n_t increase, we also have

\hat{Acov} (\sqrt{N} \hat{ρ}) = (\sum_{t = 1}^{m} {\hat{W}}_{t})^{- 1} = [\sum_{t = 1}^{m} (n_{t} / N) {\hat{Ω}}_{t}^{- 1}]^{- 1} \overset{P}{\to} {(\sum_{t = 1}^{m} h_{t} Ω^{- 1})}^{- 1} = Ω .

Consequently, the method ${GLS}_{1}$ for MASEM is also asymptotically equivalent to ${GLS}_{2}$ and ML-GLS. However, we need all the sample sizes in the m individual studies to be large enough to see the effect of this equivalence, whereas we only need for the total sample size $N = \sum_{t = 1}^{m} n_{t}$ to be large enough to see the effect of the equivalence between ${GLS}_{2}$ and ML-GLS.

ML Estimate $\tilde{γ}$ of a Structured Correlation Matrix With a Single Study

In this section, we examine the ML estimator $\tilde{γ}$ for a correlation structure model $P (γ)$ with a single study. The obtained result will be used for establishing the asymptotic equivalence of ML with ML-GLS and ${GLS}_{2}$ for m studies in the next section. The result is also interesting by itself, since correlation structures for a single group have a lot of applications and various efforts have been made in documenting the proper use of the ML method. Our results in this section indicate that GLS yields essentially the same results as ML. Thus, GLS is preferred because it is not only simpler but also avoids the conceptual and technical issues associated with the analysis of a correlation structure by ML regarding whether the sample correlation matrix can be treated as a sample covariance matrix (Bentler, 2007; Cudeck, 1989; Lee, 1985; Shapiro & Browne, 1990).

Suppose the correlation matrix $P$ is further structured as $P (γ)$ . An example is a factor model with $P (γ) = ΛΦ Λ^{'} + Ψ$ , where $diag [P (γ)] = I$ or $diag (Ψ) = I - diag(ΛΦ Λ^{'})$ , and Λ and Φ are the matrices of factor loadings and factor covariances, respectively. Then we have $Σ (θ) = D P (γ) D$ , where $θ = (γ^{'}, d^{'})^{'}$ . The information matrix for the ML estimator $\tilde{θ} = ({\tilde{γ}}^{'}, {\tilde{d}}^{'})^{'}$ obtained by minimizing Equation 8, with a structured correlation matrix, is given by

A = (\begin{matrix} A_{γ γ} & A_{γ d} \\ A_{d γ} & A_{d d} \end{matrix}),

where $A_{γγ}$ is the information matrix corresponding to γ, $A_{d d}$ is the information matrix corresponding to $d$ , and $A_{γ d}$ is the information matrix corresponding to γ joint with $d$ .

Let ${\dot{P}}_{i}$ and ${\dot{P}}_{j}$ be the matrix of partial derivatives of $P (γ)$ with respect to the ith and jth parameters in γ, respectively. It follows from Equation 9 that the $i j$ th element of $A_{γγ}$ is given by

\begin{array}{l} a_{i j} = \frac{1}{2} tr [Σ^{- 1} (D {\dot{P}}_{i} D) Σ^{- 1} (D {\dot{P}}_{j} D)] \\ = \frac{1}{2} tr (P^{- 1} {\dot{P}}_{i} P^{- 1} {\dot{P}}_{j}) \\ = \frac{1}{2} \frac{\partial vec' (P)}{\partial γ_{i}} (P^{- 1} \otimes P^{- 1}) \frac{\partial vec (P)}{\partial γ_{j}} . \end{array}

We would like to relate $A_{γγ}$ to $A_{ρρ}$ defined via Equation 12. According to the chain rule for differentiation, there exists

\frac{\partial vec (P)}{\partial γ_{i}} = G \frac{\partial ρ}{\partial γ_{i}},

where $G = \partial vec (P) / \partial ρ^{'}$ is a matrix of order $p^{2} \times p_{*}$ and each of its element is either 0 or 1. Thus, we can rewrite the $a_{i j}$ in Equation 25 as

a_{i j} = \frac{1}{2} \frac{\partial ρ^{'}}{\partial γ_{i}} [G' (P^{- 1} \otimes P^{- 1}) G] \frac{\partial ρ}{\partial γ_{j}} and A_{γ γ} = \frac{1}{2} \frac{\partial ρ^{'}}{\partial γ} [G^{'} (P^{- 1} \otimes P^{- 1}) G] \frac{\partial ρ}{\partial γ^{'}} .

Note that the saturated correlation matrix $P$ can be regarded as a structural model with $γ = ρ$ . Then, $\partial ρ / \partial γ^{'} = I$ , and it follows from Equation 26 that

A_{ρ ρ} = \frac{1}{2} G^{'} (P^{- 1} \otimes P^{- 1}) G and A_{γ γ} = {\dot{ρ}}^{'} A_{ρ ρ} \dot{ρ} .

Now we turn to matrix $A_{γ d}$ . It follows from Equation 9 that the element of $A_{γ d} = (a_{j k})$ corresponding to the jth parameter in γ and kth parameter in $d$ is given by

\begin{array}{l} a_{j k} {= 2}^{- 1} tr [Σ^{- 1} D {\dot{P}}_{j} D Σ^{- 1} {(e_{k} {e^{'}}_{k}) P D + D P (e_{k} {e^{'}}_{k})}] \\ {= 2}^{- 1} tr [D^{- 1} P^{- 1} {\dot{P}}_{j} P^{- 1} D^{- 1} {(e_{k} e_{k^{'}}) P D + D P (e_{k} {e^{'}}_{k})}] \\ {= 2}^{- 1} [tr {{\dot{P}}_{j} P^{- 1} D^{- 1} (e_{k} {e^{'}}_{k})} + tr {D^{- 1} P^{- 1} {\dot{P}}_{j} (e_{k} {e^{'}}_{k})}] \\ = b_{j k} / d_{k}, \end{array}

where $b_{j k} = {e^{'}}_{k} {\dot{P}}_{j} P^{- 1} e_{k}$ . Let $ρ_{k}$ and $ρ^{k}$ denote the vectors in the kth column of $P$ and $P^{- 1}$ , respectively. Then

b_{j k} = \frac{\partial {ρ^{'}}_{k}}{\partial γ_{j}} ρ^{k} .

Similarly, we need to relate $A_{γ d} = (a_{j k})$ to the matrices $A_{ρ d}$ defined via Equation 13 and $B_{ρ}$ defined in Equation 15. For such a purpose, we next further examine the composition of $b_{j k}$ . According to the chain rule of differentiation, there exist

\frac{\partial ρ_{k}}{\partial γ_{j}} = \frac{\partial ρ_{k}}{\partial ρ^{'}} \frac{\partial ρ}{\partial γ_{j}} and b_{j k} = \frac{\partial ρ^{'}}{\partial γ_{j}} \frac{\partial {ρ^{'}}_{k}}{\partial ρ} ρ^{k} .

Let

B_{γ} = (\frac{\partial {ρ^{'}}_{1}}{\partial ρ} ρ^{1}, \frac{\partial {ρ^{'}}_{2}}{\partial ρ} ρ^{2}, \dots, \frac{\partial {ρ^{'}}_{p}}{\partial ρ} ρ^{p}),

which is a $p_{*} \times p$ matrix. Then

A_{γ d} = \frac{\partial ρ^{'}}{\partial γ} B_{γ} D^{- 1} .

Again, consider the saturated model $γ = ρ$ , then Equation 28 becomes $A_{ρ d} = B_{ρ} D^{- 1}$ , which is essentially Equation 14. Notice that $B_{γ}$ does not depend on γ and there exists $B_{γ} = B_{ρ}$ regardless of whether the model $P (γ)$ is saturated or restricted. Thus,

A_{ρ d} = B_{ρ} D^{- 1} = B_{γ} D^{- 1}, and A_{γ d} = {\dot{ρ}}^{'} A_{ρ d} .

Example 2. Considering the case with $p = 4$ and $p_{*} = 6$ . Let $0$ be a vector of $6$ 0s, and $∊_{j}$ denote the vector of length $6$ whose jth element is 1 and others are 0s. Then, with $ρ = (ρ_{21}, ρ_{31}, ρ_{41}, ρ_{32}, ρ_{42}, ρ_{43})^{'}$ , the Jacobian matrices $\partial {ρ^{'}}_{j} / \partial ρ$ , $j = 1, 2, 3, 4$ are given by $(0, ∊_{1}, ∊_{2}, ∊_{3})$ , $(∊_{1}, 0, ∊_{4}, ∊_{5})$ , $(∊_{2}, ∊_{4}, 0, ∊_{6})$ , $(∊_{3}, ∊_{5}, ∊_{6}, 0)$ , respectively; and the vectors $(\partial {ρ^{'}}_{j} / \partial ρ) ρ^{j}$ , $j = 1, 2, 3, 4$ are given by

(\begin{matrix} ρ^{21} \\ ρ^{31} \\ ρ^{41} \\ 0 \\ 0 \\ 0 \end{matrix}), (\begin{matrix} ρ^{21} \\ 0 \\ 0 \\ ρ^{32} \\ ρ^{42} \\ 0 \end{matrix}), (\begin{matrix} 0 \\ ρ^{31} \\ 0 \\ ρ^{32} \\ 0 \\ ρ^{43} \end{matrix}), (\begin{matrix} 0 \\ 0 \\ ρ^{41} \\ 0 \\ ρ^{42} \\ ρ^{43} \end{matrix}),

respectively. The column vectors in Equation 30 are identical to those of $B_{ρ}$ in Equation 15 at $p = 4$ .

Since, with a structured $P (γ)$ , the diagonal elements of $D$ are still free to vary, the information matrix $A_{d d}$ corresponding to a structured $P (γ)$ is the same as in Equation 17. Thus, it follows from Equations 17, 24, 27, and 29 that, for the matrix $A$ in Equation 24, the top-left submatrix of $A^{- 1}$ , corresponding to $A_{γγ}$ , is given by

\begin{array}{l} A^{γ γ} = (A_{γ γ} - A_{γ d} A_{d d}^{- 1} A_{d γ})^{- 1} \\ = {({\dot{ρ}}^{'} [A_{ρ ρ} - B_{ρ} {(I + H)}^{- 1} {B^{'}}_{ρ}] \dot{ρ})}^{- 1} \\ = ({\dot{ρ}}^{'} Ω^{- 1} \dot{ρ})^{- 1}, \end{array}

where the last equality sign is due to Equations 18 and 19.

Equation 31 implies that, for the ML estimator $\tilde{γ}$ with a single study, the Acov matrix of $\sqrt{n} \tilde{γ}$ is given by $Acov (\sqrt{n} \tilde{γ}) = ({\dot{ρ}}^{'} Ω^{- 1} \dot{ρ})^{- 1}$ , with Ω being defined via Equations 1 and 2. Let $\hat{Ω}$ be the estimator of Ω evaluated at the observed $r$ for the single study. It is easy to see that the GLS estimator of γ by minimizing

[r - ρ (γ)]^{'} {\hat{Ω}}^{- 1} [r - ρ (γ)],

also has an Acov matrix $Acov (\sqrt{n} \tilde{γ}) = ({\dot{ρ}}^{'} Ω^{- 1} \dot{ρ})^{- 1}$ . Since ML for a correlation structure with a single group still involves dealing with standard deviations and/or the constraint $diag (P) = I$ in the estimation process, GLS is a lot easier to compute than ML, and they have the same asymptotic efficiency.

ML Estimate $\tilde{γ}$ of a Structured Correlation Matrix With m Studies

In this section, we will show that the estimator of γ by ML for the structural model $P (γ)$ with m studies is asymptotically equivalent to those by ML-GLS and ${GLS}_{2}$ . The technique in the proof is similar to that in the fourth section but involves the result obtained in the fifth section.

With a structured $P (γ)$ across m studies, the discrepancy function derived from the ML method is also given by Equation 7 with $Σ_{t} = D_{t} P (γ) D_{t}$ , where γ is a vector of q parameters and

θ^{(m)} = (γ^{'}, d_{11}, d_{12}, \dots, d_{1 p}, \dots, d_{m 1}, d_{m 2}, \dots, d_{m p})^{'} .

Parallel to Equation 20, let

a_{i j}^{(m)} {= 2}^{- 1} E (\frac{\partial^{2} F_{m l}^{(m)} (θ^{(m)})}{\partial θ_{i} \partial θ_{j}}),

be the element in the ith row and jth column of the information matrix $A^{(m)} = (a_{i j}^{(m)})$ corresponding to minimizing $F_{m l}^{(m)} (θ^{(m)})$ in Equation 7 but with a structured correlation matrix that is common across the m studies. Note that the structure of each $F_{m l t} (θ_{t})$ is the same as the function $F_{m l} (θ)$ for a single study with a structured $P (γ)$ examined in the previous section. In particular, $a_{i j}^{(m)} = 0$ if the ith and jth parameters are the d parameters corresponding to different studies. Thus,

A^{(m)} = \sum_{t = 1}^{m} \frac{n_{t}}{N} A_{t},

with

A_{t} = (\begin{matrix} A_{γ γ} & 0_{12} & A_{γ t} & 0_{14} \\ 0_{21} & 0_{22} & 0_{23} & 0_{24} \\ A_{t γ} & 0_{32} & A_{t t} & 0_{34} \\ 0_{41} & 0_{42} & 0_{43} & 0_{44} \end{matrix}),

where $A_{γ γ} = {\dot{ρ}}^{'} A_{ρ ρ} \dot{ρ}$ is the same as for a single study given in Equation 27;

A_{γ t} = {A^{'}}_{t γ} = {\dot{ρ}}^{'} A_{ρ t} = {\dot{ρ}}^{'} B_{ρ} D_{t}^{- 1},

which is parallel to Equation 29 with $B_{ρ}$ being given in Equation 15; and

A_{t t} = D_{t}^{- 1} (I + H) D_{t}^{- 1}

is parallel to Equation 17, with $H = (h_{j k})$ being defined previously; $0_{12} = 0_{21}'$ , $0_{14} = 0_{41}'$ , $0_{22}$ , $0_{23} = 0_{32}'$ , $0_{24} = 0_{42}'$ , $0_{34} = 0_{43}'$ , and $0_{44}$ are zero matrices with subscript 1 corresponding to q rows or columns, 2 corresponding to $(t - 1) p$ , 3 corresponding to p, and 4 corresponding to $(m - t) p$ rows or columns.

It follows from Equations 32 and 33 that

A^{(m)} = (\begin{matrix} A_{γ γ} & h_{1} A_{γ 1} & h_{2} A_{γ 2} & \dots & h_{m} A_{γ m} \\ h_{1} A_{1 γ} & h_{1} A_{11} & 0 & \dots & 0 \\ h_{2} A_{2 γ} & 0 & h_{2} A_{22} & \dots & 0 \\ ⋮ & ⋮ & ⋮ & ⋱ & ⋮ \\ h_{m} A_{m γ} & 0 & 0 & \dots & h_{m} A_{m m} \end{matrix}),

which is parallel to the information matrix in Equation 23. Let

M_{γ} = (\begin{matrix} h_{1} A_{11} & 0 & \dots & 0 \\ 0 & h_{2} A_{22} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & \dots & h_{m} A_{m m} \end{matrix}) and C_{γ} = (\begin{matrix} h_{1} A_{γ 1} & h_{2} A_{γ 2} & \dots & h_{m} A_{γ m} \end{matrix}) .

Then, it follows from Equations 34, 35, and the inverse for a block matrix that the top-left submatrix of ${(A^{(m)})}^{- 1}$ , corresponding to $A_{γγ}$ , is given by

\begin{array}{l} A^{γ γ} = (A_{γ γ} - C_{γ} M_{γ}^{- 1} {C^{'}}_{γ})^{- 1} \\ = (A_{γ γ} - \sum_{t = 1}^{m} h_{t} A_{γ t} A_{t t}^{- 1} A_{t γ})^{- 1} \\ = [A_{γ γ} - B_{γ} {(I + H)}^{- 1} {B^{'}}_{γ}]^{- 1} \\ = {({\dot{ρ}}^{'} [A_{ρ ρ} - B_{ρ} {(I + H)}^{- 1} {B^{'}}_{ρ}] \dot{ρ})}^{- 1}, \end{array}

which is identical to Equation 31. Thus, the Acov matrix of $\sqrt{N} \tilde{γ}$ obtained using the ML method for multigroup SEM is identical to that of $\sqrt{n} \tilde{γ}$ obtained by ML in a single study, which, together with the results in the previous sections, implies that ${GLS}_{2}$ , ML-GLS, and ML for MASEM are asymptotically equivalent.

Monte Carlo Results

In this section, we use the Monte Carlo method to examine the asymptotic results obtained in the previous sections. The purpose is to see whether the parameter estimates or their SEs converge to the same values as characterized by asymptotics. Our study includes both the combined correlation coefficients and the resulting structural parameter estimates of a follow-up MASEM.

The population is generated by a confirmatory factor model

y = μ + Λ ξ + ∊,

where $μ$ is a vector of population means,

Λ = (\begin{matrix} .40 & 0 \\ .50 & 0 \\ .60 & 0 \\ 0 & .55 \\ 0 & .65 \end{matrix}), Φ = Cov (ξ) = (\begin{matrix} 1.00 & .50 \\ .50 & 1.00 \end{matrix}),

$ξ \sim N (0, Φ)$ and $∊ \sim N (0, Ψ)$ are independent, with Ψ being a diagonal matrix and its elements being chosen so that $Σ = Λ Φ Λ^{'} + Ψ$ is a correlation matrix. Thus, the population values of $ρ = (ρ_{21}, ρ_{31}, ρ_{41}, ρ_{51}, ρ_{32}, ρ_{42}, ρ_{52}, ρ_{43}, ρ_{53}, ρ_{54})^{'}$ are given by

ρ = (.20,.24,.11,.13,.30,.1375,.1625,.165,.195,.3575)^{'} .

Let the structural model be the same as the two-factor model that generates the population. That is, the first three variables are loaded on the first factor with loadings $λ_{1}$ , $λ_{2}$ , and $λ_{3}$ , respectively; and the last two variables are loaded on the second factor with loadings $λ_{4}$ and $λ_{5}$ , respectively; the two factors are correlated with correlation $φ_{12}$ . Then the population values of the structural parameters $γ = (λ_{1}, λ_{2}, λ_{3}, λ_{4}, λ_{5}, φ_{12})^{'}$ are given by

γ = (.40,.50,.60,.55,.65,.50)^{'} .

All the samples share the same population distribution $N (μ, Σ)$ and consequently the condition of homogeneity holds.

There are two types of sample sizes in meta-analysis, one is the sample sizes in individual studies and the other is the total sample size. They both affect the performance of ${GLS}_{1}$ , but only the total sample size affects the performance of the other methods ( ${GLS}_{2}$ , ML-GLS, and ML) according to the results in the previous sections. Our design includes four conditions on the number of studies $m = 10$ , 20, 30, 50; and four conditions on the sample sizes in individual studies. For each value of m, the m sample sizes n_t, $t = 1$ , 2, $\dots$ , m are generated by taking the integer part of independent random draws from the uniform distribution $U [a, b]$ with $[a, b] = [30, 50]$ , [50,80], [30,100], [100, 300], respectively. We may regard the sample sizes within the range 30–50 as small, and those within the range of 100–300 as medium to large. The conditions on sample sizes allow us to see the performance of each method as n_t and m vary. In particular, the conditions $U [50, 80]$ and $U [30, 100]$ have the same expected value of $N = \sum_{t = 1}^{m} n_{t}$ but some of the n_t following $U [30, 100]$ may be smaller than those following $U [50, 80]$ . Such a design allows us to see how ${GLS}_{1}$ performs when the values of certain n_t are small, while the total sample size N remains about the same. For each set of sample sizes at a given m, 1,000 replications are used.

At each combination of conditions, m independently distributed samples with sizes n_t, $t = 1$ , 2, $\dots$ , m are drawn from the normally distributed population. The combined correlations $\hat{ρ}$ , $\bar{r}$ , and $\tilde{ρ}$ are obtained by ${GLS}_{1}$ , ${GLS}_{2}$ , and ML. Then we fit the structural model $ρ(γ)$ to $\hat{ρ}$ , $\bar{r}$ , and $\tilde{ρ}$ by GLS with a weight matrix equal to the inverse of the corresponding Acov matrix, respectively; and we also obtained the estimate of γ by ML. The empirical means and empirical SEs of each estimate of ρ and γ are then obtained across the 1,000 replications. Empirical bias of each estimate by each method is subsequently obtained. Note that we have 4 conditions of m, 4 conditions of n_t, 4 estimation methods, 10 ρ-parameters, and 6 γ-parameters, there would be 16 full pages of tables if we report the bias and SE of individual parameter estimate. Since a method may have a smaller empirical bias/SE for the estimate of the first parameter and a larger one for the estimate of the second parameter due to sampling errors, it is also rather difficult to examine the properties of each method according to individual estimate. We thus computed the averages of the empirical bias and empirical SE according to

\begin{matrix} {bias}_{ρ} = \sum_{j = 1}^{4} \sum_{i = j + 1}^{5} | {bias}_{ρ_{i j}} | / 10, S E_{ρ} = \sum_{j = 1}^{4} \sum_{i = j + 1}^{5} S E_{ρ_{i j}} / 10, \\ {bias}_{γ} = \sum_{j = 1}^{6} | {bias}_{γ_{j}} | / 6, S E_{γ} = \sum_{j = 1}^{6} S E_{γ_{j}} / 6, \end{matrix}

where ${bias}_{ρ_{i j}}$ and $S E_{ρ_{i j}}$ are the empirical bias and SEs of the estimate of $ρ_{i j}$ by one of the studied method, and ${bias}_{γ_{j}}$ and $S E_{γ_{j}}$ are the empirical bias and SE of the estimate of $γ_{j}$ by each method, respectively. In addition, for verifying the properties of asymptotic equivalence, we computed the averaged difference between SEs (DSE):

{DSE}_{ρ} = \sum_{j = 1}^{4} \sum_{i = j + 1}^{5} | S E_{ρ_{i j}}^{(1)} - S E_{ρ_{i j}}^{(2)} | / 10, {DSE}_{γ} = \sum_{j = 1}^{6} | S E_{γ_{j}}^{(1)} - S E_{γ_{j}}^{(2)} | / 6,

where $S E_{ρ_{i j}}^{(1)}$ and $S E_{ρ_{i j}}^{(2)}$ are the empirical SEs for the estimates of $ρ_{i j}$ by two different methods, and $S E_{γ_{j}}^{(1)}$ and $S E_{γ_{j}}^{(2)}$ are the empirical SEs for the estimates of $γ_{j}$ by two different methods. The averaged bias, SE, and DSE for the ρ parameters are reported in Table 1, and for the γ parameters are in Table 2. Each table also includes the information on the number of studies m, the range of sample sizes n_t, and the total sample size N. In particular, the tables are arranged according to the value of N from small to large in order for us to see the effect of total sample size. Because the values of the bias and differences between SEs are rather small, these are multiplied by 100 for us to see fine differences. For the same reason, each averaged SE is multiplied by 10 in the tables.

Table 1.

Empirical Bias ( $\times$ 100), Standard Errors (SE × 10), and Differences between SEs ( $\times 100$ ) of Different Estimates of ρ by ${GLS}_{1}$ ( $G_{1}$ ), ${GLS}_{2}$ ( $G_{2}$ ), and ML (M)

	Range of		Bias ( $\times 100$ )			SE ( $\times 10$ )			Differences Between SEs ( $\times 100$ )
m	n_t	N	${GLS}_{1}$	${GLS}_{2}$	ML	${GLS}_{1}$	${GLS}_{2}$	ML	$G_{1} - G_{2}$	$G_{1} - M$	$G_{2} - M$
10	30–50	425	4.463	.234	.211	.628	.471	.479	1.574	1.492	.082
10	50–80	651	2.653	.190	.104	.453	.379	.383	0.738	0.696	.042
10	30–100	711	2.432	.194	.082	.432	.361	.365	0.715	0.676	.040
20	30–50	776	5.349	.297	.167	.495	.350	.357	1.453	1.383	.070
30	30–50	1,168	5.512	.283	.191	.416	.286	.292	1.298	1.242	.056
20	50–80	1,337	2.812	.176	.104	.322	.264	.267	0.577	0.547	.030
20	30–100	1,350	2.865	.178	.101	.323	.262	.265	0.608	0.579	.029
30	30–100	1,755	3.594	.165	.149	.297	.228	.231	0.691	0.658	.032
10	100–300	1,815	0.907	.071	.063	.239	.223	.224	0.161	0.152	.009
50	30–50	1,927	5.725	.258	.230	.321	.218	.223	1.027	0.980	.047
30	50–80	1,939	3.003	.161	.121	.271	.219	.222	0.520	0.490	.029
50	50–80	3,196	3.102	.160	.129	.214	.172	.174	0.418	0.398	.021
50	30–100	3,353	3.053	.154	.122	.210	.169	.171	0.417	0.397	.020
20	100–300	3,972	0.854	.073	.039	.164	.154	.154	0.105	0.100	.005
30	100–300	5,919	0.872	.082	.037	.132	.125	.125	0.077	0.072	.004
50	100–300	10,235	0.856	.071	.033	.100	.094	.095	0.062	0.059	.003

Note. GLS = generalized least squares; ML = maximum likelihood

Table 2.

Empirical Bias ( $\times 100$ ), Standard Errors (SE $\times 10$ ), and Differences between SEs ( $\times 100$ ) of Different Estimates of γ by ${GLS}_{1}$ ( $G_{1}$ ), ${GLS}_{2}$ ( $G_{2}$ ), ML-GLS ( $M_{G}$ ), and ML (M)

	Range of		Bias ( $\times 100$ )				SE ( $\times 10$ )				Differences Between SEs( $\times 100$ )
m	n_t	N	${GLS}_{1}$	${GLS}_{2}$	ML-GLS	ML	${GLS}_{1}$	${GLS}_{2}$	ML-GLS	ML	$G_{1} - G_{2}$	$G_{1} - M_{G}$	$G_{1} - M$	$G_{2} - M_{G}$	$G_{2} - M$	$M_{G} - M$
10	30–50	425	5.160	.301	.713	.379	.934	.790	.795	.795	1.433	1.386	1.388	.049	.046	.016
10	50–80	651	3.130	.155	.404	.246	.702	.635	.637	.636	0.669	0.648	0.651	.024	.019	.011
10	30–100	711	2.869	.138	.372	.247	.670	.607	.609	.609	0.626	0.604	0.608	.023	.019	.009
20	30–50	776	5.739	.125	.529	.364	.707	.584	.587	.587	1.234	1.201	1.200	.035	.037	.009
30	30–50	1,168	5.743	.147	.391	.282	.577	.469	.471	.471	1.078	1.053	1.054	.033	.031	.007
20	50–80	1,337	3.085	.110	.253	.144	.477	.429	.430	.430	0.489	0.473	0.476	.019	.017	.007
20	30–100	1,350	3.139	.105	.246	.159	.481	.425	.426	.426	0.563	0.548	0.552	.017	.014	.006
30	30–100	1,755	3.821	.115	.277	.187	.430	.368	.369	.369	0.621	0.605	0.606	.018	.017	.004
10	100–300	1,815	1.078	.070	.141	.074	.374	.360	.360	.360	0.140	0.136	0.138	.005	.004	.003
50	30–50	1,927	5.858	.185	.365	.287	.445	.357	.359	.359	0.880	0.856	0.858	.025	.024	.003
30	50–80	1,939	3.230	.091	.243	.177	.400	.356	.358	.358	0.433	0.419	0.421	.015	.014	.002
50	50–80	3,196	3.272	.119	.207	.156	.314	.280	.281	.280	0.349	0.339	0.340	.011	.010	.001
50	30–100	3,353	3.222	.120	.195	.146	.306	.270	.271	.271	0.358	0.349	0.349	.010	.009	.001
20	100–300	3,972	0.960	.092	.110	.097	.256	.247	.248	.247	0.086	0.083	0.084	.003	.003	.001
30	100–300	5,919	0.961	.080	.075	.069	.206	.200	.200	.200	0.065	0.063	0.064	.003	.002	.001
50	100–300	1,0235	0.930	.080	.064	.060	.156	.151	.151	.151	0.055	0.054	0.054	.002	.002	.000

Note. GLS = generalized least squares; ML = maximum likelihood

The upper right part of Table 1 contains the SEs of the estimates of ρ by ${GLS}_{1}$ , ${GLS}_{2}$ , and ML. While the SEs under ${GLS}_{2}$ are slightly smaller than those under ML, the two columns of number are very close. As reflected by the numbrs in the lower part of Table 1, the difference in SEs between ${GLS}_{2}$ and $ML$ is in the fourth decimal place for all the studied conditions and is in the fifth decimal places when $N \geq 3, 972$ . This verifies the asymptotic equivalence between the two methods. In contrast, the SEs under ${GLS}_{1}$ differ more from the SEs under ${GLS}_{2}$ and ML than those between ${GLS}_{2}$ and ML, especially for smaller N. The numbers in the lower part of Table 1 further suggest that the difference in SEs between ${GLS}_{1}$ and ${GLS}_{2}$ or between ${GLS}_{1}$ and ML is strongly affected by the range of individual sample sizes (n_t). The differences in SEs among the three methods are in the fourth decimal place when $N \geq 5, 919$ and $n_{t} \in [100, 300]$ , which validate the conclusion that the three methods are asymptotically equivalent when all the n_t approach infinity.

Regardless of the values of m and n_t, the SEs under ML in the upper right part of Table 1 monotonically decrease with N. With only one exception (from $N = 1, 927$ to $N = 1, 939$ ), the SEs under ${GLS}_{2}$ also monotonically decrease with N. However, the size of the SEs under ${GLS}_{1}$ is clearly affected by the range of n_t in addition to N, and certain small individual sample sizes cause the estimates less efficient.

The upper left part of Table 1 contains the empirical bias of estimates for ρ by ${GLS}_{1}$ , ${GLS}_{2}$ , and ML. The bias under ${GLS}_{1}$ is much greater than those under ${GLS}_{2}$ and ML, while those under ${GLS}_{2}$ are very close to those under ML, which has the smallest value in bias. The values of the bias following none of the methods monotonically decrease with N, and all are adversely affected by the range of n_t. For example, the bias at $N = 1, 927$ (corresponding to $n_{t} \in [30, 50]$ ) by each method is much greater than the one at $N = 1, 815$ (corresponding to $n_{t} \in [100, 300]$ ).

The results in Table 2 for the estimates of γ with the structural model convey essentially the same information as by Table 1. In particular, the SEs under ${GLS}_{2}$ , ML-GLS, and ML in the upper right part of Table 2 monotonically decrease with N, whereas those under ${GLS}_{1}$ are also affected by the range of n_t. The differences in SEs between ${GLS}_{2}$ and ML-GLS, ${GLS}_{2}$ and ML, and ML-GLS and ML are in the fourth decimal place for all the conditions studied; and those between ML-GLS and ML are in the fifth decimal place when $N \geq 711$ . The differences in SEs between ${GLS}_{1}$ and the other three methods are a lot greater, especially when individual sample sizes are small (e.g., when $n_{t} \in [30, 50]$ ).

In summary, the Monte Carlo results indicate that, with a large N, ${GLS}_{2}$ , ML-GLS, and ML yield estimates having the same values in means and SEs for the first three or four decimals. The method ${GLS}_{1}$ also yields estimates very close to those by the other three methods as the smallest value of n_t increases.

Note that, although we only examined a confirmatory factor model in the simulation study in this section, the results in third section to sixth section are equally valid for other SEM models. Actually, the results in these sections were obtained without specifying the form of the model $P (γ)$ . The three methods (GLS, ML, and ML-GLS) would perform equally well empirically for other SEM models.

Violations of the Homogeneity Condition and Methods Under a Random-Effect Model

Our interest in the previous sections has been around methods of MASEM when the condition of homogeneity holds. In practice, the hypothesis $ρ_{1} = ρ_{2} = \dots = ρ_{m}$ is often rejected when tested using the statistics associated with the GLS or ML method. There are several reasons behind such a phenomenon. One is that the $ρ_{t}$ across the m studies do not equal while the raw data in all the studies are normally distributed. Another is that the $ρ_{t}$ are equal, but raw data in some or all of the m studies are nonnormally distributed. A third is that neither raw data are normally distributed nor the $ρ_{t}$ are equal across the studies. With only the sample correlations or covariances being reported in individual studies, it might be hard to distinguish the three scenarios in practice. Considering that normally distributed data are rare in practice (Blanca, Arnau, López-Montiel, Bono, & Bendayan, 2013; Cain, Zhang, & Yuan, 2017; Micceri, 1989), we suspect that the reported heterogeneity of $r_{t}$ in the literature is partially due to nonnormally distributed data. While the effects of nonnormality and heterogeneity might be confounded in the observed $r_{t}$ , when the number of studies m is not too small, an improvement in testing the homogeneity of $ρ_{t}$ is a rescaled statistic with the method ${GLS}_{1}$ (Yuan, 2016). Additional studies are needed for testing the homogeneity condition in practice when m is small.

Even when raw data in all the studies are normally distributed and the $ρ_{t}$ across the studies are different, we still need regularity conditions for combining different effect sizes. These include (1) there exists a well-defined underlying population of ρ from which different $r_{t}$ are drawn and (2) the studies are representative of the underlying population of ρ, or $ρ_{t}$ , $t = 1$ , 2, $\dots$ , m, can be regarded as a random sample from the population of ρ. When either (1) or (2) does not hold, there will be a logical issue with combining different studies. If the collected studies do not represent the underlying population well, the combined results may not be generalizable even if a large number of studies are combined.

Suppose $ρ_{t}$ , $t = 1$ , 2, $\dots$ , m, are different and they can be regarded as a random sample from a well defined population of ρ. Then, it is substantively proper to estimate the ρ, and methods have also been proposed for combining $r_{t}$ as well as for MASEM (Becker, 1992; Becker & Schram, 1994; Cheung, 2014). They rely on a model

r_{t} = ρ_{t} + e_{t} = (ρ + u_{t}) + e_{t}, t = 1, 2, \dots, m,

where $u_{t} \sim N (0, Δ)$ and $e_{t} \sim N (0, Ω_{t} / n_{t})$ are independent. Equation 36 is called a random-effect model because $ρ_{t}$ varies across studies. In contrast, the methods examined in third section to seventh section correspond to Equation 36 with $ρ_{t} = ρ$ or $Δ = 0$ , and the resulting model is called a fixed-effect model. Note that the ML method for the random-effect model based on Equation 36 is conceptually and methodologically different from the ML method for the fixed-effect model. In particular, the $F_{m l}^{(m)} (θ^{(m)})$ in Equation 7 is derived from the assumption that raw data in each study follow a multivariate normal distribution. Also, the covariance matrix $Ω_{t}$ obtained via Equations 1 and 2 is based on the assumption that the observed data in each study follow a normal distribution not the sample correlations.

Under the random-effect model in Equation 36, Becker (1992) described a method of moments for estimating Δ, using the sample covariance matrix of $r_{t}$ , $t = 1$ , 2, $\dots$ , m. The resulting $\hat{Δ}$ is then used to construct a weight matrix $W_{t} = (\hat{Δ} + {\hat{Ω}}_{t} / n_{t})^{- 1}$ in a GLS method for combining $r_{t}$ , where $\hat{Ω}$ is evaluated at $r_{t}$ according to Equations 1 and 2. The GLS method is to estimate ρ by minimizing

Q_{r} (ρ) = \sum_{t = 1}^{m} (r_{t} - ρ)^{'} W_{t} (r_{t} - ρ) .

With the random-effect model, Becker and Schram (1994) also described an EM algorithm to obtain the ML estimates of ρ and $Δ = Cov (ρ_{t})$ , based on the assumption $r_{t} \sim N (ρ, Δ + Ω_{t} / n_{t})$ . They also proposed to replace the $\hat{Δ}$ within the $W_{t}$ in Equation 36 by the ML estimate of Δ as an alternative GLS method. With unbalanced data, ρ and $W_{t}$ in Equation 37 need to be adjusted according to the observed elements in $r_{t}$ . Note that both the GLS and ML estimates of ρ under the random-effect model are approximately weighted averages of the observed $r_{t}$ , with weight ${(\sum_{t = 1}^{m} W_{t})}^{- 1} W_{t}$ for balanced data, and ${(\sum_{t = 1}^{m} A_{t^{'}} W_{t} A_{t})}^{- 1} A_{t^{'}} W_{t}$ when data are unbalanced, where $A_{t} = \partial ρ_{t} / \partial ρ'$ such that $ρ_{t} = A_{t} ρ$ . Also note that the estimates of Δ by both ML and the method of moments are consistent. Thus, GLS and ML methods for estimating ρ under the random-effect model are asymptotically equivalent. However, the equivalence needs the conditions for m to approach infinity in addition to all the n_t approaching infinity.

Under the random-effect model in Equation 36, Cheung (2014) described an ML-GLS method for MASEM. The ML estimate $\tilde{ρ}$ is obtained in the first stage, and the second stage is to fit the structural model $ρ(γ)$ to $\tilde{ρ}$ using GLS in which the weight matrix is the inverse of the covariance matrix of $\tilde{ρ}$ , obtained from the information matrix with the ML method based on $r_{t} \sim N (ρ, Δ + Ω_{t} / n_{t})$ . While GLS and ML methods for estimating $ρ(γ)$ under the homogeneity condition have been repeatedly studied, we are not aware of any documentation of GLS or direct ML for estimating a general structural model $ρ(γ)$ under the random-effect model. We might define them parallel to ${GLS}_{1}$ and ML that examined in the previous sections. That is, estimating the γ and Δ by maximizing the log likelihood function based on $r_{t} \sim N (ρ (γ), Δ + Ω_{t} / n_{t})$ , and by minimizing the quadratic function

Q_{r} (γ) = \sum_{t = 1}^{m} [r_{t} - ρ (γ)]^{'} W_{t} [r_{t} - ρ(γ)],

where $W_{t} = (\hat{Δ} + {\hat{Ω}}_{t} / n_{t})^{- 1}$ with $\hat{Δ}$ being estimated by the method of moments when data are balanced (Becker, 1992) or by the method of ML via the EM algorithm (Becker & Schram, 1994) with unbalanced data. Also, the vector $ρ(γ)$ and the matrix $W_{t}$ in the GLS function in Equation 38 need to be adjusted according to the observed elements in $r_{t}$ . Similar to estimating ρ, the GLS, ML, and ML-GLS estimates of γ under the random-effect model are approximately weighted averages of the observed $r_{t}$ , and they are asymptotically equivalent when m as well as all the n_t approach infinity.

Note that the interest of the random-effect model also includes the estimate of Δ, which gives the information as to what extent individual studies differ. Since the equivalence of ML, GLS, and ML-GLS under the random-effect model requires m and all the n_t approach infinity, the estimates by the three methods may not agree as well as their counterparts under the fixed-effect model if m or some of the n_t are not sufficiently large.

Discussion and Conclusion

Measurements in many fields do not have interpretable units, and analysis of correlations can yield more meaningful results. Like SEM, MASEM is a powerful tool for conducting various analyses and hypothetical testing on the relationship among observed variables based on the combined correlations. Under the homogeneity condition, methods for combining correlations are mostly GLS type e.g., (Becker, 1992; Hafdahl, 2007), which directly defines a method for further analysis of the combined correlations. Cheung and Chan (2005) proposed to first combine correlations using ML and then conduct the analysis of the combined correlations using GLS (ML-GLS). Oort and Jak (2016) proposed a direct ML method for MASEM. The three methods may seem to be fundamentally different. In this article, we showed that ML, ML-GLS, and the sample-size weighted average ( ${GLS}_{2}$ ) are asymptotically equivalent. The observed differences among the three methods are simply due to sampling errors or due to a not large enough total sample size N. We also showed that each of the three methods is also asymptotically equivalent to the GLS method described in Becker (1992), where the weight matrix for each study is estimated using the observed correlations of the single study. Our Monte Carlo study also showed that the three methods (ML, GLS, and ML-GLS) yield essentially the same result empirically.

The results of the asymptotic equivalence obtained in this article parallel that in covariance structure analysis where GLS and ML are asymptotically equivalent in conducting covariance structure analysis (Browne, 1974). A key condition for the asymptotic equivalence between GLS and ML in covariance structure analysis is that the model is correctly specified (Yuan & Chan, 2005). Similarly, a key condition for the asymptotic equivalence between ${GLS}_{2}$ and ML-GLS in combining correlations is homogeneity. Parallel to the results in Yuan and Chan (2005), ${GLS}_{2}$ and ML-GLS also yield estimators that have the same Acov matrix even when the normality assumption on raw data is not met. However, the covariance matrix of the combined correlations or the resulting estimates of the structural parameters is not consistently estimated by the inverse of the information matrix but a sandwich-type covariance matrix (see Yuan, 2016).

We noted that violations of homogeneity of correlations can be due to nonnormally distributed raw data and pointed out that a rescaled statistic associated with ${GLS}_{1}$ is preferred for testing the homogeneity condition. While the distribution assumption behind the ML method for the random-effect model is different from that behind the ML method for the fixed-effect model, we argued that ML, GLS, and ML-GLS under the random-effect model are also asymptotically equivalent. But the equivalence needs more conditions than that for the fixed-effect model. While the random-effect model allows us to evaluate the extent to which individual studies differ, the estimate of Δ by ML or the method of moments may be systematically biased unless raw data are normally distributed. This is because the estimates of Δ based on Equation 36 depend on the estimate of $Ω_{t}$ , and Equations 1 and 2 are obtained from a normality assumption. Thus, one needs to be cautions when intrepreting the results of the random-effect model. Regarding to the choice between random-effect and fixed-effect models in practice, Hunter and Schmidt (2004, p. 82) concluded: “If the variance of population correlations is small, then the weighted average is also always better. If the variance of population correlations across studies is large, then as long as sample size is not correlated with the size of the population correlation, the weighted average will again be superior.” The counterpart of the weighted average in MASEM is the method ${GLS}_{2}$ , which might be preferred in practice even if $ρ_{t}$ are not equal and raw data are normally distributed.

In practice, studies may not contain the same number of sample correlations. Note that the nature of unbalanced data with sample correlations might be different from missing values in raw data¹ from which $r_{t}$ is computed. Bias caused by an improper methodology in computing $r_{t}$ cannot be salvaged by an ML method in meta-analysis with either the fixed- or random-effect model. Because unbalanced data in $r_{t}$ are most likely created by design, we may assume that the mechanism is missing completely at random (MCAR, Rubin, 1976). Then ML-GLS and direct ML can proceed using the EM algorithm. For ${GLS}_{2}$ , one can use a modified formula to compute the sample-size weighted average (equation 3 in Yuan, 2016). While ML and ML-GLS yield asymptotically most efficient estimators under the normality assumption and MCAR mechanism, they are asymptotically equivalent to ${GLS}_{2}$ . Thus, ${GLS}_{2}$ is still preferred due to its simplicity in computation. Also, modified formula for estimating the covariance matrix of parameter estimates by ${GLS}_{2}$ has also been developed to account for violation of normality in raw data (Yuan, 2016). Additional studies are needed regarding the differences among the three methods when raw data contain missing values and $r_{t}$ are computed by ML rather than list-wise deletion.

The development of this article has implicitly assumed that the $r_{t}$ from different studies are based on the same variables. When different $r_{t}$ are based on different sets of variables, then it is a substantive question whether combining them is still meaningful or whether the obtained results is generalizable. Note that a statistical method does not know how the data are obtained. Under the homogeneity condition, ML, GLS, and ML-GLS are still asymptotically equivalent even when the $r_{t}$ are based on different variables. Under the condition of the random-effect model, the estimates by ML, GLS, and ML-GLS also converge to the same values as m and all the n_i increase.

The focus of this article is on the equivalence of GLS, ML-GLS, and ML in MASEM. Other issues with a correlation matrix from a single study have been studied in the SEM literature. They include whether treating a sample correlation matrix as a sample covariance matrix still generates asymptotically valid inferences (Bentler, 2007; Cudeck, 1989; Jennrich, 1970; Jöreskog, 1978; Krane & McDonald, 1978; Mels, 2000; Shapiro & Browne, 1990); how to obtain consistent SEs and more valid model inference with nonnormally distributed data (de Leeuw, 1983; Mooijaart, 1985; Savalei, 2014); and how different test statistics for correlation structures perform empirically (Fouladi, 2000; Huang & Bentler, 2015). Readers are referred to these studies for more information with specific focuses.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research was supported by the National Science Foundation under Grant No. SES-1461355.

Note

References

Bamberg

Möser

(2007). Twenty years after Hines, Hungerford, and Tomera: A new meta-analysis of psycho-social determinants of pro-environmental behavior. Journal of Environmental Psychology, 27, 14–25.

Becker

B. J.

(1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341–362.

Becker

B. J.

Schram

C. M.

(1994). Examining explanatory models through research synthesis. In Cooper

Hedges

L. V.

(Eds.), The handbook of research synthesis (pp. 357–381). New York, NY: Russell Sage Foundation.

Bentler

P. M.

(2007). Can scientifically useful hypotheses be tested with correlations? American Psychologist, 62, 772–782.

Blanca

M. J.

Arnau

López-Montiel

Bono

Bendayan

(2013). Skewness and kurtosis in real data samples. Methodology, 9, 78–84.

Browne

M. W.

(1974). Generalized least-squares estimators in the analysis of covariance structures. South African Statistical Journal, 8, 1–24.

Browne

M. W.

(1977). The analysis of patterned correlation matrices by generalized least squares. British Journal of Mathematical and Statistical Psychology, 30, 118–124.

Browne

M. W.

Shapiro

(1986). The asymptotic covariance matrix of sample correlation coefficients under general conditions. Linear Algebra and Its Applications, 82, 169–176.

Cain

Zhang

Yuan

K.-H.

(2017). Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence, influence and estimation. Behavior Research Methods, 49, 1716–1735.

10.

Cheung

M. W.-L.

(2014). Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46, 29–40.

11.

Cheung

M. W.-L.

(2015). Meta-analysis: A structural equation modeling approach. Chichester, West Sussex: John Wiley.

12.

Cheung

M. W.-L.

Chan

(2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40–64.

13.

Cheung

M. W.-L.

Hafdahl

A. R.

(2016). Special issue on meta-analytic structural equation modeling: Introduction from the Guest Editors. Research Synthesis Methods, 7, 112–120.

14.

Cudeck

(1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317–327.

15.

de Leeuw

(1983). Models and methods for the analysis of correlation coefficients. Journal of Econometrics, 22, 113–137.

16.

Fouladi

R. T.

(2000). Performance of modified test statistics in covariance and correlation structure analysis under conditions of multivariate nonnormality. Structural Equation Modeling, 7, 356–410.

17.

Gaertner

(2000). Structural determinants of job satisfaction and organizational commitment in turnover models. Human Resource Management Review, 9, 479–493.

18.

Hafdahl

A. R.

(2007). Combining correlation matrices: Simulation analysis of improved fixed-effects methods. Journal of Educational and Behavioral Statistics, 32, 180–205.

19.

Hedges

L. V.

Olkin

(1985). Statistical methods for meta-analysis. New York, NY: Academic Press.

20.

Hom

P. W.

Caranikas-Walker

Prussia

G. E.

Griffeth

R. W.

(1992). A meta-analytical structural equations analysis of a model of employee turnover. Journal of Applied Psychology, 77, 890–909.

21.

Hsu

P. L.

(1949). The limiting distribution of functions of sample means and application to testing hypotheses. In Neyman

(Ed.), Proceedings of the first Berkeley symposium on mathematical statistics and probability (pp. 359–402). Berkeley: University of California Press.

22.

Huang

Bentler

P. M.

(2015). Behavior of asymptotically distribution free test statistics in covariance versus correlation structure analysis. Structural Equation Modeling, 22, 489–503.

23.

Hunter

J. E.

Schmidt

F. L.

(2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.

24.

Jennrich

R. I.

(1970). An asymptotic

χ^{2}

test for the equality of two correlation matrices. Journal of the American Statistical Association, 65, 904–912.

25.

Jöreskog

K. G.

(1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409–426.

26.

Jöreskog

K. G.

(1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443–477.

27.

Krane

W. R.

McDonald

R. P.

(1978). Scale invariance and the factor analysis of correlation matrices. British Journal of Mathematical and Statistical Psychology, 31, 218–228.

28.

Lee

S. Y.

(1985). Analysis of covariance and correlation structures. Computational Statistics & Data Analysis, 2, 279–295.

29.

Mels

(2000). Statistical methods for correlation structures. (Unpublished doctoral dissertation). University of Port Elizabeth, Port Elizabeth, South Africa.

30.

Micceri

(1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.

31.

Mooijaart

(1985). A note on computational efficiency in asymptotically distribution-free correlational models. British Journal of Mathematical and Statistical Psychology, 38, 112–415.

32.

Nel

D. G.

(1985). A matrix derivation of the asymptotic covariance matrix of correlation coefficients. Linear Algebra and Its Applications, 67, 137–145.

33.

Olkin

Siotani

(1976). Asymptotic distribution of functions of a correlation matrix. In Ideka

(Ed.), Essays in probability and statistics (pp. 235–251). Tokyo, Japan: Sinko Tsusho.

34.

Oort

F. J.

Jak

(2016). Maximum likelihood estimation in meta-analytic structural equation modeling. Research Synthesis Methods, 7, 156–167.

35.

Pearson

Filon

L. N. G.

(1898). Mathematical contributions to the theory of evolution: IV. On the probable error of frequency constants and on the influence of random selection of variation and correlation. Philosophical Transactions of the Royal Society of London A, 191, 229–311.

36.

Rubin

D. B.

(1976). Inference and missing data (with discussions). Biometrika, 63, 581–592.

37.

Savalei

(2014). Understanding robust corrections in structural equation modeling. Structural Equation Modeling, 21, 149–160.

38.

Shapiro

Browne

M. W.

(1990). On the treatment of correlation structures as covariance structures. Linear Algebra and Its Applications, 127, 567–587.

39.

Steiger

J. H.

(1980). Testing pattern hypotheses on correlation matrices: Alternative statistics and some empirical results. Multivariate Behavioral Research, 15, 335–352.

40.

Steiger

J. H.

Hakstian

A. R.

(1982). The asymptotic distribution of elements of a correlation matrix: Theory and application. British Journal of Mathematical and Statistical Psychology, 35, 208–215.

41.

Viswesvaran

Ones

D. C.

(1995). Theory testing: Combining psychometric meta-analysis and structural equation modeling. Personnel Psychology, 48, 865–885.

42.

Yuan

K.-H.

(2016). Meta analytical structural equation modeling: Comments on issues with current methods and viable alternatives. Research Synthesis Methods, 7, 215–231.

43.

Yuan

K.-H.

Chan

(2005). On nonequivalence of several procedures of structural equation modeling. Psychometrika, 70, 791–798.

Meta-Analytical SEM: Equivalence Between Maximum Likelihood and Generalized Least Squares

Abstract

Keywords

Introduction

Overview of Existing Methods of MASEM

ML Estimate ρ ˜ With a Single Study

ML Estimate of ρ With m Studies

ML Estimate γ ˜ of a Structured Correlation Matrix With a Single Study

ML Estimate γ ˜ of a Structured Correlation Matrix With m Studies

Monte Carlo Results

Violations of the Homogeneity Condition and Methods Under a Random-Effect Model

Discussion and Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

Note

References

ML Estimate $\tilde{ρ}$ With a Single Study

ML Estimate $\tilde{γ}$ of a Structured Correlation Matrix With a Single Study

ML Estimate $\tilde{γ}$ of a Structured Correlation Matrix With m Studies