The Effects of Chance Correlations on Partial Least Squares Path Modeling

Prior Simulation Evidence on the Performance of the PLS Method

PLS is often presented as a component-based estimator for SEM. No explicit definition for component-based estimation is given in the literature, but there seems to be an implicit understanding that such estimation proceeds in two steps: (1) calculating latent variable scores as weighted linear combinations (composites) of the indicator variables and (2) estimating the relationships between the composites using separate regression analyses (Hwang, Malhotra, Kim, Tomiuk, & Hong, 2010; Lu, Kwan, Thomas, & Cedzynski, 2011; Tenenhaus, 2008). Using this definition, the most typical component-based SEM estimator would be regression with summed scales or factor scores. The distinct feature of PLS and the only difference between these more traditional analysis methods is the iterative indicator weighting system (Chin, Marcolin, & Newsted, 2003; Goodhue et al., 2012b), which is explained in more detail later in the paper.

Regardless of the chosen weighting system, a key weakness in forming composite variables of indicators that contain measurement error is that any estimates produced with these composites will generally be inconsistent, although there are special cases where consistency can be achieved (Lu & Thomas, 2008). The fact that PLS is not consistent is readily acknowledged in the PLS literature (e.g., Chin, 1998). Rather, the argument for PLS has been that there are situations where a consistent estimator (typically ML SEM) may not be available or may perform poorly because of small sample size or violation of assumptions, and in these cases PLS, while inconsistent, would be the best available alternative. In theory, there are indeed some cases where an inconsistent estimator with known bias may be preferable to an unbiased estimator if the bias is well understood and the variance of the biased estimator is small compared to the unbiased one (Lehmann & Casella, 1998, p. 84). However, one cannot categorically conclude that when ML SEM fails or performs poorly, PLS should be used instead because it is possible that one of the other alternative component-based estimation approaches (e.g., regression with summed scales or factor scores) would outperform PLS or that the data are just inappropriate for any statistical analysis.

The lack of consistency of PLS suggests that it should not perform well when compared against consistent SEM estimators. Table 1, which lists prior Monte Carlo studies of PLS, shows that this is indeed generally the case. Interestingly many of the papers also show that when compared to regression analysis with summed scales, PLS results are less biased, suggesting that PLS may have an edge over other component-based estimators. In a recent paper Goodhue et al. (2012b) argued that the difference in bias was not statistically significant, but this conclusion was based on inappropriate use of prediction intervals. When the data were reanalyzed with the appropriate tests, the difference was statistically significant (D. L. Goodhue, personal communication, April 10, 2012). In fact, all the studies that compare bias of regression with summed scales and PLS show that the latter is less biased. The analysis presented later in the paper shows why this happens, why this is the wrong thing to look at, and particularly why this cannot be used as evidence that recent criticism toward PLS would be wrong.

OPEN IN VIEWER

Table 1.

Studies Evaluating Partial Least Squares (PLS) With Monte Carlo Methods.

Study	Study Design						Number of Reported Results Where PLS Estimates Are as Follows:
				Experimental			Compared to Summed Scales:		Larger for
	Conclusions About PLS	Tested Model(s)	Parameter Values	Conditions	Estimators	Analyzed Results	Less Biased	Further From 0	Small Samples
Aguirre-Urreta, Marakas, Ellis, and Sun, 2008	SEM outperforms PLS. PLS results were biased downward. The bias decreases with increasing number of indicators and decreasing sample size, and depends on the model.	2 exog. and 2 endog. LVs with 4 paths.	Loadings: .8 Paths: .2, .3, .4, .5	Sample: 40, 80, 120, 160, 200, 260, 320 MVs: 3, 5, 7, 9, 11	PLS (unspecified)	Bias, SEs, power	NA	NA	81/120
Chin and Newsted, 1999	Partial least squares always performed better (was less biased) than the simple summed regression approach, although it did best at lower numbers of indicators.	1 exog. LV and many endog. LVs (S1). 4 exog. LVs added in S2	Loadings: .2, .4, .8 Paths: .4	Sample: 20, 50, 100, 150, 200 Endog. LVs: 2, 4, 8, 12, 16 MVs: 4, 8, 12, 16, 32	PLS-Graph, Sum scale regression	Bias, SEs, power	99/125 (S1) 34/36 (S2)	125/125 (S1) 34/36 (S2)	81/100 (S1) 12/24 (S2)
Chin, Marcolin, and Newsted, 2003	PLS was less biased and had more statistical power than regression.	A 3 LV moderation model	Loadings: .7 Paths: .3, .5,	Sample: 20, 50, 100, 150, 200 500 MVs: 2, 4, 6, 8, 10	PLS-Graph, Sum scale regression, single item regression	Bias, SEs, power	34/36	36/36	18/60 (Dir. ef.) 19/30 (Interac.)
Ciavolino and Al-Nasser, 2009	GME is a valid alternative to PLS. GME outperform the PLS in terms of MSE in some respects.	2 exog. LVs and 3 endog. LVs with 6 paths.	Loadings: .3, .6 Paths: .25, .35, .45, .6, .7, .8	Sample: 10, 20, 40 Data: ideal, missing data, outliers, multicollinearity	PLS, GME	MSE	NA	NA	NA
Goodhue, Lewis, and Thompson, 2012a	PLS estimates were close to regression estimates and the advantage in smaller bias was not statistically significant. Both PLS and regression were less accurate than SEM.	4 exog. LVs and 1 endog. LV. (S1-3). 3 exog. LVs and 4 endog. LVs with 9 paths. (S4)	Loadings: .6, .7, .8, .90 Paths: 0, .11, .27, .31, .33, .34, .36, .39, .48	Sample: 20, 40, 90, 250, 200	Sum scale regression, PLS-Graph, LISREL	Bias, SEs, power	15/20 (S1) 34/40 (S2) 7/32 (S3) 3/36 (S4)	19/20 (S1) 34/40 (S2) 26/32 (S3) 28/36 (S4)	9/16 (S1) 12/20 (S2) 12/24 (S3) 16/27 (S4)
Goodhue et al., 2007	PLS is less biased than regression with summed scales, but has larger SEs, and thus provides less statistical power. This may be explained by capitalization on chance.	A 3 LV moderation model.	Loadings: .7 Paths: .3, .5	Sample: 20, 50, 100, 150, 200 500 MVs: 2, 4, 6, 8, 10	Sum scale regression, PLS-Graph	Bias, SEs, power	35/36	36/36	21/30
Henseler and Chin, 2010	The orthogonalizing approach to interactions is recommendable under most circumstances. The 2-stage approach has a higher statistical power, so it can be used if the orthogonalizing approach does not find a significant effect.	A 3 LV moderation model.	Loadings: .7 Paths: .3, .5	Sample: 20, 50, 100, 150, 200 500 MVs: 2, 4, 6, 8, 10	PLS (proprietary, implemented with R)	Bias, power, prediction accuracy	NA	NA	48/84 (Two-S) 46/84 (PLS-PI) 51/84 (Hybr.) 65/84 (Orthog.)
Hwang, Malhotra, Kim, Tomiuk, and Hong, 2010	The performance of PLS was relatively poor in parameter recovery compared with the other two approaches. The difference was particularly clear when the model was correctly specified.	A 3 LV mediation model. The population model contained three MV cross-loadings.	Loadings: .7, .21 Paths: .6	Sample: 100, 200, 300, 400, 500 Model specification: correct, mis-specified MV distr.: normal, non-normal	PLS, ML SEM, CSGA	Rel. bias, SD of estimates, MSE	NA	NA	NA
Lu, Kwan, Thomas, and Cedzynski, 2011	PLS is positively biased for small samples and negatively biased for large samples. Bias decreases with increasing number of indicators. PLS produces confidence intervals with incorrect coverage, but has the most statistical power.	1 exog. and 4 endog. LVs with 4 paths.	Loadings: .5, .7 Paths: .14, .45, .78	Sample: 30, 60, 120, 200, 300 Loadings: .5, .7: 3, 6, 12 Paths: .14, .45, .78	PLS-Graph, ML SEM, two factor score estimators.	Rel. bias, mean err., proper solutions, coverage, power	NA	NA	11/12
Qureshi and Compeau, 2009	PLS is more likely than SEM to detect a group difference with small samples and correlated exogenous variables. Both approaches work well with large samples and have difficulties with non-normal data.	2 correlated exog. LVs and 2 endog. LVs with 4 paths.	Loadings: .7, .8, .9 Paths: .05, .35, .6, .2. (2nd group paths were an experimental condition)	Sample: 100, 500 Between groups path diff.: .05, .15, .25, .35. .45 MVs: 3, 6 Exog. corr.: 0, .40 MV distr: normal, non-normal	PLS-Graph, ML SEM	Number of significant differences between groups detected.	NA	NA	NA
Reinartz, Haenlein, and Henseler, 2009	SEM is less biased and has smaller expected estimation error than PLS, but PLS has more statistical power.	1 exog. and 5 endog. LVs with 9 paths.	Loadings: .5, .7, .9 Paths: .5, .15, .3.	Sample: 100, 250, 500, 1000 MVs: 2, 4, 6, 8 MV dist: normal, non-normal Loadings: .5, .7, .9, .5/.9	PLS (proprietary), ML SEM	Bias, proper solutions, power.	NA	NA	NA

Note: Studies collected from the ISI Web of Science, Scopus, and AISel databases using the keywords “partial least squares” and “Monte Carlo” added with a list of studies compiled by Reinartz, Haenlein, and Henseler (2009). Excludes papers that were not Monte Carlo studies, discussed PLS regression instead of PLS path modeling, used population models that contained unrealistic error-free formative indicators, were conference papers that were later published in a journal, or were not available as full-text download. If a study contained several simulations, these are labeled with parentheses. LV = latent variable, MV = manifest variables (observed variable), SE = standard error, MSE = mean squared error, GME = generalized maximum entropy, CSGA = generalized structure component analysis, exog. = exogenous, endog. = endogenous.

When two latent variables are approximated with composites, the correlation between these approximations will underestimate the true correlation because of the well-known attenuation effect, which depends on the reliability of the approximations (Bollen, 1989, pp. 166-167; Cohen, Cohen, West, & Aiken, 2003, pp. 38-39; Goodhue et al., 2012b). The effects are more complex in multiple regression models, but on average random measurement error causes underestimation of the path coefficients, also in PLS (e.g., Gefen et al., 2011). The smaller bias in PLS estimates has been interpreted earlier as evidence of better reliability of the PLS composites (e.g., Chin et al., 2003), but it will be shown later that this is an incorrect interpretation.

A few of the reviewed papers noticed that there is something peculiar in the PLS estimates. Aguirre-Urreta, Marakas, Ellis, and Sun (2008) note that the PLS estimates tend to get larger when sample size decreases, but could not explain why this happens. Goodhue et al. (2007) noted that when analyzing interaction effects, the PLS estimates of the interaction term seem to be higher than regression with summed scales. They offered capitalization on chance as a possible explanation for this finding, but did neither provide any direct evidence to support this idea nor attempt generalize it outside the interaction effect. Their latest paper (Goodhue et al., 2012b), which is a more general criticism toward PLS, does not discuss capitalization on chance at all. Schermelleh-Engel, Werner, Klein, and Moosbrugger (2010) reinterpreted the data presented by Goodhue et al. (2007) and noted that capitalization on chance may explain the sampling distribution of the interaction effect, but they too do not attempt to generalize the effect outside interactions. Rönkkö and Evermann (2013) discuss the effect of correlated errors on PLS estimates and show that in a simple two-construct case, PLS seems to be sensitive to chance correlations resulting in decreased reliability of the latent variable estimates. While they argue that there is no reason to believe that the findings would not generalize, they too fail to provide direct evidence of capitalization on chance or any evidence of the generalizability of their analysis.

The existing studies listed in Table 1 contain several anomalies that could be explained by capitalization on chance. First, if PLS inflates path coefficients by capitalizing on chance, the PLS estimates should be further from zero than summed scales estimates that have fixed indicator weights and therefore cannot capitalize on chance. The second and third columns from right compare how often the PLS estimates are less biased and how often they were simply larger than summed scales estimates, showing that the latter is more often the case. Moreover, some studies, particularly the study by Chin and Newsted (1999), show results with positive bias in simple regressions. Simply increasing reliability of the composites to overcome the attenuation effect cannot result in this type of results, but the authors fail to note and explain these anomalies. Second, because chance correlations increase with decreasing sample size, the PLS estimates should increase when sample size decreases. I analyzed the result tables of the studies included in Table 1 by counting the number of simulation conditions in which the PLS estimates were larger than in the condition with the next higher sample size. These results are listed in the last column and they show that the effect is present to some extent in all of the studies except in the study by Chin et al. (2003). One possible explanation for this anomaly would be the effect of multicollinearity in their interaction model. This explanation is consistent with the fact that in a later study with the same data generation procedure (Henseler & Chin, 2010), the effect of increasing path estimates is strong when multicollinearity is eliminated through orthogonalization. Third, sometimes there may be a strong chance correlation to the opposite direction of the effect, which should cause the sampling distribution of the PLS estimates have a long tail and a secondary mode on the opposite side of zero. This effect is seen clearly in both of the studies by Goodhue et al. (2007, 2012b) and the study by Rönkkö and Evermann (2013), which are the only reviewed studies that presented any information about the shape of the distribution of the PLS estimates.

Reviewing the existing studies, there seems to be some indirect evidence of PLS capitalizing on chance although it has rarely been interpreted as such. Because the attenuation effect causes the parameters to be underestimated and capitalizing on chance correlations causes bias that is often toward the opposite direction, PLS indeed often has smaller bias than regression with summed scales. However, interpreting this as an advantage, like many of the reviewed studies did, is seriously misguided. The next two sections show how PLS capitalizes on chance correlations between the error terms in a general case and then show direct simulation evidence that this is a very strong effect, which makes PLS unsuitable for any type of statistical inference.

Formal Analysis of How and Why the PLS Method Capitalizes on Chance Correlations

The PLS weights and therefore also the path estimates are completely determined by the sample covariance matrix (Lohmöller, 1989; Rönkkö, 2013), which is a sample realization of the population covariance matrix of the indicator variables. For a common factor model, the population covariance matrix of the indicators is (cf., Bollen, 1989, p. 35):

Σ = Λ Φ Λ ′ + Θ ,

1

where Λ is a (p × n) matrix of factor loadings, Φ is a (n × n) covariance matrix of the latent variables, and Θ is the covariance (p × p) matrix of the error terms, where n is the number of latent variables and p is the number of indicators. Because the PLS model does not allow cross-loadings or correlations between error terms, I will only focus on the case where these assumptions hold in the population. Even in this case, the sample covariance matrix of the indicators, S, is never identical to the population covariance matrix Σ because of sampling variability, but approaches Σ when the sample size increases. In SEM this does not pose problems because the assumption about correct model pertains only to the population model, and it is explicitly acknowledged that the sample covariance matrix is different from the population covariance matrix and this discrepancy follows the χ² distribution after adjusting for sample size. Although it is commonly believed that PLS has less stringent assumptions than SEM, this is incorrect in the case of correlations between the error terms. The assumption that PLS makes is that the residuals are uncorrelated (Wold, 1982) and because residuals are sample realizations, this implies a much stronger assumption than the population model assumption made in SEM.

As explained earlier, PLS uses weighted composites as approximations for latent variables. The indicators are arranged as indicator blocks so that each block is associated with exactly one composite and each indicator belongs to exactly one block according to a (p × n) matrix of outer weights W. Starting with unit weights, the weights are adjusted iteratively in two steps called inner estimation and outer estimation until convergence. To simplify the equations, I will assume that all the variables, including the latent variables in the population, the observed data, and the calculated composites are always standardized. The correlation matrix between the weighted composites is

r C = W ′ S W .

2

During inner estimation, this matrix is used to calculate a (n × n) matrix of inner weights E. The PLS literature describes three alternative inner estimation schemes, but because these have been shown to produce nearly identical results (Noonan & Wold, 1982), I describe only the simplest one, the centroid weighting system. In this scheme the cell E_ij is set to sign of r C i j if two latent variables, η_i and η_j are connected with a regression path in the model, and zero otherwise. This leads to the diagonal of E being always zero. ¹ After this, a new set of composite variables are simply sums of the indicators weighted by a product of W and E. During outer estimation we calculate a correlation matrix between the new values of the composite variables and the indicator variables and use these correlations as new weights W. This is called Mode A estimation in the PLS literature and it is commonly used when the population model is assumed to be a common factor model (“reflective” in the PLS terminology). ² Because W and E are always scaled so that the resulting composites are standardized, we can write the correlations between the indicators and the composites as:

r x C = S W E ,

3

where r x C is a (p × n) matrix of correlations between indicators and composites. We can substitute S with the sample correlations between observed variables, latent variables, and error terms and rearrange as follows:

r x C = λ ϕ λ ′ W E + θ W E ,

4

where λ are the sample correlations between indicators and latent variables, φ are the sample correlations between latent variables, and θ are the sample correlations between the error terms. (Some effects of sample correlations between error terms and latent variables are omitted for clarity.) Equation 4 contains two terms, which I call structural effects term and error correlations term. Because the diagonal of E is zero, the correlations between indicators in the same block are ignored and only correlations across indicator blocks are used. In the asymptotic case, θ is a diagonal matrix. Each row of W has exactly one non-zero element and the elements on the diagonal of E are all zero, and therefore all the elements that are non-zero in W are zero in WE. Multiplying WE with a diagonal θ will simply scale each cell of WE with a scalar. As a result, the elements θ that are used in the indicator weighting process are zeros and thus the error correlations term has no impact on the indicator weights in the asymptotic case.

In the structural effects term, the factor loadings λ are multiplied by ϕ λ ′ W E , which is an n by n matrix. Because there are no cross-loadings, there is exactly one non-zero term on each row of λ in the asymptotic case, which means that each factor loading in an indicator block is scaled with the same scalar. In other words, in the asymptotic case the indicators are weighted proportionally to their factor loadings (Dijkstra, 1981, pp. 55, 73). This is different from principal component weights or the any of the commonly used factor score weights, but is instead a common way to erroneously calculate factor scores (Glass & Maguire, 1966).

In a finite sample, things get more complicated because the correlations between the errors are almost never exactly zero in a sample because of random sampling variations (θ is no longer diagonal). This results in indicators with error correlations in the same direction as the correlation caused by the latent variables receiving higher weights. Because the factor loadings are multiplied by ϕ λ ′ W E , but correlations between error terms only by WE, one unit difference in error correlations has a much higher impact on the indicator weights than one unit difference in factor loadings. In an extreme case where the latent variables are uncorrelated in the population, the indicator weights are almost completely determined by the off-diagonal elements of θ.

After the indicator weights are calculated, the correlation matrix between the composites, which completely determines the latent variables, is calculated according to Equation 2. Substituting S with the same sample correlations as earlier and rearranging yields:

r C = W ′ λ ϕ λ ′ W + W ′ θ W .

5

I will again call these two terms structural effects term and error correlations term. The equation shows that if r_C are used as estimates of the correlations between the latent variables (ϕ) there are two sources of error. In the structural effects term the W ′ λ is a (n × n) matrix of square roots composite reliabilities. ³ The product of square roots of two reliabilities is the coefficient of attenuation (cf. Cohen et al., 2003, pp. 55-56), which results in the well-known attenuation effect described earlier.

The second source of error is the error correlations term. When using equal weights, this second source of error increases the variance of the parameter estimates, but because the expected value of each error correlation is zero and they are mutually independent, the expected value of their sum is zero resulting in no bias. With PLS, the effect is different because the indicator weights depend on the error correlations, as shown in Equation 4. Consider again that a population path is positive. In this case, indicators with positive error correlations receive larger weights, and the indicators with negative error correlations receive smaller weights resulting in non-zero expected value and a positive bias in the results.

Monte Carlo Study of the Sampling Distribution of PLS Estimates

The previous described a potential problem in the PLS algorithm but did not address the magnitude of the problem. This section address the severity of the problem with a Monte Carlo simulation. The full R code for the study is included in Appendix 1, available online. Also, a picture is worth a thousand words, and the distribution plots that will soon follow communicate the problem in a much more clear way than the formal analysis presented in the previous section.

Simulation Design

Prior studies have argued that the structure of the estimated model can affect the bias (Aguirre-Urreta et al., 2008; Cassel et al., 2000) and the number of false positives (Goodhue et al., 2012b) in a PLS analysis. Therefore, to avoid capitalizing on idiosyncrasies of one particular model, I chose to generate a new, random population model for each simulation round. These random models consisted of six latent variables with eight paths reflecting typical empirical applications of PLS (Hair, Sarstedt, et al., 2012; Ringle et al., 2012). The path values were randomly set using values –.3, –.2, –.1, .1, .2, and .3 with equal probabilities. The measurement model varied according to the first two experimental factors listed in Table 2. The population values of the factor loadings were set by taking the factor loading based value and adding a random number from uniform distribution [–.1, .1] to each loading to simulate uneven reliability of the indicators. Although the population factor loading of .4 is low, such low values can be mistakenly accepted as reliable measurements because PLS severely overestimates low factor loadings (Rönkkö & Evermann, 2013).

OPEN IN VIEWER

Table 2.

Monte Carlo Simulation Design Factors and Values.

Design Factor	Values
Number of indicators	3, 4, 6
Factor loading base value	.4, .6, .8
Sample size	50, 100, 250
Estimated model	Correct, minor misspecification (1 path removed and 1 added), severe misspecification (4 paths removed and 4 added).

I generated the data by first generating a sample of latent variable true scores by drawing multivariate normal samples using a covariance matrix calculated from the population model. Then I generated the indicator base data by multiplying the latent variable true scores with the factor loadings and used each base data to generate two sets of indicator data. The original data were created by adding error terms drawn randomly from multivariate normal distribution with zero population correlations scaled so that the population variances of the indicators were one. The manipulated data were generated by taking the error terms used in the original data and orthogonalizing them maintaining their variances before adding them to the indicator base data. The manipulated data were thus in all respects identical to the original data with the exception that chance correlations between error terms were artificially removed.

I used each set of original data to estimate three models according to the fourth experimental factors: the correct model, which was identical to the population model, and two misspecified models that were created by removing paths and adding nonexisting paths randomly. This resulted in a full factorial design of 81 cells (3 × 3 × 3 × 3). For each cell, 500 independent replications were used. Each model was estimated with PLS, SEM, and regression analysis with summed scales. The simulation was conducted using the R statistical programming environment (version 2.15.2, R Core Team, 2013) using the plspm package (version 0.3.7, Sanchez & Trinchera, 2013) to for the PLS analysis and lavaan (version 0.5-12, Rosseel, 2012) for the SEM analysis.

Simulation Results

A key argument by the PLS proponents is that the way in which PLS weights the indicators does maximize the reliability of the composites or minimize the effect of random errors (e.g., Chin et al., 2003; Gefen et al., 2011), but there is no direct evidence that this would be the case (Rönkkö & Evermann, 2013). I tested this argument by comparing the squared correlations between the composites and the latent variable true scores that they were approximating. The unweighted summed scales were more reliable in every one of the 81 experimental conditions.

Then I compared the path estimates from these two methods and SEM. The PLS estimates were always less biased and always on average further from zero than estimates from regression with summed scales. The mean squared error (MSE), which measures the inaccuracy of an estimator (i.e., risk, Lehmann & Casella, 1998, pp. 5-7), was statistically significantly higher for PLS in 71 out of 81 conditions. In the remaining 10 conditions MSEs were identical to the third decimal between the methods. Comparing PLS and SEM revealed that SEM was almost always less biased than PLS but had larger MSE, particularly with smaller sample sizes. This result is expected because the ML estimator is inaccurate with small samples (Bollen, 1989). Comparing PLS estimates across conditions revealed that on average, the PLS estimates were further from zero with smaller samples. All these results are consistent with the earlier studies shown in Table 1 and support the argument that PLS capitalizes on chance. The full set of comparison tables is included in Appendix 2.

Figure 2, Figure 3, and Figure 4, showing the distribution of path estimates for PLS, regression with summed scales, and SEM for the cases in which the population parameter is 0 (i.e., no path exists in the population), has an absolute value of .1, or has an absolute value .3, respectively. The purpose of these analyses is not to show that chance correlations have an effect on the PLS weights per se, but to assess the impact that the chance correlations have on the path coefficients when the indicator weights have been calculated with realistic data where chance correlations exists in the sample. The distributions marked with solid lines are calculated using the original data, where the errors were uncorrelated in the population. The second set of results, marked with dashed lines, are calculated using the indicator weights from the analysis using original data but using the manipulated data, where the error terms were orthogonalized in the sample, to calculate the final composites. This is identical to removing the error correlations term from Equation 5 and can therefore be used to analyze how strongly chance correlations affect the results. Because SEM does not use indicator weights, estimating the effect of chance correlations this way was not possible for this method and therefore only one set of results is included for SEM.

OPEN IN VIEWER

Figure 2.

Distribution of the path estimates (kernel density estimates) when the population parameter is 0.

OPEN IN VIEWER

Figure 3.

Distribution of the path estimates (kernel density estimates) when the population parameter is .1.

OPEN IN VIEWER

Figure 4.

Distribution of the path estimates (kernel density estimates) when the population parameter is .3.

In Figure 2, the PLS estimates and the summed scales estimates are approximately unbiased because they are distributed evenly around zero. For the summed scales, the shape of the distribution is similar for both sets of data, but the variance of the estimates is slightly smaller when chance correlations between errors were removed. This scenario is expected because the error caused by chance correlations has a non-zero variance but has an expected value of zero. In contrast, with PLS composites, the variance of the estimates increases substantially, and the distribution changes into a bimodal form, which resembles the distribution reported for the two latent variable model by Rönkkö and Evermann (2013), but does not follow any known probability distribution. This is very problematic because virtually all current PLS studies use the method for null hypothesis significance testing of the path coefficients comparing the ratio of the parameter estimate and its standard error (t-statistic) with t-distribution. However, because the underlying distribution of parameter estimates is non-normal, the ratio of the parameter estimate and its standard error cannot follow the t-distribution regardless of how the standard error is estimated. Because the t-statistic does not follow the t-distribution, using the t-distribution to obtain a p-value, as is the current practice, is not appropriate (Rönkkö & Evermann, 2013).

Figure 3 and Figure 4 depict the same analysis for paths with an absolute value of .1 or .3 in the population, respectively. In these figures, the PLS estimates are slightly smaller than the summed scale estimates obtained when the effect of chance correlations between the error terms is eliminated from the path coefficients. This is expected because PLS composites were less reliable and the strength of the attenuation effect depends on reliability. With the original data, the effect of the error correlations biasing the PLS estimates away from zero, which was already observed in Figure 2, is again visible. When the true parameter value is small (absolute value is .1), the peak of the distribution is to the right of the true value, meaning that the parameters are mostly overestimated. With the larger population values (absolute value is .3), the peak of the PLS estimates is closer to the population value, but the paths are mostly underestimated. These results show that the widely held belief that PLS underestimates path coefficients (e.g., Chin, 1998) is incorrect because small path coefficients are clearly overestimated.

The figures show that in addition to the well-known attenuation effect, the PLS estimates suffer from also another source of bias that is to the opposite direction and these two sources of bias can in some instances cancel out each other. The question thus becomes is creating a new and thus far unknown source of bias the best way to compensate for the attenuation effect? There are at least three reasons why this is definitely not the case: First, the absolute size of the attenuation effect depends on the true value of the regression path and the factor loadings, whereas the bias caused by the error correlations does not, at least not directly. Second, the correction for attenuation should not depend on the strength of the error correlations, unlike the bias in the PLS results. Third, capitalizing on chance correlations prevents significance testing of path estimates because the sampling distribution of estimates corrected this way is no longer known. The problem of attenuation is well known and we have decades of research on how the bias can be corrected resulting in unbiased estimates without sacrificing the shape of the distribution of the estimates (Charles, 2005; Le, Schmidt, & Putka, 2009; Lu & Thomas, 2008; Muchinsky, 1996; Zimmerman, 2007; Zimmerman & Williams, 1997). I compared the PLS estimates with disattenuated regression estimates confirming that also in these simulation conditions the results from disattenuated regression were substantially less biased than PLS estimates. While the correction for attenuation is a statistically sound approach and has been shown to produce unbiased estimates in fairly general conditions, the resulting estimator is inefficient (Zimmerman & Williams, 1997) and SEM with latent variables has superseded this approach for many practical applications (Cohen et al., 2003, Chapter 12).

Discussion and Conclusions

Like other indicator weighting systems that can be used with regression analysis, PLS suffers from the well-known attenuation problem. In addition to this known problem, this article has showed that PLS suffers from an additional problem of amplifying the effect of chance correlations between error terms. While there is some earlier evidence that PLS does not have the advantages that the proponents of the method argue (e.g., Goodhue et al., 2012b; Rönkkö & Evermann, 2013), the conclusions of these studies have been relatively mild. For example, Goodhue et al. (2012b) concluded their paper by stating that “PLS is still a convenient and powerful technique that is appropriate for many research situations” (p. 999) and Rönkkö and Evermann (2013) merely state that their results cast strong doubts on the usefulness of the method and make it difficult to justify the use of the method.

Understanding the effects of chance correlations on the PLS estimates is also important because the proponents argue that while this may be a problem, it is limited in scope. “Capitalization on chance is indeed an issue, but is mainly limited to very small models with weakly related constructs,” but also continue that “It is pivotal to understand under which conditions this phenomenon occurs” (Rai, Goodhue, Henseler, & Thompson, 2013, p. 2). This paper presents results that hopefully bring about that pivotal understanding: The results show that the apparent advantage of PLS is a fallacy caused by ignoring the effect of chance correlations on the PLS results. These findings also provide an explanation for the positive conclusions about the statistical power of PLS that some studies Table 1 make. Because capitalization on chance makes PLS estimates non-normally distributed, these conclusions may just be results of inappropriate use of the t-test for non-normal data and not reporting false positives when doing so (Rönkkö & Evermann, 2013). It is rather unlikely that anyone would deliberately want to amplify the effect of chance correlations that are idiosyncratic to a sample in their study, particularly when this means also sacrificing the ability to test the statistical significance of the resulting parameter estimates.

The results also show that the existing attempts of fixing PLS by addressing its asymptotic properties and consistency (Dijkstra & Henseler, 2012; Dijkstra, 2014) are unlikely to succeed as long as they ignore an equally severe problem of amplifying the effect of chance correlations that only manifests in finite samples. The best way to eliminate this effect would be to substitute the model-dependent weighting system with equal weights or factor scores weights. In other words, the best way to fix PLS would be to abandon the PLS indicator weighting method altogether and instead focus improving existing component-based SEM methods that do not share the key weakness of PLS exposed in this paper.

The results raise two important questions: (1) On a higher level of abstraction, why does PLS perform poorly when it is applied to data originating from a common factor model? (2) How has the effect of amplifying chance correlations gone unnoticed for so long? The answer to the first question is that PLS was initially developed with a strong focus on prediction. In fact, in analyzing the work of Wold, the developer of PLS, it is difficult to find any recommendations for using the method for statistical inference. In contrast, Dijkstra, Wold’s student, concluded his dissertation on PLS by specifically recommending against using PLS for statistical inference (Dijkstra, 1981, p. 191). While PLS has been shown to produce construct scores that predict one another better than summed scales (Chin, 2010), prediction ability of a model and construct validity that is important when working with theory are two fundamentally different things (cf. Nunnally, 1978, Chapter 3). Thus, by maximizing prediction, PLS sacrifices construct validity and the ability to do null hypothesis significance testing.

The reason why this effect was not noted earlier is most likely because PLS, although used extensively in some disciplines, has not been extensively studied. For example, in Table 1, all but two studies were published in the past five years, and only three were published in research methods journals, the remainder of which were book chapters and papers in applied disciplines. In fact a review of several current beliefs about the PLS method by Rönkkö and Evermann (2013) indicated that many of the prevalent beliefs about the PLS method fit the definition of methodological myths and urban legends: Instead of being based on known statistical principles or rigorous simulation studies, most papers about the PLS method just repeat what earlier similar papers have stated. The fact that PLS has started to enter some of the mainstream textbooks about statistical analysis (Hair, 2010, Chapter 15; Kline, 2010, pp. 287-288; Rigdon, in press), makes these results that expose a previously unknown flaw in the PLS method even more important and timely.