Confirmatory Latent Class Analysis

Person-Centered Methodologies Under an Inductive Versus Deductive Research Framework

For direct applications of finite mixture modeling, investigators do assume, a priori, that the overall population is heterogeneous, consisting of a mixture of a finite number of latent, and substantively distinct, homogenous subpopulations (latent classes). However, beyond the assumption of population heterogeneity that motivates use of a person-centered approach and theoretical rationale that may guide indicator selection and class interpretation, the class enumeration/model selection process to date has been primarily data-driven in the sense that constraints are not placed or formalized during model selection (Meyer & Morin, 2016). In organizational research as well as more broadly, appraisal of the applied literature quickly demonstrates that class enumeration typically involves statistical comparisons of a series of unconstrained k class models, with statistical fit the primary criterion for the smaller number candidate models selected for further theoretical interpretation. The best fitting class structure is frequently synthesized in the context of an established theoretical literature and hypotheses may be made a priori regarding the number or nature of the latent subgroups; however, in the absence of model constraints, there are limits to the specificity of hypotheses that may be made, as well as the risk of data exploration methods later being presented as confirmatory (Jebb et al., 2017). Subsequent confirmatory-type model constraints used for construct validation (i.e., latent class regression) are therefore based on a measurement model that was primarily data-driven. We are not critical of this standard approach and are not intending to imply that prior applications have been atheoretical. When person-centered analyses are applied for the first time within a given research context, empirically based decision making is arguably the more principled and conservative approach. Nonetheless, if deductive applications are characterized as those in which the entire domain of theory must be specified a priori (Jebb et al., 2017), then applications of person-centered approaches firmly fit toward the inductive side of the spectrum. As person-centered analyses grow in popularity within organizational research, fully exploratory methods may become inadequate for advancing the literature. Application of model constraints within LCA/LPA that move these techniques closer to a deductive/confirmatory realm may be useful for improved theory refinement, to facilitate empirical comparison of competing theories, and to formally guide replication and validation efforts (Jebb et al., 2017; Laudy et al., 2005).

There have been some informal attempts to extend beyond data-driven approaches to class enumeration and evaluation of class structure. For example, there exist review studies that aim to synthesize results of various LCA/LPA applications within a specific topic area, but these tend to be of a descriptive nature without clear empirical guidelines for aggregating findings. An inherent problem with synthesizing results across latent class analyses is that various studies have different methods, including different indicators (number, quality, and makeup of indicators), different sample sizes, and different covariates/predictors, all of which can influence class enumeration and model specification (Wurpts & Geiser, 2014). There has also already been some formal attention to applying confirmatory principles to categorical latent variable models (Hoijtink, 2001). McCutcheon (1987) provides the first description, of which we are aware, of potential modeling constraints that may allow for confirmatory tests of a latent class model, and an early application of LCA was confirmatory in that a two-class model was hypothesized and tested, rather than basing class enumeration on relative fit of k class models (Rindskopf & Rindskopf, 1986). Laudy et al. (2005) discuss how inequality constraints may be applied to LCA models to provide more focused empirical testing of competing theoretical models, and Finch and Bronk (2011) provide an overview of confirmatory LCA (CLCA), discuss its underutilization in research, and provide examples on how to set parameter constraints to conduct CLCA. In recent years, a handful of substantive studies have applied one or more types of modeling constraints to arrive at a CLCA model (Donovan & Chung, 2015; Hettema, Aggen, Kubarych, Neale, & Kendler, 2015; Monahan et al., 2015).

Despite these advances, confirmatory approaches to LCA/LPA have not been well-integrated into the applied literature, and strengths and limitations of various forms of model constraint setting have not been sufficiently addressed in the methodological literature. Indeed, the small number of substantive studies that have used CLCA each used differing methods for constraint setting and model evaluation, indicating a lack of standard practice. As confirmatory approaches to LCA/LPA are more frequently utilized and methods better understood, applied investigators may wish to decide whether to maintain the common practice of estimating a series of k class unconstrained measurement models, or if constrained or partially constrained models may be more appropriate (akin to the complexities of decision making within the continuum of exploratory to confirmatory factor models in structural equation modeling). Decisions about whether or not to apply modeling constraints require content knowledge within a specific substantive research area, though broad considerations include (a) data availability, such as whether a dataset is sufficiently large to validate one’s model using a split sample; (b) extant empirical work, such as whether past studies have performed person-centered analysis of the same construct with similar/identical indicators to allow externally informed, empirically driven model constraints; (c) explicit theoretical advancement, where the theoretical literature conceptualizes a construct with enough specificity to inform theoretically driven model constraints; and (d) study goal (e.g., validation or replication of prior results, theoretical model testing, hypothesis generating).

If sufficient conditions are met to warrant a confirmatory approach, then investigators must decide how to operationalize model constraints and interpret constrained models. This is not a trivial undertaking, given that there are many ways to approach constraint setting and an existing lack of standardization around placement of constraints and evaluation of constrained models. This article provides a comprehensive framework for operationalizing and evaluating model constraints. In the context of two analytic examples, we illustrate several methods for placing model constraints to facilitate movement toward a confirmatory framework for class enumeration and specification of class structure. The data illustrations are specific to LCA, though extensions to LPA are considered in discussion. The first approach was a dual sample analysis, in which a completely exploratory LCA was conducted in one sample and then validated in a second sample using modeling constraints. The second example demonstrated a confirmatory approach drawn from the same research context as the dual sample analysis, but tested a model structure that was hypothesized a priori. The goal of including two types of illustrations was to cast as wide of a net as possible in providing a set of tangible, illustrative examples of CLCA. Applying both approaches to similar data sets within the same research context allowed common ground in underscoring commonalities and differences between empirically driven model constraints versus those based on specific hypotheses.

Model Parameterization in LCA

We briefly review the parameterization of an unconditional LCA measurement model to facilitate illustration of potential model constraints. Consider M binary indicators, u ₁, u ₂, u ₃,…, u_M , where u_mi represents the response to item m for individual i. The M items are assessed as indicators of an underlying latent categorical latent class variable, c, with K classes (c_i = k if individual i belongs to Class k). The K classes representing the categorical latent variable are unordered, with π_k representing the proportion of individuals in Class k (Pr(c = k)). The overall joint distribution of the observed indicators is expressed as a finite mixture of the K classes, with each class having its own class-specific distribution for the indicators, as expressed by

Pr ( u 1 i , u 2 i , … , u M i ) = ∑ k = 1 K [ π k · Pr ( u 1 i , u 2 i , . … u M i | c i = k ] .

1

The K classes are exhaustive and mutually exclusive such that each individual has membership in only one of the latent classes and the sum of the π_ks is equal to 1. The relationship between each indicator and the categorical variable is typically parameterized as either class-specific indicator probabilities or as single thresholds (τ) on the inverse logit scale (such that negative threshold values are associated with higher probabilities that an item is endorsed within a class), as given by

Pr ( u m i = 1 | c i = k) = 1 1 + exp ( τ m k ) .

2

In the illustration, we demonstrate several methods of placing parameter constraints on model thresholds. In this unconditional measurement model, the structural parameters are limited to the class proportions, parameterized as intercepts on the inverse multinomial logit scale. We don’t place constraints on these structural parameters, though devote greater attention to structural parameters in discussion. As a final note, this model parameterization assumes conditional independence, or that latent class membership explains all shared variance among the observed indicators. This assumption can be at least partially relaxed in some situations, a topic revisited in the discussion in the context of extensions to LPA.

Placement of Model Constraints to Facilitate a Confirmatory Structure

Parameter constraints are intrinsic to confirmatory modeling approaches, though a unique aspect of person-centered analyses is the myriad of ways in which constraints may be operationalized. We first distinguish between two primary sources informing the constraints—empirical or theoretical. We further distinguish constraints related to class enumeration (i.e., the number of classes) and constraints related to the classes structure (i.e., the nature of the classes). Class enumeration in a confirmatory framework avoids a fully data driven model selection approach, though confirmatory testing of the fit of a single, final model without considering plausible alternative models is likely impractical and potentially limiting given reliance on relative fit indices. Instead, our confirmatory approach to class enumeration still focused on model comparison, though within a narrower range (e.g., k, k-1, and k+1 classes), and in the case of the dual sample, convergence of model selection between samples. Class structure was operationalized as degree of homogeneity and separation among indicators, where homogeneity reflects similarity with respect to a given item response among individuals belonging to the same class and separation reflects lack of similarity in item response patterns between those belonging to different classes (Collins & Lanza, 2010; Masyn, 2013). Typically class-specific response probabilities of approximately >0.70 or <0.30 indicate high within-class homogeneity. High separation in LCA is reflected by odds ratios of item endorsement between classes of approximately >5.0 or <0.2. Each latent class does not need to be homogenous or separated with regards to every indicator. However, ideally, each class should be homogeneous with respect to at least one indictor and every class should be separated from every other latent class by at least one indictor.

We propose four types of constraints and suggest general guidelines for their use. Table 1 provides a description of these constraints, how they were operationalized in the current analyses for the dual sample and the hypothesis-driven example (as appropriate), whether/how they target class enumeration and/or class structure, and research scenarios where a particular constraint may be most appropriate. Table 1 underscores the sheer amount of decision making faced by applied investigators wishing to carry out CLCA. The strictest constraints, fixed thresholds, were those based on model validation principles that fixed threshold values to a precise estimate generated from the empirical example (see Masyn, 2013), an approach not deemed practical/appropriate for the hypothesis-driven example. Other threshold constraints ranged in level of strictness and targeted homogeneity and/or separation; the general principles for these constraints were the same across both data illustrations. We targeted item homogeneity by setting boundary constraints on threshold values, and targeted item (non)separation by constraining item thresholds to equality. Homogeneity and separation were also addressed simultaneously using boundary and equality constraints within the same model. All threshold constraints were generated empirically in the dual sample illustration and conceptually in the hypothesis-driven example, with the specific operationalization more explicitly described below in the context of each illustration. Finally, we show how LCA in a multiple group framework (see Finch, 2015; Morin, Meyer, Creusier, & Biétry, 2016) can be used to assess the convergence of an observed solution between samples.

OPEN IN VIEWER

Table 1.

Methods of Placing Empirically Driven and/or Hypothesis-Driven Threshold Constraints.

Constraint	Operationalization in Current Analyses		Analytic/Theoretical Considerations	Research Context Guidelines
	Example	Description
Fixed thresholds	Dual sample	All threshold (τ) values fixed to precise estimates obtained in exploratory sample	Attempt at full validation of primary model based on measurement validation standards Most strict/specific of all threshold constraint options	Requires dual sample data or datasets with highly similar indicators/research populations Validation/replication of class enumeration and class structure Too precise for theoretical model testing; limited utility for replication of published findings
Threshold boundary constraints	Dual sample	Boundary constraints placed when τ significantly different than zero in first sample Significantly positive τ bounded to be > the lower limit of the 95% CI; significantly negative τ bounded to be < upper limit of the 95% CI τ within 95% CI containing zero in first sample not constrained	Addresses item homogeneity Constraint set at lower or upper bound, but degree of homogeneity empirically estimated Heterogeneous indicators unconstrained If model observed threshold values occur at boundary, suggests boundary too rigid Least strict method of placing constraints; parameter estimates empirically estimated within prespecified range	Reasonable to extrapolate potential constraints from models obtained using divergent methods, indicators, or research populations Appropriate when homogeneity is observed empirically or when expected theoretically Partial confirmation of class structure (i.e., addresses aspects of configural replication) No formal confirmation of class enumeration, though k+1 and k-1 sensitivity checks recommended
	Hypothesized model	τ constrained to be >.70 or <.30 based on hypothesized homogeneity τ without strong homogeneity hypotheses freely estimated
Threshold equality constraints	Dual sample	τ constrained to equality if statistically equivalent in first sample Nonequivalent τ from first sample freely estimated No constraints made on absolute values of τ	Addresses item separation Equality constraints placed for lack of separation, but degree of separation empirically estimated Assumes lack of significant difference reflects equality Less strict than fixed threshold method but more strict than threshold boundaries	Reasonable to extrapolate potential constraints from models obtained using divergent methods, indicators, or research populations Appropriate when lack of separation is observed empirically or when expected theoretically Partial confirmation of class structure (i.e., addresses aspects of configural replication) No formal confirmation of class enumeration, though k+1 and k-1 sensitivity checks recommended
	Hypothesized model	τ constrained to equality when hypothesized to be similar between classes τ not expected to be similar freely estimated
Threshold equality and boundary constraints	Dual sample/hypothesized model	Combination of all equality and boundary constraints Any remaining values of τ freely estimated	Addresses both homogeneity and separation More strict than equality or threshold boundary constraints imposed individually	Reasonable to extrapolate potential constraints from models obtained using divergent methods, indicators, or research populations Appropriate when homogeneity/separation are observed empirically or when expected theoretically Confirmation of class structure No formal confirmation of class enumeration, though k+1 and k-1 sensitivity checks recommended
Multiple groups approach	Dual sample	Fit multiple group model with all values of τ constrained to equality between samples Fit multiple group model with all values of τ unconstrained between samples Compare fit between unconstrained and constrained models	Method of assessing convergence of model across subpopulations Simultaneous analysis of samples	Requires raw data from both/all groups May be used in dual sample context or for confirmation of model with identical indicators across subpopulations Not applicable to theoretical model testing or replication of published findings

Note: τ = threshold; CI = confidence interval.

Method

Data were from N = 2,733 graduate nurse residents nested within N = 50 organizations who participated in the University Health Consortium/American Association of Colleges of Nursing Post Baccalaureate Residency Program (Fink, Krugman, Casey, & Goode, 2008) in 2009. Nurses were measured as they were transitioning within the organization from a student role to that of a skilled employee. Nurses completed the Casey-Fink Graduate Nurse Experience Survey (Casey, Fink, Krugman, & Propst, 2004). The full scale consists of 24 items covering four domains that were previously identified, empirically and theoretically, by the scale developers: communication, patient safety, satisfaction, and support. For illustration purposes, we selected two representative items from each domain: 1. Communication: “I feel confident communicating with physicians” and “I feel comfortable making suggestions for changes to the nursing plan of care”; 2. Patient safety: “I am having difficulty prioritizing patient care needs” and “I feel I may harm a patient due to my lack of knowledge and experience”; 3. Satisfaction: “I am satisfied with my chosen nursing specialty” and “I feel my work is exciting and challenging”; 4. Support: “I feel my preceptor provides encouragement and feedback about my work” and “My preceptor is helping me to develop confidence in my practice.” Responses were dichotomized into endorsement (e.g., agree/strongly agree) versus nonendorsement (e.g., disagree/strongly disagree); negatively worded items were recoded so that item endorsement reflected positive behaviors. The data were randomly split into an exploratory dataset (n = 1,366) and a confirmatory dataset (n = 1,367). All latent class modeling was carried out using Mplus Version 7.4 (L. K. Muthén & Muthén, 2015), and annotated syntax files are provided in the online Supplemental Materials. The type = complex command was used to address nurses nested within organization, which corrects standard errors for nonindependence of observations using a sandwich estimator.

Results

Exploratory Approach

We estimated between 1 class and 7 class models, using multiple random perturbations of start values. All models converged, there were no red flags with regard to model identification, and the log likelihood value was replicated for all solutions. Model fit indices (shown in Table A1 of the online Supplemental Materials) overwhelmingly supported the 5-class solution. Based on measures of absolute and relative fit, and adequate classification precision, we interpreted the 5-class model results. Figure 1 depicts the model-estimated item response probabilities for each of the 8 indicators of the 5-class solution. The first class (“Lack Support”) was homogeneous with low endorsement rates for the satisfaction and support indicators and was well-separated from all other classes with respect to support. Class 2 (“Low Efficacy/Dissatisfied”) was homogeneous with low endorsement rates of the communication and satisfaction items and, to a lesser degree, of the safety items. Class 3 (“High Efficacy/Dissatisfied”) was homogeneous with high endorsement rates of the communication, safety, and support items and low endorsement rates of satisfaction items. Class 4 (“Satisfied and Supported”) was the most difficult class to characterize; this class was homogeneous with high endorsement rates of the satisfaction and support items, but heterogeneous in terms of communication and safety indicators. It was also not well-separated from most other classes with regard to high rates of support, nor from Class 5 with regard to high rates of satisfaction. Finally, Class 5 was homogeneous with high endorsement of items and was labeled the “Highly Effective” class. In summary, the 5-class model was supported as the optimally fitting model in relative and absolute terms, the average classification probabilities tended to be high, and overall the observed class structure was theoretically interpretable. We chose this as our final model for demonstration of confirmatory approaches to LCA in the second (split) sample.

OPEN IN VIEWER

Figure 1.

Model estimated item response probabilities based on the five-class unconditional LCA using exploratory (n = 1,366) and confirmatory (n = 1,367) subsamples.

Confirmatory Approaches

We tested a series of modeling constraints to confirm model fit and composition in the second subsample. These constraints were introduced at a conceptual level in Table 1, and coding details may be found in the annotated Mplus syntax as part of the online Supplemental Materials. The fixed threshold model constrained all thresholds in the confirmation sample to the precise corresponding maximum likelihood estimates from the 5-class model obtained using the exploratory sample. The threshold boundary model addressed homogeneity by placing constraints on threshold values in the confirmatory sample if the value was significantly different than zero in the exploratory sample. If the 95% CI did not include zero for a given threshold in the exploratory sample, significantly positive thresholds were constrained in the confirmatory sample model to be greater than the lower bound of the 95% CI and significantly negative thresholds were constrained in the confirmatory sample model to be less than the upper bound of the 95% CI. Equality constraints addressed separation by constraining class-specific thresholds to be equal if the difference between them was nonsignificant (at α = .05) in the exploratory sample. The fourth model included a combination of threshold boundary and equality constraints. Table 2 depicts results of model testing comparing the 5-class unconstrained solution in the confirmatory sample relative to the four proposed models with threshold constraints. The likelihood ratio χ² test associated with each model provides some indication of absolute fit, but relative fit of the model in comparison to competing models is critical for model evaluation. With the exception of the fully constrained fixed threshold model, all other equality and boundary constraints demonstrated good fit to the data, both at an absolute level and relative to the unconstrained model. Overall this lends support to a 5-class solution that is constrained in accordance with the results of the exploratory sample.

OPEN IN VIEWER

Table 2.

Comparisons Among Constrained and Unconstrained Models as an Empirical Test of the Stability of the 5-Class Solution Across Exploratory and Confirmatory Subsamples.

Model	LL	npar	Adjusted χ2 LR (df), p Value	Difference Test	AIC	BIC	CAIC	AWE
Confirmation sample
Unconstrained	–5506.74	44	205.19 (211), .60	—	11101	11331	11375	11781
Fixed thresholds	–5562.55	4	291.58 (251), .04	<.001	11133	11154	11158	11195
Boundary constraints	–5509.85	41	207.22 (214), .62	0.57	11102	11316	11357	11735
Equality constraints	–5516.57	31	218.92 (224), .58	0.39	11095	11257	11288	11574
Boundary and equality constraints	–5523.57	28	230.30 (227), .43	0.07	11103	11249	11277	11535
Exploratory sample
Unconstrained	–5360.38	44	172.36 (211), .98	—	10809	11038	11082	11488
Fixed thresholds	–5405.47	4	249.62 (251), .51	<.001	10819	10840	10844	10881
Boundary constraints	–5360.51	39	171.86 (216), .99	>0.99	10799	11003	11042	11401
Equality constraints	–5389.48	20	223.12 (235), .70	0.01	10819	10923	10943	11128
Boundary and equality constraints	–5393.00	20	228.53 (235), .61	0.001	10826	10930	10950	11135
Multiple groups framework
Thresholds unconstrained across groups	–12761.90	85	346.97 (426), >.99	—	25694	26196	26281	27124
Thresholds equal across groups	–12787.30	45	394.81 (466), .99	0.18	25665	25931	25976	26422

Note: LL = log likelihood value; npar = number of parameters; LR = likelihood ratio; AIC = Akaike information criterion; BIC = Bayesian information criterion; CAIC = consistent Akaike information criterion; AWE = approximate weight of evidence criterion.

With empirical confirmation using a dual sample approach, the goal is centered on convergence of a solution across samples, rather than the prioritization of the solution obtained in one sample over the other. Dual cross-validation methods are recommended as supportive of the final model in both samples (cf. Masyn, 2013). As an initial validation step, we carried out the same steps undertaken in the exploratory sample in the confirmatory sample. Model fit statistics for 1-class to 7-class solutions in the confirmation sample paralleled those for the exploratory sample in terms of a 5-class solution (Table A1 in the online Supplemental Materials), and provided initial evidence for replication of the class enumeration process in the confirmation sample. We next compared model constraints applied to the exploratory sample, using parameter estimates from the confirmatory sample. Table 2 also shows that each of the four constrained models demonstrated satisfactory absolute fit and, to some degree, better relative fit compared to the unconstrained model. As a final test of the convergence of fit between the confirmatory and exploratory samples, we drew from Morin et al. (2016) and Finch (2015), who demonstrated methods for comparing the similarity of class structures using a multiple groups approach. Sample was treated as a “knownclass” variable in Mplus and model parameters were estimated simultaneously between samples. The model where all thresholds were constrained to equality did not significantly differ from one where all thresholds were freely estimated (Table 2, p = .18), supporting model convergence between samples.

Brief Discussion

Using a large dataset that could be randomly split in half, we demonstrated a fully exploratory approach to LCA in the first sample, followed by several methods to confirm the obtained solution in the second sample. We demonstrated several methods for obtaining constraints empirically, if there is not sufficient theory to drive a confirmatory model structure. The first confirmation method was the most strict, placing precise parameter constraints on the confirmation sample using the observed threshold estimates from the exploratory sample. This fully constrained model was not unequivocally supported, though this method of placing specific parameter constraints was discussed in the context of measurement model validation (Masyn, 2013), and may therefore be too rigorous an approach when the focus is on theoretical confirmation of model structure and configuration. This may be a particularly high standard when investigators are attempting to confirm a model using two different data sources, rather than a split sample as we had here where variability was already reduced. We then presented alternative methods of model confirmation based on equality constraints and/or threshold constraints, as well as multiple group methods. Each of these approaches supported the 5-class model with similar structure in the confirmation sample. Despite taking a confirmatory approach in the second subsample, we fully estimated all 1-class to k-class (here, 7-class) models, given the necessity of using relative fit indices and because evaluating one’s final model in the context of plausible competing models strengthens confidence in the final solution. We used dual cross-validation techniques, though recent work demonstrating k-fold validation within mixture modeling is a promising future direction (Grimm, Mazza, & Davoudzadeh, 2016).

Method

The first data illustration approach assumed a sufficiently large dataset from which to create a split sample, or two or more distinct datasets with like indicators. There are instances where neither of these conditions may be met, yet the theoretical underpinnings of a model are advanced to a point to render a fully exploratory approach unsatisfying. An alternative CLCA approach is to set model constraints in accordance with hypotheses generated from a previously expounded theory or competing theories. This illustration used the same survey and general method as the first approach, with an independent sample of a randomly chosen subset of participants drawn from 2010 data collection (n = 1,483), as opposed to the first approach that drew from 2009 data. We used the same indicators as the dual sample illustration to more effectively draw parallels between commonalities and differences when constraints are empirically versus hypothesis-driven, but note that the hypothesized model presented here does not originate from published theory. However, the hypothesized structure was conjured independently from and naïve to the results obtained under the dual sample approach.

There are varying degrees of specificity that may drive a hypothesized model structure. It is realistic to approach the model with expectations about the number of classes manifest in the data, as well as with hypotheses about how these classes are characterized based on specific, distinguishable indicators. However, homogeneity and separation of all indicators across all classes is likely not tenable, nor possibly even theoretically useful, making it unrealistic in most applications to generate strong hypotheses for every class-specific measurement parameter. Constraints around structural parameters, here expected class sizes, are also typically not central to confirmatory measurement model testing. There are also decision points around each type of constraint to best match theoretical expectations. Finch and Bronk (2011) demonstrated several of these options, including equality constraints between indicators of two or more classes and values of one parameter relative to the values of another class (e.g., a given threshold may be half the value in one class than in another). Within these broader categories (equality/threshold), decisions around level of strictness are required. Investigators may wish to constrain two parameters to equality, may specify rankings among them, or may specify them as statistically equivalent, with a predefined notion of what constitutes equivalence. Parameters can be fixed at specific values, or may be specified in terms of expected range (e.g., item probability >0.80). Refer back to Table 1 for explication of threshold boundary constraints, equality constraints, and their combination, used in this illustration.

Results

We tested a series of models to evaluate the performance of model parameter constraints to arrive at our hypothesized model structure. We started with the unconstrained 4-class solution as a point of reference for our model constraints; in this model, we were conforming to the aspect of our hypothesized structure that suggested a 4-class solution, but were not placing additional constraints on model parameters. We then imposed threshold boundary constraints, equality constraints, and combination of the two to arrive at three possible confirmatory-type models. Given the importance of ruling out alternative model structures, we also estimated model performance for unconstrained 3-class and 5-class solutions, and we considered constraints on the 3- and 5-class models in accordance with the hypothesized structure. The 3-class model was constrained by removing the smallest class observed from the 4-class solution (following empirical examination of this solution); the 5-class model was constrained by placing threshold constraints on 4 of the 5 classes and allowing the 5th class to be freely estimated. This final model relaxes assumptions about class enumeration, but specifies a minimum of 4 classes that conform to hypothesized expectations.

Fit statistics for all models under consideration are presented in Table 3, and Figure 2 depicts model estimated item probabilities using either threshold or equality constraints. Table 3 shows that the unconstrained 4-class model demonstrated acceptable absolute fit to the data and may also be a reasonable candidate model in terms of relative fit. Although it is clear from Figure 2 that we were successful in specifying model constraints in accordance with our hypotheses, 4-class models with threshold and/or equality constraints demonstrated poor absolute and relative fit to the data, leading us to conclude low support for our hypothesized 4-class solution. The unconstrained and constrained 3- and 5-class models provide additional context for interpreting our hypothesized 4-class solution. The unconstrained 3-class model fit the data poorly, whereas the unconstrained 5-class model fit well. Fit of the 3-class model was significantly improved by fixing model parameters to match the three largest classes observed in the 4-class solution, lending some support to the proposed class composition. The 5-class solution with 4 of the 5 classes constrained in accordance with our original hypotheses is arguably the best fitting model. This provides partial confirmation of the hypothesized structure, where we confirmed expectations around having at least 4 classes that followed our expected structure, while allowing for additional population heterogeneity that was not specified a priori.

OPEN IN VIEWER

Table 3.

Model Fit Results for Hypothesized Confirmatory Model and Alternative, Competing Models (n = 1,483).

Model	Adjusted χ2 LR	df	p Value	AIC	BIC	CAIC	AWE
4 class, unconstrained	250.45	220	.08	11732	11917	11952	12278
4 class, boundary constraints	298.14	220	<.001	11795	11981	12016	12342
4 class, equality constraints	304.22	236	.002	11768	11869	11888	12065
4 class, boundary and equality constraints	314.45	236	<.001	11781	11882	11901	12078
3 class, unconstrained	377.54	229	<.001	11874	12012	12038	12280
3 class, smallest class removed	174.48	231	>.99	12182	12192	12194	12213
5 class, unconstrained	193.17	211	.81	11676	11909	11953	12363
5 class, 4 classes constraints	195.45	212	.79	11676	11904	11947	12347

Note: LR = Likelihood ratio; AIC = Akaike information criterion; BIC = Bayesian information criterion; CAIC = consistent Akaike information criterion; AWE = approximate weight of evidence criterion.

OPEN IN VIEWER

Figure 2.

Model estimated item response probabilities generated using threshold and equality constraints to match hypothesized model structure (n = 1,483).

Brief Discussion

We attempted several approaches for placing constraints to arrive at our hypothesized model structure, and we were able to demonstrate procedurally how one might undertake this process. However, there were also several indications based on absolute and relative model fit and model interpretation suggesting that our hypothesized model was at best only partially supported. In addition, although solutions were obtained for all models we presented, we encountered a greater number of difficulties in model estimation under this approach compared to the first illustration, where the best log likelihood value was less likely to be replicated and thoughtful consideration of start values was required. These difficulties in fully confirming a hypothesized model structure could be an artifact of this structure being generated for illustration purposes rather than from strong theoretical underpinnings, though we suspect that a majority of theoretically driven mixture models face challenges in this arena. Theoretical model testing requires balance between the specificity and the complexity of the theory. Even models that are highly grounded in theory may still encompass several unknowns (or weaker hypotheses) that may not be well captured through setting model constraints; on the flip side, even theories that lack specificity are still quite complex. The sweet spot between theory specificity and complexity may indeed be very narrow. Unless the proposed theoretical model is easily specified and interpretable, an investigator might find themselves in an exploratory or a partially confirmatory realm, at minimum to understand sources and explanations for misfit/estimation problems. This is not to imply that investigators should avoid the development of theoretically driven person-centered models, but careful consideration of theoretical specificity and principled evaluation of alternative models are necessary; partially confirmatory models may be a more realistic goal. Even when undertaking a fully confirmatory approach, we estimated a greater number of models than we would have under a strictly exploratory approach. Although one approach would be to simply compare the hypothesized k-class model that included threshold/equality constraints to the same k-class model without constraints, we posit such an approach is not fully adequate for ruling out plausible alternative model structures; instead we recommend examining models that include upper and lower bounds (e.g., at least k-1 and k+1), both unconstrained and then constrained in accordance with theory, as demonstrated here.