How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models

Abstract

This contribution deals with effect measures for covariates in ordinal data models to address the interpretation of the results on the extreme categories of the scales, evaluate possible response styles, and motivate collapsing of extreme categories. It provides a simpler interpretation of the influence of the covariates on the probability of the response categories both in standard cumulative link models under the proportional odds assumption and in the recent extension of the Combination of Uncertainty and Preference of the respondents models, the mixture models introduced to account for uncertainty in rating systems. The article shows by means of marginal effect measures that the effects of the covariates are underestimated when the uncertainty component is neglected. Visualization tools for the effect of covariates are proposed, and measures of relative size and partial effect based on rates of change are evaluated by the use of real data sets.

Keywords

cumulative link models cumulative logits extreme categories marginal effects mixture models proportional odds rating data uncertainty visualization tools

Ordinal data models based on a rating procedure are common in different disciplines such as economics, marketing, medicine, and psychology, see, for example, Agresti (2010). Traditional methods for their analysis are generalized linear models that employ nonlinear link functions to cumulative probabilities (McCullagh 1980).

A recent literature deals with an alternative class of models that use a mixture distribution to mimic the decision-making process. More precisely, this class of mixture models considers the selection of a response category as a combination of a deliberate choice based on the preference of the respondent (preference component) and an uncertainty in the process of response (uncertainty component); see Piccolo et al. (2019), Piccolo and Simone (2019), Iannario and Piccolo (2016), Tutz et al. (2017) and reference therein. The preference component accounts for reasoned judgments toward the object/item under evaluation as well as the set of emotions, sentiments, and perceptions logically connected with it. The uncertainty component accounts for other unreasonable elements such as the unconscious willingness to please the interviewer and difficulty in expressing a rating regarding a specific object/item about which the interviewed has not a clear opinion.

In this contribution, we focus on Combination of Uncertainty and Preference of the respondents (cup) models recently introduced by Tutz et al. (2017). These models represent a special case in the framework of the GEneralized Mixture (gem) model with uncertainty (Iannario and Piccolo 2016); thus, the preference component is combined to the uncertainty one via a mixture.

Any model for ordinal data can be used to represent the preference component of a cup model. A natural candidate is the cumulative link model under the proportional odds assumption (McCullagh 1980), that is, the proportional odds model (pom). The pom is the most largely used model for the analysis of ordinal data; see, for example, Agresti (2010) and Tutz (2012) for a critical discussion.

For the uncertainty component, a uniform distribution is usually envisaged. Nevertheless, other distributions, instead of the Uniform one, maybe considered; supplement learning on this issue is in Gottard et al. (2016), Colombi et al. (2018), and Tutz and Schneider (2019).

In the present article, we focus on the standard cup model, which combines the pom with a uniform distribution. Its data generating process will be discussed in the following section. We present simple ways to interpret the effects of the covariates (continuous, discrete), and possible interaction, on the rating process. We introduce average and global marginal effect (ME) measures for cup, and we describe proper visualization tools. Furthermore, we deal with a way to face the somewhat challenging computational task of extracting the quantity of interest from regression results. We propose simple effect measures that could be used to evaluate how a change in the covariates affects the probability of the response categories, especially for the extreme values of the scale. Frequently, in fact, with ordinal responses, a special interest focuses on the highest and lowest response categories, the most extreme outcomes. In some specific fields, those categories represent a noteworthy state, such as the “best” or “worst” outcome (e.g., complete recovery vs. death). Indeed, as any explanatory variable on the observed rating increases, cumulative link models imply monotonicity in the extreme-category probabilities but not in the other probabilities. This status is merely altered by the uncertainty component shielding the monotonicity. Thus, to summarize the effect of the covariates on the observed rating, it can be useful to report the rate of change in the probability of an extreme response category, as a function of the covariates. Finally, following the approach of Long and Mustillo (2018) for binary data, we develop attractive graphical devices for ordinal data.

The plan of the article is as follows. In the second section, we discuss the data generating process that motivates the use of cup models, then in the third section, we present, with reference to a motivating example, the measures suggested to evaluate the effect of the covariates. In the same section, we also consider simple comparisons of the probability of extreme response outcomes at extreme values of an explanatory variable, measures of average rates of change of the extreme response probabilities, and group comparisons. Fourth section is devoted to the analysis of four different case studies for illustrating the use of the proposed measures and their implementation in different contexts. Some concluding remarks end the article while technical details and illustrative R code for the implementation of the proposed measures are confined in the Supplemental Material (which can be found at http://smr.sagepub.com/supplemental/).

Data Generating Process and CUP Models

Finite mixtures have been advanced by several authors for analyzing ordinal data; see Wedel and DeSarbo (1995), Greene and Hensher (2010), Grün and Leisch (2008), Breen and Luijkx (2010), Iannario and Piccolo (2016), among others. They are generally introduced for improving fitting accuracy, and they are motivated by the development of the data generating process. Finite mixture models for rating data can be broadly classified into two types.

– The models of the first type mimics the behavior of standard mixture models. Respondents can be considered as members of different clusters characterized by alternative rating procedures, each of which is described by a suitable probability distribution. The final mixture is given by a convex combination of these probability distributions.

– The rating assigned by an individual on a specific topic is the final outcome of a complex activity based on knowledge of the topic, instinct, and emotion of the individual. Hence, the mixture can be considered as a combination of the distributions of a discretized version of the underlying continuous latent variables describing these different components. This is the philosophy embraced by the gem model.

As previously mentioned, cup models are a special case of gem models and consequently fall within the second type of mixture models. According to gem models, there exist two components resulting from two separate approaches that respondents unconsciously combine in their mind in order to express their choice (preference and uncertainty component).

Original motivations for the selection of the random variables marking out the two components of the cup model were mostly based on a heuristic criterion. The choice of a cumulative model for the preference component is based on the will of using a classical ordinal response model well-known by researchers working on ordinal data, while the uniform distribution has been introduced as the most extreme among all discrete alternatives and accounts for the inherent uncertainty/heterogeneity of the choice. Furthermore, the uniform distribution was the one originally considered in the baseline cub model, combination of a discrete uniform and a shifted binomial random variable (Piccolo 2003), which motivated the gem structure.

The cup mixture allows to improve results with respect to the classical assumption of cumulative link models by means of the added value of the uncertainty component. The latter measures both subjective indecision and heterogeneity, and it is not related to the randomness, which is a concept related to sampling variability of surveys. A deeper discussion on this topic is provided in the paper by Tutz et al. (2017). In the same paper, authors show that mixture models with an uncertainty component typically present better fit and performance in terms of Akaike information criterion (AIC), Bayesian information criterion (BIC), and prognostic measures than standard ordinal models. More specifically, when the uncertainty component is neglected, the strength of the covariates tends to be underestimated. In addition, when uncertainty is very high, the study of the preference component without the assessment of the uncertainty one causes a loss of information and a misspecification of the model. Although cup models present some edges, the difficulty of a direct interpretation of parameters, analogously to the standard cumulative model, persists. This motivates the present contribution that establishes a useful and intuitive way for the interpretation of the effect of covariates on the cup models.

Now, we briefly recall the main characteristics of cup models; for more details, see Tutz et al. (2017). As previously mentioned in a cup model, the observed rating r is a realization of a random variable R modeled via a combination of an ordered response model and a discrete uniform distribution. More precisely, let R be a k-category ordinal response variable that represents the discrete measurement of two underlying (continuous) latent variables $Y_{i}^{*}$ and $U_{i}^{*}$ such that, for any ith subject, for $i = 1, 2, . . ., n$ , and for given (row) vectors of deterministic covariates $x_{i} = (x_{i 1}, . . ., x_{i j}, . . . x_{i p})$ and $w_{i} = (w_{i 1}, . . ., w_{i j}, . . . w_{i q})$ , affecting the preference and uncertainty component, respectively,

P (R_{i} = r | x_{i}, w_{i}) = π_{i} P_{M} (Y_{i} = r | x_{i}) + (1 - π_{i}) P (U_{i} = r), r = 1, 2, . . ., k

with $π_{i} = π_{i} (w_{i})$ .

The probability distribution of Y_i, that represents the discrete measurement of $Y_{i}^{*}$ , is determined by $P_{M} (Y_{i} = r | x_{i})$ which can be any ordinal model M, while a discrete Uniform distribution, $P (U_{i} = r) = 1 / k$ is used to represent the discrete measurement of the latent trait $U_{i}^{*}$ .

The subject propensity to adhere to a well-structured response behavior rather than to a random choice is modeled by mixing the previous components via the uncertainty parameter $π_{i}$ . The vector of covariates w _i may have a nonempty intersection with x _i . A logit link is usually applied to model the effect of covariates on the uncertainty component. That is, $\log i t (π_{i}) = β_{0} + w_{i} β$ with $β$ denoting the parameter vector for the uncertainty component. The choice of a logit link is not compulsory, but it is preferred to alternative ones for easiness of interpretation and robustness properties; see Iannario et al. (2017) for a detailed explanation.

In several implementations of cup models, covariates solely for the preference component are considered assuming a global level of uncertainty for the whole sample of respondents; hence, we set $π_{i} = π$ . In this case, the uncertainty level $(1 - π)$ , which is the weight of the uniform distribution assumed for the indecision in the responses, summarizes the global heterogeneity of the responses with respect to the examined rating variable. The parameter $π$ can be considered as the average of the individual parameters $π_{i}, i = 1, 2, . . ., n$ . See the examples Survey on Household Income and Wealth (SHIW) Data and American National Election Study (ANES) Data in the Case Studies section below.

As earlier pointed out and motivated, we describe the preference component of the cup model via a cumulative link model. We consider

P_{M} (Y_{i} \leq r | x) = F (α_{r} - x_{i} γ) 1 = 1, 2, . . ., n; r = 1, 2, . . ., k - 1

where $γ$ is the parameter vector for the preference component and $F (\cdot)$ is the cumulative distribution function whose common specifications are the normal distribution, the logistic distribution, and the extreme value distributions which correspond to the probit, logit, and log-log link (see Agresti 2010). In this article, we consider the logit one for the robustness properties previously mentioned.

Using the latent variable interpretation of cumulative link models, the preference component can be written as

P_{M} (Y_{i} = r | x_{i}) = P (α_{r - 1} < Y_{i}^{*} \leq α_{r}) = \{\begin{array}{l} F (α_{r} - x_{i} γ) & r = 1 \\ F (α_{r} - x_{i} γ) - F (α_{r - 1} - x_{i} γ) & r = 2, . . ., k - 1, \\ 1 - F (α_{r - 1} - x_{i} γ) & r = k \end{array}

where $- \infty = α_{1} < . . . < α_{k - 1} = + \infty$ are the thresholds of the scale of the latent variable $Y^{*}$ ; $Y_{i}^{*} = x_{i} γ + ∊_{i},$ for $i = 1, 2, . . ., n,$ and $∊_{i} \sim F (\cdot)$ . In this contribution, we assume that covariates have the same effect on the cumulative odds regardless of the category of the response

logit \{P_{M} (Y_{i} \leq r | x_{i})\} = log \{\frac{P_{M} (Y_{i} \leq r | x_{i})}{P_{M} (Y_{i} > r | x_{i})}\} = α_{r} - x_{i} γ .

For given x _i , the logit is altered only by the intercepts $α_{r}$ leading to the proportional odds assumption and then the considered pom. The standard cumulative model represents a special case of (1) with $π = 1$ .

Effect Measures for Covariates in Ordinal Data Models

For cup models, as a consequence of the nonlinearity of the considered link functions, model parameters are not as simple to interpret as slopes and correlations for ordinary linear regression. The model effect parameters, related to measures such as odds ratios, may not be easily understood or can even be misinterpreted. This enhances the need to introduce simpler ways to interpret the effects of the covariates. A recent paper (Agresti and Tarantola 2018) reviews different methods to easily interpret effects in cumulative link models. Following the suggestion of the previous authors, we evaluate the effect of each explanatory variable, on the preference and on the uncertainty component, using the so-called ME measures; see, for example, Greene (2008). These measures gauge how a change in a specific covariate $x_{i j}$ ( $w_{i j}$ ) affects the response variable when other covariates are fixed at certain values $x_{i \ j}^{*}$ ( $w_{i \ j}^{*}$ ). For more details, see, for example, Greene (2008), and see Greene and Hensher (2010) for the interpretation of ME in ordered response models.

We report below the ME measure of a continuous variable $x_{i j}$ involved in the preference component of the model. The ME on $P (R_{i} = r)$ is given by the partial derivative of $P (R_{i} = r)$ with respect to $x_{i j}$

{ME}_{{R_{i} = r, x_{i j}}} = \frac{\partial P [R_{i} = r | x_{i} = (x_{i j}, x_{i \ j}^{*}), w_{i}]}{\partial x_{i j}} = π_{i} \frac{\partial P_{M} [Y_{i} = r | x_{i} = (x_{i j}, x_{i \ j}^{*})]}{\partial x_{i j}} .

In equation (3), the partial derivative of $P (Y = r)$ with respect to $x_{i j}$ indicates the rate of change in $P (Y_{i} = r)$ with respect to $x_{i j}$ when other covariates are fixed at value $x_{i \ j}^{*}$ . It can be obtained as

\frac{\partial P [Y_{i} = r | x_{i} = (x_{i j}, x_{i \ j}^{*},)]}{\partial x_{i j}} = \{\begin{array}{l} - γ_{j} f (α_{r} - x_{i} γ) & r = 1 \\ - γ_{j} f (α_{r} - x_{i} γ) + γ_{j} f (α_{r - 1} - x_{i} γ) & r = 2, . . ., k - 1 \\ γ_{j} f (α_{r - 1} - x_{i} γ) & r = k \end{array},

where $f (\cdot)$ is the density function corresponding to the examined cumulative model.

If $x_{i j}$ is a categorical variable, we need to calculate the discrete change. For a dichotomous variable, it is given by

{ME}_{{R_{i} = r, x_{i j}}} = π_{i} [P (Y = r | x = (1, x_{i \ j}^{*})) - P (Y = r | x = (0, x_{i \ j}^{*}))] .

If the number of possible values is greater than two, the discrete change is computed as the difference in the predicted probabilities for cases in one category relative to the reference level.

We now describe how we can obtain the ME of a continuous variable $w_{i j}$ involved in the uncertainty component of the model. As mentioned, the effect of covariates on the uncertainty component is commonly obtained by means of the logit link; thus, the ME of $w_{i j}$ is given by

{ME}_{{R_{i} = r, w_{i j}}} = \frac{\partial P (R_{i} = r | x_{i}, w_{i} = (w_{i j}, w_{i \ j}^{*}))}{\partial w_{i j}} = \frac{\partial π_{i}}{\partial w_{i j}} (P_{M} (Y_{i} = r | x_{i} = x_{i}) - 1 / k) .

ME	Effect	Standard Error	z Value	p Value
$ME .1$
Gender	−.0623	.0082	−7.5593	.0000
Country	.0318	.0066	−4.7867	.0000
$ME .5$
Gender	.0358	.0045	7.9878	.0000
Country	−.0182	.0037	−4.9719	.0000

Case Studies

In this section, we introduce four case studies to motivate the need of the ME measures proposed in the previous section. The first example underlines the utility of the implementation and the analysis of the ME measures on the extreme categories when the examined model fits poorly for them; the second one shows the possible impact of the uncertainty component on ME measures of the preference one. The third example summarizes the impact of the interaction between covariates improving the main contents of Agresti and Tarantola’s (2018) paper. This example emphasizes as the interaction effects capture the impact of one explanatory variable on the ME measure of another explanatory variable. In fact, while in linear models the effect of a marginal change in the interaction term is equal to the interaction effect, this equality generally does not hold in nonlinear specifications. The last one reports the ME of nominal variables using a well-known data set regarding American presidential elections (Faraway 2006).

Survey of Health, Ageing and Retirement in Europe (SHARE) Data

We consider a set of data from wave 1 (2004) of the SHARE. It is a multidisciplinary and cross-national panel database of microdata on health, socioeconomic status, and social and family networks regarding individuals aged 50 or older. It covers 27 European countries and Israel. Up to now, seven survey waves have been collected within the period 2004–2017. The examined survey analyzes how different expectations and attitudes of the 3,458 respondents influence their quality of life. We analyze the rating assigned by the respondents on their pain perception. We use a rating on a five-point Likert-type scale (1 = no pain, 2 = mild, 3 = moderate, 4 = serious, and 5 = severe); covariates introduced for the analysis are Gender (0 = male, 1 = female) and Age (from $50$ to $80$ years old, with average = $62.15$ and SD = $8.24$ ). The observed distribution of the ratings is shown in Figure 1. It is characterized by low heterogeneity of the ratings, with the majority of the scoring classified in the first three classes (for reference, the case of a uniform rating is plotted as a dotted line).

Figure 1.

Relative frequency distribution of perceived pain—Survey of Health, Ageing and Retirement in Europe data.

Table 2 lists estimated parameters ${\hat{γ}}_{j}$ ( $j = 1, 2$ ) and cut points ( ${\hat{α}}_{r}, r = 1, 2, 3, 4$ ) with asymptotic standard errors in parentheses, and $BIC$ index for the preference component of cup models. The estimated weight is equal to $\hat{π} = 0.959$ with asymptotic standard error of 0.025.

Table 2.

The CUP Model Fitted to Perceived Pain Assessment—SHARE Data.

${\hat{γ}}_{1}$ (Gender)	${\hat{γ}}_{2}$ (Age)	${\hat{α}}_{1}$	${\hat{α}}_{2}$	${\hat{α}}_{3}$	${\hat{α}}_{4}$	$BIC$
$.6708 (. 0692)$	$.0213 (. 0041)$	$1.0167 (. 2685)$	$2.6136 (. 2821)$	$4.3771 (. 3431)$	$6.8675 (1.0200)$	$8, 958.0390$

Note: CUP = Combination of Uncertainty and Preference of the respondents; SHARE = Survey of Health, Ageing and Retirement in Europe. Asymptotic standard errors in parentheses.

Given the sign convention, as expected, it is possible to observe a slightly positive effect on the female (Gender = 1; they perceived more pain than male) and on increasing level of Age of perceived pain. Females have a higher perception of pain, and the older a person is, the higher is their pain perception.

Thus, the probability distributions for varying Age are plotted, given the value of Gender (Figure 2). Here, the weight of the uncertainty component is constant for the whole distribution; $P (Y = 1)$ and $P (Y = 5)$ have opposite trend since these probabilities decrease/increase with Age, respectively, for any value of Gender. Notice that the probability of a mild pain presents a swap of the two groups when Age = 65, and the probability of the last category (severe) is very low. Furthermore, a modification of Gender does not change the probability of the evaluation of the pain for the extreme values of Age. This is confirmed by the average ratings calculated at representative values of Age (minimum, mean, maximum), classified by gender. The results for the cup model are reported in Table 3.

Figure 2.

Probability of assessment for pain as function of Age for given Gender (male, dashed blue; female, solid red)—Survey of Health, Ageing and Retirement in Europe data.

Table 3.

Average Ratings Calculated at Representative Values of Age Classified by Gender—Survey of Health, Ageing and Retirement in Europe Data.

Age	Male	Female
$50$ (minimum)	$1.7783$	$2.0787$
$62$ (mean)	$1.8895$	$2.2041$
$80$ (maximum)	$2.0643$	$2.3938$

We then compared the observed ratings with the ones predicted by the cup model. For each individual, we consider the predicted value with the highest probability (the modal value). Indeed, using the modal value as a synthesis of the matrix related the individual prediction for each category of response, predicted versus observed ratings for the cup model are computed, and results are summarized in Figure 3. The percentage of correct responses for the investigated cup model is $37.4$ with a McFadden pseudo R² of $.508$ . A careful examination of this figure shows that the worst prediction of the models happens for last categories. The estimated model does not allow to predict categories highest than the second one; that is, the cup model is not able to predict response categories higher than that related to the mild pain (same similar results are for pom). Furthermore, when we implement the ME measures, we observe a not significant result for the last extreme category 5 (severe; Table 4). The inspection of Figure 2 suggests a similar behavior by showing predicted probabilities related to the last category (severe) close to zero for both female and male by varying age.

Figure 3.

Observed and predicted responses for the Combination of Uncertainty and Preference of the respondents model—Survey of Health, Ageing and Retirement in Europe data.

Table 4.

ME Measures for the CUP Model—SHARE Data.

ME	Effect	Standard Error	z Value	p Value
$ME .1$
Gender	−.1409	.0141	−9.9769	.0000
Age	−.0045	.0009	−5.1922	.0000
$ME .5$
Gender	.0038	.0030	1.1411	.2538
Age	.0001	.0001	1.1513	.2496

Note: ME = marginal effect; CUP = Combination of Uncertainty and Preference of the respondents; SHARE = Survey of Health, Ageing and Retirement in Europe.

In Figure 4, we report the receiver operating characteristics (ROC) curve for pain diagnostic following Tosteson et al.’s (1994) approach. The curve has been constructed using a dichotomized version of the observed ratings. It reports the false positive and true positive rates obtained as the decision threshold is varied on the different categories. Figure 4 shows that the last category ( $k = 5$ ) is overlapped with the no-discrimination line (the light blue two dashed line). The first category (black dashed line) is the one that discriminates more.

Figure 4.

Receiver operating characteristics curves for the diagnosis of perceived pain versus categorized perceived pain—Survey of Health, Ageing and Retirement in Europe data. Note: First category is the black dashed line. Second category is the red dot line. Third category is the green dot-dashed line. The fourth category is the blue long dashed line. The fifth category is the light blue two dashed line. Areas under the curves are not significantly different with exception of the area related to the first category. Category 5 is overlapped with 45 degrees orthogonal line (no-discrimination).

The poor fitting results in terms of prediction/realization tables summarized in Figure 3, the evidence of Figure 4, and the not significant value of the ME performed on the last category point out the usefulness of merging the last two categories. Collapsing categories may be also suggested by small frequencies in the last category ( $0.014$ ). Estimated results of this new cup model, obtained collapsing the last two categories, are reported in Table 5. New ME measures are in Table 6.

Table 5.

CUP Model Fitted to Perceived Pain Assessment With Collapsed Categories—SHARE Data.

Model	${\hat{γ}}_{1}$ (Gender)	${\hat{γ}}_{2}$ (Age)	${\hat{α}}_{1}$	${\hat{α}}_{2}$	${\hat{α}}_{3}$	$BIC$
CUP	$.6448 (. 0893)$	$.0220 (. 0043)$	$1.0166 (. 3063)$	$2.5824 (. 3429)$	$4.2200 (. 4327)$	$8, 694.7350$

Note: CUP = Combination of Uncertainty and Preference of the respondents; SHARE = Survey of Health, Ageing and Retirement in Europe. Asymptotic standard errors in parentheses.

Table 6.

ME Measures for the CUP Model With Collapsed Categories—SHARE Data.

ME	Effect	Standard Error	z Value	p Value
$ME .1$
Gender	−0.1395	.0193	−7.2150	.0000
Age	−0.0048	.0009	−5.0452	.0000
$M E .4$
Gender	.0461	.0057	8.0981	.000
Age	.0016	.0003	5.2862	.000

Note: ME = marginal effect; CUP = Combination of Uncertainty and Preference of the respondents; SHARE = Survey of Health, Ageing and Retirement in Europe.

The choice to collapse the last two categories enhances a discussion on the loss of efficiency (Iannario et al. 2021) because it also induces a loss of information which is reflected in larger asymptotic standard deviations (Johnson and Albert 1999; Whitehead 1993). Since the variance of the estimators is a decreasing function of k, the opportunity of merging categories should be carefully evaluated. In this case, the efficiency ratio between the before merging estimator and the after merging estimator of ${\hat{γ}}_{1}$ (Gender) and ${\hat{γ}}_{2}$ (Age) is around $1.0017$ and $1.0012$ , respectively; hence, the loss of efficiency can be considered negligible.

In Figure 5, we present the group comparison (male and female vs. Age) in terms of MER measures for the estimated model in Table 5. The top panel shows the trend of Gender MERs (for the lowest and the highest rates) at varying Age, and bottom panel shows Age MERs (for the lowest and the highest rates) at varying Age. The MER measures of the two groups are represented in different colors: male in blue and female in red. Figure 5 displays the highest perceived pain for females in all cases and in both the extreme categories. It also underlines the previously mentioned observation on the Age: Elderly people perceived highest pain.

Figure 5.

Group comparisons (male and female vs. Age) assuming marginal effects (MEs) at representative values of covariates (MER)—Survey of Health, Ageing and Retirement in Europe data. Note: First ME on left panel, last in right panel. Top panel is about Gender ME plotted for varying Age, and bottom panel shows Age ME plotted for different age and the two groups (male, blue; female, red).

We proceed with the analysis of this data set with collapsed categories of pain perception evaluating the effect of the introduction of a covariate on the uncertainty component. As a covariate, we consider the handgrip (Grip) which is a measure of physical functioning and a predictor of morbidity, disability, and mortality. Handgrip strength was measured twice on each hand using a dynamometer for describing the status of respondent’s energy. The results for the cup model are reported in Table 7.

Table 7.

CUP Model Fitted to Perceived Pain Assessment With Covariate on the Uncertainty Component—SHARE Data.

Preference component	${\hat{γ}}_{1}$ (Gender)	${\hat{γ}}_{2}$ (Age)	${\hat{α}}_{1}$	${\hat{α}}_{2}$	${\hat{α}}_{3}$
	$0.5457 (0.0881)$	$.0193 (. 0051)$	$0.8900 (. 3300)$	$2.6147 (. 3555)$	$4.7495 (. 5183)$
Uncertainty component	${\hat{β}}_{0}$ (Constant)	${\hat{β}}_{1}$ (Grip)			$BIC$
	$- 2.9442 (1.2827)$	$.1480 (. 0594)$			$8, 628.8040$

Note: CUP = Combination of Uncertainty and Preference of the respondents; SHARE = Survey of Health, Ageing and Retirement in Europe. Asymptotic standard errors in parentheses.

Before discussing the results, we would like to remind to the reader that the weight of the uncertainty component of the mixture is given by $(1 - π_{i})$ with $logit (π_{i}) = β_{0} + β_{1} Gri p_{i}$ ; thus, the performance of the in(de)creasing level of the variable is reversed with respect to the sign of the coefficient ${\hat{β}}_{1}$ . The uncertainty reduces for people with high Grip; that is, respondents with higher strength are resolved with respect the declaration of their level of perceived pain, whereas results of the variables related to the preference component remain almost the same. The new ME measures are in Table 8. It is possible to observe that the ME measures of Age are quite similar to the ones reported in Table 6, also with respect to the pom ( $- 0.0050 / 0.0020$ for the first/last category). This points out that the introduction of a continuous covariate in the uncertainty component does not affect the ME measures of the continuous covariates included in the preference component of the model.

Table 8.

ME Measures for the CUP Model With Collapsed Categories—SHARE Data.

ME	Effect	Standard Error	z Value	p Value
$M E .1$
Grip	.0012	.0001	5.9430	.0000
Gender	−.1228	.0149	−8.2515	.0000
Age	−.0043	.0009	−4.7795	.0000
$M E .4$
Grip	−.0035	.0001	−17.0001	.0001
Gender	.0201	.0051	3.9491	.000
Age	.0007	.0003	2.4505	.0143

Note: ME = marginal effect; CUP = Combination of Uncertainty and Preference of the respondents; SHARE = Survey of Health, Ageing and Retirement in Europe.

European Social Survey (ESS) Data

As a further illustration, we consider a set of data from round 5 (2010) of the ESS. The sample of 38,641 respondents is available at http://ess.nsd.uib.no/ess/round5/. The ESS is a cross-national survey that has been conducted across Europe on biennial basis since 2001. Face-to-face interviews were conducted to measure the attitudes, beliefs, and behavior patterns of the examined populations. We apply a cup model to describe self-perceived physical and mental health status collected on a five-point Likert-type scale (1 = very good, 2 = good, 3 = fair, 4 = bad, and 5 = very bad).

The observable variable in Figure 6 is analyzed by means of a cup model whose covariates affecting the preference component are Gender (male = 0, female = 1) and gross domestic products (GDP), whereas Age of respondents ranging between 14 and 98 years represents a relevant factor for explaining the uncertainty in the process of response. The observed distribution of the ratings is asymmetric with a modal value corresponding to score 2 = good (for reference, the case of a uniform rating is plotted as a dotted line). The estimates of the model parameters are reported in Table 9.

Figure 6.

Relative frequency distribution of perceived health—European Social Survey data.

Table 9.

CUP Model Fitted to Perceived Health—ESS Data.

Preference component	${\hat{γ}}_{1}$ (Gender)	${\hat{γ}}_{2}$ (GDP)	${\hat{α}}_{1}$	${\hat{α}}_{2}$	${\hat{α}}_{3}$	${\hat{α}}_{4}$
	$- 0.2613 (. 0209)$	$.1063 (. 0060)$	$- .9560 (. 0264)$	$.9623 (. 0272)$	$3.2468 (. 0443)$	$6.7040 (. 2314)$
Uncertainty component	${\hat{β}}_{0}$ (Constant)	${\hat{β}}_{1}$ (Age)				$BIC$
	$7.8287 (. 2450)$	$- 0.0980 (. 0036)$				$98, 628.2600$

Note: CUP = Combination of Uncertainty and Preference of the respondents; ESS = .European Social Survey; GDP = gross domestic product. Asymptotic standard errors in parentheses.

In this analysis, we focus on the role of the uncertainty component. In particular, we show how neglecting the effect of possible covariates we have an adverse effect on the MEs of the covariates employed to describe the preference component of the model.

Starting with a model without covariates affecting the uncertainty component with a log-likelihood of $- 49, 935.28$ , we then analyze the extended model in Table 9. The extended model presents a log-likelihood of $- 49, 271.88$ . The deviance (two times the log-likelihood ratio of the full model compared to the reduced model) is $1, 326.8$ with a p value lower than 0.001, and this motivates the use of model whose results are reported in Table 9.

Results show a negative effect on the GDP (respondents perceive lower physical and mental health condition when they are located in countries with higher GDP) and a critical health status for male. Age affects the uncertainty component increasing the quality of the answer (the certainty) for elderly people. The corresponding ME measures are reported in Table 10.

Table 10.

ME for CUP Models—ESS Data.

ME	Effect	Standard Error	z Value	p Value
$ME .1$
Age	−.0002	.0000	−13.6609	.0001
Gender	.0461	.0033	13.9946	.0000
GDP	−.0188	.0009	−20.2804	.0019
$M E .6$
Age	.0014	.0001	485.0718	.0001
Gender	−.0004	.0000	−4.4826	.0000
GDP	.0002	.0000	4.8575	.0001

Note: ME = marginal effect; CUP = Combination of Uncertainty and Preference of the respondents; ESS = European Social Survey; GDP = gross domestic product.

The ME measures of the covariates affecting the preference component along with the asymptotic confidence intervals are displayed in Figure 7. Here, in the left panel, ME measures of the model with a fixed uncertainty component are displayed (without covariates); in the right panel, instead, there are the ME measures of the model with a covariate on the uncertainty component. The analysis underlines the loss of information obtained when a model with covariates affecting only on the preference component is examined.

Figure 7.

Marginal effect measures for the first category (high perceived health status) and last category (low perceived health status) by varying Gender and GDP—European Social Survey data. Note: Left panel is about the Combination of Uncertainty and Preference of the respondents (CUP) model with a fixed uncertainty component. Right panel is about the CUP model with Age affecting the uncertainty.

Survey on Household Income and Wealth (SHIW) Data

The SHIW has been conducted by the Bank of Italy since 1965 to collect information on the economic behavior of Italian households and specifically to measure income and wealth components. The basic statistical unit is the household, defined as a group of individuals linked by ties of blood, marriage, or affection, sharing the same dwelling and pooling all or part of their incomes. Data collection is entrusted to a specialized company, and the interview stage is preceded by a series of meetings at which officials from the Bank of Italy and representatives of the company give instructions directly to the interviewers. The sample includes approximately 8,000 households and is drawn using a two-stage sample design. The questionnaire also collects information on demographics, consumption, savings, and several other topics. The number of validated observations for the empirical analysis of 2016 sample survey is 7,420 individuals. Among the several variables, the survey asks whether respondents consider their income sufficient to see the family through to the end of the month: This ordinal variable, named Family condition, ranges from 1 (with great difficulty) to 6 (very easily). Figure 8 shows the distribution of this variable, people mostly express an average expectation (mean, mode, and median = 3) about their perceived family condition.

Figure 8.

Relative frequency distribution of perceived Family condition—Survey on Household Income and Wealth data.

We analyze these data applying a cup model with a constant level of uncertainty and two covariates on the preference component: Cons the logarithm of consumption (enclosing different types of consumption such as food consumption, expenses for housing, health, insurance, spending on durable goods) and Child a dichotomous variable indicating the presence or not of children in the family. Furthermore, we include in the model an interaction term $Con s_{i} \times Chil d_{i}$ .

In particular, we consider the following latent model for the preference component

Y_{i}^{*} = γ_{0} + γ_{1} C o n s_{i} + γ_{2} C h i l d_{i} + γ_{3} (C o n s_{i} \times C h i l d_{i}) + ε_{i}, i = 1, 2 . . ., n .

Results of estimation are reported in Table 11 (with a low level on uncertainty $1 - \hat{π} = 0.05$ ), whereas Table 12 reports ME measures.

Table 11.

CUP Model Fitted to Perceived Family Condition—SHIW Data.

${\hat{α}}_{1}$	${\hat{α}}_{2}$	${\hat{α}}_{3}$	${\hat{α}}_{4}$	${\hat{α}}_{5}$
$18.0826 (. 5865)$	$19.2171 (. 5910)$	$20.8957 (. 6024)$	$22.8926 (. 6203)$	$24.7321 (. 6422)$
${\hat{γ}}_{1}$ (Cons)	${\hat{γ}}_{2}$ (Child)	${\hat{γ}}_{3}$ (Cons×Child)			$BIC$
$- 2.9664 (. 8658)$	$2.0780 (. 0610)$	$0.2019 (. 0867)$			20,997.370

Note: CUP = Combination of Uncertainty and Preference of the respondents; SHIW = Survey on Household Income and Wealth. Asymptotic standard errors in parentheses.

Table 12.

ME Measures for the CUP Model—SHIW Data.

ME	Effect	Standard Error	z Value	p Value
$ME .1$
Child	.321	.093	3.441	.001
Cons	−.233	.007	−32.188	.000
Inter	−.022	.009	−2.337	.019
$ME .6$
Child	−.055	.017	3.441	.001
Cons	.040	.003	32.188	.000
Inter	.004	.002	−2.337	.023

Note: ME = marginal effect; CUP = Combination of Uncertainty and Preference of the respondents; SHIW = Survey on Household Income and Wealth.

Given the sign convention, it is possible to observe a positive effect of Consumption (Cons) (respondents who are more likely to consume perceive higher family condition) and a decreasing effect on the presence of children. Thus, the probability distributions for varying Cons are plotted, given the value of Child (Figure 9); here $P (R = 1)$ and $P (R = 6)$ have opposite trend since these probabilities decrease/increase with Cons, respectively, for any value of Child with a peculiar performance for the intermediate categories $(k = 3, 4)$ . Notice that these probabilities present a swop of the distribution of two groups for highest level of consumption.

Figure 9.

Probability of assessment for perceived family condition as function of consumption for given child (no child, blue; having child, red)—Survey on Household Income and Wealth data.

American National Election Study (ANES) Data

Studies of presidential election attract increasing attention in the field of political, economic, and other social sciences. One of the traditional research questions is how party identification relates to respondents’ voting behavior (Bartels 2000; Miller 1991) or perception of their self left–right placement, an element influencing their behavior (Lesschaeve 2017).

We consider a set of data from the 1996 ANES project developed by the Institute for Social Research of the University of Michigan (https://www.icpsr.umich.edu/icpsrweb/ICPSR/series/00003). It regard voting preferences on Clinton political position. It is also available in the R package faraway (Faraway 2006).

The examined data frame consists of 944 observations. We analyze these data applying a cup model with a constant level of uncertainty and we consider only party identification (PID) as covariate on the preference part. PID is an ordered factor with five levels from “strong Democrat” to “strong Republican” (this is a revised version of the original classification).

The ordinal response variable is self left–right placement expressed on a seven-point Likert-type scale from 1 = extremely liberal to 7 = extremely conservative (Figure 10 shows the relative frequency distribution). Parameter estimation of the cup model is reported in Table 13. Here, it is possible to notice a coherence of party identification and perceived left–right placement. It means that respondents of Republican Party report a conservative position while Democratic respondents are oriented on liberal opinions. The global level of uncertainty in the process of assessing the self left–right placement is generally low ( $1 - \hat{π} ≃ 0.2$ ).

Figure 10.

Relative frequency distribution of self left–right placement—American National Election Study data.

Table 13.

CUP Models Fitted to Self Left–Right Placement—ANES Data.

${\hat{α}}_{1}$	${\hat{α}}_{2}$	${\hat{α}}_{3}$	${\hat{α}}_{4}$	${\hat{α}}_{5}$	${\hat{α}}_{6}$
$- 3.4453 (. 2890)$	$- 1.1767 (. 1158)$	$0.0260 (. 1038)$	$1.7081 (. 1430)$	$2.9066 (. 1797)$	$5.6066 (. 3217)$
${\hat{γ}}_{1}$	${\hat{γ}}_{2}$	${\hat{γ}}_{3}$	${\hat{γ}}_{4}$
(Indep-Dem)	(Indep-Indep)	(Indep-Rep)	(Strong Rep)		$BIC$
$0.4133 (. 1999)$	$0.9662 (. 3122)$	$2.3300 (. 2389)$	$3.1533 (. 2153)$		2,889.1540

Note: CUP = Combination of Uncertainty and Preference of the respondents; ANES = American National Election Study. Asymptotic standard errors in parentheses.

Figure 11 reports the probability of the extreme categories of the ordinal variables given PID by showing the opposite trend for the two extremes. It illustrates a lower heterogeneity in the last two categories (independent-Republican and strong Republican) when the response is “extremely conservative’ and for the first one (independent-Democrat) when the response is “extremely liberal.” A comparatively highest weight of the ME for the lowest value of party identification (Democrat) when extremely liberal is scored and for the highest value of party identification (Republican) when extremely conservative is selected is reported in Table 14. Figure 12 illustrates the effects of ME measures on the extreme values of the ordinal scale for the two models (pom [dots] and cup [diamonds]) by stressing the role of the uncertainty. We can notice that neglecting the uncertainty (pom), we obtain ME measures very close to each other when the extremely conservative score is considered (the blue dots present very similar values).

Figure 11.

Individual marginal effects at representative values of covariates ( $M E R$ ) for the first category (extremely liberal)—upper panel—and last category (extremely conservative)—bottom panel—by varying party identification—American National Election Study data.

Table 14.

ME for the CUP Model—ANES Data.

ME	Effect	Standard Error	z Value	p Value
$ME .1$
Indep-Dem	−.0292	.0081	−3.5968	.0003
Indep-Indep	−.0149	.0062	−2.4127	.0158
Indep-Rep	−.0359	.0102	−3.5288	.0004
Strong Rep	−.0486	.0131	−3.6987	.0002
$ME .7$
Indep-Dem	.0745	.0145	5.1240	.0000
Indep-Indep	.0296	.0108	2.7507	.0059
Indep-Rep	.0714	.0144	4.9698	.0000
Strong Rep	.0967	.0177	5.4643	.0000

Note: ME = marginal effect; CUP = Combination of Uncertainty and Preference of the respondents; ANES = American National Election Study.

Figure 12.

Marginal effect (ME) for the first category “extremely conservative’ (blue) and last category “extremely liberal” (red) by varying party identification—American National Election Study data. Note: Dots represent ME measures computed on the standard proportional odds model without taking into account the uncertainty component. Diamonds represent ME measures computed on the Combination of Uncertainty and Preference of the respondents model. The size of diamond is scaled on the uncertainty measure.

Concluding Remarks

The aim of this article has been to introduce effect measures that can be easier to interpret than model parameters for ordinal response models with uncertainty component. The measures discussed in this article extend to the examined context some previous results and implement new topics in case of interaction among covariates.

The article highlights by means of the analysis of ME measures that when the uncertainty component is neglected, the strength of the covariates tends to be underestimated. Readers who find challenging to understand cumulative link models with corresponding summary measures such as odds ratios undoubtedly find such generalized models even more demanding. When effects are monotone, simple summaries such as changes over the range in estimated ordinal extreme-category probabilities could be useful to help readers understand the substantive importance of the effects, and they can be presented with simple graphical devices. Extreme ME measures may also help to understand possible response styles and motivate collapsing of extreme categories (as the example on SHARE data shows).

The reported examples illustrate different use of ME measures changing the explanatory covariates and the main topic by stressing the role of these measures and their possible implementation in different contexts. A natural extension of this work may be to present effect measures for other models such as generalized additive models for ordinal responses with uncertainty or nominal response models.

Furthermore, by considering a recent development that replaces the uniform in a cup model with a β binomial random variable to process the uncertainty (Tutz and Schneider 2019) is possible to consider the implementation of ME in that context. Tutz and Schneider’s proposal deals with a more flexible distribution that allows to generalize our approach on the extreme categories distinguishing between a tendency to middle categories and a tendency to extreme categories.

An alternative approach may be the analysis of ME measures for ordinal data models that take into account the “don’t know” option as in Iannario et al. (2020) or ME interpretation in case where the data generating process results into too many zeroes as for zero-inflated ordinal data models (see Harris and Zhao 2007). Finally, although this article is developed in a frequentist framework, it could be of interest to study the corresponding counterpart measures in a Bayesian setting. Naturally, this next study should take into account Bayesian model estimation and selection and the corresponding computational issues.

Supplemental Materials

Supplemental Material, sj-eps-1-smr-10.1177_0049124120986179 - How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models

Supplemental Material, sj-eps-1-smr-10.1177_0049124120986179 for How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models by Maria Iannario and Claudia Tarantola in Sociological Methods & Research

Supplemental Materials

Supplemental Material, sj-eps-2-smr-10.1177_0049124120986179 - How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models

Supplemental Material, sj-eps-2-smr-10.1177_0049124120986179 for How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models by Maria Iannario and Claudia Tarantola in Sociological Methods & Research

Supplemental Materials

Supplemental Material, sj-eps-3-smr-10.1177_0049124120986179 - How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models

Supplemental Material, sj-eps-3-smr-10.1177_0049124120986179 for How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models by Maria Iannario and Claudia Tarantola in Sociological Methods & Research

Supplemental Materials

Supplemental Material, sj-tex-1-smr-10.1177_0049124120986179 - How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models

Supplemental Material, sj-tex-1-smr-10.1177_0049124120986179 for How to Interpret the Effect of Covariates on the Extreme Categories in Ordinal Data Models by Maria Iannario and Claudia Tarantola in Sociological Methods & Research

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Claudia Tarantola

Supplemental Materials

Supplemental material for this article is available online.

References

Agresti

2010. Analysis of Ordinal Categorical Data. 2nd ed. Hoboken: J. Wiley.

Agresti

Tarantola

. 2018. “Simple Ways to Interpret Effects in Modeling Ordinal Categorical Data.” Statistica Neerlandica 72:210–23.

Bartels

L. M.

2000. “Partisanship and Voting Behavior, 1952-1996.” American Journal of Political Science 44:35–50.

Breen

Luijkx

. 2010. “Mixture Models for Ordinal Data.” Sociological Methods & Research 39:3–24.

Colombi

Giordano

Gottard

Iannario

. 2018. “Hierarchical Marginal Models with Latent Uncertainty.” Scandinavian Journal of Statistics 46:595–620.

Faraway

J. J.

2006. Extending the Linear Model with R. London, England: Chapman & Hall.

Franses

P. H.

Paap

. (2004). Quantitative Models in Marketing Research. Cambridge: Cambridge University Press.

Gottard

Iannario

Piccolo

. 2016. “Varying Uncertainty in CUB Models.” Advances in Data Analysis and Classification 10:225–44.

Greene

W. H.

2008. Econometric Analysis. 6th ed. Upper Saddle River, NJ: Pearson Prentice Hall.

10.

Greene

W. H.

Hensher

D. A.

. 2010. Modeling Ordered Choices: A Primer. Cambridge, England: Cambridge University Press.

11.

Grün

Leisch

. 2008. “Identifiability of Finite Mixtures of Multinomial Logit Models with Varying and Fixed Effects.” Journal of Classification 25:225–47.

12.

Harris

N. M.

Zhao

. 2007. “A Zero-inflated Ordered Probit Model, with an Application to Modelling Tobacco Consumption.” Journal of Econometrics 141:1073–99.

13.

Iannario

Manisera

Piccolo

Zuccolotto

. 2020. “Ordinal Data Models for No-opinion Responses in Attitude Survey.” Sociological Methods & Research 49:250–76.

14.

Iannario

Monti

A. C.

Piccolo

Ronchetti

. 2017. “Robust Inference for Ordinal Response Models.” Electronic Journal of Statistics 11:3407–45.

15.

Iannario

Monti

A. C.

Scalera

. 2021. “The Number of Response Categories in Rating Scales.”

16.

Iannario

Piccolo

. 2016. “A Comprehensive Framework of Regression Models for Ordnal Data.” Metron 74:233–52.

17.

Johnson

V. E.

Albert

J. H.

. 1999. Ordinal Data Modeling. New York: Springer-Verlag.

18.

Lesschaeve

2017. “The Predictive Power of the Left-right Self-placement Scale for the Policy Positions of Voters and Parties.” West European Politics 40:357–77.

19.

Long

J. S.

1997. Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks, CA: Sage.

20.

Long

J. S.

Freese

. 2014. Regression Models for Categorical Dependent Variables Using Stata. College Station, TX: Stata Press.

21.

Long

J. S.

Mustillo

A. S.

. 2018. “Using Predictions and Marginal Effects to Compare Groups in Regression Models for Binary Outcomes.” Sociological Methods & Research. doi: 10.1177/0049124118799374.

22.

McCullagh

1980. “Regression Models for Ordinal Data (with Discussion).” Journal of the Royal Statistical Society, Series B 42:109–142.

23.

Miller

W. E.

1991. “Party Identification, Realignment, and Party Voting: Back to the Basics.” American Political Science Review 85:557–68.

24.

Mood

Carina

. 2010. “Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do about It.” European Sociological Review 26:67–82.

25.

Piccolo

2003. “On the Moments of a Mixture of Uniform and Shifted Binomial Random Variables.” Quaderni di Statistica 5:85–104.

26.

Piccolo

Simone

. 2019. “The Class of CUB Models: Statistical Foundations, Inferential Issues and Empirical Evidence.” Statistical Methods & Applications 28:389–435.

27.

Piccolo

Simone

Iannario

. 2019. “Cumulative and CUB Models for Rating Data: A Comparative Analysis.” International Statistical Review 87:207–36.

28.

Sun

2015. Empirical Research in Economics: Growing up with R. Starkville, MS: Pine Square LLC.

29.

Tosteson

A. N.

Weinstein

M. C.

Wittenberg

Begg

C. B.

. 1994. “ROC Curve Regression Analysis: The Use of Ordinal Regression Models for Diagnostic Test Assessment.” Environmental Health Perspectives 102:73–78.

30.

Tutz

2012. Regression for Categorical Data. Cambridge, England: Cambridge University Press.

31.

Tutz

Schneider

. 2019. “Flexible Uncertainty in Mixture Models for Ordinal Responses.” Journal of Applied Statistics 46:1582–1601.

32.

Tutz

Schneider

Iannario

Piccolo

. 2017. “Mixture Models for Ordinal Responses to Account for Uncertainty of Choice.” Advances in Data Analysis and Classification 11:281–305.

33.

Wedel

DeSarbo

. 1995. “A Mixture Likelihood Approach for Generalized Linear Models.” Journal of Classification 12:21–55.

34.

Whitehead

1993. “Sample Size Calculations for Ordered Categorical Data.” Statistics in Medicine 12:2257–71.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB

0.01 MB

0.02 MB