Partitioned conditional generalized linear models for categorical responses

Abstract

Abstract:

In categorical data analysis, several regression models have been proposed for hierarchically structured responses, such as the nested logit model, the two-step model or the partitioned conditional model for partially ordered set. The specifications of these models are heterogeneous and they have been formally defined for only two or three levels in the hierarchy. Here, we introduce the class of partitioned conditional generalized linear models (PCGLMs) that encompasses all these models and is defined for any number of levels in the hierarchy. The hierarchical structure of these models is fully specified by a partition tree of categories. Using the genericity of the recently introduced $(r, F, Z)$ specification of generalized linear models (GLMs) for categorical responses, it is possible to use different link functions and explanatory variables for each partitioning step. PCGLMs thus constitute a very flexible framework for modelling hierarchically structured categorical responses including partially ordered responses.

Keywords

Specification of GLMs Hasse diagram Hierarchical structure among categories partially ordered variable partition tree

1 Introduction

Response categories are often hierarchically organized, that is, partitioned into subsets of related categories. Several partitioned conditional regression models have been proposed in different applied fields, including econometrics, medicine and psychology. This is the case, for instance, when the assessment of a disease after a treatment is measured using a coarse scale with ordered categories—‘worse’, ‘same’ and ‘improvement’—and a fine scale to decompose the category ‘improvement’ into several degrees. (Tutz (1989) introduced the two-step model to take account of the hierarchy among ordinal choices in medicine (see also Morawitz and Tutz, 1990). Although the hierarchical structure among categories may seem natural for ordinal responses, it also makes sense for nominal responses. This is the case in econometrics, when an individual compares different alternatives, making its decision in several steps. This is illustrated by the well-known example of travel demand with the four alternatives: ‘air’, ‘bus’, ‘car’ and ‘train’. The nested logit model, introduced by McFadden (1978), enables to decompose the decision mechanism into two steps: (a) ‘air’ versus other alternatives and (b) choice among the three alternatives ‘bus’, ‘car’ and ‘train’. Zhang and Ip (2012) introduced the partitioned conditional model for partially ordered set (POS-PCM). We will show that introducing a hierarchical structure among partially ordered categories is very relevant for analyzing in plants the influence of growth on branching pattern.

Until now, partitioned conditional models have been formally defined for only two or three levels in the hierarchy. Furthermore, they all assume that the hierarchy among categories is a priori known. Our first contribution is to use partition trees to specify the hierarchy among categories. This enables us to define partitioned conditional models for an arbitrary number of levels in the hierarchy. Moreover, using the genericity of the $(r, F, Z)$ specification (Peyhardi et al., 2015), we develop an extended class of partitioned conditional models for nominal, ordinal but also partially ordered responses. It is, thus, possible to use different link functions and also different explanatory variables at each level of the hierarchy, leading to more flexible models with often a better fit. Finally, in the case of ordinal responses, instead of considering that the hierarchy among categories is a priori known, we propose a way to learn it.

In Section 2, the $(r, F, Z)$ specification of generalized linear models (GLMs) for categorical responses is recalled and partition trees are defined. We use these two main building blocks to define the class of partitioned conditional GLMs (PCGLMs) and propose estimation methods for this class of models. Section 3 (respectively 4 and 5) focuses on hierarchy among nominal (respectively ordinal and partially ordered) response categories. For these three cases, existing hierarchically structured models are first specified using the proposed PCGLM framework, and then some extensions are proposed and compared on datasets. In the case of ordinal responses, a model selection procedure which selects the partition tree and the explanatory variables at the same time is presented. Finally, our methodology for partially ordered responses is illustrated using the pear tree example.

2 Partitioned conditional GLMs

2.1 (r, F, Z) specification of GLMs for categorical responses

Consider the regression context, where the response $Y$ has $J \geq 2$ categories and the vector of $Q$ explanatory variables $x = (x_{1}, \dots, x_{Q})$ is defined in a general form, that is, categorical variables are represented by indicator vectors. The dimension of $x$ is, thus, denoted by $p$ with $p \geq Q$ . The definition of a GLM includes the specification of a link function $g$ which is a diffeomorphism from the simplex $Δ = {π \in] 0, 1 [^{J - 1} | \sum_{j = 1}^{J - 1} π_{j} < 1}$ to an open subset $E$ of $ℝ^{J - 1}$ . This function links the expectation $π = E (Y | X$ = $x)$ and the linear predictor $η = (η_{1}, . . ., η_{J - 1})^{t}$ . It also includes the parametrization of the linear predictor $η$ , which can be written as the product of the design matrix $Z$ (as a function of $x$ ) and the vector of parameters $β$ (tutz, 2012). All the classical link functions $g = (g_{1}, \dots, g_{J - 1})$ rely on the same structure (Peyhardi et al., 2015)

\begin{matrix} g_{j} = F^{- 1} \circ r_{j}, j = 1, \dots, J - 1, \end{matrix}

(2.1)

where

F

is a continuous and strictly increasing cumulative distribution function (cdf) and

r = (r_{1}, \dots, r_{J - 1})^{t}

is a diffeomorphism from the simplex

Δ

to an open subset

P

of the hypercube

] 0, 1 [^{J - 1}

. Finally, given

x

, we propose to summarize a GLM for a categorical response variable by the

J - 1

equations

r (π) = F (Z β),

where

F (η) = (F (η_{1}), \dots, F (η_{J - 1}))^{t}

. In the following text, we describe with more details the components

r

F

and

Z

. The linear predictor

η

is not directly related to the expectation

π

but to a particular transformation

r

of the vector

π

which is called the ratio. In the following text we will consider four particular diffeomorphisms. The adjacent, cumulative and sequential ratios, respectively defined by

r_{j} (π) = π_{j} / (π_{j} + π_{j + 1})

r_{j} (π) = π_{1} + \dots + π_{j}

and

r_{j} (π) = π_{j} / (π_{j} + \dots + π_{J})

for

j = 1, \dots, J - 1

, all rely on an ordering assumption among categories. The reference ratio, defined by

r_{j} (π) = π_{j} / (π_{j} + π_{J})

for

j = 1, \dots, J - 1

, is devoted to nominal responses. The logistic and normal cdfs are the most commonly used symmetric cdfs, but Laplace and Student cdfs may also be useful. The Gumbel and Gompertz cdfs are the most commonly used asymmetric cdfs. Exponential and Pareto cdfs have to be used carefully because of their support. Adjusting the tail weight and asymmetry of distributions may markedly improve model fit.

Each linear predictor has the form $η_{j} = α_{j} + x^{T} δ_{j}$ with $β = (α_{1}, \dots, α_{J - 1}, δ_{1}^{T}, \dots, δ_{J - 1}^{T})^{T}$ $\in ℝ^{(J - 1) (1 + p)}$ . For a given vector of explanatory variables $x$ , each constrained space is represented by a specific design matrix. For example, the complete design $(J - 1) \times (J - 1) (1 + p)$ -matrix $Z_{c}$ (without constraint) and the proportional design $(J - 1) \times (J - 1 + p)$ -matrix $Z_{p}$ (with common slope) have the following forms

Z_{c} = (\begin{matrix} 1 & x^{T} \\ ⋱ & ⋱ \\ 1 & x^{T} \end{matrix}), Z_{p} = (\begin{matrix} 1 & x^{T} \\ ⋱ & ⋮ \\ 1 & x^{T} \end{matrix}) .

The $(r, F, Z)$ triplet will play a key role in this article, since each GLM for categorical responses will be specified by one of these triplets; see Appendix A for illustrations. A single estimation procedure based on Fisher's scoring algorithm can be used for all the GLMs specified by an $(r, F, Z)$ triplet. The score function can be decomposed into two parts, where the first, unlike the second, depends on the $(r, F, Z)$ triplet

\frac{\partial l}{\partial β} = \underset{(r, F, S) d e p e n d e n t ​ p a r t}{\underset{︸}{Z^{t} \frac{\partial F}{\partial η} \frac{\partial π}{\partial r}}} \underset{(r, F, Z) i n d e p e n d e n t ​ p a r t}{\underset{︸}{C o v {(Y | x)}^{- 1} (y - π)}} .

(2.2)

We only need to evaluate the associated density function values

{f (η_{j})}_{j = 1, \dots, J - 1}

to compute the diagonal Jacobian matrix

\partial F / \partial η

. For details on computation of the Jacobian matrix

\partial π / \partial r

for each ratio, see (Peyhardi et al. 2015).

2.2 Definition of PCGLMs

We first need to introduce definitions and notations of partition trees.

Definition A directed tree $T$ is said to be a partition tree of ${1, \dots, J}$ if

${1, \dots, J}$ is the root of $T$ ,

child vertices constitute a partition of their parent vertex and

every singleton ${j}$ belongs to $T$ .

Let

V^{*}

denote the set of non-terminal vertices of

T

. For each

v \in V^{*}

let

Ch (v) = {Ω_{1}^{v}, \dots, Ω_{J_{v}}^{v}}

denote the set of indexed children of

v

. The children must be indexed because the GLMs are not necessarily invariant under permutation of the response categories (Peyhardi et al. 2015). Children

Ω_{1}^{v}, \dots, Ω_{J_{v}}^{v}

are presented from left to right in the figures, and

Ω_{J_{v}}^{v}

is considered as the reference child. Moreover, for each vertex

v

(except the root),

Pa (v)

denotes the parent of

v

and

{An}^{*} (v)

denotes the ancestors set of

v

except the root. Let us remark that partition trees are equivalent to fragmentation trees (McCullagh et al., 2008), but we prefer a non-recursive definition which is more convenient to handle PCGLMs.

Definition Let $J \geq 2$ and $1 \leq k \leq J - 1$ . A $k$ -partitioned conditional GLM of categories ${1, \dots, J}$ ( $k$ -PCGLM) is specified by

a partition tree $T$ of ${1, \dots, J}$ with $card (V^{*}) = k$ ,

a collection of models $C = {(r^{v}, F^{v}, Z^{v}) : v \in V^{*}}$ for each conditional probability vector $π^{v} = (π_{1}^{v}, \dots, π_{J_{v} - 1}^{v})$ , where $π_{j}^{v} = P (Y \in Ω_{j}^{v} | Y \in v; x)$ .

Therefore, we have for each category

P (Y = j | x) = P (Y = j | Y \in Pa (j), x) P (Y \in Pa (j) | x),

where

P (Y = j | Y \in Pa (j), x)

is modelled by the GLM of

C

associated with vertex

Pa (j)

. The probability of each category

j

is then recursively obtained

P (Y = j | x) = P (Y = j | Y \in Pa (j), x) \prod_{v \in {An}^{*} ({j})} P (Y \in v | Y \in Pa (v), x) .

(2.3)

The class of PCGLMs for categorical response variables is the set of $k$ -PCGLMs for $1 \leq k \leq J - 1$ . The limit cases are classical GLMs. For $k = 1$ , the root is the only non-terminal vertex of $T$ ; thus, the $1$ -PCGLM is a classical GLM for categories ${1, \dots, J}$ ; see Figure 1(a). For $k = J - 1$ , $T$ is a binary tree and thus $C$ is a collection of $J - 1$ GLMs for binary responses. In this case, all the ratios are identical. Moreover, if the depth of $T$ is $J - 1$ and all vertices $v \in V^{*}$ have common cdf $F$ , then the $(J - 1)$ -PCGLM is exactly the (sequential, $F$ , complete) model; see Figure 1(b).

Figure 1

(a) $1$ -partition tree and (b) $(J - 1)$ -partition tree.

There are exactly $J - 1$ independent equations to define a GLM for categorical responses. As noticed by Zhang and Ip (2012), we must check the identifiability of the PCGLMs.

Proposition Let $J \geq 2$ . There are exactly $J - 1$ independent equations for any $k$ -PCGLM of categories ${1, \dots, J}$ with $1 \leq k \leq J - 1$ .

See Appendix B for the proof. Henceforth, we will specify any PCGLM by its graphical representation, with each non-terminal vertex being labelled by an $(r, F, Z)$ triplet and a specific subset of the explanatory variables $x$ . In the case of a binary response, all ratios $r$ and design matrices $Z$ lead to the same model; thus, only $F$ and the explanatory variables $x$ are specified. In the case of a minimal response model (i.e., without explanatory variables), the $(r, F, Z)$ triplet does not play any role and, therefore, no label is given. For instance, Figure 2 fully defines a PCGLM for response variable $Y$ with six categories and three explanatory variables $x = (x_{1}, x_{2}, x_{3})^{T}$ . Using equation (2.3), we obtain for category $3$

\begin{matrix} P (Y = 3 | x) & = P (Y = 3 | Y \in {2, 3}) \times P (Y \in {2, 3} | Y \in {2, 3, 4}; x_{2}) \\ \times P (Y \in {2, 3, 4} | x_{1}, x_{2}, x_{3}), \end{matrix}

where

P (Y = 3 | Y \in {2, 3})

is the proportion of category

3

among categories 2 and 3,

P (Y \in {2, 3} | Y \in {2, 3, 4}; x_{2}) = F_{L} (α + δ x_{2})

with

F_{L}

the Laplace cdf and

P (Y \in {2, 3, 4} | x_{1}, x_{2}, x_{3}) = Φ (α_{{2, 3, 4}}^{'} + δ_{1}^{'} x_{1} + δ_{2}^{'} x_{2} + δ_{3}^{'} x_{3}) - Φ (α_{{1}}^{'} + δ_{1}^{'} x_{1} + δ_{2}^{'} x_{2} + δ_{3}^{'} x_{3})

with

Φ

the normal cdf.

Figure 2

Graphical representation of a PCGLM.

2.3 Estimation of PCGLMs

Using the partitioned conditional structure of the model, the log-likelihood can be decomposed as follows

l = \sum_{v \in V^{*}} l^{v},

(2.4)

where

l^{v}

represents the log-likelihood of model

(r^{v}, F^{v}, Z^{v})

. If all parameters are different from one vertex to another, then each component

l^{v}

can be maximized separately on the sub-dataset

{(y, x) | y \in v}

using equation (2.2).

Concavity of $l$ with respect to $β$ is equivalent to concavity of $l$ with respect to $η$ since

\frac{\partial^{2} l}{\partial β^{t} \partial β} = Z^{t} \frac{\partial^{2} l}{\partial η^{t} \partial η} Z .

Using equation (2.4), it can be remarked that the Hessian matrix

\partial^{2} l / \partial η^{t} \partial η

is block diagonal

\frac{\partial^{2} l}{\partial η^{t} \partial η} = diag {(\frac{\partial^{2} l^{v}}{\partial η^{vt} \partial η^{v}})}_{v \in V^{*}} .

Concavity of

l

is thus equivalent to concavity of

l^{v}

for all

v \in V^{*}

. Results of concavity for simple

(r, F, Z)

models can be used to obtain concavity for several PCGLMs. Here, we focus on two particular cases:

Binary partition trees: In the Bernoulli regression case, the log-likelihood is strictly concave when $log F$ and $log (1 - F)$ are strictly concave (Pratt 1981). Thus, for all binary PCGLMs—that is, relying on a binary partition tree—the log-likelihood is strictly concave when $log F^{v}$ and $log (1 - F^{v})$ are strictly concave for all $v \in V^{*}$ ; this concerns in particular sequential models. This was shown for normal, Laplace, Gumbel and Gompertz distributions (Bergstrom and Bagnoli, 2005) using results of convex analysis. Sequential models can be viewed as binary PCGLMs.

Canonical GLMs: In the case of a canonical GLM for categorical responses—that is, (reference, logistic, $Z$ ) model—the log-likelihood is strictly concave because the observed information and the Fisher's information matrices are equal; see McCullagh and Nelder (1989,43). Therefore, the log-likelihood is strictly concave for all PCGLMs defined with a collection of canonical GLMs.

3 PCGLMs for nominal responses

3.1 PCGLM specification of the nested logit model

The nested logit model for nominal responses was introduced by McFadden (1978) in the framework of individual choice behaviour. This model was designed in order to avoid the inconsistency of the independence of irrelevant alternatives (IIA) property in some situations. This is well illustrated by the paradox of the classical blue and red buses example (Debreu 1960). In the following model, the nested logit model is presented with only two levels. Let $L$ be the number of nests obtained by partitioning the set of $J$ alternatives (i.e., categories).

{1, \dots, J} \cup_{l = 1}^{L} N_{l} .

j

denotes an alternative belonging to the nest

N_{l}

, then the probability of alternative

j

is decomposed as follows

P (Y = j | x) = P (Y = j | Y \in N_{l}; x^{l}) P (Y \in N_{l} | x^{0}, IV),

(3.1)

where

IV = ({IV}_{1}, \dots, {IV}_{L})

denotes the vector of ‘inclusive values’ described thereafter,

x^{0}

are the characteristics (i.e., explanatory variables) of the first level,

x^{0}

are the characteristics of the nest

N_{l}

and

x = (x^{0}, x^{1}, \dots, x^{L})

. The two terms in equation (3.1) are determined by the multinomial logit models

P (Y = j | Y \in N_{l}; x^{l}) = \frac{\exp (η_{j}^{l})}{\sum_{k ε N_{l}} \exp (η_{k}^{l})},

and

P (Y \in N_{l} | x^{0}, IV) = \frac{exp (η_{l}^{0} + λ_{l} {IV}_{l})}{\sum_{k = 1}^{L} exp (η_{k}^{0} + λ_{k} {IV}_{k})},

with

{IV}_{l} = ln \{\sum_{k \in N_{l}} exp (η_{k}^{l})\} .

The deterministic utilities (predictors)

η_{j}^{l}

are functions of characteristics

x^{l}

and

η_{l}^{0}

are functions of characteristics

x^{0}

. In practice, they are linear with respect to

x

. If some characteristics depend on alternative, the conditional logit model has to be used (McFadden 1974).

Multinomial and conditional logit models, that share the canonical link function, are specified by the (reference, logistic, $Z$ ) triplet where the design matrix depends on individual and/or choice characteristics (Peyhardi 2015). Therefore, a nested logit model is a PCGLM with a collection of canonical models. The particularity of the nested logit model is the use of different characteristics for each non-terminal vertex and mainly the use of inclusive values. The inclusive values are necessary to obtain a random utility maximization (RUM) model. It has been shown that the nested logit model can be considered as a RUM model if and only if $0 < λ_{l} \leq 1$ for $l = 1, \dots, L$ (\citealt mcfadden78; the particular case of $λ_{l} = 1$ leads to the simple multinomial logit model). But the inclusive values affect the estimation procedure (log-likelihood can not be separately maximized) and associated parameters are difficult to interpret. We propose to relax the RUM assumption by removing the inclusive values. In this framework, each characteristic is not restricted to only one vertex. Moreover, all (reference, $F$ , $Z$ ) models, appropriate for nominal response (Peyhardi et al. 2015), can be used at each non-terminal vertex. This new class of regression models for hierarchically structured choices is more flexible and easier to estimate than the original nested logit model.

3.2 Illustration on the travel demand benchmark dataset

The dataset (Greene 2003) contains $210$ observations of choice among $J = 4$ travel modes between Sydney and Melbourne (Australia): ‘air’ ( $1$ ), ‘bus’ ( $2$ ), ‘train’ ( $3$ ) and ‘car’ ( $4$ ). The individual characteristic is the household income $x$ . The two choice characteristics are the terminal waiting time $ω^{t} = (ω_{1}^{t}, \dots, ω_{4}^{t})$ $(with ω_{4}^{t} = 0 for car)$ and the in-vehicle cost $ω^{c} = (ω_{1}^{c}, \dots, ω_{4}^{c})$ . Sampling is choice-based so as to balance among the four choices, knowing that the true population is dominated by drivers.

Table 1:
Estimates of the nested logit model and three PCGLMs.

Nested logit PCGLM 1 PCGLM 2 PCGLM 3

FIML LIML

Root Intercept air 6.042 $-$ 0.0647 $-$ 1.9225 1.2238 5.3465

Income 0.01533 0.02079 0.02612 0.02606 0.04451

IV $_{{1}}$ 0.586 0.2266

IV $_{{2, 3, 4}}$ 0.389 0.1587

Mean cost 0.02541 0.05453

Mean TTime $-$ 0.08741 $-$ 0.2661

Vertex {2, 3, 4} Intercepts bus 4.096 3.105 3.1047 3.1047 2.097

train 5.065 4.464 4.4637 4.4637 2.904

Cost $-$ 0.03159 $-$ 0.06368 $-$ 0.06368 $-$ 0.06368 $-$ 0.03823

TTime $-$ 0.1126 $-$ 0.0699 $-$ 0.06988 $-$ 0.06988 $-$ 0.0632

Log-likelihood $-$ 193.656 $-$ 203.274 $-$ 202.606 $-$ 169.69 $-$ 152.669

BIC 430.089 449.324 437.294 382.148 348.116

			Nested logit	PCGLM 1	PCGLM 2	PCGLM 3
Root	Intercept	air	6.042	$-$ 0.0647	$-$ 1.9225	1.2238	5.3465
	Income		0.01533	0.02079	0.02612	0.02606	0.04451
	IV $_{{1}}$		0.586	0.2266
	IV $_{{2, 3, 4}}$		0.389	0.1587
	Mean cost					0.02541	0.05453
	Mean TTime					$-$ 0.08741	$-$ 0.2661
Vertex {2, 3, 4}	Intercepts	bus	4.096	3.105	3.1047	3.1047	2.097
		train	5.065	4.464	4.4637	4.4637	2.904
	Cost		$-$ 0.03159	$-$ 0.06368	$-$ 0.06368	$-$ 0.06368	$-$ 0.03823
	TTime		$-$ 0.1126	$-$ 0.0699	$-$ 0.06988	$-$ 0.06988	$-$ 0.0632
Log-likelihood			$-$ 193.656	$-$ 203.274	$-$ 202.606	$-$ 169.69	$-$ 152.669
BIC		430.089	449.324	437.294	382.148	348.116

Figure 3:

(a) PCGLM specification of a nested logit model and (b) PCGLM 1 for the travel demand between Melbourne and Sydney.

In this context, the hierarchy among response alternatives is a priori fixed. We used a classical hierarchy for this dataset: the first level separates air from other alternatives and the second level separates bus, train and car. The nested logit model (Hensher and Greene 2002) can be specified in the PCGLM context by a partition tree and a collection of $(r, F, Z)$ models (Figure 3(a)). It should be remarked that the first level is a binary regression model and thus the ratio is not useful and the design matrix $Z$ is reduced to a design vector $z$ . The nested logit model was estimated using the limited information maximum likelihood (LIML) method and the full information maximum likelihood (FIML) method (Greene 2003). The LIML estimates are obtained in two steps. Parameters of each nest are first estimated, and then used to compute inclusive values and parameters of the root can thus be estimated. The FIML estimates, obtained by maximizing the full log-likelihood, gave better results (Table 1). After removing the two inclusive values, we obtained a model referred to as PCGLM 1 Figure 3(b), defined with two canonical GLMs for which the log-likelihoods are concave. Fisher's scoring algorithm can be applied in parallel for these two canonical GLMs since they do not share common parameters. This corresponds to the LIML method. It can be remarked (see Table 1) that we obtain the same parameter estimates for the vertex {2, 3, 4} as in Greene (2003). We then proposed to replace the inclusive values directly by the characteristics which composed them (see PCGLM 2 in Figure 4a). We first computed these choice characteristics for the non-terminal vertex {2, 3, 4} by their mean over Ch({2, 3, 4}), that is, $ω_{2, 3, 4} = (ω_{2} + ω_{3} + ω_{4}) / 3$ for cost and time. It was thus possible to estimate the effect of cost and time on air choice versus other alternatives. The corresponding log-likelihood was $- 169.69$ for eight parameters. The last step was the selection of the cdf $F$ among the logistic, normal, Laplace, Cauchy, Gumbel and Gompertz distributions. The best model see PCGLM 3 in Figure 4(b) was obtained with the Gumbel distribution for the root and the Cauchy distribution for the vertex {2, 3, 4}.

Figure 4:

(a) PCGLM 2 and (b) PCGLM 3 for the travel demand between Melbourne and Sydney.

Looking at the log-likelihoods, we see that replacing the inclusive values by the characteristics which composed them improved the fit. Moreover, the corresponding parameters are easy to interpret (effect of cost and transfer time on choice between air and ground). Let us remark that PCGLMs 1 and 2 locally respect the RUM principle since each conditional model of the collection $C$ is a classical logit model. Finally, the gain in log-likelihood for PCGLM 3 compared to the nested logit model (estimated with FIML) was $41.01$ with the same number of parameters. The flexibility of PCGLMs allowed us to better fit the data but also to obtain different parameter estimates leading to different interpretations. Parameters cannot be compared directly between models with different link functions, but ratios of parameters can. For instance, for the root model, the ratio $δ_{TTime} / δ_{Cost}$ is approximatively $- 3.44$ with the logistic distribution and $- 4.88$ with the Cauchy distribution. It means that the transfer time has the strongest effect on travel choice compared to the cost in PCGLM 3.

4 PCGLMs for ordinal responses

4.1 PCGLM specification and extension of the two-step model

The two-step or compound model was defined by Tutz (1989) in order to decompose the latent mechanism of an ordinal response into two levels. Ordinal responses are commonly used in medicine and psychology, for instance, to assess a patient's condition. This ordinal scale is often built from a coarse and a fine scale.

For the back pain prognosis dataset (Doran and Newell 1975), the response variable $y$ is the assessment of back pain after three weeks of treatment using the six ordered categories: worse (1), same (2), slight improvement (3), moderate improvement (4), marked improvement (5) and complete relief (6). Categories 3, 4 and 5 can be aggregated into a general category improvement. Thus, the coarse scale corresponds to the categories: worse, same, improvement and complete relief, and the fine scale corresponds to the categories: slight improvement, moderate improvement and marked improvement (see Figure 5). The three selected categorical explanatory variables observed at the beginning of the treatment period were $x_{1} =$ length of previous attack (short, long), $x_{2} =$ pain change (1 = getting better, 2 = same, 3 = worse) and $x_{3} =$ lordosis (absent/decreasing, present/increasing).

Figure 5:

Two-scale back pain assessment.

The two-step model can be extended in different ways. A partition tree with a depth greater than two, that conserves the ordering among categories, can be used. Furthermore, different link functions appropriate for ordinal responses can be used for each non-terminal vertex. For instance, the (adjacent, $F$ , $Z$ ) models, with $F \neq$ logistic, can be used for ordinal responses (Peyhardi et al., 2015). A two-step model and a PCGLM were estimated on the back pain prognosis dataset using the same partition tree; see Figure 6. The PCGLM is more flexible because the link function (i.e., $r$ and $F$ ) and the explanatory variables can be different between the two levels. The Akaike Information Criterion (AIC) was 336.903 and the Bayesian information criterion (BIC) was 357.824 for the PCGLM to be compared to an AIC of 338.439 and a BIC of 367.205 for the two-step-model. BIC favoured the PCGLM which is also more parsimonious than the two-step model.

Figure 6:

(a) PCGLM specification of a two-step model and (b) PCGLM for the back pain prognosis example.

As we saw in our first two examples, the hierarchy is often a priori fixed according to the relations between response categories. In the case of a symmetric scale among item response, Thissen-Roe and Thissen (2013) proposed to automatically determine the hierarchy, decomposing the decision into two steps. At the second level, the two child vertices have a common distribution and thus the likelihood cannot be separately maximized. Note that the hierarchical structure of this two-decision model does not respect the partition tree definition when there is an odd number of response categories.

4.2 Indistinguishability of response categories

In this section, we extend the indistinguishability procedure of Anderson (1984) to select the hierarchy among ordered response categories. He introduced the stereotype model derived from the classical multinomial logit model

P (Y = j | x) = \frac{exp (α_{j} + x^{T} δ_{j})}{1 + \sum_{k = 1}^{J - 1} exp (α_{k} + x^{T} δ_{k})},

using different parametrizations for the slopes

δ_{j}

. For instance, he defined the one-dimensional stereotype model using the particular parametrization of slopes

δ_{j} = ϕ_{j} δ,

for

j = 1, \dots, J - 1

, where

ϕ_{j}

are scalars and

δ

is a vector.

Anderson (1984) proposed a testing procedure to identify consecutive categories that can be clearly distinguished by the explanatory variables $x$ . Consecutive categories are said to be indistinguishable with respect to the explanatory variables $x$ when they do not have significantly different effects on them. He proposed to aggregate the corresponding consecutive slope $δ_{j}$ and then use a deviance test. More precisely, he proposed an iterative procedure to locate the best splits between categories $1, \dots, J$ with respect to $x$ . The minimal number of splits is zero, corresponding to the simple model without explanatory variable (null hypothesis $H_{0}$ ), and the maximal number of splits is $J - 1$ , corresponding to the classical multinomial logit model with $J - 1$ different slopes. The first step is to locate the best partition into two groups of categories. The hypothesis $H_{(2; r)}$ is then introduced

H_{(2; r)} : δ_{1} = \dots = δ_{r}; δ_{r + 1} = \dots = δ_{J} = 0,

for

r = 1, \dots, J - 1

. Comparing the corresponding log-likelihood values,

l_{(2; r)}

yields the best splitting point

r^{*}

such that

l_{2} = l_{(2; r^{*})} = max_{r} l_{(2; r)}

. The hypothesis

H_{(2; r^{*})}

is tested against

H_{0}

, using the deviance statistic

2 (l_{2} - l_{0})

which follows a

χ_{p}^{2}

distribution under

H_{0}

. Finally, if the splitting point

r^{*}

is accepted, the procedure must be restarted in parallel for the two groups

{1, \dots, r^{*}}

and

{r^{*} + 1, \dots, J}

in order to obtain the best partition into three groups. When the procedure is restarted on group

{r^{*} + 1, \dots, J}

, the tested hypothesis

H_{(3; r^{*}, s)}

H_{(3; r^{*}, s)} : δ_{1} = \dots = δ_{r^{*}}; δ_{r^{*} + 1} = \dots = δ_{r}; δ_{s + 1} = \dots = δ_{J} = 0 .

This is a dichotomous partitioning procedure with at most

J (J - 1) / 2

different parametrizations to test. It should be noted that this procedure is simplified for the one-dimensional stereotype model, since the equality between slopes

δ_{1} = \dots = δ_{r}

becomes equality between scalar parameters

ϕ_{1} = \dots = ϕ_{r}

. In practice, only this particular case of the procedure is used.

Figure 7:

PCGLM specification of indistinguishability hypothesis $H_{(3, r, s)}$ .

The indistinguishability procedure can be viewed as a partitioning procedure using the PCGLM specification; see Figure 7 and Appendix C for details. Since the partition tree $T$ has to respect categories ordering (only successive categories are aggregated), the space of possible partition trees is reduced. But the root (reference, logistic, complete) model does not rely on the ordering assumption among the groups of categories. Thus, we can define the same procedure with an ordinal root model, that is, adjacent, cumulative or sequential model. In our example, the categories are defined on a scale ordering. The sequential model is thus discarded since it is not reversible (Peyhardi et al., 2015). Then cumulative models are preferred to adjacent models, since they can be interpreted using latent variable. We choose to keep the logistic distribution and use the proportional design matrix to be comparable with the method of Anderson. Indeed, the stereotype logit model is defined with the logistic distribution and a number of parameters between $J - 1 + p$ and $(J - 1) (1 + p)$ parameters. Now assuming that we applied the procedure to determine the best root partition according to $x$ , we can say that response categories of the same non-terminal vertex are indistinguishable with respect to $x$ . But what about indistinguishability with respect to a subset of $x$ ? We, therefore, propose to select the best subset of $x$ for each non-terminal vertex. If this subset is non-empty, the procedure is restarted, otherwise the procedure is stopped. A final refinement step is then used to select $F$ in each non-terminal vertex to obtain a better fit. We now illustrate our partitioning procedure using the back pain prognosis example.

First level: Note that all the PCGLMs with only a root proportional model (and minimal response models for other non-terminal vertices) have exactly the same number of parameters: $J - 1 + p = 8$ . Thus, we simply used the log-likelihood to compare these models. The procedure was started with the simple model $M_{0} =$ (cumulative, logistic, proportional) that can be seen as a $1$ -PCGLM. The corresponding log-likelihood was $l_{0} = - 159.046$ . We then looked for the best splitting point $r \in {1, 2, 3, 4, 5}$ for explanatory variables $x_{1}$ , $x_{2}$ and $x_{3}$ . Note that model $M_{0}$ corresponds exactly to the splitting point $r = J - 1 = 5$ since all the $J - 1$ slopes are equal for this model. The best model was obtained for $r^{*} = 4$ with log-likelihood $l_{r^{*}} = - 158.132$ . Since $l_{r^{*}} > l_{0}$ , the splitting point $r^{*}$ was selected. We then looked for the best splitting point $s \in {1, 2, 3} \cup {5}$ that gives three vertices. The best model was obtained for $s^{*} = 1$ with $l_{s^{*}, r^{*}} = - 155.756$ . Since $l_{s^{*}, r^{*}} > l_{r^{*}}$ , the second splitting point $s^{*}$ was also selected. As all partitions in four groups were rejected, the best root partition was ${1} \cup {2, 3, 4} \cup {5, 6}$ for explanatory variables $x_{1}$ , $x_{2}$ and $x_{3}$ . Second level: We then focused on the non-terminal vertices $v_{1} = {2, 3, 4}$ and $v_{2} = {5, 6}$ . We first selected the subset of influential explanatory variables for these two vertices, using again the simple (cumulative, logistic, proportional) model with the BIC. The explanatory variable $x_{2}$ was selected for vertex $v_{1}$ with log-likelihood $l_{0}^{v_{1}} = - 54.561$ , and no variable was selected for vertex $v_{2}$ with $l^{v_{2}} = - 28.841$ . We then looked for the best splitting point $t \in {2, 3, 4}$ of vertex $v_{1}$ for the explanatory variable $x_{2}$ . The best model was obtained for $t^{*} = 3$ with $l_{t^{*}}^{v_{1}} = - 54.31$ . Since $l_{t^{*}}^{v_{1}} > l_{0}^{v_{1}}$ , the splitting point $t^{*}$ was selected. This was the last possible partition according to $x_{2}$ for the second level of the partition tree. The selection procedure of the partition tree and the explanatory variables was, therefore, stopped. The minimal response model was estimated for the vertex $v_{3} = {2, 3}$ with $l^{v_{3}} = - 21.93$ . Refinement step: The log-likelihood was $l = - 153.418$ with nine parameters, with the logistic cdf for each vertex. We then applied a refinement step by selecting the best cdf $F$ for each vertex and obtained the model $M_{*}$ (see Figure 8) with $l_{*} = - 152.727$ and nine parameters (AIC = 323.454 and BIC = 346.99). Figure 8:

PCGLM selected by the extended indistinguishability procedure with the back pain prognosis dataset.

Figure 9:

PCGLM specification of a one-dimensional stereotype model of Anderson (1984) with the back pain prognosis dataset.

It can be seen that the original categories need to be aggregated in order to efficiently describe the back pain. At the first level, our results are similar to those of Anderson, that is, the partition (worse), (same, little improvement, moderate improvement) and (slight improvement, complete relief) for the three explanatory variables. Anderson (1984) obtained a log-likelihood of $- 154.39$ for nine parameters with the one-dimensional stereotype model (AIC = 326.78 and BIC = 350.316); see Figure 9. Remark that in that case, the predictor $η$ is no longer a product of the design matrix $Z (x)$ and the parameter vector $β$ , since $η_{j} = ϕ_{j} δ^{T} x$ . Our methodology allows us to go a step further and to find a separation between (same, little improvement) and (moderate improvement) according to pain change $x_{2}$ . But the successive categories $2$ , $3$ and $5$ , $6$ stay indistinguishable; see Figure 8. We, therefore, propose a new ordinal scale with four categories: worse $= (1)$ , same $= (2, 3)$ , improvement $= (4)$ and relief $= (5, 6)$ , which seems to be better suited.

5 PCGLMs for partially ordered responses

In categorical data analysis, the case of nominal and ordinal responses has already been investigated in depth, while the case of partially ordered responses has been comparatively neglected. Zhang and Ip (2012) introduced the partitioned conditional model for partially ordered set. In the following text, the method of Zhang and Ip (2012) and our method will be described and compared using the pear tree dataset.

5.1 The pear tree dataset

The branching process in temperate trees is a sequential process where successive developmental events are modulated by the growth of the parent shoot (made of a succession of internodes separated by nodes where the axillary shoots are located). We here focus on the immediate axillary shoots, that is, developed without delay with respect to the parent node establishment date. The first event $Y_{1}$ is the initiation of the axillary shoot which is assumed to be modulated by the growth rate of the parent shoot. The internode length which can be easily measured retrospectively is assumed to be a proxy of the parent shoot growth rate. In our context, the successive internode lengths were smoothed using a symmetric smoothing filter; see Peyhardi et al., (2013)Peyhardi, Costes, Caraglio, Lauri, Trottier, and Guédon for justifications. This smoothing can be interpreted as an averaging over the internodes that elongate at a given time. The second event $Y_{2}$ is the elongation of the axillary shoot that distinguishes short (not-elongated) from long shoots. The third event $Y_{3}$ is the transformation of the axillary shoot apex into spin. This transformation is assumed to be influenced by the parent shoot growth arrest. The data thus consists of the categorical response branching variable $Y$ representing the different types of axillary production and two quantitative explanatory variables: the smoothed parent shoot internode length $x_{1}$ and the distance (in nodes) from the axillary shoot insertion node to the parent shoot end $x_{2}$ assumed to be a proxy of the degree of synchronism between the axillary shoot growth and the parent shoot growth arrest. The five categories of axillary production are latent bud (l), unspiny short shoot (u), unspiny long shoot (U), spiny short shoot (s) and spiny long shoot (S); see Figure 10.

Figure 10:

Successive nodes and axillary productions of pear tree.

5.2 Partially ordered variable

Let us assume that every partially ordered variable $Y$ can be expressed in terms of elementary ordinal or nominal latent variables $Y_{1}, \dots, Y_{K}$ (with at least one ordinal variable). For the pear tree example, $Y_{1}$ is an ordinal variable with categories latent bud (0) and branching (1), $Y_{2}$ is an ordinal variable with categories short (0) and long (1) and $Y_{3}$ is a nominal variable with categories unspiny (0) and spiny (1). The correspondence between the latent process $(Y_{1}, Y_{2}, Y_{3})$ and the response $Y$ is given in Table 2. There exist two classical ways to define a partial order among $K$ -tuples according to the ordering relations between the $K$ variables: the Cartesian product order or the lexicographic order. From a biological point of view, the initiation $Y_{1}$ clearly precedes the elongation $Y_{2}$ and the transformation into spin $Y_{3}$ . In this case, the lexicographic order $⪯_{L}$ is appropriate:

(y_{1}, y_{2}, y_{3}) \leq_{L} (y_{1}^{'}, y_{2}^{'}, y_{3}^{'}) if y_{1} \leq y_{1}^{'} or {y_{1} = y_{1}^{'} and (y_{2}, y_{3}) ⪯ (y_{2}^{'}, y_{3}^{'})} .

The difficulty lies in the definition of the ordering relation

⪯

between

(y_{2}, y_{3})

and

(y_{2}^{'}, y_{3}^{'})

. The most obvious assumption consists in saying that the elongation

Y_{2}

precedes the transformation into spin

Y_{3}

(hypothesis a) and thus the lexicographic order has to be used. The resulting partial order among response categories is summarized by the Hasse diagram in Figure 11(a); the order relation

j ⪯ k

between two response categories is represented by an edge between the two corresponding vertices,

k

being above

j

. But if

Y_{2}

and

Y_{3}

are considered as non-ordered mechanisms (hypothesis b), the Cartesian product order

⪯_{C}

has to be used:

(y_{2}, y_{3}) ⪯_{C} (y_{2}^{'}, y_{3}^{'}) if y_{2} \leq y_{2}^{'} and y_{3} \leq y_{3}^{'} .

Let us remark that

y_{3} \leq y_{3}^{'}

is equivalent to

y_{3} = y_{3}^{'}

since

Y_{3}

is nominal. The resulting partial order among response categories is summarized by the Hasse diagram in Figure 11(b).

Table 2:
Correspondence between the latent process $(Y_{1}, Y_{2}, Y_{3})$ and the response $Y$ .

( $y_{1},$ $y_{2},$ $y_{3})$ $y$

(0, -, -) latent bud (l)

(1, 0, 0) unspiny short shoot (u)

(1, 1, 0) unspiny long shoot (U)

(1, 0, 1) spiny short shoot (s)

(1, 1, 1) spiny long shoot (S)

( $y_{1},$ $y_{2},$ $y_{3})$	$y$
(0, -, -)	latent bud (l)
(1, 0, 0)	unspiny short shoot (u)
(1, 1, 0)	unspiny long shoot (U)
(1, 0, 1)	spiny short shoot (s)
(1, 1, 1)	spiny long shoot (S)

Figure 11:

Hasse diagrams obtained assuming that (a) the elongation $Y_{2}$ precedes the transformation into spin $Y_{3}$ and (b) the elongation $Y_{2}$ and the transformation into spin $Y_{3}$ are non-ordered.

5.3 POS-PCM

The main idea of Zhang and Ip (2012) was to propose an algorithm that builds a partitioned conditional model from any Hasse diagram.

Proposition A finite partially ordered set can always be partitioned into antichains that are totally weakly ordered (Zhang and Ip 2012).

An antichain is a set of pairwise incomparable elements. In our example, the partially-ordered set summarized by the Hasse diagram in Figure 11(a) is partitioned into the three totally ordered antichains (l), (u, s) and (U, S). Zhang and Ip (2012) proposed to use the odds proportional logit model, considering the antichains as ordered categories, and then used a multinomial logit model within each antichain. The corresponding partitioned conditional model is specified in Figure 12 (BIC = 4 333.939). It should be noted that (tutz, 2012) used a weak ordering relation between sets of categories and, thus, some information is lost during the partition tree construction. In our example, the same POS-PCM would be obtained using the second Hasse diagram in Figure 11(b).

Figure 12:

POS-PCM obtained from the Hasse diagram in Figure 11(a) and equivalently from the Hasse diagram in Figure 11(b).

5.4 PCGLM

We propose to directly use the latent variables to build a PCGLM bypassing the Hasse diagram construction. We assume that the elongation $Y_{2}$ precedes the transformation into spin $Y_{3}$ (hypothesis a). Therefore, each level $k$ of the partition tree corresponds to the latent variable $Y_{k}$ . Appropriate $(r, F, Z)$ models are used to take account of the ordinal/nominal nature of $Y_{k}$ (Peyhardi et al., 2015). Each variable $Y_{k}$ being binary in our example, all ratios $r$ and all design matrices $Z$ lead to the same model. We just have to select the explanatory variables and the cdf $F$ for each non-terminal vertex of the partition tree. As there are only two explanatory variables, the four combinations were compared with BIC using canonical models for each vertex. Finally the best cdfs were selected among the logistic, normal, Laplace, Cauchy, Gumbel and Gompertz distributions. We ended up with the PCGLM specified in Figure 13 (BIC = 4 286.976); see Table 3 for parameters estimates. Now, if the elongation $Y_{2}$ and the transformation into spin $Y_{3}$ are considered as non-ordered (hypothesis b), another partition tree is built. Using the same model selection approach, the PCGLM specified in Figure 14 was selected with BIC = 4 311.849. Finally, the PCGLM specified in Figure 13 is the best according to BIC. The sequential aspect of this model eases the parameter interpretation. We see that at the first level (a) the higher the parent shoot growth rate, the more frequent the axillary shoot initiation, at the second level (b) the higher the parent shoot growth rate, the more the axillary shoot elongates and at the third level (c) the more the axillary shoot position is close to the parent shoot end, the more the axillary shoot apex transforms into spin.

Figure 13:

PCGLM for the latent process $Y_{1}$ , $Y_{2}$ , $Y_{3}$ , assuming that the elongation $Y_{2}$ precedes the transformations into spin $Y_{3}$ .

Figure 14:

PCGLM for the latent process $Y_{1}$ , $Y_{2}$ , $Y_{3}$ , assuming that the elongation $Y_{2}$ and the transformation into spin $Y_{3}$ are non-ordered.

Table 3:

Parameter estimates for the PCGLM specified in Figure 13.

Root	Intercept	$-$ 0.0112
	Internode length ( $x_{1}$ )	$-$ 0.2191
Vertex {u, U, s, S}	Intercept	2.2773
	Internode length ( $x_{1}$ )	$-$ 0.2481
	Distance to parent shoot end ( $x_{2}$ )	$-$ 0.0156
Vertex {u, s}	Intercept	$-$ 0.1636
	Distance to parent shoot end ( $x_{2}$ )	0.0337
Vertex {U, S}	Intercept	$-$ 1.8005
	Distance to parent shoot end ( $x_{2}$ )	0.0853
Log-likelihood		$-$ 2109.7957
BIC		4 286.976

6 Discussion

PCGLMs constitute a flexible and interpretable framework for analyzing categorical responses. Explanatory variables can be selected for each non-terminal vertex. An explanatory variable may thus have an effect on one partition of categories, not on another. It should be kept in mind that the non-effect of a variable is as interesting as the effect. As for other regression models, various variable selection procedures can be applied to PCGLMs. Because of the small number of explanatory variables ( $K = 2, 3$ ), we used BIC and tested all the combinations in our examples. With a higher number of explanatory variables regularization methods should be used (Tutz, 2012). Moreover, the decomposition into several steps makes the interpretation easier, using a sequential latent mechanism approach, and also leads to a better fit. After rearranging the data by partitioning and conditioning, classical estimation algorithms can be applied for each sub-dataset. The different algorithms may be parallelized and running time is thus reduced.

An important issue with PCGLMs is the selection of the partition tree. The proposed indistinguishability procedure for jointly selecting the partition tree and the explanatory variables could be used in the supervised classification context with ordered classes. This procedure selects the best split between categories, starting from the entire set ${1, \dots, J}$ . Alternatively, we may aggregate consecutive categories, starting from singletons ${j}$ .

Appendices

A Examples of ( $r, F, Z$ ) specification

Table A1
$(r, F, Z)$ specification of four generalized linear models for categorical responses

Multinomial logit model
$P (Y = j) = \frac{exp (α_{j} + x^{T} δ_{j})}{1 + \sum_{k = 1}^{J - 1} exp (α_{k} + x^{T} δ_{k})}$	(reference, logistic, complete)
Adjacent logit model
$log \{\frac{P (Y = j)}{P (Y = j + 1)}\} = α_{j} + x^{T} δ_{j}$	(adjacent, logistic, complete)
Proportional odds logit model
$log \{\frac{P (Y \leq j)}{1 - P (Y \leq j)}\} = α_{j} + x^{T} δ$	(cumulative, logistic, proportional)
Proportional hazard model
(Grouped Cox Model)
$log \{- log P (Y > j ∣ Y \geq j)\} = α_{j} + x^{T} δ$	(sequential, Gompertz, proportional)

B Proof of Proposition 1

The cardinal of vertex $v$ is denoted by $| v |$ . For each vertex $v \in V^{*}$ , $M^{v}$ denotes the associated GLM and $M_{v}$ the PCGLM associated with the subtree rooted at vertex $v$ . Finally, $| M |$ denotes the number of independent regression equations of $M$ . Here, we reason recursively on $k$ , the cardinal of $V^{*}$ .

Initialization: For $k = 1$ , the $1$ -PCGLM of any subset $v$ of ${1, \dots, J}$ is a simple GLM for categorical responses with $| v | - 1$ regression equations and so the desired result.

Recursion: For $k < J - 1$ , let us assume, considering any subset $v$ of ${1, \dots, J}$ , that all the $m$ -PCGLMs of $v$ , such that $m \leq k$ , contain exactly $| v | - 1$ independent regression equations. Let $M$ be a $(k + 1)$ -PCGLM of ${1, \dots, J}$ . Noting $r$ the root vertex, we obtain the following decomposition:

| M | = | M^{r} | + \sum_{v \in Ch (r) \cap V^{*}} | M_{v} | .

Since the root model

M^{r}

is a

1

-PCGLM of the root's children,

| M^{r} | = | Ch (r) | - 1

. Since each model

M_{v}

is a

m

-PCGLM of

v

such that

m \leq k

, we can use the recursive assumption and obtain

| M_{v} | = | v | - 1

. Therefore, the number of independent equations of

M

\begin{matrix} | M | & = | Ch (r) | - 1 + \sum_{v \in Ch (r) \cap V^{*}} (| v | - 1) \\ = | Ch (r) | - 1 + \sum_{v \in Ch (r)} (| v | - 1) \\ = - 1 + \sum_{v \in Ch (r)} | v | \\ = J - 1 . \end{matrix}

C Indistinguishability procedure

C1 Indistinguishability procedure with $(r, F, Z)$ specification

Here, we express the indistinguishability procedure in terms of canonical models by simply changing the design matrix. In fact, the hypothesis $H_{(3; r, s)}$ corresponds to the canonical (reference, logistic, $Z_{r, s}$ ) model with

Z_{r, s} = [\begin{matrix} 1 & x^{t} \\ ⋱ & ⋮ \\ ⋱ & x^{t} \\ ⋱ & ⋮ \\ 1 \end{matrix}],

the design matrix with

r

repetitions of

x^{t}

for the first block and

s - r

repetitions of

x^{t}

for the second block. The indistinguishability procedure, specified in terms of the

(r, F, Z)

triplet, can be seen as a design matrix selection procedure.

C2 Indistinguishability procedure with PCGLM specification

Here we express the indistinguishability procedure in terms of PCGLM by simply changing the partition tree. In fact, any canonical (reference, logistic, $Z$ ) model with a block-structured design matrix $Z$ is equivalent to a PCGLM of depth $2$ with the canonical (reference, logistic, complete) model for the root and minimal response models for other non-terminal vertices. Let us describe this result in details using the block-structured design matrix $Z_{r, s}$ .

Proposition The canonical model (reference, logistic, $Z_{r, s}$ ) is equivalent to the PCGLM specified in A1.

Figure A1

PCGLM specification of indistinguishability hypothesis $H_{(3, r, s)}$ .

Proof: Assume that the distribution of

Y

is defined by the canonical (reference, logistic,

Z_{r, s}

) model. We thus obtain

\frac{π_{j}}{π_{J}} = \{\begin{matrix} exp (α_{j} + x^{t} δ_{1}), & 1 \leq j \leq r, \\ exp (α_{j} + x^{t} δ_{2}), & r < j \leq s, \\ exp (α_{j}), & s < j \leq J - 1 . \end{matrix}

(C.1)

Let

T

denote the partition tree of Figure A1 and

Ω_{1}

Ω_{2}

and

Ω_{3}

the children of the

T

’s root. We thus obtain

\frac{π_{Ω_{1}}}{π_{Ω_{3}}} = \frac{π_{1} + \dots + π_{r}}{π_{s + 1} + \dots + π_{J}} .

Using equation (C.1), we obtain

\frac{π_{Ω_{1}}}{π_{Ω_{3}}} = \frac{\{\sum_{j = 1}^{r} exp (α_{j} + x^{t} δ_{1})\} π_{J}}{\{1 + \sum_{j = s + 1}^{J - 1} exp (α_{j})\} π_{J}},

and thus

\frac{π_{Ω_{1}}}{π_{Ω_{3}}} = exp (α_{1}^{'} + x^{t} δ_{1}^{'}),

using the following parametrization

\{\begin{matrix} α_{1}^{'} = log \{\frac{\sum_{j = 1}^{r} exp (α_{j})}{1 + \sum_{j = s + 1}^{J - 1} exp (α_{j})}\} . \\ δ_{1}^{'} = δ_{1} . \end{matrix}

Similarly, we obtain

π_{Ω_{2}} / π_{Ω_{3}} = exp (α_{2}^{'} + x^{t} δ_{2}^{'})

with the parametrization

\{\begin{matrix} α_{2}^{'} = log \{\frac{\sum_{j = r + 1}^{s} exp (α_{j})}{1 + \sum_{j = s + 1}^{J - 1} exp (α_{j})}\} . \\ δ_{2}^{'} = δ_{2} . \end{matrix}

Therefore, the root model is exactly the canonical (reference, logistic, complete) model. We want to ensure that we have a minimal response model for each non-terminal vertex of the second level. For the non-terminal vertex

Ω_{1} = {1, \dots, r}

, we have

\frac{π_{j}}{π_{r}} = \frac{π_{j}}{π_{J}} \frac{π_{J}}{π_{r}} = exp (α_{j} + x^{t} δ_{1}) exp (- α_{r} - x^{t} δ_{1}) = exp (α_{j} - α_{r}),

for

j < r

. These

r - 1

ratios do not depend on

x

and, therefore, correspond exactly to the minimal response model. Similarly, we have

π_{j} / π_{s} = exp (α_{j} - α_{s})

for

r < j < s

and

π_{j} / π_{J} = exp (α_{j})

for

s < j < J

. Then,

Y

follows exactly the expected PCGLM. As the parametrization is invertible, we obtain the equivalence.

Using this proposition, the canonical (reference, logistic, $Z_{r, s}$ ) model is easily estimated. In fact, we need to transform the data, aggregating the response categories according to the partitioning sets $Ω_{1} = {1, \dots, r}$ , $Ω_{2} = {r + 1, \dots, s}$ and $Ω_{3} = {s + 1, \dots, J}$ . We then simply need to estimate the canonical (reference, logistic, complete) model using this new dataset (and also the three minimal response models of vertices $Ω_{1}$ , $Ω_{2}$ and $Ω_{3}$ ).

Footnotes

Acknowledgments

The authors thank Yves Caraglio for the pear tree dataset and for the representation of pear tree axillary production.

References

Anderson

(1984) Regression and ordered categorical variables. Journal of the Royal Statistical Society. Series B (Methodological) , 46, 1–30.

Bergstrom

Bagnoli

(2005) Log-concave probability and its applications. Economic theory , 26, 445–469.

Debreu

(1960) Review of rd luce, individual choice behavior: A theoretical analysis. American Economic Review , 50, 186–188.

Doran

and Newell

(1975) Manipulation in treatment of low back pain: A multicentre study. British Medical Journal , 2, 161.

Greene

(2003) Econometric Analysis . 5th edn. New York: Pearson Education.

Hensher

and Greene

(2002) Specification and estimation of the nested logit model: Alternative normalisations. Transportation Research Part B: Methodological , 36, 1–17.

McCullagh

Nelder

(1989) Generalized Linear Models . 2nd edn. New York: Chapman & Hall/CRC.

McCullagh

, Pitman

Winkel

(2008) Gibbs fragmentation trees. Bernoulli , 14, 988–1002.

McFadden

(1974) Conditional logit analysis of qualitative choice behavior. Frontiers in Econometrics , 105–142.

10.

McFadden

(1978) Modelling the choice of residential location. In Karlvist

, Lundqvist

, Snickars

Weibull

, eds. Spatial interactions theory and planning models , pages 75–96. Amsterdam: North-Holland.

11.

Morawitz

Tutz

(1990) Alternative parameterizations in business tendency surveys. Mathematical Methods of Operations Research , 34, 143–156.

12.

Peyhardi

(2015) New qualitative choice models incorporating individual and choice characteristics. In Friedl

Wagner

, eds., 30th International Workshop on Statistical Modelling , volume 1, pages 319–323.

13.

Peyhardi

, Costes

, Caraglio

, Lauri

P-E

, Trottier

Guédon

(2013) Integrative models for analyzing jointly shoot growth and branching patterns. In 7th International Workshop on Functional Structural Plant Models , pages 207–209.

14.

Peyhardi

, Trottier

and Guédon

(2015) A new specification of generalized linear models for categorical responses. Biometrika , 102, 889–906.

15.

Pratt

(1981) Concavity of the log likelihood. Journal of the American Statistical Association , 76, 103–106.

16.

Thissen-Roe

Thissen

(2013) A two-decision model for responses to likert-type items. Journal of Educational and Behavioral Statistics , 38, 522–547.

17.

Tutz

(1989) Compound regression models for ordered categorical data. Biometrical Journal , 31, 259–272.

18.

Zhang

and Ip

(2012) Generalized linear model for partially ordered data. Statistics in Medicine , 31, 56–68.

Partitioned conditional generalized linear models for categorical responses

Abstract

Abstract:

Keywords

1 Introduction

2 Partitioned conditional GLMs

2.1 (r, F, Z) specification of GLMs for categorical responses

(a) 1 -partition tree and (b) ( J − 1 ) -partition tree.

Graphical representation of a PCGLM.

3.1 PCGLM specification of the nested logit model

(a) PCGLM specification of a nested logit model and (b) PCGLM 1 for the travel demand between Melbourne and Sydney.

(a) PCGLM 2 and (b) PCGLM 3 for the travel demand between Melbourne and Sydney.

4.1 PCGLM specification and extension of the two-step model

Figure 5:

Two-scale back pain assessment.

(a) PCGLM specification of a two-step model and (b) PCGLM for the back pain prognosis example.

Figure 7:

PCGLM specification of indistinguishability hypothesis H ( 3 , r , s ) .

PCGLM selected by the extended indistinguishability procedure with the back pain prognosis dataset.

PCGLM specification of a one-dimensional stereotype model of Anderson (1984) with the back pain prognosis dataset.

5.1 The pear tree dataset

Figure 10:

Successive nodes and axillary productions of pear tree.

Table 2: Correspondence between the latent process ( Y 1 , Y 2 , Y 3 ) and the response Y . ( y 1 , y 2 , y 3 ) y (0, -, -) latent bud (l) (1, 0, 0) unspiny short shoot (u) (1, 1, 0) unspiny long shoot (U) (1, 0, 1) spiny short shoot (s) (1, 1, 1) spiny long shoot (S)

Hasse diagrams obtained assuming that (a) the elongation Y 2 precedes the transformation into spin Y 3 and (b) the elongation Y 2 and the transformation into spin Y 3 are non-ordered.

Figure 12:

POS-PCM obtained from the Hasse diagram in Figure 11(a) and equivalently from the Hasse diagram in Figure 11(b).

Figure 13:

PCGLM for the latent process Y 1 , Y 2 , Y 3 , assuming that the elongation Y 2 precedes the transformations into spin Y 3 .

PCGLM for the latent process Y 1 , Y 2 , Y 3 , assuming that the elongation Y 2 and the transformation into spin Y 3 are non-ordered.

Appendices

A Examples of ( r , F , Z ) specification

Table A1 ( r , F , Z ) specification of four generalized linear models for categorical responses

C Indistinguishability procedure

C1 Indistinguishability procedure with ( r , F , Z ) specification

C2 Indistinguishability procedure with PCGLM specification

PCGLM specification of indistinguishability hypothesis H ( 3 , r , s ) .

Footnotes

Acknowledgments

References

(a) $1$ -partition tree and (b) $(J - 1)$ -partition tree.

PCGLM specification of indistinguishability hypothesis $H_{(3, r, s)}$ .

Table 2:
Correspondence between the latent process $(Y_{1}, Y_{2}, Y_{3})$ and the response $Y$ .

( $y_{1},$ $y_{2},$ $y_{3})$ $y$

(0, -, -) latent bud (l)

(1, 0, 0) unspiny short shoot (u)

(1, 1, 0) unspiny long shoot (U)

(1, 0, 1) spiny short shoot (s)

(1, 1, 1) spiny long shoot (S)

Hasse diagrams obtained assuming that (a) the elongation $Y_{2}$ precedes the transformation into spin $Y_{3}$ and (b) the elongation $Y_{2}$ and the transformation into spin $Y_{3}$ are non-ordered.

PCGLM for the latent process $Y_{1}$ , $Y_{2}$ , $Y_{3}$ , assuming that the elongation $Y_{2}$ precedes the transformations into spin $Y_{3}$ .

PCGLM for the latent process $Y_{1}$ , $Y_{2}$ , $Y_{3}$ , assuming that the elongation $Y_{2}$ and the transformation into spin $Y_{3}$ are non-ordered.

A Examples of ( $r, F, Z$ ) specification

Table A1
$(r, F, Z)$ specification of four generalized linear models for categorical responses

C1 Indistinguishability procedure with $(r, F, Z)$ specification

PCGLM specification of indistinguishability hypothesis $H_{(3, r, s)}$ .