Approximate and Exact Optimal Designs for Paired Comparison Experiments

1. Introduction

In a paired comparison experiment, usually two competing objects (alternatives) are presented to a respondent who must trade off one alternative against the other and state his/her preferences. Data arising from a paired comparison task can either be qualitative^[1] or quantitative^[2] depending on the response format used. This article adopts the corresponding method by which the data is generated according to a quantitative response and, when additionally, the degree of preference for the alternatives is scored or indicated on a rating scale. For example, scoring alternative one when compared to alternative two on a seven point scale (3, 2, 1, 0, −1, −2, −3) means the following: 3, strong preference for alternative one to alternative two; 2, moderate preference for alternative one to alternative two, 1; slight preference for alternative one to alternative two, 0; no preference for either alternatives, −1; slight preference for alternative two to alternative one, −2; moderate preference for alternative two to alternative one and −3; strong preference for alternative two to alternative one.

The method of paired comparisons has received considerable attention in many fields of applications like psychology, health economics, environmental valuation, transportation economics and marketing to study people’s preferences for goods or services. In application a situation may arise where information on main effects and two and three and four attribute interactions is worthwhile.^{[3, 4]} However, in the behavioural and social sciences there are relatively little empirical investigation of such higher order interactions.^[5] This paper which is theoretically worthwhile is motivated by the situation where the designs enable estimation of main effects and higher order interactions. These proposed designs can serve as a benchmark for assessing the performance of any paired comparison design for estimating main effects and two and three and four attribute interactions.

In order to reduce information overload as frequently encountered in applications when a respon dent has to compare alternatives described by a large number of attributes, comparisons are often restricted to only a subset of the attributes with potentially different levels and the remaining attributes are usually set to zero. The number of attributes that are shown in this restricted settings is called the profile strength,^[6] and the set of alternatives described by this profile strength is known as par tial profiles, while for full profiles there are no restrictions on the number of attributes that are shown.^{[7, 8]}

The aim of this paper is to construct optimal approximate designs that estimate main effects and interactions for the situation of both full and partial profiles when all attributes have common general number of levels, and also to generate exact designs with reduced pairs for the particular situation of two level attributes. Corresponding optimality results in the case of approximate designs for main effects and first order and second order interactions can be found,^{[6, 9, 10]} whereas results in the case of exact designs for main effects and first order interactions can also be found.^{[11, 12]} Here we provide results in the case of both approximate and exact designs in the presence of third order interactions.^[13] For the particular case of two level attributes, corresponding results for approximate designs can be found.^[14]

It is worthwhile mentioning that under the indifference assumption of equal choice probability the designs considered in this paper carry over to the Bradley and Terry^[1] type choice experiments.^[12]

This paper is organized as follows. In Section 2 the general linear model is introduced to motivate the linear paired comparison model. The third order interactions model in the presence of general common number levels for both full and partial profiles in presented in Section 3. The optimal approximate and exact designs results obtained are presented in Section 4 and Section 5, respectively, and the final Section 6 offers some conclusions. All major proofs are deferred to the Appendix.

2. Preliminaries

In this section, we consider an experimental situation in which K attributes are of influence such that the kth attribute has i_k levels (i_k = 1, …, v) selected from a set I k = { 1 , … , v } . In this setting, any (direct) observation Y_n(i) (as frequently encountered in the context of standard design problems) of a single alternative i = (i₁, …, i_K) from a set I = I k × ⋯ × I K under which we obtain an nth observation where i_k is the component of the kth attribute, k = 1, …, K of influence can be formalized by a general linear model. However, in the present context of paired comparison experiments, we only obtain (comparative) observations of alternatives i = (i₁, …, i_K ) and j = (j₁, …, j_K ), defining a pair (i, j) selected from an experimental (design) region X = I × I under which we obtain an nth observation. Denote this observation by Y_n(i, j). To motivate a model for Y_n(i, j), it is useful to first consider the observations under each i and j separately. Denote these observations by Y_n(i) and Y_n(j). As was already pointed out, these direct observations can be formalized by a general linear model

Y n ( i ) = μ n + f ( i ) ⊤ β + ε n ( i ) Y n ( j ) = μ n + f ( j ) ⊤ β + ε n ( j )

(2.1)

where µ_n is the block effect, the index n denotes the nth presentation, n = 1, …, N and as earlier the alternatives i and j are chosen from the design region X = I × I , f : X → ℝ p is a vector of p known regression functions and β ∈ ℝ ^p is the vector of unknown parameters of interest, and the random errors ε_n(i) and ε_n(j) are assumed to be uncorrelated with constant variance and zero mean. It should be noted that for each attribute k, the corresponding regression functions f _k = f coincide with the one way layout.^{[6, Section 3]}

Now, for observation Y_n(i, j), the resulting model is obtained under the assumption that the preference between pair (i, j) alternatives is defined as the difference between the utilities of i and j. So, a model for Y_n(i, j) is properly described by the linear paired comparison model as follows:

Y n ( i , j ) = [ f ( i ) − f ( j ) ] ⊤ β + ε n ( i , j )

(2.2)

where [f(i) − f(j)] is the derived regression function and the random errors ε_n(i, j) = ε_n(i) − ε_n(j) asso ciated with the different pairs (i, j) are assumed to be uncorrelated with constant variance and zero mean. Here, the block effects u_n becomes immaterial.

The performance of the statistical analysis depends on the pairs (i, j) that are chosen from the design region X = I × I . The choice of such pairs i 1 , j 1 , … , i N , j N ∈ X makes up the design for the study. The goodness of a design is measured by its information matrix:

M i 1 , j 1 , … , i N , j N = ∑ n = 1 N M i n , j n ,

(2.3)

where M((i, j)) = (f(i) − f(j))(f(i) − f(j))^⊤ is the elemental information of a single pair (i, j).

In the optimal design literature, two types of designs are studied: approximate (or continuous) designs and exact designs. This article focuses on both types of designs. Approximate designs are essentially probability measures defined on a design region.^[15] Every approximate design ξ that assigns only rational weights ξ (i, j) to all pairs ( i , j ) ∈ X can be realized as an exact design ξ_N of size N consisting of the pairs (i₁, j₁), …, (i _N , jN). The normalized information matrix M(ξ_N) for an exact design ξ_N coincides with the information matrix M(ξ) of the corresponding approximate design ξ.^[6]

The D optimality criterion is considered, which can be regarded as a scalar measure of design quality. An approximate design ξ ^∗ is D optimal if it maximizes the determinant of the information matrix, that is, if detM(ξ ^∗) ≥ detM(ξ) for every approximate design ξ on X .

In this section, we give a characterization of the third order interactions model under consideration. In this setting a minimum of four attributes are required to enable identifiability of the interactions. As was already pointed out, to motivate a model for paired observations Y_n(i, j) where n denotes the nth presentation, n = 1, …, N, it is useful to first consider the (direct) observation Y_n(i). In the following third order interactions model the direct response Y_n(i), can be formulated as

Y n ( i ) = μ n + ∑ k = 1 K f i k ⊤ β k + ∑ k < l f i k ⊗ f i l ⊤ ⊤ β k l + ∑ k < l < m f i k ⊗ f i l ⊗ f i m ⊤ β k l m + ∑ k < l < m < r f i k ⊗ f i l ⊗ f i m ⊗ f i r ⊤ β k l m r + ε n ( i ) ,

(3.1)

where i_k = 1, …, v is the levels of the kth attribute, k = 1, …, K of influence and i = (i₁, …, i_K) is selected from the set I = I k × ⋯ × I K , I k = { 1 , … , v } . Here µ_n is the block effect, ⊗ denotes the Kronecker product of vectors or matrices, β k = β i k ( k ) i k = 1 , … , ν − 1 denotes the main effect of the kth and ℓth attribute, β k l = β i k i l ( k l ) i k = 1 , … , v − 1 , i l = 1 , … , v − 1 is the first order interaction of the kth and ℓth attribute, β i k i l i m ( k l m i k = 1 , … , v − 1 , i l = 1 , … , v − 1 , i m = 1 , … , v − 1 is the second order interaction of the kth, ℓth and mth attribute, and β k l m r = β i k i l i m i r ( k l m r ) i k = 1 , … , v − 1 , i l = 1 , … , v − 1 , i m = 1 , … , v − 1 , i r = 1 , … , v − 1 is the third order interaction of the kth, ℓth, mth and rth attribute. The vectors (β_k)_1≤k≤K of main effects have parameter p 1 = K ( v − 1 ) , β k l 1 ≤ k < l ≤ K of first order interactions have parameter p 2 = ( 1 / 2 ) K ( K − 1 ) ( v − 1 ) 2 , β k l m 1 ≤ k < l < m ≤ K of second order interactions have parameter p 3 = ( 1 / 6 ) K ( K − 1 ) ( K − 2 ) ( v − 1 ) 3 , and β k l m r 1 ≤ k < l < m < r ≤ K of third order interactions have parameter p₄ = (1/24)K(K − 1)(K − 2)(K − 3)(v − 1)⁴. These vector of parameters sum up to the complete parameter vector β ∈ ℝ p of dimension p = p₁ + p₂ + p₃ + p₄. Here the corresponding p-dimensional vector of regression functions f : X → ℝ p is given by

f ( i ) = f i 1 ⊤ , … , f i K ⊤ , f i 1 ⊤ ⊗ f i 2 ⊤ , … , f i K − 1 ⊤ ⊗ f i K ⊤ , f i 1 ⊤ ⊗ f i 2 ⊤ ⊗ f i 3 ⊤ , … , f i K − 2 ⊤ ⊗ f i K − 1 ⊤ ⊗ f i K ⊤ , f i 1 ⊤ ⊗ f i 2 ⊤ ⊗ f i 3 ⊤ ⊗ f i 4 ⊤ , … , f i K − 3 ⊤ ⊗ f i K − 2 ⊤ ⊗ f i K − 1 ⊤ ⊗ f i K ⊤ , ⊤

(3.2)

where the first components f(i₁), …, f(i_K) are associated with the main effects, the second components f(i₁) ⊗ f(i₂), …, f(i_K−1) ⊗f(i_K) are associated with the first order interactions, the third components f(i₁) ⊗ f(i₂) ⊗ f(i₃), …, f(i_K−2) ⊗ f(i_K−1) ⊗ f(i_K) are associated with the second order interactions and the remaining components f(i₁) ⊗ f(i₂) ⊗ f(i₃) ⊗ f(i₄), …, f(i_K−3) ⊗ f(i_K−2) ⊗ f(i_K−1) ⊗ f(i_K) of f(i) are associated with the third order interactions.

As was already pointed out, in order to reduce information overload as frequently encountered in applications when one has to compare alternatives described by a large number of attributes, comparisons are often restricted to only a subset of the attributes with potentially different levels and the remaining attributes are usually set to zero or not shown to respondents. The number of attributes in this restricted settings is referred to as the profile strength, say, S and alternatives described by S are referred to as partial profiles. In this setting a direct observation can be described by (3.1) even for a partial profile i from the set

I ( S ) = i ; i k ∈ { 1 , … , v } for at most S indices k ,

(3.3)

where level i_k = j_k = 0 for K − S component of i, which means no showing of this level to respondents. Notice that for the situation of full profiles, S = K and that the corresponding set may be specified by I ( K ) = I ( S ) .

For observations Y_n(i, j) in linear paired comparisons the resulting model is given by

Y n ( i , j ) = ∑ k = 1 K f i k − f j k ⊤ β k + ∑ k < l f i k ⊗ f i l − f j k ⊗ f j l ⊤ β k l + ∑ k < l < m f i k ⊗ f i l ⊗ f i m − f j k ⊗ f j l ⊗ f j m ⊤ β k l m + ∑ k < l < m < r f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⊤ β k l m r + ε n ( i , j ) .

(3.4)

In this case, the corresponding set (3.3) can be specified as

X ( S ) = ( i , j ) ; i k , j k ∈ { 1 , … , v } for at most S indices k

(3.5)

where the design region X ( K ) = I ( K ) × I ( K ) in the case of full profiles (S = K), and all pairs (i, j) of alternatives are K tuples. Here the pair (i, i) with identical levels should not be included in the set since it results in zero information.

In the present setting, we construct optimal designs under the third order interactions paired comparison model (3.4) with corresponding regression functions f(i) given by (3.2). Denote d as the comparison depth^[6] which describes the number of attributes in which the two alternatives presented differ satisfying d = 1, …, S. For this situation, the design region X ( S ) in (3.5) can be partitioned into disjoint sets

X d ( S ) = ( i , j ) ∈ X ( S ) ; i k ≠ j k for exactly d components

(4.1)

These sets constitute the orbits with respect to permutations of both the levels i_k, j_k = 1, …, v within the attributes as well as among attributes k = 1, …, K, themselves. In the present work, the problem of finding D optimal designs is restricted to the class of invariant designs,^{[16, Section 3.2]} which are uniform on the orbits of fixed comparison depth d ≤ S. According to Kiefer and Wolfowitz,^[17] a design ξ with non singular information matrix M(ξ) is D optimal if the variance function is defined as V ((i, j), ξ) = (f(i) − f(j))^⊤M(ξ)⁻¹(f(i) − f(j)) ≤ p for all ( i , j ) ∈ X . Moreover, for the corresponding design region (4.1), let N d = K S S d v S ( v − 1 ) d be the number of different pairs in X d ( S ) which vary in exactly d attributes and denote ξ ¯ d as the uniform approximate design which assigns equal weights ξ ¯ d ( i , j ) = 1 / N d to each pair (i, j) in X d ( S ) and weight zero to all remaining pairs in X ( S ) . It should be noted that the uniform design ξ ¯ d may need a huge number of pairs. For example, the number of pairs N_d = 210000 for K = 7, S = 3, d = 2 and v = 5. In the following we present the information matrix for the corresponding invariant designs. Here Id _m is the identity matrix of order m for every m, and M = 2 v − 1 I d v − 1 + 1 v − 1 1 v − 1 ⊤ is the information matrix of the one way layout.^[6]

Lemma 1. The uniform design ξ ¯ d on the set X d ( S ) of comparison depth d has block diagonal information matrix

M ξ ¯ d = d i a g h q ( d ) I d p q ⊗ M ⊗ q q = 1 , … , 4

where M^⊗q denotes the q fold Kronecker product of M and

h 1 ( d ) = d K , h 2 ( d ) = d ( ( d − 1 ) ( v − 2 ) + 2 ( S − d ) ( v − 1 ) ) 2 v K ( K − 1 ) , h 3 ( d ) = d λ 1 ( d ) 4 v 2 K ( K − 1 ) ( K − 2 ) , h 4 ( d ) = d λ 2 ( d ) 8 v 3 K ( K − 1 ) ( K − 2 ) ( K − 3 ) , λ 1 ( d ) = ( d − 1 ) ( d − 2 ) v 2 − 3 v + 3 + 3 ( S − d ) ( d − 1 ) ( v − 1 ) ( v − 2 ) + 3 ( S − d ) ( S − d − 1 ) ( v − 1 ) 2 λ 2 ( d ) = ( d − 1 ) ( d − 2 ) ( d − 3 ) v 3 − 4 v 2 + 6 v − 4 + 4 ( S − d ) ( d − 1 ) ( d − 2 ) v 2 − 3 v + 3 ( v − 1 ) + 6 ( S − d ) ( S − d − 1 ) ( d − 1 ) ( v − 1 ) 2 ( v − 2 ) + 4 ( S − d ) ( S − d − 1 ) ( S − d − 2 ) ( v − 1 ) 3

Without loss of generality, we mention that an invariant design ξ ¯ can be written as a convex combination of uniform designs on the comparison depths d with positive weights w_d, which sum up to 1. Hence, every invariant design has diagonal information matrix.

Lemma 2. The invariant design ξ ¯ on the design region X ( S ) has information matrix of the form

M ( ξ ¯ ) = d i a g h q ( ξ ¯ ) I d p q ⊗ M ⊗ q q = 1 , … , 4

where h q ( ξ ¯ ) = ∑ d = 1 S w d h q ( d ) , q = 1 , 2 , 3 , 4 .

First we consider optimal designs for the main effects, the first-, second and third order interaction terms separately, by maximizing the entries h_q (d), q = 1, 2, 3, 4 in the corresponding information matrix M ( ξ ¯ d ) . The resulting designs can optimize every invariant design criterion if interest is in the full parameter vector of the main effects and interactions. In particular, in Table 1, the values of d^∗ recorded in brackets where the first, second and third entries correspond to the first, second and third order interactions, respectively, were obtained by first calculating the values of h_q(d) and then determining the maximum. Zero entries in the table indicate that the minimum number of attributes required for identifiability of the interactions is not available. It is worthwhile mentioning that the optimal comparison depth d^∗ = S for the case of main effects, while for the case of first order interactions, d^∗ = S/2 for S as well as d^∗ = (S − 1)/2 for S odd in the presence of very moderate values of v (v = 2, for example) and d^∗ = S − 1 for sufficiently large values of v (v = 20, for example). Further, for the case of second order interactions, d^∗ = S, but this is not true for the situation where S = K = 3. Moreover, for the case of third order interactions, d^∗ = S − 3 for sufficiently large values of v (v = 20, for example). This means that for the corresponding main effects and interactions, only those pairs of alternatives should be used that differ in the comparison depth d^∗ subject to the profile strength S:

Theorem 1. The uniform design ξ ¯ d * is D-optimal for the parameter vector of the third order interaction effects.

For the corresponding results presented in Table 1 generated by maximizing the entries h_q (d) for q = 1, 2, 3, 4 in M ( ξ ¯ d ) , obviously, no design exists that simultaneously optimizes the information of the whole parameter vector. As a result, we restrict attention to the D-criterion to derive optimal designs for the corresponding whole parameter vector. To reach this goal, it suffices to mention that for invariant designs ξ ¯ the corresponding variance function V ((i, j), ξ ¯ ) is also invariant with respect to permutations and, hence, constant on the orbits X d ( S ) of fixed comparison depth d. Hence, the value of the variance function for an invariant design ξ ¯ evaluated at comparison depth d may be denoted by V (d, ξ ¯ ), say, where V ( d , ξ ¯ ) = V ( ( i , j ) , ξ ¯ ) on X d ( S ) .

Lemma 3. For every invariant design ξ ¯ the variance function V (d, ξ ¯ ) is given by

V ( d , ξ ¯ ) = d ( v − 1 ) 1 h 1 ( ξ ¯ ) + v − 1 4 v h 2 ( ξ ¯ ) ( ( d − 1 ) ( v − 2 ) + 2 ( S − d ) ( v − 1 ) ) + ( v − 1 ) 2 24 v 2 h 3 ( ξ ¯ ) ( d − 1 ) ( d − 2 ) v 2 − 3 v + 3 + 3 ( S − d ) ( d − 1 ) ( v − 1 ) ( v − 2 ) + 3 ( S − d ) ( S − d − 1 ) ( v − 1 ) 2 + ( v − 1 ) 4 192 v 3 h 4 ( ξ ¯ ) ( d − 1 ) ( d − 2 ) ( d − 3 ) v 3 − 4 v 2 + 6 v − 4 + 4 ( S − d ) ( d − 1 ) ( d − 2 ) v 2 − 3 v + 3 ( v − 1 ) + 6 ( S − d ) ( S − d − 1 ) ( d − 1 ) ( v − 1 ) 2 ( v − 2 ) + 4 ( S − d ) ( S − d − 1 ) ( S − d − 2 ) ( v − 1 ) 3

On a single comparison depth d^′, the corresponding variance function V (d, ξ ¯ ) simplifies to the following.

OPEN IN VIEWER

Table 1.

Values of the Optimal Comparison Depths of the D -optimal Uniform Designs for S = K.

	v
S	2	3	4	5	6	7	8	9	10	20
2	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)	(1, 0, 0)
3	(1, 1, 0)	(2, 1, 0)	(2, 1, 0)	(2, 1, 0)	(2, 1, 0)	(2, 1, 0)	(2, 1, 0)	(2, 1, 0)	(2, 1, 0)	(2, 1, 0)
4	(2, 4, 1)	(2, 4, 1)	(3, 4, 1)	(3, 4, 1)	(3, 4, 1)	(3, 4, 1)	(3, 4, 1)	(3, 4, 1)	(3, 4, 1)	(3, 4, 1)
5	(2, 5, 1)	(3, 5, 1)	(3, 5, 1)	(4, 5, 2)	(4, 5, 2)	(4, 5, 2)	(4, 5, 2)	(4, 5, 2)	(4, 5, 2)	(4, 5, 2)
6	(3, 6, 1)	(4, 6, 2)	(4, 6, 2)	(4, 6, 2)	(5, 6, 2)	(5, 6, 3)	(5, 6, 3)	(5, 6, 3)	(5, 6, 3)	(5, 6, 3)
7	(3, 7, 1)	(4, 7, 2)	(5, 7, 3)	(5, 7, 3)	(5, 7, 3)	(6, 7, 3)	(6, 7, 3)	(6, 7, 4)	(6, 7, 4)	(6, 7, 4)
8	(4, 8, 2)	(5, 8, 3)	(6, 8, 3)	(6, 8, 4)	(6, 8, 4)	(6, 8, 4)	(7, 8, 4)	(7, 8, 4)	(7, 8, 4)	(7, 8, 5)
9	(4, 9, 2)	(6, 9, 3)	(6, 9, 4)	(7, 9, 4)	(7, 9, 5)	(7, 9, 5)	(7, 9, 5)	(8, 9, 5)	(8, 9, 5)	(8, 9, 6)
10	(5, 10, 2)	(6, 10, 4)	(7, 10, 4)	(8, 10, 5)	(8, 10, 5)	(8, 10, 6)	(8, 10, 6)	(8, 10, 6)	(9, 10, 6)	(9, 10, 7)

Source: The authors.

OPEN IN VIEWER

Table 2.

Optimal Comparison Depths and Optimal Weights for S = K.

	v
S	2	3	4	5	6	7	8
5	(2, 4, 0.667)	2	2	2	2	2	2
6	(2, 5, 0.714)	(2, 5, 0.878)	3	3	3	3	3
7	(2, 6, 0.750)	3	3	3	3	4	4
8	(3, 6, 0.667)	3	4	4	4	4	4
9	(3, 7, 0.700)	4	4	5	5	5	5
10	(3, 8, 0.727)	4	5	5	6	6	6

Source: The authors.

Corollary 1. For a uniform design ξ d ' the variance function is given by the following:

V d , ξ ¯ d ′ = d d ′ p 1 + p 2 ( d − 1 ) ( v − 2 ) + 2 ( S − d ) ( v − 1 ) d ′ − 1 ( v − 2 ) + 2 S − d ′ ( v − 1 ) + p 3 λ 1 ( d ) λ 1 d ′ + p 4 λ 2 ( d ) λ 2 d ′ .

For d = d^′, we obtain V (d, ξ d ' ) = p, which shows the D-optimality of ξ d ' on X d ( S ) in view of the

Kiefer Wolfowitz equivalence theorem.^[17]

Becausethe variance function is a polynomial of degree four in the comparison depth, we will require at most four different comparison depths to obtain our results. Denote these different comparison depths by d^∗, d 1 * , d^∗ + 1 and d 1 * + 1 . The following theorem gives the maximum number of comparison depths required for a D-optimal design by virtue of the equivalence theorem.

Theorem 2. The D-optimal design ξ^* is supported on, at most, four different comparison depths: d^∗, d 1 * , d^∗ + 1 and d 1 * + 1 .

The optimal design for the full parameter vector are presented in Table 2, where numerical com putations indicate that at most two different comparison depths d^∗ and d 1 * may be required for D-optimality. The corresponding optimal designs with their optimal comparison depths d^∗ (in boldface) and their corresponding weights w d * * for various choices of attributes K = 5, …, 10 and levels v = 2, …, 8 are exhibited in Table 2 where entries of the form (d^∗, d 1 * , w d * * ) indicate that invariant designs ξ * = w d * * , ξ d * + ( 1 − w d * * ) ξ d 1 * have to be considered, while for single entries, d^∗ the optimal design ξ * = ξ d * * has to be considered that is uniform on the optimal comparison depth d^∗. For the particular case S = K = 4 of full profiles, the uniform design on all possible comparisons proves to be optimal.^{[6, Theorem 4]} It is worth noting that for the case of binary attributes, the corresponding design, which is presented in the next section in the form of (exact) designs as a generalization of the method in Großmann et al. ^[11] for S = K = 4, possesses a D-efficiency of 0.909. Numerical results presented in Table 3 of Nyarko^[14] show that D-efficiency of at least 90 per cent can be obtained in general. The values of the nor malized variance function V(d, ξ^∗)/p, which show D-optimality of the design ξ^∗ in view of the Kiefer and Wolfowitz^[17] equivalence theorem, are recorded in Table 3, where maximal values less than or equal to 1 establish optimality. It can be seen that for moderate values of v(v = 2, for example), two types of pairs have to be used in which the numbers of distinct attributes are symmetric with respect to about half of the profile strength to obtain a D-optimal design for the whole parameter vector,^[14] while for large values of v(v > 2, for example), only one type of pair is sufficient, but this is not true for the case S = 6 and v = 3.

In this section, we construct an exact design with reduced number of pairs. Let ξ_N,d be an exact design with N pairs that have distinct levels for the d (so called comparison depth) of the binary attributes K (or K two level attributes), which allow for the estimation of main effects and two, three and four attribute interactions. The construction generates two N × K matrices I and J with rows i₁, …, i _N and j₁, …, j _N , respectively, where N is the treatment combinations (sample size). This treatment combination is essentially a collection of pairs (i _n , j _n ) for n = 1, …, N in the design region X d .

For given K and d, the method requires three building blocks:

A Hadamard matrix H of order t ≥ d,

These building blocks are used to construct designs ξ_N,d with N = bmt treatment combinations (pairs) in X _d . The construction is summarized as follows:

Step 1: Let A be a t × d matrix obtained by selecting d columns from H and let F be an m × (K − d) matrix corresponding to the regular fractional factorial design.

OPEN IN VIEWER

Table 3.

Values of the Variance Function V (d, ξ^∗) for ξ^∗ from Table 2 for S = K.

		d
S	v	1	2	3	4	5	6	7	8	9	10
5	2	0.938	1	0.938	1	0.938
	3	0.881	1	0.961	1	0.987
	4	0.858	1	0.965	0.985	0.981
	5	0.845	1	0.970	0.982	0.980
	6	0.837	1	0.974	0.982	0.981
	7	0.832	1	0.977	0.983	0.982
	8	0.828	1	0.980	0.984	0.983
6	2	0.850	1	0.950	0.950	1	0.850
	3	0.793	1	0.988	0.970	1	0.977
	4	0.777	0.999	1	0.977	0.995	0.987
	5	0.570	0.984	1	0.979	0.990	0.986
	6	0.734	0.975	1	0.982	0.989	0.987
	7	0.723	0.969	1	0.984	0.989	0.988
	8	0.715	0.964	1	0.985	0.989	0.988
7	2	0.792	1	0.982	0.952	0.982	1	0.792
	3	0.723	0.973	1	0.972	0.971	0.997	0.965
	4	0.679	0.945	1	0.984	0.976	0.990	0.980
	5	0.657	0.930	1	0.993	0.983	0.992	0.987
	6	0.643	0.921	1	0.999	0.989	0.995	0.993
	7	0.634	0.914	0.998	1	0.991	0.995	0.994
	8	0.625	0.906	0.994	1	0.991	0.995	0.994
8	2	0.759	0.998	1	0.954	0.954	1	0.998	0.759
	3	0.650	0.928	1	0.990	0.973	0.981	0.998	0.964
	4	0.612	0.898	0.993	1	0.986	0.984	0.995	0.984
	5	0.585	0.873	0.982	1	0.990	0.986	0.993	0.988
	6	0.567	0.858	0.974	1	0.994	0.989	0.994	0.991
	7	0.559	0.848	0.969	1	0.997	0.992	0.995	0.994
	8	0.552	0.841	0.965	1	0.999	0.994	0.996	0.996
9	2	0.693	0.958	1	0.966	0.945	0.966	1	0.958	0.693
	3	0.596	0.885	0.989	1	0.983	0.976	0.987	0.997	0.960
	4	0.550	0.841	0.967	1	0.995	0.986	0.987	0.995	0.984
	5	0.528	0.819	0.954	0.998	1	0.992	0.991	0.996	0.992
	6	0.512	0.801	0.941	0.993	1	0.994	0.992	0.996	0.993
	7	0.501	0.789	0.932	0.989	1	0.996	0.993	0.996	0.995
	8	0.493	0.780	0.927	0.986	1	0.997	0.994	0.996	0.996
10	2	0.644	0.925	1	0.985	0.958	0.958	0.985	1	0.925	0.644
	3	0.544	0.836	0.965	1	0.994	0.982	0.981	0.991	0.996	0.960
	4	0.501	0.791	0.936	0.991	1	0.993	0.987	0.990	0.996	0.985
	5	0.478	0.764	0.916	0.982	1	0.997	0.992	0.993	0.996	0.992
	6	0.464	0.748	0.904	0.976	0.999	1	0.996	0.995	0.998	0.995
	7	0.453	0.735	0.893	0.969	0.997	1	0.996	0.995	0.997	0.996
	8	0.446	0.726	0.885	0.965	0.995	1	0.997	0.996	0.996	0.997

Source: The authors.

Note: Boldface 1 corresponds to the optimal comparison depths d ^∗ and d ^∗.

OPEN IN VIEWER

Table 4.

Exact Design ξ_32,1 for Four Attributes and Comparison Depth One.

n	Pair (i n , jn)
1	(( 1, 1, 2, 2),	( 2, 1, 2, 2))
2	(( 1, 2, 2, 2),	( 2, 2, 2, 2))
3	(( 1, 1, 2, 1),	( 2, 1, 2, 1))
4	(( 1, 2, 2, 1),	( 2, 2, 2, 1))
5	(( 1, 1, 1, 2),	( 2, 1, 1, 2))
6	(( 1, 2, 1, 2),	( 2, 2, 1, 2))
7	(( 1, 1, 1, 1),	( 2, 1, 1, 1))
8	(( 1, 2, 1, 1),	( 2, 2, 1, 1))
9	(( 1, 1, 2, 2),	( 1, 2, 2, 2))
10	(( 2, 1, 2, 2),	( 2, 2, 2, 2))
11	(( 1, 1, 1, 2),	( 1, 2, 1, 2))
12	(( 2, 1, 1, 2),	( 2, 2, 1, 2))
13	(( 1, 1, 2, 1),	( 1, 2, 2, 1))
14	(( 2, 1, 2, 1),	( 2, 2, 2, 1))
15	(( 1, 1, 1, 1),	( 1, 2, 1, 1))
16	(( 2, 1, 1, 1),	( 2, 2, 1, 1))
17	(( 2, 2, 1, 1),	( 2, 2, 2, 1))
18	(( 2, 2, 1, 2),	( 2, 2, 2, 2))
19	(( 1, 2, 1, 1),	( 1, 2, 2, 1))
20	(( 1, 2, 1, 2),	( 1, 2, 2, 2))
21	(( 2, 1, 1, 1),	( 2, 1, 2, 1))
22	(( 2, 1, 1, 2),	( 2, 1, 2, 2))
23	(( 1, 1, 1, 1),	( 1, 1, 2, 1))
24	(( 1, 1, 1, 2),	( 1, 1, 2, 2))
25	(( 2, 2, 1, 1),	( 2, 2, 1, 2))
26	(( 2, 2, 2, 1),	( 2, 2, 2, 2))
27	(( 2, 1, 1, 1),	( 2, 1, 1, 2))
28	(( 2, 1, 2, 1),	( 2, 1, 2, 2))
29	(( 1, 2, 1, 1),	( 1, 2, 1, 2))
30	(( 1, 2, 2, 1),	( 1, 2, 2, 2))
31	(( 1, 1, 1, 1),	( 1, 1, 1, 2))
32	(( 1, 1, 2, 1),	( 1, 1, 2, 2))

Source: The authors.

For illustrative purposes, we now construct a set of pairs that differ in only one attribute to compare products with K = 4 attributes. Suppose in an experimental situation, an experimenter is interested in constructing a design ξ_N,d with N = 32 pairs that differ in only one attribute to compare products K = 4 attributes. For this situation, the pairs (i_n, j _n ) for n = 1, …, 32 with effects coded levels ±1, where for convenience in notation, the effects coded first and last level of each attribute is assigned with the actual levels 1 and 2, respectively (Table 4). The design was constructed from a Hadamard matrix of order t = 2, a regular half fraction of a 2³ full factorial design and an incomplete block design with blocks {1}, {2}, {3} and {4}. In the 1 − 8 pairs, the levels of attribute 1 in each alternative are determined by a column from the combined rows of the Hadamard matrix and the regular half fraction of the 2³ full factorial design. For pairs 9 − 16, 17 − 24 and 25 − 32, the levels of the attributes in columns 2, 3 and 4, respectively, are determined by the corresponding column of the combined rows of the Hadamard matrix and the regular half fraction of the factorial design, while the levels of the other remaining attributes are the same in both alternatives and depend on the regular half fraction of the 2³ full factorial design.

It should be noted that by identifying ξ_N,d with the aforementioned approximate design ξ, the D-efficiency of the design ξ_N,d can be computed. As was already noted, the final design ξ_{32 , 1} in Table 4 has D-efficiency of 0.909 for estimating main effects and two, three and four attribute interactions. Notice that for given values of K and d, and by choosing a Hadamard matrix H of appropriate order d and a regular two level fractional factorial design F with m rows or treatment combinations, similar exact designs (or numerical results) with N = bmd pairs of the aforementioned design can be constructed by performing a computer search^[12] over appropriately selected block B, which represents a balanced incomplete block design for K treatments k = 1, …, K in b blocks of size d.

For the situation of linear paired comparisons, when a continuous response is available for the amount of preferences, the problem of generating optimal approximate designs (when the attributes are common general level factors) and efficient exact designs (when the attributes are two level factors) that allow the identification of main effects and two, three and four attribute interactions from either full or partial profile data is considered. The resulting designs are also optimal under the indifference assumption of equal choice probabilities for a multinomial logit model when the preference tasks or choice sets are pairs.

In this article, it is shown that, for the approximate designs, at most two types of pair alternatives have to be used, depending on the level of the attributes, in which the numbers of distinct attributes or the comparison depths are symmetric with respect to about half of the profile strength. Optimal designs may be concentrated on at most four different comparison depths, depending on the number of the profile strengths and corresponding levels.^[14] Further, for the case when the corresponding paired comparison designs involve alternatives characterized by a common set of two level attributes, we have presented efficient exact design with practical (or reduced) numbers of pairs. The designs proposed in the current article can be used as a benchmark to compare any design for estimating main effects and two, three and four attribute interactions.

Appendix

Proof of Lemma 1. First we note that the quantities h_q(d) for q = 1, 2, 3 are identicaLl to the terms in Graßhoff et al. ^[6] and Nyarko and Schwabe.^[10] For h₄(d) we proceed by first noting that ∑ i = 1 v f ( i ) f ( i ) ⊤ = ν − 1 2 M and ∑ i ≠ j f ( i ) f ( j ) ⊤ = − v − 1 2 M .

For the third order interactions we consider attributes k, ℓ, m and r, say, and distinguish between pairs (i_ki_ℓi_mi_r) and (j_kj_ℓj_mj_r) in which all the four associated attributes k, ℓ, m and r differ, pairs (i_ki_ℓi_mi_r) and (j_kj_ℓj_mj_r) which differ in three of these attributes k, ℓ and m, say, pairs (i_ki_ℓi_mi_r) and (j_kj_ℓj_mj_r) which differ in two of these attributes k and ℓ, say, and finally, pairs (i_ki_ℓi_mi_r) and (j_kj_ℓj_mj_r) which differ in only one of the attributes k, say. Hence

∑ i k ≠ j k ∑ i l ≠ j l ∑ i m ≠ j m ∑ i r ≠ j r f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⋅ f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⊤ = 2 v − 1 4 ∑ i k = 1 v f i k f i k ⊤ ⊗ ∑ i l = 1 v f i l f i l ⊤ ⊗ ∑ i m = 1 v f i m f i m ⊤ ⊗ ∑ i m = 1 v f i r f i r ⊤ − 2 ∑ i k ≠ j k f i k f j k ⊤ ⊗ ∑ i l ≠ j l f i l f j l ⊤ ⊗ ∑ i m ≠ j m f i m f j m ⊤ ⊗ ∑ i r ≠ j r f i r f j r ⊤ = 1 8 ν ν − 1 4 ν 3 − 4 ν 2 + 6 ν − 4 M ⊗ q ,

(A.1)

also

∑ i k ≠ j k ∑ i l ≠ j l ∑ i m ≠ j m ∑ i r = j r f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⋅ f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⊤ = 2 v − 1 3 ∑ i k = 1 v f i k f i k ⊤ ⊗ ∑ i l = 1 v f i l f i l ⊤ ⊗ ∑ i m = 1 v f i m f i m ⊤ ⊗ ∑ i m = 1 v f i r f i r ⊤ − 2 ∑ i k ≠ j k f i k f j k ⊤ ⊗ ∑ i l ≠ j l f i l f j l ⊤ ⊗ ∑ i m ≠ j m f i m f j m ⊤ ⊗ ∑ i r = j r f i r f j r ⊤ = 1 8 v v − 1 4 v 2 − 3 v + 3 M ⊗ q ,

(A.2)

further

∑ i k ≠ j k ∑ i l ≠ j l ∑ i m = j m ∑ i r = j r f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⋅ f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⊤ = 2 v − 1 2 ∑ i k = 1 v f i k f i k ⊤ ⊗ ∑ i l = 1 v f i l f i l ⊤ ⊗ ∑ i m = 1 v f i m f i m ⊤ ⊗ ∑ i m = 1 v f i r f i r ⊤ − 2 ∑ i k ≠ j k f i k f j k ⊤ ⊗ ∑ i l ≠ j l f i l f j l ⊤ ⊗ ∑ i m = j m f i m f j m ⊤ ⊗ ∑ i r = j r f i r f j r ⊤ = 1 8 v v − 1 4 ( v − 2 ) M ⊗ q ,

(A.3)

and, finally

∑ i k ≠ j k ∑ i l = j l ∑ i m = j m ∑ i r = j r f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⋅ f i k ⊗ f i l ⊗ f i m ⊗ f i r − f j k ⊗ f j l ⊗ f j m ⊗ f j r ⊤ = 2 v − 1 ∑ i k = 1 v f i k f i k ⊤ ⊗ ∑ i l = 1 v f i l f i l ⊤ ⊗ ∑ i m = 1 v f i m f i m ⊤ ⊗ ∑ i m = 1 v f i r f i r ⊤ − 2 ∑ i k ≠ j k f i k f j k ⊤ ⊗ ∑ i l = j l f i l f j l ⊤ ⊗ ∑ i m = j m f i m f j m ⊤ ⊗ ∑ i r = j r f i r f j r ⊤ = 1 8 v v − 1 4 M ⊗ q ,

(A.4)

where M^⊗q, q = 1, 2, 3, 4 is the q-fold Kronecker product of M.

Now for the given attributes k, ℓ, m and r the pairs with distinct levels in the four attributes occur K − 4 S − 4 S − 4 d − 4 v S − 4 ( v − 1 ) d − 4 times in X d ( S ) , while those which differ in three attributes occur 4 K − 4 S − 4 S − 4 d − 3 v S − 4 ( v − 1 ) d − 3 times in X d ( S ) , while those which differ in two attributes occur 6 K − 4 S − 4 S − 4 d − 2 v S − 4 ( v − 1 ) d − 2 times in X d ( S ) . Finally, those which differ only in one attribute occur 4 K − 4 S − 4 S − 4 d − 1 v S − 3 ( v − 1 ) d − 1 times. Hence, the diagonal blocks for the interactions are given by

1 N d K − 4 S − 4 1 8 S − 4 d − 4 v S − 3 v − 1 d v 3 − 4 v 2 + 6 v − 4 M ⊗ q + 1 2 S − 4 d − 3 v S − 3 v − 1 d + 1 v 2 − 3 v + 3 M ⊗ q + 3 4 S − 4 d − 2 v S − 3 v − 1 d + 2 v − 2 M ⊗ q + 1 2 S − 4 d − 1 v S − 3 v − 1 d + 3 M ⊗ q = d 8 v 3 K ( K − 1 ) ( K − 2 ) ( K − 3 ) d − 1 d − 2 d − 3 v 3 − 4 v 2 + 6 v − 4 + 4 S − d d − 1 d − 2 v 2 − 3 v + 3 v − 1 + 6 S − d S − d − 1 d − 1 v − 1 2 v − 2 + 4 S − d S − d − 1 S − d − 2 v − 1 3 M ⊗ q .

The off diagonal elements all vanish because the terms in the corresponding entries sum up to zero due to the effects type coding. □

Proof of Theorem 3. First we note that the inverse of the information matrix M ξ ¯ of the design ξ ¯ is given by