Formal Modeling of the Resistance to Peer Influence Questionnaire: A Comparison of Adolescent Boys and Girls With and Without Mild-to-Borderline Intellectual Disability

Participants

RPIQ data were available for 179 TD and 190 MBID adolescents. However, participants with missing data on one of more item(s) of the RPIQ (N_TD = 2; N_MBID = 10) or whose gender was not recorded (N_MBID = 6) were excluded from all analyses. This was done because the dendrify function that was used to transform the data (see “Model Estimation and Selection: Methods” section) cannot handle missing item scores. ¹ As a result, in the current study, data of 177 TD (N_{Boys_TD} = 117; 66.10%; SPM IQ: M [SD] = 106.00 [14.45]) and 174 MBID adolescents (N_{Boys_MBID} = 103; 59.20%; SPM IQ: M [SD] = 66.72 [11.32]), aged between 12 and 19 years (M [SD] = 15.21 [1.50]), were included. An analysis of variance (ANOVA), with intellectual ability (two levels: TD, MBID) and gender (two levels: boys, girls) as independent variables and age as dependent variable, indicated that MBID adolescents (M [SD] = 15.47 [0.11]) were significantly older than TD adolescents (M [SD] = 14.87 [0.12]), F(1, 347) = 13.76, p < .001; and that boys (M [SD] = 15.33 [0.10]) were marginally, but significantly older than girls (M [SD] = 15.01 [0.13]), F(1, 347) = 3.96, p = .047; whereas the interaction between intellectual ability and gender was not significant (p > .05). Participants were recruited through regular or special education schools in the Netherlands. Assignment to the TD or MBID group consisted of two phases: an initial phase and a refined phase. Initial assignment was based on school type. TD adolescents were selected from regular secondary education schools that prepare students for community college or higher education. MBID adolescents were selected from two types of special education schools: (1) special vocational education (“Praktijkonderwijs”) and (2) special education for adolescents with an intellectual disability. Special vocational education schools have the following admittance criteria: (1) an IQ between 60 and 85, tested no longer than 2 years prior to admittance and (2) learning delays of 50% or more in at least two of the following areas: mathematics, reading accuracy and fluency, reading comprehension, and spelling; one of these delays must be in mathematics or reading comprehension. Special education schools for adolescents with an intellectual disability have the following admittance criteria: (1) an IQ less than 70, tested no longer than 2 years prior to admittance; (2) a severe learning delay, that has not been sufficiently improved after remediation provided at the previous school; and (3) severely limited (social) adaptive skills. Refined assignment was based on the most recent IQ information from teachers or school files. That is, group assignment to the TD group was checked by asking teachers to indicate students who they thought had lower than average IQ, which was not the case for any of the selected students. Group assignment to the MBID group was refined by checking standardized IQ scores from school files and excluding participants who had a standardized IQ greater than 85. Prior to participation, written informed consent was obtained from adolescents and their caregivers. All procedures were approved by the local Ethical Committee of the University of Amsterdam.

Materials

As part of a larger study, that aimed to investigate risk taking in TD and MBID adolescents, both the RPIQ and an experimental risk-taking task were administrated. Only data on the RPIQ were included in the current study; information on the experimental task can be obtained from the corresponding author.

Procedure

Participants were individually tested at school in an empty classroom, by one of several trained (under)graduate students. After completion of an experimental task, participants were asked to fill out the RPIQ. Scales with items that have a similar tree-based structure as items of the RPIQ have been successfully administrated in research among adolescents with learning disabilities or behavior disorders (e.g., Harter, Whitesell, & Junkin, 1998). In addition, care was taken to make sure that participants understood (the structure of) the items and instructions. That is, instructions were read to participants aloud. Participants were then asked to read aloud the example item and to verbally indicate how they would answer the item. This procedure was repeated, if necessary, until participants understood (the structure of) the items and understood what was required from them. Participants then completed the questionnaire, whereas the experimenter stayed in another part of the room to answer questions when needed.

Model Estimation and Selection: Methods

As indicated in the “Introduction” section and illustrated in Figure 2, we modeled choices to items of the RPIQ with a three-node branching model (De Boeck et al., 2011; De Boeck & Partchev, 2012). Node1 refers to the highest branching level: the choice between the more or less peer-resistant statement. Node2 and node3 refer to the lowest branching levels; choices between strongly (“Really true”) or weakly (“Sort of true”) agreeing with the statement chosen at node1.

In order to explain the model conceptually, it is convenient to first consider the case in which participants are only presented with the node1 subitem between the more or less peer-resistant statement. Responses to such two-choice items can be modeled using IRT (De Boeck & Wilson, 2004, 2015; Embretson & Reise, 2000; Hambleton & Swaminathan, 1985; Lord, 1980; Thissen & Wainer, 2001). In such a model, the probability that a participant opts for the more peer-resistant statement is modeled as a function of two types of parameters. The first type is a set of item parameters, which reflects, for each item, the probability of opting for the peer-resistant statement given that the participant has an average level of RPI. That is, if the item parameter of a certain item is high compared with other item parameters, participants will overall be more likely to choose the peer-resistant option on this item. The second type of parameter is a person parameter, which is indicative of the participant’s level of RPI. Independent of the item at hand, participants with a high person parameter will be more likely to choose the peer-resistant option. An IRT model with these types of parameters is referred to as a Rasch model (see Bond & Fox, 2007). This model, in which item parameters are fixed and person parameters vary random over participants, is a specific case of the more general family of generalized linear mixed models (Cho et al., 2013; De Boeck et al., 2011; De Boeck & Partchev, 2012; De Boeck & Wilson, 2004, 2015; Molenaar, Tuerlincks, & van der Maas, 2015; Partchev & De Boeck, 2012; Willoughby et al., 2011).

However, in case of the RPIQ, participants are presented with two out of three potential subitems—reflected by the three different nodes in Figure 2. The generalized linear mixed model (GLMM) framework should therefore be extended such that, for each item, three (potential) choices are modeled instead of one. Note that for each participant (N = 351), for each item, a subresponse to either the node2 or node3 subitem is missing, depending on the participant’s choice to the node1 subitem. Since this missing response is only dependent on the participant’s response to node1, conditioning on this response generates a situation in which the missing data are missing at random (MAR; Little & Rubin, 1987), and therefore the generalized linear mixed model (GLMM) framework can still be used (cf. De Boeck & Partchev, 2012).

Based on its internal structure, we hypothesized that items of the RPIQ tap a multidimensional instead of unidimensional construct (Hypothesis 1a). In its most extensive version, a multidimensional conceptualization of items of the RPIQ becomes a three-dimensional model, in which the three choices to the node1, node2, and node3 subitems are reflective of three different underlying dimensions (Hypotheses 1b and 1c; Figure 2B). This three-dimensional conceptualization can be estimated by a full model, with for each of the three subitems a separate set of item parameters and a separate person parameter. In a less extensive version, the multidimensional conceptualization of items of the RPIQ becomes a two-dimensional model, in which choices to the highest branching-level node (node1) are reflective of an underlying dimension that is distinct from the dimension that underlies choices to the lowest branching-level nodes (node2 and node3), which are themselves indicative of the same underlying dimension (Figure 2B with the only difference that the dimensions underlying choices to the node2 and node3 subitems are identical). This two-dimensional conceptualization can be arrived at by constraining the sets of item parameters and the person parameters to be equal for choices to the node2 and node3 subitems. One additional item parameter should then be estimated, that reflects the overall (collapsed over items) differential probability of opting for the right-sided versus left-sided branch at the node2 versus node3 subitems.

Alternatively, scoring items of the RPIQ on a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007) suggests that responses to each item are reflective of one underlying dimension, in that the three choices to the node1, node2, and node3 subitems tap the same process, which could be captured by one set of item parameters and one person parameter. This unidimensional conceptualization (Figure 2A) can be arrived at by constraining the sets of item parameters and the person parameters to be equal over all three nodes. Two additional item parameters should then be estimated, that reflect the overall (collapsed over items) differential probability of opting for the right-sided versus left-sided statement at the node1 versus node2 subitems, and at the node1 versus node3 subitems.

By systematically constraining parameters, we fitted a series of three models: a unidimensional model with one set of item parameters and one person parameter (Figure 2A), a two-dimensional model with two sets of item parameters and two person parameters (Figure 2B with the constraint that parameters are constrained to be equal over the node2 and node3 subitems), and a three-dimensional model with three sets of item parameters and three person parameters (Figure 2B).

For all models, parameterization was chosen to be in line with the earlier proposed usage of a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007), in that for all three subitems the probability to opt for the higher Likert-type scale score (i.e., right-sided branch) versus the lower Likert-type scale score (i.e., left-sided branch) was modeled. ² Model estimation was performed with the glmer function as implemented in the lme4 package in R (Bates & Maechler, 2009). For this purpose, original data were transformed (see, De Boeck et al., 2011) such that they could be modeled as a two-level branching IRT model by using the R function dendrify (De Boeck & Partchev, 2012). As the estimated models were identically parameterized and only differed in the number of freely estimated sets of item parameters and person parameters, model fit of these models could be compared by measures of comparative model fit. Both Akaike information criterion (AIC; Akaike, 1973) and Bayesian information criterion (BIC; Schwarz, 1978) were inspected. As lower AIC as well as BIC indicate better model fit, the model with the lowest AIC and BIC was selected. In case of conflict, additional analyses were performed.

Model Estimation and Selection: Results

AICs of the estimated uni-, two-, and three-dimensional models with, respectively, one, two, or three set(s) of item parameters and one-, two-, or three-person parameter(s), were 8,657 (unidimensional), 8,610 (two-dimensional), and 8,354 (three-dimensional). BICs of these models were 8,746 (unidimensional), 8,775 (two-dimensional), and 8,601 (three-dimensional). Thus, a three-dimensional model, with three sets of item parameters and three person parameters, had both the lowest AIC and lowest BIC, and was initially selected for further analysis.

Interpretation of Multidimensionality: Methods

For the selected three-dimensional model, two interpretations of its three dimensions exist, which are depicted in Figure 3A and B, respectively. First, all three dimensions might be reflective of more or less RPI. This multidimensional conceptualization is in line with the earlier proposed usage of a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007) and will be referred to as the “3RPI conceptualization.” In this conceptualization, all three dimensions tap a different aspect of RPI, such that these cannot be reduced to one underlying RPI dimension (which would be a unidimensional model). Alternatively, and in line with our hypotheses (Hypotheses 1b and 1c), choices to the highest branching-level node (node1) might be reflective of more or less RPI, whereas choices to the lowest branching-level nodes (node2 and node3) might instead be reflective of Response Polarization. This alternative multidimensional conceptualization is conflicting with usage of a 4-point Likert-type scale (cf., Steinberg & Monahan, 2007) and will be referred to as the “1RPI&2POL conceptualization.” This model encompasses two distinct Response Polarization dimensions, in that the tendency to opt for the “Really true” versus “Sort of true” indicator after an initial choice for the more peer-resistant statement (i.e., node3) taps a construct that is different from the tendency to opt for the “Really true” versus “Sort of true” indicator after an initial choice for the less peer-resistant statement (i.e., node2). These two dimensions thus cannot be reduced to one Response Polarization dimension (which would be a two-dimensional model).

OPEN IN VIEWER

Figure 3.

Conceptual visualization of an item of the Resistance to Peer Influence Questionnaire (RPIQ) in terms of two different interpretations of a three-dimensional branching model. For the interpretation of the nodes at each branching level, please see the legend to Figure 2. (A) 3RPI conceptualization in which choices to all branching levels are reflective of three distinct RPI dimensions. (B) 1RPI&2POL conceptualization in which choices to the highest branching-level node (node1) are reflective of an RPI dimension, whereas choices to the lowest branching-level nodes (node2 and node3) are reflective of two distinct Response Polarization dimensions.

To decide which alternative best matched the data, we inspected the correlations between all three nodes. ³ If the 3RPI conceptualization best described the multidimensional structure, we would observe all three nodes to be positively correlated, in that higher RPI at one node would be associated with higher RPI at another node. Instead, if the 1RPI&2POL conceptualization best described the multidimensional structure, we would not necessarily observe the highest branching-level node (node1) to be correlated with the lowest branching-level nodes (node2 and node3). More important, however, we would observe the lowest branching-level nodes (node2 and node3) to be negatively correlated. That is, a tendency to respond more polarized after the initial choice to be less peer resistant (i.e., lower Likert-type scale score/left-sided choice to the node2 subitem) would be associated with a tendency to respond more polarized after the initial choice to be more peer resistant (i.e., higher Likert-type scale score/right-sided choice to the node3 subitem).

Interpretation of Multidimensionality: Results

As can be verified in Table 1, we did not observe a positive correlation between all three nodes, rendering the 3RPI conceptualization, which is in line with usage of a 4-point Likert-type scale, not very plausible. In contrast, we did observe low-to-moderate correlations between node1 on one hand and node2 and node3 on the other, and, in accordance with the 1RPI&2POL conceptualization, a negative correlation between node2 and node3. We thus tentatively concluded that, as hypothesized (Hypotheses 1b and 1c), choices to the highest branching-level node (node1) are reflective of more or less RPI, whereas choices to the lowest branching-level nodes (node2 and node3) are reflective of two distinct, but associated, Response Polarization dimensions.

OPEN IN VIEWER

Table 1.

Correlations Between the Three Nodes, as Estimated Under the Initially Selected Three-Dimensional Model.

	Person effect node1	Person effect node2	Person effect node3
Person effect node1	1.000
Person effect node2	.414	1.000
Person effect node3	.409	−.661	1.000

Refined Model Estimation and Selection: Methods

As outlined in the “Model Estimation and Selection: Methods” section, the initially chosen model parameterization was in line with the earlier proposed usage of a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007; 3RPI conceptualization). This choice was inconsistent with the idea that choices to the lowest branching-level nodes (node2 and node3) are reflective of tendencies to respond more or less polarized, that is Response Polarization (1RPI&2POL conceptualization). Since the correlational structure of the initially selected three-dimensional model supports this later 1RPI&2POL conceptualization, model estimation and selection was repeated after reparameterization of the model, such that it was consistent with this conceptualization. This reparameterization entailed that the modeled probabilities for choices to the node2 subitem were mirrored.

After this reparametrization, we fitted, as before, a series of three models: the uni-, two-, and three-dimensional models. As, after reparameterization, the estimated models were identically parameterized and only differed in the number of freely estimated sets of item parameters and person parameters, model fit of these models could again be compared by measures of comparative model fit; model fit of models after reparameterization was not directly compared with model fit of models before reparameterization. Again, both AIC and BIC were inspected, and the model with the lowest AIC and BIC was selected. In case of conflict, additional analyses were performed.

Refined Model Estimation and Selection: Results

AICs of the estimated uni-, two-, and three-dimensional models were 8,625 (unidimensional), 8,416 (two-dimensional), and 8,354 (three-dimensional). BICs of these models were 8,714 (unidimensional), 8,581 (two-dimensional), and 8,601 (three-dimensional). Thus, whereas AIC favored a three-dimensional model, with three sets of item parameters and three person parameters, BIC favored a two-dimensional model, with two sets of item parameters and two person parameters. To resolve this discrepancy, two additional models were estimated. In these models either (instead of both) the sets of item parameters or the person parameters were constrained to be equal over the node2 and node3 subitems. Following previous studies (e.g., Partchev & De Boeck, 2012), likelihood-ratio tests were used to test whether imposing these constraints was tenable. This proved not to be the case; constraining either the sets of item parameters, χ²(9) = 41.98, p < .001, or person parameters, χ²(3) = 55.58, p < .001, to be equal over the node2 and node3 subitems led to a significant worsening in model fit. Therefore, based on this result and in line with AIC, the reparameterized, three-dimensional model, with three sets of item parameters and three person parameters, was retained and finally selected for further analysis. The correlations between nodes for this reparameterized, three-dimensional model are presented in Table 2.

OPEN IN VIEWER

Table 2.

Correlations Between the Three Nodes, as Estimated Under the Finally Selected Reparameterized, Three-Dimensional Model.

	Person effect node1	Person effect node2	Person effect node3
Person effect node1	1.000
Person effect node2	−.414	1.000
Person effect node3	.409	.661	1.000

Effects of Intellectual Ability and Gender: Methods

For comparative purposes, we first performed a regular ANOVA, with intellectual ability (two levels: TD, MBID) and gender (two levels: boys, girls) as between-subjects factors and traditional RPI total scores (based on summing Likert-type scale scores to all items) as dependent variable.

Subsequently, we tested for intellectual ability and gender differences on the RPI and the two Response Polarization dimensions by including effects of intellectual ability (two levels: TD, MBID), gender (two levels: boys, girls), and their interaction at all three nodes of the finally selected reparameterized, three-dimensional branching model. Significance of these main and interaction effects added at each node was evaluated using Wald tests as implemented in the lme4 package (Bates & Maechler, 2009).

A unidimensional branching model is comparable to the linear model that underlies an ANOVA (cf. De Boeck et al., 2011). Therefore, we calculated and compared effect sizes of the effects of intellectual ability and gender on RPI in the unidimensional branching model on one hand and at the node1 subitem of the three-dimensional branching model on the other. Effect sizes, in terms of Cohen’s d, were defined as the ratio between the parameter estimate of the intellectual ability or gender effect and the standard deviation of the (node1) person effect, which was used as an estimate of the population standard deviation of the effect of interest. Note that these branching model effect sizes are measured on a different scale than ANOVA effect sizes, and therefore these cannot be directly compared with one another.

Effects of Intellectual Ability and Gender: Results

The ANOVA on traditional total scores yielded main effects of intellectual ability, F(1, 347) = 14.44, p < .001, d = 0.41; and gender, F(1, 347) = 31.13, p < .001, d = 0.62; whereas the interaction effect was not significant, F(1, 347) < 1, p = .798. MBID adolescents (M [SD] = 28.64 [5.45]) reported to be significantly less RPI than TD adolescents (M [SD] = 30.38 [3.80]); and boys (M [SD] = 28.53 [4.55]) reported to be significantly less RPI than girls (M [SD] = 31.18 [4.66]).

With respect to the multidimensional branching model analysis, it was found that at none of the three nodes the intellectual ability by gender interaction effect reached significance (ps > .05). Therefore, this effect was omitted from the model. There were significant main effects of intellectual ability and gender. More specifically, at the node1 subitem, MBID, compared with TD adolescents, had a significantly lower probability to opt for the more peer-resistant statement. With respect to the lowest branching-level choices, MBID, compared with TD adolescents, were significantly more likely to respond more polarized, both after an initial choice for the more (node3) and the less (node2) peer-resistant statement. In addition, at the node1 subitem, girls, compared with boys, had a significantly higher probability to opt for the more peer-resistant statement. With respect to the lowest branching-level choices, after an initial choice for the more peer-resistant statement (node3), the probability for a more polarized response was significantly higher for girls than for boys. However, after an initial choice for the less peer-resistant statement (node2), no significant gender differences emerged. A similar pattern of findings was obtained after controlling for individual differences in age, by including age as a continuous covariate at each node.

Effect sizes indexing effects of intellectual ability and gender on RPI were smaller for the unidimensional (intellectual ability: d = 0.13, gender: d = 0.38), compared with three-dimensional, branching model (intellectual ability: d = −0.38, gender: d = 0.41). That is, they increased with 65.43% and 7.34%, respectively. The branching model parameter estimates, associated standard errors (SEs), test statistics, levels of significance, and effect sizes of effects of intellectual ability and gender are presented in Table 3.

OPEN IN VIEWER

Table 3.

Parameter Estimates, Standard Errors (SEs), Test Statistics, Levels of Significance, and Effect Sizes of Fixed Effects of Intellectual Ability and Gender, Included in the Reparameterized, Unidimensional Model and in the Reparameterized, Three-Dimensional Model.

Effect	Estimate	SE of estimate	Wald statistic	p	Cohen’s d
Unidimensional model
Intellectual Ability	0.087	0.045	1.939	.053	0.130
Gender	0.314	0.076	4.128	<.001	0.379
Three-dimensional model, node1
Intellectual Ability	−0.283	0.057	−4.961	<.001	−0.376
Gender	0.308	0.060	5.139	<.001	0.409
Three-dimensional model, node2
Intellectual Ability	0.994	0.114	8.747	<.001	0.930
Gender	−0.085	0.110	−0.775	.438	−0.080
Three-dimensional model, node3
Intellectual Ability	0.332	0.075	4.454	<.001	0.317
Gender	0.312	0.076	4.082	<.001	0.297

Note. SE = standard error; TD = typically developing; MBID = mild-to-borderline intellectual disability. Parameterization was chosen such that TD adolescents and boys served as a reference. In the unidimensional model, the intellectual ability and gender parameter estimates then indicate the additional effect of being, respectively, an MBID adolescent or a girl to the probability of opting for the more peer-resistant statement. In the three-dimensional model, at the node1 subitem, the intellectual ability and gender parameter estimates then indicate the additional effect of being, respectively, an MBID adolescent or a girl to the probability of opting for the more peer-resistant statement. At the node2 and node3 subitems, the intellectual ability and gender parameter estimates indicate the additional effect of being, respectively, an MBID adolescent or a girl to the probability of opting for the more extreme statement.

Together, both the ANOVA and multidimensional branching model analysis revealed that, as hypothesized (Hypothesis 2a), adolescents with MBID report to be less peer resistant than adolescents without MBID. Also, these analyses revealed that, as hypothesized (Hypothesis 2b), girls report to be more peer resistant than boys. Moreover, a comparison of effect sizes between a unidimensional versus multidimensional branching model revealed that differences in RPI between TD versus MBID adolescents and boys versus girls were substantially and somewhat larger, respectively, when the multidimensionality of items of the RPIQ was considered, compared with when it was not. In addition, the multidimensional branching model analysis showed that, as hypothesized (Hypothesis 3a), independent of their initial choice, adolescents with MBID responded more extremely than adolescents without MBID. Finally, this analysis revealed that girls responded more extremely than boys, but only in case they initially indicated to be more peer resistant.