Abstract
It has been evident for some time that the Boredom Proneness Scale (BPS), a commonly used measure of trait boredom, does not constitute a single scale. Factor analytic studies have identified anything from two to seven factors, prompting Vodanovich and colleagues to propose an alternative two factor, short form version Boredom Proneness Scale–Short Form (BPS-SR). The present study further investigates the factor structure and validity of both the BPS and the BPS-SR. The two-factor solution obtained for the BPS-SR appears to be an artifact of item wording of reverse-scored items. These same items may also have contributed to the earlier complexity and inconsistency of results for the full BPS. An eight-item scale of only consistently worded items (i.e., those not requiring reverse scoring) was developed. This new scale demonstrated unidimensionality and the scale score had good internal consistency and construct validity comparable to the original BPS score.
Keywords
Boredom represents a negative experience commonly arising in situations deemed deficient in meaning, interest, and challenge, and is thought to motivate us to remediate these deficiencies by modifying behaviors or situations (Sansone, Weir, Harpster, & Morgan, 1992; Smith, Wagaman, & Handley, 2009; van Tilburg & Igou, 2011; van Tilburg, Igou, & Sedikides, 2013). Indeed, it has been suggested that the function of boredom is to regulate behavior by acting as a cue to seek more meaningful alternate goals (Bench & Lench, 2013; Elpidorou, 2014). It is further argued that individuals differ in their propensity to experience boredom. Although several measures of boredom have been developed, the widely used Boredom Proneness Scale (BPS; Farmer & Sundberg, 1986) is the only instrument that attempts to capture the general tendency to experience boredom. Farmer and Sundberg (1986) suggested that the BPS assesses the extent to which an individual is connected to their environment and can access resources to effectively develop mastery. This measure has been associated with negative affect such as depression (Carriere, Cheyne, & Smilek, 2008; Farmer & Sundberg, 1986; Goldberg, Eastwood, LaGuardia, & Danckert, 2011), anxiety (Sommers, & Vodanovich, 2000, Vodanovich, Verner, & Gilbride, 1991), and anger (Dahlen, Martin, Ragan, & Kuhlman, 2004; Rupp & Vodanovich, 1997). Boredom proneness has also been implicated in attention lapses, difficulty sustaining attention, and increased attention deficit hyperactivity disorder (ADHD) symptoms (Carriere et al., 2008; Malkovsky, Merrifield, Goldberg, & Danckert, 2012). Furthermore, it is associated with a number of problem behaviors such as pathological gambling (Blaszczynski, McConaghy, & Frankova, 1990; Mercer-Lynn & Eastwood, 2010) and procrastination (Vodanovich & Rupp, 1999).
The assessment of boredom also has clinical relevance. Boredom proneness has been found to predict quality of life in patients diagnosed with schizophrenia, suggesting the clinical relevance of assessing boredom separately from other commonly assessed amotivational conditions (Gerritsen, Goldberg, & Eastwood, 2015). Likewise, Schubert (1978) suggested that identifying boredom proneness in patients, may aid psychiatrists in understanding patient reactions during treatment. In an open label study of treatment of depression in cancer patients, depression was found to be ameliorated well before boredom. Thus, boredom is likely an important outcome measure, and failure to address boredom may mean we miss important factors potentially relevant to relapse (Theobald, Kirsh, Holtsclaw, Donaghy, & Passik, 2003). Among other things, this clinical relevance highlights the importance of developing an accurate tool to assess boredom. Despite the relationships observed between boredom proneness and a number of negative outcomes and clinically relevant factors, without a more accurate measure, our ability to interpret these relationships is limited.
We and others have suggested that higher levels of boredom proneness are associated with distinct self-regulatory profiles (Blunt & Pychyl, 1998; Mercer-Lynn, Bar, & Eastwood, 2014; Mercer-Lynn, Flora, Fahlman, & Eastwood, 2013; Struk, Scholer, & Danckert, 2015). For example, individuals vulnerable to experiencing boredom are more sensitive to punishment (i.e., a higher sensitivity to the behavioral inhibition system; Mercer-Lynn et al., 2014; Mercer-Lynn, Hunter, & Eastwood, 2013). Finally, boredom proneness has been associated with a general failure to self-regulate behavior (Struk, et al., 2015). In light of growing interest in boredom and the broad relationships observed between boredom proneness and negative psychological and physical outcomes, it has become imperative to have available to researchers a convenient, reliable, and consistent individual-difference measure to capture the construct of boredom proneness.
The BPS is not without problems. There is a considerable volume of research finding that the BPS does not constitute a single scale, but appears to be a multifactorial inventory (Ahmed, 1990; Vodanovich & Kass, 1990). At the same time, the BPS factor structure has proven to be highly unstable, with previous studies having identified anywhere from two to five factors (Melton & Schulenberg, 2009; Vodanovich, 2003; see Supplementary Table S1 [all supplementary tables are available online at http://journals.sagepub.com/doi/suppl/10.1177/1073191115609996] for a full list of all prior factor analytic studies of BPS). Although there remains a lack of agreement as to the number of factors needed to fully capture the variance in the BPS, at least two factors, with somewhat consistent item loadings, have been observed with some regularity across multiple studies. See Table 1, for which items have loaded on these two factors in prior factor analytic solutions. These factors have been interpreted as representing a lack of internal stimulation (internal factor) and lack of external stimulation (external factor; Vodanovich & Kass, 1990; Vodanovich, Wallace, & Kass, 2005). Vodanovich and Kass (1990) suggest that the external factor assesses an individual’s inability to satisfy a high need for excitement, challenge, and change. In other words, individuals may be motivated but are unable to engage in satisfying activities. In contrast, the internal factor assesses an individual’s inability to self-generate interest and engagement. The internal versus external distinction is alleged to contrast two subjective states of boredom: (a) an apathetic state of ennui and (b) an agitated state in which the individual is motivated to engage in their environment but is unable to do so (Danckert, 2013; Greenson, 1953). To address the complexity and inconsistency in the factor structure of the full BPS, Vodanovich et al. (2005) created a short, 12-item version of the scale Boredom Proneness Scale–Short Form (BPS-SR), consisting of two factors, with six items from the original scale contributing to each factor. Both confirmatory factor analysis of the BPS-SR and prior factor structures have, however, yielded a poor model fit, suggesting that the BPS-SR does not have a replicable factor structure (Melton & Schulenberg, 2009). The conceptual internal–external distinction is, moreover, complicated by a methodological problem, as one of the factors—the lack of internal stimulation—consistently comprises only items requiring reverse scoring; as is evident from Table 1, all items that have consistently been found to load on the lack of internal stimulation factor are worded to require reverse scoring.
BPS Items and Prior Factor Analyses Solutions.
Note. E = External Stimulation; I = Internal Stimulation; 1 = Ahmed (1990); 2 = Vodanovich and Kass (1990); 3 = Vodanovich et al. (2005); 4 = Gordon, Wilkinson, McGown, and Jovanoska (1997); 5 = Gana and Akremi (1998); 6 = Melton and Schulenberg (2009). Factor analyses results only include two largest factors. Reverse-scored items are bolded. Items that predominantly load onto the internal stimulation factor are italicized.
Given previous issues identifying and replicating a consistent set of factors in the BPS, the goals of the present study were threefold: (a) to explore the inconsistencies in the factor structure of the original BPS, as well as the BPS-SR; (b) to test whether the two-factor structure of the BPS-SR is based on an artifact; and (c) to test the hypothesis that modification or removal of items contributing to the artificial factor structure will yield a single factor scale. To achieve these goals. we modified the wording of the previously reverse-scored items such that no items were worded in a way that required reverse coding. While reverse wording is often used to combat a number of participant response biases, a number of studies have raised similar concerns about whether this technique is effective or, to the contrary, reduces scale score reliability and validity and creates artificial factor structures (e.g., Harvey, Billings, & Nilan, 1985; Schriesheim & Hill, 1981; van Sonderen, Sanderman, & Coyne, 2013). Accordingly, we anticipated that rewording the reverse-scored items would eliminate an extraneous source of variance contributing to a spurious factor. Furthermore, to determine whether factor structure instability arose from ambiguous items, item response theory (IRT) analysis was employed to evaluate the discriminative ability of each individual item. Guided by the foregoing analyses, we develop and present evidence for a unidimensional Short Boredom Proneness Scale (SBPS) and assess the construct validity of the new scale score.
In reducing the BPS to the SBPS, we felt it was important to determine whether or not the construct validity of the SBPS and its relation to other behavioral measures would be comparable to the full BPS. To achieve this, we used a measure of aggression which in past research has demonstrated a strong positive relationship with boredom proneness as measured by the full BPS. (Dahlen et al., 2004; Rupp & Vodanovich, 1997). In other words, to address the construct validity of the SBPS, we examined the relationship between boredom and aggression using both the full BPS and our newly constructed SBPS, in four separate samples. We performed the same kind of construct validity analyses comparing the full scale with our new SBPS across a number of other variables known to be associated with boredom. Past research has consistently demonstrated strong relationships between boredom (as measured by the original BPS) and negative affective states such as depression and anxiety. Finally, there is a strong and consistent relationship between boredom proneness and failures of attentional functioning, leading to the suggestion that boredom represents a disengaged attentional state (Berlyne, 1960; Cheyne, Carriere, & Smilek, 2006; Eastwood, Frischen, Fenske, & Smilek, 2012; Fisher, 1993; Hamilton, 1981; Harris, 2000). Therefore, to test whether our new scale continues to demonstrate the relationships consistently observed when the full scale BPS is used, we assessed the relationship between the SBPS, measures of anxiety, depression, and stress, as well as a number of attention variables such as ADHD symptoms, spontaneous mind-wandering, and lapses of attention. We chose these factors and measures of aggression to validate our new scale as they were available to us in all samples tested. Another common finding obtained with the full scale BPS is that men score higher than women (Polly, Vodanovich, Watt, & Blanchard, 1993; Sundberg, Latkin, Farmer, Saoud, 1991; Tolor, 1989). In particular, men score higher on the external stimulation factor than do women (Dahlen et al., 2004; McLeod & Vodanovich, 1991; Studak & Workman, 2004; Vodanovich & Kass, 1990; Watt & Vodanovich, 1999; although see Vodanovich et al., 2005 for evidence that men score more highly than women on both the external and internal stimulation scales). Given these gender differences, we felt it was important to test whether the newly developed scale behaves similarly across both genders. We thus monitored how scores differed across both genders in all scale versions tested here.
Study 1
Prior inconsistencies in factor structure of the full BPS and poor model fit of BPS-SR suggest that either versions of the scale does not have a replicable factor structure. Prior to modification of the scale, we wanted to confirm these inconsistencies by conducting an exploratory factor analysis. Furthermore, we wanted to acquire a baseline of relationships between the two BPS-SR factors and other known related constructs in order to monitor how the scale behaves after modifications made in the following study.
Method
Participants and Procedure
Analyses were conducted for two separate samples of undergraduate students who completed mass testing surveys. The first sample consisted of 2,714 participants (802 males, mean age = 20.2 years, SD = 3.50), from whom data were collected during the fall term of 2012. A total of 47.4% identified themselves as Caucasian, 22.1% as East Asian, 12% as other Asian groups, and 18.5% were other groups. The second sample consisted of 2,348 participants (752 males, mean age = 20.4 years, SD = 4.63), from whom data were collected during the winter term of 2013. A total of 47.3% identified themselves as Caucasian, 23.9% as East Asian, 12% as South Asian, and 16.8% were other groups. The participants were students in psychology courses at the University of Waterloo. The use of these data was approved by the University of Waterloo, Office of Research Ethics. Exploratory factor analyses were conducted to investigate the consistency of BPS-SR factor structure. The correlations between BPS-SR subscales and other constructs were determined to monitor any changes in scale behavior after modifications.
Measures
The BPS (Farmer & Sundberg, 1989) is a self-report questionnaire consisting of 28 items (Table 1) rated on a 7-point Likert-type scale ranging from strongly disagree to strongly agree. High scores indicate a higher tendency to experience boredom. In addition to an overall score, the BPS can also be used to obtain scores for one’s lack of internal and external stimulation, the two scales that constitute the BPS-SR. The internal stimulation factor was composed of Items 1, 8, 11, 13, 22, and 24 and the external stimulation factor was composed of Items 6, 9, 19, 25, 27, and 28 (Table 1; Vodanovich et al., 2005).
The Aggression Questionnaire (AQ; Buss & Perry, 1992) consists of 29 items (e.g., “I have become so mad that I have broken things”) rated on 5-point Likert-type scale ranging from extremely uncharacteristic of me to extremely characteristic of me. High scores indicate a higher tendency to experience aggression. The questionnaire consists of four subscales: Physical Aggression, Verbal Aggression, Anger, and Hostility.
Results
Exploratory Factor Analysis
Exploratory factor analysis, employing a principal axis extraction method with a standard varimax rotation, was conducted on the Vodanovich 12-item Boredom Proneness Scale–Short Form (six internal and six external stimulation items; Vodanovich et al., 2005) for the two samples separately. The number of factors were determined using parallel analysis (Horn, 1965), as well as using a minimum average partial test (Zwick & Velicer, 1982, 1986). Both methods converged on a two-factor solution in both samples. Both samples yielded a factor structure similar to that previously reported for the BPS-SR (Table 2; Vodanovich et al., 2005). One item previously designated as “Internal” also had an almost equally strong cross-loading on the “External” factor in both samples. Thus, the two-factor structure was largely corroborated, albeit with one ambiguous item.
Factor Solutions for Exploratory Factor Analysis With Varimax Rotation for the BPS-SR Items for Samples 1 and 2.
Note. BPS = Boredom Proneness Scale, BPS-SR = Boredom Proneness Scale–Short Form. All factor loadings >.4 are in bold.
Descriptive Statistics and Gender Differences
A series of t tests were performed comparing males and females on all study variables in both samples. Sample sizes, means, standard deviations, and t scores for all variables are presented by gender in Supplementary Table S2 for Samples 1 and 2. In both samples, males scored significantly higher than females on the BPS, BPS-SR: External, Physical Aggression, and Verbal Aggression scales. Overall sample sizes, means, standard deviations, and t scores for all variables in Samples 1 and 2 are presented in Supplementary Table S3. The samples differed significantly on all variables except for BPS-SR: External Stimulation factor. All measures of aggression were significantly higher in the Winter 2013 than Fall 2012 sample. Overall BPS score and BPS-SR: Internal Stimulation scores were significantly lower in the Winter 2013 term sample than in the Fall 2012 term sample.
Construct Validity Analysis
To assess BPS-SR score validity, we conducted zero-order correlations between BPS: Overall, BPS-SR: Internal, and BPS-SR: External scores and AQ measures in each sample separately (Table 3). Samples 1 and 2 demonstrated similar correlations both in terms of direction and magnitude for both boredom proneness factors. To directly test whether the correlations differed between the two samples, all correlations were contrasted between groups using z scores (DeCoster, 2007). After Bonferroni correction for multiple comparisons, only Physical versus Verbal, Physical versus Anger, Physical versus Hostility, and Verbal versus Anger correlations differed significantly between samples. In contrast to the External Stimulation factor, Internal Stimulation appears to be a poorly associated with aggression. To directly test whether these correlations differed significantly, we contrasted them in each sample independently using a z test for dependent correlations (DeCoster, 2007). Even after correcting for multiple comparisons, all correlations were significantly higher for External Stimulation and AQ scales than Internal Stimulation, in both samples. Furthermore, it is worth noting that the Internal and External scales were only weakly correlated with one another. This study confirms the inconsistency of BPS-SR factor structure, and provides a measure of relationships between the two BPS-SR factors and measures of aggression, which will be necessary to monitor how the scale behaves once it is modified.
Partial Correlations Between BPS and AQ scores, Controlling for Gender, for Samples 1 (Above Major Diagonal) and 2 (Below Major Diagonal).
Note. BPS = Boredom Proneness Scale; BPS-SR = Boredom Proneness Scale–Short Form; AQ = Aggression Questionnaire. In Sample 1, N varies from 2,562 to 2,679, and in Sample 2, N varies from 2,323 to 2,345. All correlations that are significant at .01 alpha level are in bold.
Study 2
The Internal Stimulation scale is exclusively composed of reverse-scored items. It is therefore possible that these items form a factor simply by virtue of their reverse wording, relative to items comprising the External Stimulation factor (Table 1). It is also plausible that the reverse wording changes the focus from boredom to active engagement and, as such, this factor may simply describe the absence of dimensions such as curiosity, imaginativeness, or active engagement that may have complex relations with boredom proneness, or none at all. If so, rewording these items positively may shift the focus back to boredom per se, attenuating or eliminating this factor.
To investigate the possibility that the two-factor structure of the BPS-SR is an artifact of item wording (i.e., the presence of reverse-worded items), we reworded these items such that the entire scale is now worded in a manner consistent with the remainder of the scale—that is, as positively endorsed statements (see Supplementary Table S4 for revised items). To assess the effects of scale rewording, we collected a new sample of data with the modified scale and, as in Study 1, conducted an exploratory factor analysis on the BPS-SR. We also correlated the overall scale score, as well as scores from individual BPS-SR factors (as they have been defined in the original version of the scale) with scales of AQ, and evaluated the utility of the full set of 28 BPS items, again after rewording, using IRT analyses.
Method
Participants, Measures, and Analyses
The third sample consisted of 1,007 participants (376 male, mean age = 21.2 years) who completed the modified BPS scale (with all reverse-scored items now directly worded; Supplementary Table S4) in the spring term of 2013. A total of 31.5% identified themselves Caucasian, 38% as East Asian, 17.1% as South Asian, and 13.4% were other groups. Participants also completed the AQ. An exploratory factor analysis was conducted on the modified BPS-SR items (with internal subscale items reworded) to investigate possible differences in the factor loadings obtained from the original and modified versions of the scale. The correlation between modified BPS-SR subscales and other constructs was assessed to examine the effects of scale modification on score validity. In addition, IRT analyses were utilized to determine the contribution of each BPS item in discriminating between different levels of trait boredom. Based on the results of these analyses, estimated item parameters were used to develop a shorter, more reliable version of the BPS and a confirmatory factor analysis was conducted to test the unidimensionality of the shortened scale. Finally, measurement invariance of the final SBPS was assessed.
Results
Exploratory Factor Analysis
Exploratory factor analysis, employing a maximum likelihood extraction method with a standard varimax rotation was conducted on the modified 12 BPS-SR (six internal items—reworded—and six external items). The number of factors were determined using parallel analysis, as well as using a minimum average partial test, with both methods converging on a single factor solution (Table 4). These factor loadings suggest the modified scale behaves like a single factor. Furthermore, the total variance explain by this single factor is greater than the sum of the variance explained by both factors of the unmodified BPS-SR, suggesting that the modified scale does indeed have more utility.
Single-Factor Solution for Exploratory Factor Analysis for the Modified BPS-SR Items.
Note. BPS = Boredom Proneness Scale, BPS-SR = Boredom Proneness Scale–Short Form. All factor loadings >.4 are in bold.
Descriptive Statistics and Gender Differences
A series of t tests was performed between males and females on all study variables. Sample sizes, means, standard deviations, and t scores for all variables are presented by gender in Supplementary Table S5, (see Supplementary Table S6 for descriptive statistics for the combined sample). Males scored significantly higher than females on BPS, BPS-SR: External, BPS-SR: Internal, Physical Aggression, and Verbal Aggression, as was the case in our first two samples (Supplementary Tables S2 and S3), with exception of BPS-SR: Internal which began to differentiate between gender in its modified form.
Construct Validity of BPS-SR With the Reverse-Worded Scale
To investigate whether modifying the BPS-SR (rewording internal stimulation factor items) altered the relationship between the scale and other measures, we conducted zero-order correlations between scores from the original BPS-SR factors and AQ scales, for both the original and the modified scales (Table 5). Both External and Internal Stimulation scales appear to yield similar correlations with aggression variables in terms of both direction and magnitude. To directly test whether these correlations differed significantly, we contrasted the correlations using a z test for dependent correlations (DeCoster, 2007). After Bonferroni corrections for multiple comparisons, only the External versus Verbal Aggression correlation was significantly higher than the Internal versus Verbal Aggression correlation. Thus, rewording items of the original Internal stimulation scale meant that both it and the External Stimulation factors behaved more like each other and more like the total BPS score (see Table 5 for comparison).
Partial Correlations Between Modified BPS and AQ scores, Controlling For Gender.
Note. BPS = Boredom Proneness Scale, BPS-SR = Boredom Proneness Scale–Short Form, AQ = Aggression Questionnaire. N varies from 991 to 1,003. All correlations are significant, p < .01.
Estimated Item Parameters
The two prominent factors previous studies have identified as comprising the majority of the BPS items and comprising the entirety of the BPS-SR (i.e., a lack of external and internal stimulation) began to behave in a similar manner after the reverse-worded items were changed to be directly worded. This analysis clearly fails to replicate the previous internal–external stimulation distinction. Furthermore, the large correlation between the scores of the BPS-SR: External and BPS-SR: Internal (r = .70 as opposed to a weaker r = .23-.31 found in Studies 1 and 2, respectively), renders the utility of retaining a distinction between internal and external sources of boredom in the BPS questionable. It is also worth noting that previous factor analytic studies have not established a good model fit for the two-factor (or indeed multifactor) BPS (Melton & Schulenberg 2009; Vodanovich et al., 2005). We believe this gives us sufficient grounds to suggest that the full BPS is assessing a single coherent construct, though obscured by several nuisance factors, such as the reverse-wording problem and highly correlated item pairs.
To investigate this directly, we explored the utility of individual items in the full BPS, again with all previously reverse-scored items reworded, using IRT (de Ayala, 2009; Embretson & Reise, 2000). IRT examines the capacity of individual items to discriminate among participants at different trait levels, as well as exploring how much information each item contributes to the whole scale. This analysis was conducted using the ltm package (v 0.9-9; Rizopoulos, 2006) in the R Environment for Statistical Computing (v 3.0.2; R Development Core Team, 2008). Category response curves illustrating the relation between an individual’s estimated trait level and the probability of choosing a given response on a particular item are shown in Figure 1. We employed a 7-point rating scale response format in our use of the BPS, and hence a graded response model (de Ayala, 2009; Embretson & Reise, 2000; Ostini & Nering, 2006) was an appropriate model to explore scale items. The graded response model provides an estimation of an item’s discrimination parameter (αi), which represents an estimate of an item’s ability to differentiate between individuals of varying trait levels (see appendix for complete list of item parameter estimates). Values of αi = 0.01 to 0.24 are considered very low, 0.25 to 0.63 low, 0.65 to 1.34 moderate, 1.35 to 1.69 high, and >1.7 very high (Baker, 2001). Thus, the most optimally discriminable items tend to produce a roughly U-shaped pattern in Figure 1, where lower response options are highly probable at low trait levels, and higher response options are highly probable at high trait levels, with very little overlap.
Category response curves for all 28 items of the Boredom Proneness Scale.
Item discrimination parameters spanned from 0.45 to 2.71, ranging from low to very high (Supplementary Table S7), for example Item 20 as a low discrimination item, “I would like more challenging things to do in life” and Item 13 as a high discrimination item, “In most situations, it is hard for me to find something to do or see to keep me interested.” The average item discrimination parameter was 1.43, suggesting overall high scale discriminability. However, our results demonstrated that many items (1, 2, 3, 6, 15, 17, 18, 20, 21, 22, 24, 26, and 27) had moderate-to-low discriminability and thus may reflect ambiguous items that contribute to the previously observed factor structure instability. Accordingly, the 15 items with the highest discriminability (α > 1.35), accounting for 71% of the total information in the scale (i.e., overall discriminability across all response options within items), were tested for unidimensionality using confirmatory factor analyses.
Confirmatory Factor Analyses
To test the unidimensionality of the proposed model, which consists of the 15 items of highest discriminability, we conducted a confirmatory factor analysis using AMOS v 21.0.0 (Arbuckle, 2012) on the current sample. The comparative fit index (CFI; Bentler, 1990), Tucker–Lewis index (TLI; Bentler & Bonett, 1980), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR) were all used to assess the model fit. Hu and Bentler (1999) recommend the use of the following criteria to establish a good model fit: CFI ≥ .95, TLI ≥ .95, RMSEA ≤ .06, SRMR ≤ .08.
Using these criteria, the proposed 15-item model demonstrated poor fit to the observed data: χ2(90) = 898.55, p < .001, CFI = .879, TLI = .859, RMSEA = .099. The SRMR (.053) was the only measure that indicated a reasonable fit. Overall, the analysis suggests that the 15-item scale does not reflect a unidemensional model of boredom proneness. A poor observed model fit often results from unexplained covariance among certain pairs or sets of items within a questionnaire. In our case, an examination of the residual covariances revealed seven pairs of items with significant unexplained covariance between them. In these cases, the items with relatively lower discriminability in each pair were removed in an attempt to produce a more parsimonious and well-fitting model. Removal of high residual covariance item pair members was conducted in a stepwise fashion. In each step, we first selected the item pair with the highest covariance, and within that pair, the item with the lowest level of discrimination was removed (see Supplementary Table S8 for item pairs in question). We then reevaluated the residual covariance for the remaining items. This was repeated until there were no highly covaried item pairs. This led to eight items being included in the SBPS (Items 4, 8, 9, 10, 11, 13, 14, and 25 of the modified BPS; see appendix) for all further analyses.
A second confirmatory factor analysis assessed the unidimensionality of the eight-item version of the BPS. Fit indices generally indicated a good fit of the single factor model to the observed data: CFI = .976, TLI = .967, RMSEA = .062, SRMR = .026. Although the chi-square result, χ2(20) = 97.65, p < .001, suggests poor fit, the usefulness of this measure has been debated, especially for large sample sizes (Bentler & Bonnet, 1980; Brown, 2006). Overall, the model can be regarded as having a good fit.
Next, we assessed the measurement equivalence of the SBPS by testing configural, metric, and scalar invariance across gender (Steenkamp & Baumgartner, 1998; Vandenberg & Lance, 2000). As recommended by Cheung and Rensvold (2002), to compare models with different levels of measurement invariance constraints, we assessed, in addition to CFI, Steiger’s (1989) gamma hat (GH) and McDonald’s (1989) noncentrality index (NCI) for each invariance model. If in the sequence of progressively constrained invariance tests, two models demonstrate a decrease in value of CFI, GH, and NCI greater than or equal to .01, .01, and .02, respectively, then the more restrictive model should be rejected (Cheung & Rensvold, 2002; Milfont & Fischer, 2010). To achieve configural invariance, the model has to provide an adequate fit across both groups, while factor loadings are free to vary (Horn & McArdle, 1992). Configural model equivalence indicated a good fit with the data: χ2(40) = 125.08, CFI = .974, TLI = .964, RMSEA = .046, SRMR = .033, GH = .976, NCI = .966, suggesting that both males and females display a similar factor structure. In order to test for metric invariance, the factor loadings are constrained to be equal across both groups (Horn & McArdle, 1992). Metric model equivalence also indicated a good fit, χ2(48) = 136.20, CFI = .973, TLI = .969, RMSEA = .043, SRMR = .042, GH = .981, NCI = .961. Furthermore, when compared with the configural model, changes in CFI, GH, and NCI indicated no change in model fit, suggesting that factor loadings were invariant across gender. Finally, to test for scalar invariance, all item intercepts are constrained to be equal across groups (Steenkamp & Baumgartner, 1998). Scalar model equivalence also indicated a good fit, χ2(56) = 161.59, CFI = .968, TLI = .968, RMSEA = .043, SRMR = .043, GH = .978, NCI = .949. Again, when compared with the metric model, the changes in CFI, GH, and NCI indicated that the degree of fit does not change with additional constraints, suggesting that differences in item means are genuinely due to differences in the latent variable.
Study 3
Having reduced the 28-item BPS to an eight-item, single-factor SBPS in Study 2, we sought to again evaluate its psychometric properties and confirm the validity of the scale score in an additional large sample of participants. We also chose to assess correlations among the SBPS and measures of aggression, depression, anxiety, stress, mindfulness, and attention, since these variables have all previously demonstrated significant relations with the full BPS (Carriere et al., 2008; Dahlen et al., 2004; Farmer & Sundberg, 1986; Goldberg et al., 2011; Harris, 2000; LePera, 2011; Malkovsky et al., 2012; Merrifield & Danckert, 2014; Rupp & Vodanovich, 1997; Sommers, & Vodanovich, 2000; Vodanovich et al., 1991).
Method
Participants
The present sample consisted of 2,592 undergraduate student participants at the University of Waterloo (710 males, mean age = 20.21 years), collected during a mass testing session in the fall term of 2013. A total of 48.9% identified themselves as Caucasian, 23.3% as East Asian, 16.4% as other Asian groups, and 11.4% was other groups.
Measures and Analyses
As in the previous studies, we assessed both the SBPS and the AQ. In addition to these measures, the present study also included the short form Depression Anxiety Stress Scale (DASS-21; Lovibond & Lovibond, 1995). This scale consists of 21 items (e.g., “I felt that life was meaningless”) rated on a 4-point Likert-type scale ranging from did not apply to me at all to applied to me very much, or most of the time. The scale consists of three factors with seven items each, measuring depression, anxiety, and stress. High scores indicate elevated levels of symptoms.
This study also included the Adult ADHD Self-Report Scale Screener (ASRS-Screener; Kessler et al., 2005) which consists of six items (e.g., “How often do you feel overly active and compelled to do things, like you were driven by a motor?”) rated on a 5-point Likert-type scale ranging from not at all to very often.
The Mind-Wandering: Spontaneous Scale (MWS; Carriere, Seli, & Smilek, 2013) was also included in this study. This scale consists of four items (e.g., “I find my thoughts wandering spontaneously.”) rated on a 7-point Likert-type scale ranging from rarely to a lot. High scores indicated increased tendency to mind-wander unintentionally—a form of lapsed attention.
The Five Facet Mindfulness Questionnaire–Acting with Awareness Subscale (FFMQ-A; Baer, Smith, Hopkins, Krietemeyer, & Toney, 2006), was also used here. This scale consists of five items (e.g., “I find it difficult to stay focused on what’s happening in the present.”) rated on a 5-point Likert-type scale ranging from never or very rarely true to very often or always true. This scale is a short form of the Mindful Attention Awareness Scale–Lapses Only (Carriere et al., 2008). High scores on the FFMQ-A are associated with a lower tendency to experience lapses of attention.
To replicate the analyses of our previous study, a confirmatory factor analysis, as well as measurement invariance analyses were conducted on the SBPS. Finally, we assessed the reliability of the final SBPS score and examined its construct validity in terms of its relations with the AQ, DASS, ASRS, MWS, FFMQ-A.
Results
Descriptive Statistics
A series of t tests was performed between males and females on all study variables. Sample sizes, means, standard deviations, and t scores for all variables are presented by gender in Supplementary Table S10 (see Supplementary Table S11 for descriptive statistics for the combined sample). Males scored significantly higher than females on SBPS, Physical Aggression, Verbal Aggression, and ASRS, and significantly lower than females on the Anxiety and Stress measures.
Confirmatory Factor Analysis
A confirmatory factor analysis assessed the unidimensionality of the SBPS. As in Study 2, fit indices indicated a good fit of the single factor model to the observed data, CFI = .978, TLI = .970, RMSEA = .061, SRMR = .025, with the exception of the chi-square result, χ2(20) = 205.57, p < .001; however, as mentioned earlier, use of the chi-square test may be problematic for large samples.
Next, we assessed the measurement equivalence of the SBPS by testing configural, metric, and scalar invariance across gender. As in Study 2, configural, metric, and scalar model equivalence all indicated a good fit with the data (configural model: χ2(40) = 218.00, CFI = .979, TLI = .971, RMSEA = .042, SRMR = .026, GH = .984, NCI = .959; metric model: χ2(48) = 256.26, CFI = .975, TLI = .971, RMSEA = .042, SRMR = .040, GH = .981, NCI = .957; scalar model: χ2(56) = 314.05, CFI = .970, TLI = .970, RMSEA = .043, SRMR = .039, GH = .978, NCI = .949). Progressive constraints on the invariance models did not affect the fit of each model, as assessed by change in CFI, GH, and NCI, providing support for measurement invariance.
Construct Validity and Reliability of the Final Model Score
To test the construct validity of the new eight-item SBPS score, we conducted zero-order correlations between the scores of SBPS and Aggression and DASS subscales separately, as well as ASRS, MWS, and FFMQ-A (Table 6). Importantly, the new scale is appropriately and significantly correlated with all aggression measures, as well as with depression, anxiety, and stress, ADHD symptoms, spontaneous mind-wandering, and lapses of attention. Cronbach’s alpha for the eight-item scale was 0.88, suggesting excellent internal consistency.
Pearson Product-Moment Correlation Coefficients for SBPS, AQ, DASS, ASRS, MWS, and FFMQ-A scores for Sample 4.
Note. SBPS = Short Boredom Proneness Scale; AQ = Aggression Questionnaire; DASS = Depression Anxiety Stress Scale; ASRS = ADHD Self-Report Scale; MWS = Mind-Wandering: Spontaneous Scale; FFMQ-A = Five Facet Mindfulness Questionnaire–Acting with Awareness Subscale. N varied between 2,440 and 2,502.
Discussion
Our study provides substantial evidence that the inconsistencies in the results of previous factor structure analyses of the BPS can be largely accounted for by the use of reverse-scored items, as well as similarly worded item pairs that tended to cluster together to form separate factors. After rewording items to create a consistently worded scale, the previous two-factor structure observed for the BPS-SR was markedly compromised and the resulting Internal and External Stimulation scales found to be largely redundant. In addition, by resolving the issue of item pairs having high residual covariance, we were able to construct a robust, unidimensional measure of boredom proneness. Analysis of the new eight-item SBPS provided good evidence of unidimensionality, and that its score is a highly reliable and valid measure of boredom proneness.
These results cast further doubt on the necessity, utility, or even coherence of using the external/internal factor distinction or multifactor interpretations of the BPS. Considering the evidence that the BPS factor structure may be an artifact of poor item wording, results of prior studies that considered the BPS to be a multifactorial measure, including our own studies, should be interpreted with care. Although we cannot rule out the possibility that multiple boredom proneness dimensions exist, it does nonetheless seem to be the case that the original full BPS does not reliably or consistently capture them. As such, further research on the multidimensionality of the BPS would seem unlikely to be productive unless guided by a novel theoretical model specifying such hypothetical factors.
Our present analyses also provide robust evidence that the Internal versus External Stimulation distinction postulated in prior factor analyses of the BPS and BPS-SR is also an artifact of particular item wording. After modifying problematic reverse-scored items, we were able to identify a core set of items yielding a single factor solution and a scale that appears to reflect a single dimension of boredom proneness. Although this finding questions the utility of the BPS factor structure, we expect that this does not change prior known relationships between the full BPS and other constructs.
Pekrun, Goetz, Daniels, Stupnisky, and Perry (2010) identified a lack of optimally stimulating and valued activities as antecedents of boredom. Likewise, Goetz et al. (2014) demonstrated that boredom often occurred when individuals desired, sought, or attempted to engage in meaningful activities. Consistent with these theoretical accounts of boredom, Eastwood et al. (2012) suggest that boredom is best thought of in terms of wanting, but not being able, to engage in a satisfying activity. We argue that items that are included in the SBPS are consistent with this conceptualization of boredom. Furthermore, SBPS items seem to assess an individual’s capacity or situational disposition to access and find satisfactory outlets for engagement. Face validity analysis of the items comprising the SBPS is also consistent with current models characterizing state boredom as a cue to regulate behavior, in order to engage in more satisfactory alternatives, and trait boredom as reflecting a failure to regulate oneself in a manner which would result in effective engagement (Bench & Lench, 2013; Eastwood et al., 2012; Elpidorou, 2014, Struk et al., 2015). Consistent with these views, we found scores on the SBPS to be highly related to poor attentional ability and negative affect, as would be expected from individuals who are unable to effectively engage attention and satisfy their needs and desires.
Limitations and Implications
A major limitation of the studies reported here is that they have all been based on samples of undergraduate students attending the same university. Differences between the original BPS-SR structure and that observed in Study 1 may depend on a variety of demographic (e.g., cultural backgrounds) variables not explicitly tested here. This also implies that the factor structure of the newly developed SBPS may be sample specific, and thus warrants further testing across a wide range of samples that differ in terms of key factors such as culture, age bracket, and so on. The scale developed here would clearly benefit from further validation, by establishing criterion validity, discrimination validity, and by confirming that other objective indicators of boredom proneness correlate with the newly developed measure. More generally, advances in conceptualization of boredom proneness itself are needed to fully establish the validity of the SBPS. Finally, a high overlap in category response curves derived from the IRT analysis was observed, suggesting that individuals have difficulty differentiating between response options on a 7-point Likert-type scale. We thus recommend to use fewer response options (e.g., 5-point Likert-type scale) for future applications. Nevertheless, the results shown here have the potential to clarify the construct of boredom proneness suggesting it is better characterized as comprising a single component. We suggest this component is characterized by an individual’s capacity (or failure) to engage in sufficiently satisfying activities. Furthermore, this unidimensional construct may be adequately captured using the SBPS. This scale may serve as a shorter and more reliable tool to assess boredom proneness than the full BPS, which would be especially helpful in clinical settings given the often onerous diagnostic tests patients undergo.
Footnotes
Appendix
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Natural Sciences and Engirneering Research Council of Canada funding to JD, AS.