Abstract
Commensurate measures of alcohol-related consequences across countries and cultures are critical for addressing the global burden of hazardous alcohol use. The Rutgers Alcohol Problem Index (RAPI), developed and validated in the United States, is a popular measure of alcohol problems. This study examined measurement invariance of the RAPI across samples of U.S. and Swedish high school seniors. Latent mean differences in alcohol problems across countries and differences in associations between alcohol problems with alcohol use and protective behavioral strategies (PBS) were also examined. The RAPI was scalar invariant. Swedish students reported fewer problems than U.S. students (latent mean difference = −0.19, p = .047). In both samples, the RAPI was positively correlated with alcohol use frequency and quantity (ps < .001), and negatively correlated with PBS use (ps < .05). Overall, the RAPI demonstrated measurement invariance, and we found evidence for its validity across samples of U.S. and Swedish high school seniors.
Alcohol use peaks in young adulthood with a substantial proportion (26%) of 12th graders in the United States (U.S.) reporting using alcohol in the past 30 days despite it being illegal (Johnston et al., 2022). Furthermore, 11.8% of 12th graders in the U.S. report having engaged in binge drinking in the past 2 weeks (Johnston et al., 2022). Similarly, heavy episodic drinking (defined as consuming 60 or more grams of pure alcohol on at least one occasion) is relatively common in Sweden, with 28% of the population (15 years of age and older) reporting an episode of heavy episodic drinking in the past 30 days (World Health Organization, 2018). Early onset of alcohol use increases the likelihood that an individual will develop an alcohol use disorder (a continued condition involving impaired control over alcohol use that often increases over time, despite significant impairment or distress; American Psychiatric Association, 2013) and is associated with increased negative mental health and social problems over time (Marshall, 2014; Poudel & Gautam, 2017; Richmond-Rakerd et al., 2017). Furthermore, alcohol use, particularly heavy use, has been associated with a number of proximal (e.g., hangover, assault, risky sexual behavior, suicide, drink-driving) and distal (e.g., academic failure, relationship difficulties, physical illness, impaired adult cognitive functioning) negative consequences (Allen et al., 2011; Marshall, 2014; National Research Council and Institute of Medicine, 2004; Schwebel et al., 2019).
The Young Adult Alcohol Consequences Questionaire (YAACQ; Read et al., 2006) and the Rutgers Alcohol Problem Index (RAPI; White & Labouvie, 1989) are two of the most commonly used measures of negative alcohol-related consequences. The YAACQ focuses on a broad spectrum of consequences while the RAPI focuses on more severe consequences. The RAPI is a relatively brief, unidimensional self-report measure widely used in the U.S. to assess negative alcohol-related consequences among adolescents and young adults (Prince et al., 2018). The RAPI is one of the oldest measures of negative alcohol-related consequences. It has been validated among clinical (White et al., 1988) and non-clinical (White & Labouvie, 1989) samples and has high internal consistency (Cronbach’s alpha = .92). RAPI scores are substantially correlated with alcohol use intensity (0.20–0.57) but low enough to suggest it provides additional information beyond self-reported levels of drinking (White et al., 1988). The RAPI has been shortened to decrease bias (Earleywine et al., 2008) and has been modified for use across substances (Johnson & White, 1995). Although most RAPI-related research has occurred in the U.S. (e.g., Skalisky et al., 2019; Villarosa-Hurlocker & Madson, 2020; Woloshchuk et al., 2022), it has also been used internationally. For example, the RAPI has been adapted and validated for use in Spain (López-Núñez et al., 2012) and with First Nations populations in Canada (Noel et al., 2010). In addition, the RAPI has been used in longitudinal and epidemiological studies in New Zealand (Fergusson et al., 2002), Norway (Pedersen & Skrondal, 1998), Russia (Koposov et al., 2002), and Finland (Pascale et al., 2022; Viken et al., 2007). The international and translated versions of the RAPI have consistently differentiated between low- and high-problem drinkers (Koposov et al., 2002; López-Núñez et al., 2012; Noel et al., 2010), and the level of psychological distress (López-Núñez et al., 2012) is regularly used as an outcome variable (Fergusson et al., 2002; Koposov et al., 2002; Pascale et al., 2022; Pedersen & Skrondal, 1998; Viken et al., 2007).
Although international validation efforts of the RAPI have generally been successful, it is important to continue to examine whether the RAPI is equivalent across young adults in different countries representing distinct cultures. It is especially important to validate the RAPI cross-culturally for young adults given it includes items related to academic and social consequences (e.g., “missed a class because of drinking” and “had a fight with a roommate because of drinking”). Establishing the equivalency (e.g., measurement invariance) of the RAPI is necessary for making meaningful comparisons in the construct of interest across groups. Once equivalency is established, researchers can test substantive questions that are critical for culturally modifying alcohol interventions to address the global burden of hazardous alcohol use among young adults. Potential substantive questions include, are young adults in certain countries at greater risk of alcohol problems? And, do certain risk (e.g., indicators of alcohol use) and protective factors (e.g., protective behavioral strategies [PBS]) have similar relations with alcohol problems across young adults in different countries? PBS, or the use of behaviors before, during, or after drinking to reduce alcohol use, intoxication, and negative consequences, are of particular interest as they are a possible area of intervention. PBS use has been related to decreased alcohol consumption and negative alcohol-related consequences (Pearson, 2013; Pearson et al., 2013). Examination of PBS use among adolescents outside the U.S. are limited. One study adapted the Protective Behavioral Strategies Scale (PBSS; Martens et al., 2005) in Spanish and found this version to be reliable (Sánchez-García et al., 2020). Grazioli et al. (2015) found initial support for the generalizability of PBS among Swedish high school drinkers.
Despite the need to establish measurement invariance of the RAPI to evaluate cross-cultural associations between risk and protective factors and consequences, to date, only one study has compared measurement of negative alcohol-related consequences among college students using the RAPI in the U.S. and Mexico (Armenta & Cooper, 2019). They found the latent consequences variable functioned well in measuring negative alcohol-related consequences in both countries and that alcohol-related problems were positively correlated with drinking frequency, quantity, and intensity across their sample. Further examinations of RAPI measurement invariance in other countries are merited.
Swedish secondary school students completing their final year (equivalent to U.S. high school seniors) make for an interesting comparison with their U.S. equivalent. Similar college drinking trajectories noted by Schulenberg et al. (1996) in the U.S. have been found in Swedish samples (Andersson et al., 2007; Johnsson et al., 2008). In addition, numerous legal policies and structural similarities and differences make a U.S. and Sweden comparison theoretically interesting. In the U.S., the minimum legal drinking and purchasing age is 21 years, and in most states, beer, wine, and spirits can be purchased from private sellers (e.g., grocery/convenience stores). In Sweden, the legal purchasing age is 20 at Systembolaget (the government-controlled retail alcohol monopoly) and 18 in restaurants (Boman & Engdahl, 2006). In Washington state, you can get a driver’s license at age 16, and the blood alcohol concentration legal limit is 0.02% until age 21 years, after which it is 0.08%; in Sweden, you can get a driver’s license at age 18, and the blood alcohol concentration limit is 0.02% at any age. The European School Survey Project on Alcohol and Other Drugs (ESPAD) estimated lifetime alcohol use, past 30-day alcohol use, and past 30-day alcohol intoxication rates among 15- and 16-year-olds as 47%, 22%, and 10% in the U.S. and 65%, 26%, and 9% in Sweden, respectively (ESPAD Group, 2016). Similar results have been found among college students in the U.S. and Sweden, with Swedish students drinking more on average than U.S. students, but U.S. sorority/fraternity students (i.e., heaviest drinkers in U.S. universities) drinking more than Swedish residence hall students (i.e., heaviest drinkers in Swedish universities) (Ståhlbrandt et al., 2008). Furthermore, cultural differences in the relationship to alcohol exist between the U.S. and Sweden. Drinking remains illegal for U.S. high school seniors, whereas drinking during the final year of secondary school is common (and legal) in Sweden with organized trips to foreign countries with inexpensive alcohol commonly occurring in the Skåne region of Sweden (Larimer et al., 2021). Given these similarities and differences, it is important to ascertain that the RAPI is measuring the same constructs in both the U.S. and Sweden.
Purpose of Present Study
The RAPI was developed and validated primarily in U.S. adolescent samples. As one of the oldest, most widely used, and well-validated measures of negative alcohol-related consequences for adolescents, a prerequisite to the use of this measure with adolescents outside of the U.S. is the establishment of measurement invariance. The present study sought to examine the following questions: (a) Is the RAPI measure commensurate across adolescents in the U.S. and Sweden (i.e., measurement invariant)? (b) Are there differences in the extent to which alcohol problems are experienced across countries? And (c) are there differences in the magnitude of associations of alcohol problems with risk (i.e., alcohol use) and protective factors (i.e., PBS) across countries? Measurement invariance would suggest that the RAPI assesses negative alcohol-related consequences commensurately among Swedish adolescents, and therefore comparisons with U.S. samples are appropriate.
Method
Participants and Procedures
Participants in this secondary analysis were students recruited from 40 high schools in the U.S. (n = 22) and Sweden (n = 18) to participate in a longitudinal study of alcohol use during the transition from secondary school (i.e., high school). For the complete recruitment procedures, see Larimer et al. (2021). Participants were included in the analytic sample if they completed the RAPI and reported consuming alcohol during the past 3 months. The final sample included 2,084 high school seniors from the U.S. (n = 415; 56.1% female) and Sweden (n = 1,669; 55.5% female). In the U.S. sample, most participants were White (77.6%). Racial/ethnic background was not assessed among Swedish participants as this information is typically not assessed by studies in Sweden (Larimer et al., 2021). The average number of negative alcohol-related consequences reported were 4.51 and 2.88 in the U.S. and Sweden, respectively. In the U.S., the average number of drinking days per week was 2.57 (SD = 1.66), the average typical number of drinks consumed was 5.95 (SD = 6.05), and the average peak number of drinks consumed was 5.77 (SD = 5.01); in Sweden, the average number of drinking days per week was 2.82 (SD = 1.82), the average typical number of drinks consumed was 6.23 (SD = 3.75), and the average peak number of drinks was 9.03 (SD = 5.67). In terms of PBS used during the previous 6 months, U.S. students reported an average of 11.9, and Sweden students reported an average of 11.0. Participants in the U.S. received $20, and participants in Sweden received a cinema voucher for 100 SEK (approximately $14) to compensate them for their time in participating.
Measures
Measures in the current study were part of a larger battery of assessments. Only baseline data were used. All measures were translated into Swedish and back-translated to check for translation accuracy.
Negative Alcohol-Related Consequences
Negative alcohol-related consequences were assessed using a 26-item version of the RAPI (White & Labouvie, 1989) that included three items added to assess severity of drink-driving at doses ranging from one to four or more drinks (see Supplemental Appendix S1 for full list of items). Responses were measured on a five-point response scale (0 = Never, 4 = More than 10 times). In the original psychometric study, White & Labouvie found the RAPI had excellent internal consistency (α = .92). Other studies have found strong internal consistency and concurrent validity (Ginzler et al., 2007) and criterion-related validity (Shono et al., 2018). Cronbach’s alpha was .95 for this study.
Alcohol Consumption
The Quantity/Frequency/Peak Alcohol Use Index (Dimeff et al., 1999; Marlatt et al., 1998) was used to assess peak and typical number of alcoholic drinks consumed on one occasion over the past month. The Daily Drinking Questionnaire (Collins et al., 1985) was used to assess typical number and peak number of drinks consumed per occasion and the average number of days per week participants consumed alcohol. Both questionnaires have been previously used in studies of young adults in the U.S. (e.g., D’Amico & Fromme, 2000) and Sweden (e.g., Gajecki et al., 2014).
Protective Behavioral Strategies
PBS were assessed using a 15-item version of the PBSS (Martens et al., 2005). Participants were asked to respond “How often they engaged in the following behaviors when using alcohol or ‘partying’ in the past six months” using a six-point scale (1 = Never, 6 = Always). Items are averaged to create three subscales: limiting/stopping drinking (seven items; “Avoid mixing different types of alcohol”), manner of drinking (five items; “Stop drinking at a pre-determined time”), and serious harm reduction (three items; “Use a designated driver). Internal consistency estimates (Cronbach’s alpha) for the limiting/stopping drinking (U.S.: α = .86, Sweden: α = .79) and manner of drinking (U.S.: α = .65, Sweden: α = .75) subscales were adequate, whereas internal consistency estimates for the serious-harm-reduction subscale were inadequate (U.S.: α = .65, Sweden: α = .62); these estimates are consistent with prior studies on the PBSS (e.g., Martens et al., 2005).
Analysis Plan
Prior to testing measurement invariance, we first examined whether a single common factor model of the RAPI provided a good fit to the data for U.S. and Sweden participants separately. Given strong support for a single common factor model for the RAPI in the extant literature, we did not consider testing alternative models. We conducted a confirmatory factor analysis (CFA) with weighted least square mean and variance (WLSMV)-adjusted estimation given the ordinal nature of the data. Simulation studies have shown that WLSMV is robust to modest violations of underlying normality (Flora & Curran, 2004). The latent variable was given a metric by setting the first factor loading to one. Furthermore, we estimated correlations among the residuals of the three drink-driving items as we expected that they would have shared variability given their interdependency (e.g., endorsing Item 26 requires also endorsing Items 24 and 25) beyond that explained by the latent variable. To evaluate model fit, we used joint fit criteria suggested by Hu and Bentler (1999) including Comparative Fit Index (CFI) ≥.90 (acceptable) and ≥.95 (optimal), Tucker–Lewis Index ≥.90 (acceptable) and ≥.95 (optimal), root mean square error of approximation (RMSEA) ≤.06, and standardized root mean square residual ≤.08.
To evaluate measurement invariance, we conducted a series of multi-group CFAs placing increasingly restrictive equality constraints across the groups (U.S. and Sweden). Specifically, we tested three levels of measurement invariance: configural (whether the factor structure is equivalent across groups), metric (whether the factor loadings are equivalent across groups), and scalar (whether the item thresholds are equivalent across groups). If scalar invariance was established, a further restricted model could be tested where the correlations among the three items assessing driving after drinking were set to equality across groups. Latent mean differences were examined contingent upon achieving scalar invariance as measurement artifacts that may account for group differences are ruled out, thereby reflecting true differences in the latent construct. The procedures for these multi-group CFAs were the same as for the single-group CFAs described previously. In addition, both latent means were set to zero in the configural invariance model, and the latent mean for the U.S. group was set to zero but estimated for the Sweden group to examine latent mean differences in the metric and scalar models. Configural invariance is met if the model provides a good fit to the data, which was also determined using the joint fit criteria proposed by Hu and Bentler. Metric, scalar, and the scalar plus equivalent residual correlations invariance models were evaluated based on whether they resulted in significant decrement to model fit, a reduction in CFI ≥ .01 as higher values indicate better model fit, and an increase in RMSEA ≥ .015 as lower values indicate better model fit (Cheung & Rensvold, 2002). The level of invariance under consideration is achieved if the associated equality constraints do not result in a significant decrement to model fit according to the aforementioned criteria . All factor analyses were conducted using Mplus version 8.3 (Muthén & Muthén, 1998–2017).
After testing and establishing measurement invariance, we examined correlations of the observed scores on the RAPI with risk (i.e., indicators of alcohol consumption) and protective factors (i.e., PBS) for alcohol problems to assess construct validity of the RAPI by sample. These correlations were computed for U.S. and Sweden participants separately to examine differences in the magnitude of these associations across groups.
Results
Confirmatory Factor Analysis
Table 1 summarizes results of the CFAs testing the single common factor model of the RAPI for the U.S. and Sweden groups separately. As shown, a one-factor solution provided a good fit to the data in both samples. Each item loaded saliently onto the common factor (U.S.: standardized λs = .47–.98, Sweden: standardized λs = .60–.96). Furthermore, as expected, correlations among residuals of the three driving-after-drinking items were substantial in both the U.S. (r = .81–.95, all ps < .001) and Sweden samples (r = .81–.90, all ps < .001). Given good fit to the data, we proceeded with measurement invariance testing of the one-factor model for the RAPI across the U.S. and Sweden samples.
RAPI Measurement Invariance Model Fit Indices and Model Fit Comparisons.
Note. SB X2 = Satorra-Bentler adjusted X2; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMSEA = root mean square error of approximation; CI = confidence interval, UB = upper bound, LB = lower bound; SRMR = standardized root mean square residual; RAPI = Rutgers Alcohol Problem Index.
Measurement Invariance Testing
For measurement invariance testing with categorical indicators, each response option must be selected in each sample in order to calculate thresholds for comparison. For two items (Items 16 and 19), no one in the U.S. sample endorsed “4 = 10 or more times.” Therefore, scores of “3” or “4” were combined in the Swedish subsample. Note that we also conducted analyses after dropping these two items, and substantive results did not differ.
Table 1 summarizes results of the multi-group CFAs testing measurement invariance. As shown, each increasingly restrictive model did not result in a significant decrement in model fit, indicating the single common factor model for the RAPI was scalar invariant across the U.S. and Sweden samples. Therefore, we examined latent mean differences across the groups in the most restrictive model (scalar invariance plus equal residual correlations among the three driving-after-drinking items) and found the Sweden sample reported experiencing fewer alcohol problems than the U.S. sample (latent mean difference = −0.19, p = .047).
Construct Validity
Table 2 reports the bivariate correlations of the observed scores of the RAPI with alcohol consumption and PBS. As hypothesized, indicators of alcohol consumption were positively correlated with the RAPI, suggesting more frequent and higher quantities of consumption are associated with experiencing more alcohol problems. These correlations were greater in magnitude among the U.S. sample. Also consistent with hypotheses, each PBS subscale was negatively correlated with RAPI scores. This suggests more frequent use of PBS is associated with experiencing fewer alcohol problems; however, again, these correlations were larger for the U.S. subsample.
Bivariate Correlations of RAPI Observed Scores and Antecedents of Drinking and Protective Behavioral Strategies.
Note. PBS = Protective Behavioral Strategies; RAPI = Rutgers Alcohol Problem Index.
Discussion
Cultural factors could influence the operation of instruments; thus, measurement invariance testing is essential in cross-cultural studies (Jeong & Lee, 2019). The purpose of this study was to assess if a Swedish version of the RAPI (White & Labouvie, 1989) was equivalent to a U.S. version. Specifically, the present study examines measurement invariance of a 26-item version of the RAPI, including three additional questions on alcohol and driving, and where all questions were answered on a five-point nominal scale. Participants were U.S. and Swedish high school seniors participating in a cross-cultural intervention study targeting hazardous drinking (Larimer et al., 2021).
Despite essential cultural and legal differences between the two countries, a single common factor model was scalar invariant across U.S. and Swedish high school seniors. Support for the proposed structure adds to a growing literature demonstrating the utility of the RAPI for the assessment of drinking consequences across cultures (Armenta & Cooper, 2019). Drinking consequences are a common outcome for interventions and can also be used to differentiate between low- and high-problem drinkers. Accurate measurement is essential for measuring intervention effectiveness and identifying high-risk drinkers.
We found U.S. high school seniors reported more alcohol problems than their Swedish counterparts. This finding is supported by previous comparisons of self-reported alcohol problems in U.S. and Swedish college students (Ståhlbrandt et al., 2008) and replicates findings in the same data set that used observed scores (Larimer et al., 2021). Importantly, in the present study, we first established the prerequisite of measurement invariance for making meaningful comparisons and compared mean differences in latent as opposed to observed variables, which has statistical benefits such as removing measurement error. The fact that U.S. young adults repeatedly report more alcohol-related problems than young adults in Sweden could be the result of cultural differences between the two countries with respect to legal alcohol access and acceptability which may influence drinking patterns or contexts related to more or less consequences. It should be considered that alcohol problems are known to be affected by specific cultural factors, relating to beliefs, attitudes, norms, and expectancies about drinking (Bo, 2022; Heath, 1981), and young adults in Sweden tend to drink more than their U.S. counterparts (Ståhlbrandt et al., 2008).
To assess construct validity, we examined the relationship between the RAPI and quantity-frequency of alcohol use using measures frequently used to assess drinking patterns in both countries. As expected, we found both quantity and frequency of drinking were positively correlated with alcohol problems in both countries, although strength of these correlations was greater in the U.S. sample than in the Swedish sample. Among U.S. high school seniors, the correlation was greatest for frequency, followed by typical and peak quantity. In the Swedish sample, the correlation was greatest for peak quantity, followed by frequency and typical quantity. Generally, quantity of drinking, and especially heavy episodic or “binge” drinking, is associated with higher risks than frequency of drinking, particularly among college students (Dawson et al., 2005). However, in this high school student sample (at baseline), and thus underage drinkers, more frequent drinking may be especially problemtic.
Also, in line with our expectations, the RAPI had an overall negative correlation to different types of PBS, although the strength of the association differed between the two countries. Correlations were stronger in the U.S. sample than in the Swedish sample. Among U.S. and Swedish high school seniors, the strength of correlations between PBS subscales and RAPI was greatest for serious harm reduction and manner of drinking, followed by limiting/stopping drinking. A possible explanation for the disparity in correlation strength is that PBS were designed and validated in the U.S. and protective strategies may vary based on cultural context (Schwebel et al., 2022). For example, it is possible that strategies like drinking water and putting extra ice in a drink are not used as often in Sweden as in the U.S. because it is not typical to mix ice or water into alcoholic beverages in Sweden or because more high school students drink beer in Sweden. The overall interpretation is that self-reported alcohol problems are negatively associated with the use of PBS. It should be noted it has previously been reported that U.S. participants use PBS significantly more over time, and the use of protective strategies is related to fewer alcohol-related consequences over time in both U.S. and Sweden (Grazioli et al., 2015). Given the typically strong relationship between limiting/stopping drinking and negative alcohol-related consequences, future efforts to adapt these items for use in Sweden might be of benefit.
Limitations
The present study is not without limitations. The study sample only included high school seniors who predominantly are in the age range of 18–19 years in both the U.S. and Sweden. The narrow age range limits our ability to generalize findings, nor may findings be assumed to be generalizable to high school seniors in the remaining Scandinavian countries. Even if these countries have similar languages, cultural differences are still to be considered. The manner of drinking PBS subscale in the U.S. had suboptimal reliability. Data were collected in 2012, before the 20-item PBSS was developed which improved subscale reliability (Treloar et al., 2015). Furthermore, measurement invariance across gender subsamples was not analyzed, and the present study did not investigate predictive validity, that is, whether RAPI is a useful tool for the identification of future problematic alcohol involvement.
Conclusion
These findings suggest a measure of alcohol-related negative consequences developed in the U.S. is psychometrically valid for use in Sweden. The present study provides additional support for utilizing the RAPI in diverse cultural contexts. It also provides further support that frequent use of PBS reduces the risk of negative alcohol-related consequences and extends these results to a sample of Swedish high school seniors.
Supplemental Material
sj-docx-1-asm-10.1177_10731911231195834 – Supplemental material for A Cross-Cultural Comparison of Negative Alcohol-Related Consequences in the United States and Sweden: Measurement Invariance of the Rutgers Alcohol Problem Index
Supplemental material, sj-docx-1-asm-10.1177_10731911231195834 for A Cross-Cultural Comparison of Negative Alcohol-Related Consequences in the United States and Sweden: Measurement Invariance of the Rutgers Alcohol Problem Index by Frank J. Schwebel, Dylan K. Richards, Claes Andersson and Mary E. Larimer in Assessment
Footnotes
Acknowledgements
Thank you to Matthew R. Pearson for providing statistical consultation.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) (R01 AA018276-05). FJS was supported in part by the NIAAA (T32AA018108) and the National Institute on Drug Abuse (NIDA; RM1DA055301-02S1). DKR was supported the NIAAA (F32AA028712).
Supplemental Material
Supplemental material for this article is available online.