Establishing Psychometric Validity of the PBSS-20 for Sexual Minority College Students

Abstract

This study psychometrically evaluated the Protective Behavioral Strategies Scale-20 (PBSS-20) in a nationally recruited sample of sexual and non-sexual minority students. Participants included 1,730 college students (M_age = 19.41) from 12 universities, who were predominantly White non-Hispanic (63.6%), female (70%), and heterosexual (67.6%). Participants reported demographics and completed measures of hazardous alcohol use, alcohol-related negative consequences, and alcohol protective behavioral strategy use. Exploratory structural equation modeling (ESEM) and bifactor ESEM revealed an adequate model fit of the PBSS-20, supporting the hypothesized factor structure. Invariance testing revealed a similar factor structure for both sexual and non-sexual minority students. The PBSS-20 total and subscale scores were negatively associated with all alcohol outcome measures for all groups. This study provides additional support for the generalizability and utility of the PBSS-20, with suggestions for further refinement of the measure. This study has implications for measuring safe drinking behaviors among sexual and non-sexual minority college students.

Keywords

alcohol use sexual minority invariance testing college students harm reduction

Get full access to this article

View all access options for this article.

References

Babor

T. F.

Higgins-Biddle

J. C.

Robaina

(2017). The Alcohol Use Disorders Identification Test, adapted for use in the United States: A guide for primary care practitioners. https://sites.education.miami.edu/wp-content/uploads/sites/3/2018/07/USAUDIT-Guide_2016_final-1.pdf

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606. https://doi.org/10.1037/0033-2909.88.3.588

Blanchard

B. E.

Stevens

A. K.

Littlefield

A. K.

(2021). Differences in alcohol-related protective behavioral strategies among female and male college students. Addictive Behaviors, 120, 106969. https://doi.org/10.1016/j.addbeh.2021.106969

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. Guilford Press.

Browne

M. W.

Cudeck

(1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005

Centers for Disease Control and Prevention (2014). Planning and implementing screening and brief intervention for risky alcohol use: A step-by-step guide for primary care practices. Centers for Disease Control and Prevention, National Center on Birth Defects and Developmental Disabilities. https://www.cdc.gov/ncbddd/fasd/documents/alcoholsbiimplementationguide.pdf

Centers for Disease Control and Prevention. (2022). Terminology. Centers for Disease Control and Prevention, Division of Adolescent and School Health (DASH). https://www.cdc.gov/healthyyouth/terminology/sexual-and-gender-identity-terms.htm

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

Clark

D. A.

Bowles

R. P.

(2018). Model fit and item factor analysis: Overfactoring, underfactoring, and a program to guide interpretation. Multivariate Behavioral Research, 53(4), 544–558. https://doi.org/10.1080/00273171.2018.1461058

10.

Collins

R. L.

Parks

G. A.

Marlatt

G. A.

(1985). Daily Drinking Questionnaire. PsycTESTS Dataset, 153, 189–200. https://doi.org/10.1037/t02146-000

11.

Coulter

R. W. S.

Marzell

Saltz

Stall

Mair

(2016). Sexual-orientation differences in drinking patterns and use of drinking contexts among college students. Drug and Alcohol Dependence, 160, 197–204. https://doi.org/10.1016/j.drugalcdep.2016.01.006

12.

Cox

M. J.

DiBello

A. M.

Jones

E. P.

Gette

Godbole

Barcenas

Roudebush

Simensky

Mancini

Gheewalla

Pannu

(2024). A systematic review of the associations between protective behavioral strategies and heavy alcohol consumption and consequences among young adults. Psychology of Addictive Behaviors, 38(4), 488–506. https://doi.org/10.1037/adb0001002

13.

DeSimone

J. A.

Harms

P. D.

(2017). Dirty data: The effects of screening respondents who provide low-quality data in survey research. Journal of Business and Psychology, 33(5), 559–577. https://doi.org/10.1007/s10869-017-9514-9

14.

Ebersole

R. C.

Moorer

K. D.

Noble

J. J.

Madson

M. B.

(2015). Protective behavioral strategies, alcohol consumption, and alcohol-related negative consequences among lesbian, gay, and bisexual college students. Journal of Gay & Lesbian Mental Health, 19(2), 184–200. https://doi.org/10.1080/19359705.2014.991957

15.

Ebersole

R. C.

Noble

J. J.

Madson

M. B.

(2012). Drinking motives, negative consequences, and protective behavioral strategies in lesbian, gay, bisexual, and transgender college students. Journal of LGBT Issues in Counseling, 6(4), 337–352. https://doi.org/10.1080/15538605.2012.725650

16.

Eisenberg

Wechsler

(2003). Substance use behaviors among college students with same-sex and opposite-sex experience. Addictive Behaviors, 28(5), 899–913. https://doi.org/10.1016/s0306-4603(01)00286-6

17.

Fenkl

E. A.

Jones

S. G.

Aronowitz

Messmer

Olafson

E. A.

Simon

S. R.

Framil

C. V.

(2020). Risky sex and other personal consequences of alcohol and drug use among LGBTQ college students. Journal of the Association of Nurses in AIDS Care, 31(4), 476–482. https://doi.org/10.1097/jnc.0000000000000161

18.

Gegenfurtner

(2022). Bifactor exploratory structural equation modeling: A meta-analytic review of model fit. Frontiers in Psychology, 13, Article 1037111. https://doi.org/10.3389/fpsyg.2022.1037111

19.

Gilmore

A. K.

García-Ramírez

Fortson

Salamanca

N. K.

Nicole Mullican

Metzger

I. W.

Leone

R. M.

Kaysen

D. L.

Orchowski

L. M.

Cue Davis

(2024). The association between alcohol use and sexual assault victimization among college students differs by gender identity and race. Addictive Behaviors, 149, 107892. https://doi.org/10.1016/j.addbeh.2023.107892

20.

Graupensperger

Jaffe

A. E.

Fleming

C. N.

Kilmer

J. R.

Lee

C. M.

Larimer

M. E.

(2021). Changes in college student alcohol use during the COVID-19 pandemic: Are perceived drinking norms still relevant? Emerging Adulthood, 9(5), 531–540. https://doi.org/10.1177/2167696820986742

21.

Grazioli

V. S.

Studer

Larimer

M. E.

Lewis

M. A.

Marmet

Lemoine

Daeppen

Gmel

(2019). Protective Behavioral Strategies Scale-20: Psychometric properties of a French and German version among young males in Switzerland. International Journal of Methods in Psychiatric Research, 28(3), e1777. https://doi.org/10.1002/mpr.1777

22.

Higgins-Biddle

J. C.

Babor

T. F.

(2018). A review of the Alcohol Use Disorders Identification Test (AUDIT), AUDIT-C, and USAUDIT for screening in the United States: Past issues and future directions. The American Journal of Drug and Alcohol Abuse, 44(6), 578–586. https://doi.org/10.1080/00952990.2018.1456545

23.

Hoover

S. M.

Jordan

H. R.

Mohn

R. S.

Lui

P. P.

Madson

M. B.

(2023). Assessing reasons for limiting heavy drinking among college students: Development and preliminary analysis. Journal of Substance Use, 29(3), 450–456. https://doi.org/10.1080/14659891.2023.2173099

24.

Bentler

P. M.

(1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453. https://doi.org/10.1037/1082-989X.3.4.424

25.

Hughto

J. M.

Quinn

E. K.

Dunbar

M. S.

Rose

A. J.

Shireman

T. I.

Jasuja

G. K.

(2021). Prevalence and co-occurrence of alcohol, nicotine, and other substance use disorder diagnoses among us transgender and cisgender adults. JAMA Network Open, 4(2), Article e2036512. https://doi.org/10.1001/jamanetworkopen.2020.36512

26.

Hurlocker

Madson

M. B.

Lui

P. P.

Dvorak

Ham

L. S.

Leffingwell

Looby

Meier

Montes

Napper

L. E.

Prince

M. A.

Skewes

Zamboanga

B. L.

(2022). Mental health risk profiles and related substance use during coronavirus pandemic among college students who use substances. International Journal of Mental Health and Addiction, 21(6), 1–18. https://doi.org/10.1007/s11469-022-00813-1

27.

IBM Corp. (2026). IBM SPSS Statistics for Windows (Version 27.0) [Computer software]. https://www.ibm.com/products/spss-statistics

28.

Jackson

K. M.

Merrill

J. E.

Stevens

A. K.

Hayes

K. L.

White

H. R.

(2021). Changes in alcohol use and drinking context due to the COVID-19 pandemic: A multimethod study of college student drinkers. Alcoholism: Clinical and Experimental Research, 45(4), 752–764. https://doi.org/10.1111/acer.14574

29.

Jasinski

J. L.

Ford

J. A.

(2007). Sexual orientation and alcohol use among college students: The influence of drinking motives and social norms. Journal of Alcohol and Drug Education, 51(4), 63–82.

30.

Jordan

H. R.

Colvin

K. F.

Kim

K. Y.

Martin

J. L.

Madson

M. B.

(2021). Psychometric validation of the protective drinking practices scale in college students across the United States. Experimental and Clinical Psychopharmacology, 29(3), 251–260. https://doi.org/10.1037/pha0000471

31.

Jordan

H. R.

Madson

M. B.

Nicholson

B. C.

Bravo

A. J.

Pearson

M. R.

(2019). Posttraumatic stress disorder symptoms and problematic alcohol use in college students: The moderating role of alcohol protective behavioral strategies and gender. Psychological Trauma: Theory, Research, Practice, and Policy, 11(3), 247–255. https://doi.org/10.1037/tra0000417

32.

Kahler

C. W.

Hustad

Barnett

N. P.

Strong

D. R.

Borsari

(2008). Validation of the 30-day version of the Brief Young Adult Alcohol Consequences Questionnaire for use in longitudinal studies. Journal of Studies on Alcohol and Drugs, 69(4), 611–615. https://doi.org/10.15288/jsad.2008.69.611

33.

Kahler

C. W.

Strong

D. R.

Read

J. P.

(2005). Toward efficient and comprehensive measurement of the alcohol problems continuum in college students: The Brief Young Adult Alcohol Consequences Questionnaire. Alcoholism: Clinical and Experimental Research, 29(7), 1180–1189. https://doi.org/10.1097/01.alc.0000171940.95813.a5

34.

Kalb

Gillis

R. J.

Goldstein

A. L.

(2018). Drinking to cope with sexual minority stressors: Understanding alcohol use and consequences among LGBQ emerging adults. Journal of Gay & Lesbian Mental Health, 22(4), 310–326. https://doi.org/10.1080/19359705.2018.1476277

35.

Kazak

A. E.

(2018). Editorial: Journal article reporting standards. American Psychologist, 73(1), 1–2. https://doi.org/10.1037/amp0000263

36.

Kerr

D. L.

Ding

Chaya

(2014). Substance use of lesbian, gay, bisexual and heterosexual college students. American Journal of Health Behavior, 38(6), 951–962. https://doi.org/10.5993/ajhb.38.6.17

37.

LaBrie

J. W.

Lac

Kenney

S. R.

Mirza

(2011). Protective behavioral strategies mediate the effect of drinking motives on alcohol use among heavy drinking college students: Gender and race differences. Addictive Behaviors, 36(4), 354–361. https://doi.org/10.1016/j.addbeh.2010.12.013

38.

Lemoine

P. G.

Whitley

R. B.

Jordan

H. R.

Madson

M. B.

(2020). College alcohol beliefs and alcohol outcomes: The mediating effects of alcohol protective behavioral strategies. Substance Use & Misuse, 55(14), 2389–2394. https://doi.org/10.1080/10826084.2020.1817085

39.

Litt

D. M.

Lewis

M. A.

Blayney

J. A.

Kaysen

D. L.

(2013). Protective behavioral strategies as a mediator of the generalized anxiety and alcohol use relationship among lesbian and bisexual women. Journal of Studies on Alcohol and Drugs, 74(1), 168–174. https://doi.org/10.15288/jsad.2013.74.168

40.

Madson

M. B.

Arnau

R. C.

Lambert

S. J.

(2013). Development and psychometric evaluation of the Revised Protective Behavioral Strategies Scale. Psychological Assessment, 25(2), 556–567. https://doi.org/10.1037/a0031788

41.

Madson

M. B.

Moorer

K. D.

Zeigler-Hill

Bonnell

M. A.

Villarosa

(2013). Alcohol expectancies, protective behavioral strategies, and alcohol-related outcomes: A moderated mediation study. Drugs: Education, Prevention and Policy, 20(4), 286–296. https://doi.org/10.3109/09687637.2013.766788

42.

Marsh

H. W.

Liem

G. A. D.

Martin

A. J.

Morin

A. J. S.

Nagengast

(2011). Methodological measurement fruitfulness of exploratory structural equation modeling (ESEM): New approaches to key substantive issues in motivation and engagement. Journal of Psychoeducational Assessment, 29, 322–346.

43.

Martens

M. P.

Ferrier

A. G.

Sheehy

M. J.

Corbett

Anderson

D. A.

Simmons

(2005). Development of the Protective Behavioral Strategies Survey. Journal of Studies on Alcohol, 66(5), 698–705. https://doi.org/10.15288/jsa.2005.66.698

44.

Martens

M. P.

Pedersen

E. R.

LaBrie

J. W.

Ferrier

A. G.

Cimini

M. D.

(2007). Measuring alcohol-related protective behavioral strategies among college students: Further examination of the Protective Behavioral Strategies Scale. Psychology of Addictive Behaviors, 21(3), 307–315. https://doi.org/10.1037/0893-164x.21.3.307

45.

Martens

M. P.

Taylor

K. K.

Damann

K. M.

Page

J. C.

Mowry

E. S.

Cimini

M. D.

(2004). Protective behavioral strategies when drinking alcohol and their relationship to negative alcohol-related consequences in college students. Psychology of Addictive Behaviors, 18(4), 390–393. https://doi.org/10.1037/0893-164x.18.4.390

46.

Martin

J. L.

Colvin

K. F.

Madson

M. B.

Zamboanga

B. L.

Pazienza

(2020). Optimal assessment of protective behavioral strategies among college drinkers: An item response theory analysis. Psychological Assessment, 32(4), 394–406. https://doi.org/10.1037/pas0000799

47.

Martin

J. L.

Zamboanga

B. L.

Haase

R. F.

Buckner

L. C.

(2019). Measurement invariance of the Protective Behavioral Strategies Scale across racial groups. Measurement and Evaluation in Counseling and Development, 53(1), 17–33. https://doi.org/10.1080/07481756.2019.1640615

48.

McCabe

S. E.

Boyd

Hughes

T. L.

d’Arcy

(2003). Sexual identity and substance use among undergraduate students. Substance Abuse, 24(2), 77–91. https://doi.org/10.1080/08897070309511536

49.

McCabe

S. E.

Hughes

T. L.

Bostwick

Boyd

C. J.

(2005). Assessment of difference in dimensions of sexual orientation: Implications for substance use research in a college-age population. Journal of Studies on Alcohol, 66(5), 620–629. https://doi.org/10.15288/jsa.2005.66.620

50.

McCabe

S. E.

Hughes

T. L.

Boyd

C. J.

(2004). Substance use and misuse: Are bisexual women at greater risk? Journal of Psychoactive Drugs, 36(2), 217–225. https://doi.org/10.1080/02791072.2004.10399732

51.

Meade

A. W.

Craig

S. B.

(2012). Identifying careless responses in survey data. Psychological Methods, 17, 1–19.

52.

Morin

A. J.

Myers

N. D.

Lee

(2020). Modern Factor Analytic Techniques. In Tenenbaum

Eklund

R. C.

(Eds.), Handbook of sport psychology (pp. 1044–1073). John Wiley & Sons. https://doi.org/10.1002/9781119568124.ch51

53.

Muthén

L. K.

Muthén

B. O.

(2021). Mplus user’s guide (8th ed.).

54.

Patrick

Schulenberg

Miech

Johnston

O’Malley

Bachman

(2022). Monitoring the future panel study annual report: National data on substance use among adults ages 19 to 60, 1976-2021. https://doi.org/10.7826/isr-um.06.585140.002.07.0001.2022

55.

Pearson

M. R.

(2013). Use of alcohol protective behavioral strategies among college students: A critical review. Clinical Psychology Review, 33(8), 1025–1040. https://doi.org/10.1016/j.cpr.2013.08.006

56.

Pituch

K. A.

Stevens

(2016). Applied multivariate statistics for the Social Sciences: Analyses with SAS and IBM’s SPSS. Routledge/Taylor & Francis Group.

57.

Prince

M. A.

Carey

K. B.

Maisto

S. A.

(2013). Protective behavioral strategies for reducing alcohol involvement: A review of the methodological issues. Addictive Behaviors, 38(7), 2343–2351. https://doi.org/10.1016/j.addbeh.2013.03.010

58.

Reise

S. P.

Bonifay

W. E.

Haviland

M. G.

(2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140. https://doi.org/10.1080/00223891.2012.725437

59.

Reise

S. P.

Moore

T. M.

Haviland

M. G.

(2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. https://doi.org/10.1080/00223891.2010.496477

60.

Richards

D. K.

Puentes

R. P.

Gonzales

Smith

J. C.

Field

C. A.

Morera

O. F.

(2018). A psychometric evaluation of the Protective Behavioral Strategies Scale-20 among internet samples of adult drinkers. Addictive Behaviors Reports, 8, 71–78. https://doi.org/10.1016/j.abrep.2018.08.002

61.

Sánchez-García

Lozano-Rojas

Ó. M.

Díaz-Batanero

Carmona-Márquez

Rojas-Tejada

A. J.

Fernández-Calderón

(2020). Spanish adaptation of the Protective Behavioral Strategies Scale-20 (S-PBSS-20) and evaluation of its psychometric properties in university students. Psicothema, 32(4), 598–606. https://doi.org/10.7334/psicothema2020.172

62.

Substance Abuse and Mental Health Services Administration. (2023a). Key substance use and mental health indicators in the United States: Results from the 2022 national survey on drug use and health (HHS Publication No. PEP23-07-01-006, NSDUH Series H-58). Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/report/2022-nsduh-annual-national-report

63.

Substance Abuse and Mental Health Services Administration. (2023b). Lesbian, gay, and bisexual behavioral health: Results from the 2021 and 2022 national surveys on drug use and health (SAMHSA Publication No. PEP23-07-01-001). Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/report/LGB-Behavioral-Health-Report-2021-2022

64.

Swami

Maïano

Morin

A. J. S.

(2023). A guide to exploratory structural equation modeling (ESEM) and bifactor-ESEM in body image research. Body Image, 47, 101641. https://doi.org/10.1016/j.bodyim.2023.101641

65.

Teasdale

O. D.

Schepis

T. S.

(2025). Simultaneous alcohol and marijuana use versus alcohol use only on protective behavioral strategies: The moderating influences of coping and enhancement motives. Substance Use & Misuse, 61(3), 336–344. https://doi.org/10.1080/10826084.2025.2556254

66.

Treloar

H. R.

Martens

M. P.

McCarthy

D. M.

(2014). Testing measurement invariance of the Protective Behavioral Strategies Scale in college men and women. Psychological Assessment, 26(1), 307–313. https://doi.org/10.1037/a0034471

67.

Treloar

Martens

M. P.

McCarthy

D. M.

(2015). The Protective Behavioral Strategies Scale-20: Improved content validity of the serious harm reduction subscale. Psychological Assessment, 27(1), 340–346. https://doi.org/10.1037/pas0000071

68.

van Zyl

L. E.

ten Klooster

P. M

. (2022). Exploratory structural equation modeling: Practical guidelines and tutorial with a convenient online tool for Mplus. Frontiers in Psychiatry, 12, Article 795672. https://doi.org/10.3389/fpsyt.2021.795672

69.

Villarosa

M. C.

Messer

M. A.

Madson

M. B.

Zeigler-Hill

(2018). Depressive symptoms and drinking outcomes: The mediating role of drinking motives and protective behavioral strategies among college students. Substance Use & Misuse, 53(1), 143–153. https://doi.org/10.1080/10826084.2017.1327974

70.

Villarosa

M. C.

Moorer

K. D.

Madson

M. B.

Zeigler-Hill

Noble

J. J.

(2014). Social anxiety and alcohol-related negative consequences among college drinkers: Do protective behavioral strategies mediate the association? Psychology of Addictive Behaviors, 28(3), 887–892. https://doi.org/10.1037/a0037628

71.

Walters

S. T.

Roudsari

B. S.

Vader

A. M.

Harris

T. R.

(2007). Correlates of protective behavior utilization among heavy-drinking college students. Addictive Behaviors, 32(11), 2633–2644. https://doi.org/10.1016/j.addbeh.2007.06.022

72.

White

A. M.

Tapert

Shukla

S. D.

(2018). Binge drinking. Alcohol Research, 39(1), 1–3.

73.

Woodford

M. R.

Kulick

(2014). Academic and social integration on campus among sexual minority students: The impacts of psychological and experiential campus climate. American Journal of Community Psychology, 55(1–2), 13–24. https://doi.org/10.1007/s10464-014-9683-x