Factor Analysis of the Administrator-Research Campus Climate Collaborative (ARC3) Survey

Abstract

Sexual assault, dating violence, stalking, and sexual harassment present serious challenges to the health and well-being of college students across the United States. An estimated one in five female college students is sexually assaulted during college. Under the federal protection of Title IX, institutions of higher education are responsible for addressing sexual harassment and sexual misconduct as forms of discrimination, thus campus climate surveys are increasingly being conducted in campus settings to examine student beliefs and experiences about sexual assault, dating violence, stalking, sexual harassment, and related campus prevention and response efforts. The Administrator-Researcher Campus Climate Collaborative (ARC3) Campus Climate Survey was designed to assess a range of Title IX violations that include sexual harassment, dating violence, and sexual misconduct victimization and perpetration. This article used a sample of students drawn from seven universities to assess the psychometric properties of the ARC3 survey. Specifically, factor analysis (exploratory and confirmatory) was used to examine the factor structure of each module of the ARC3 survey. Results indicated that the majority of modules within the ARC3 survey were valid and reliable. Recommendations are provided for university administration to utilize a scorecard approach in order to get a brief but comprehensive view of campus safety status.

Keywords

campus climate dating violence sexual misconduct stalking

Sexual assault, dating violence, stalking, and sexual harassment present serious challenges to the health and well-being of college students across the United States. These problems are known to have a negative health impact, both short and long terms in nature (Breiding et al., 2014). Despite a flurry of media attention and growing efforts by lawmakers and administrators of institutions of higher education (IHEs) to rein in these problems, sexual assault and other types of sexual misconduct continue to occur at an alarming rate. Campus climate surveys are an important tool for understanding and preventing campus sexual assault (Lindquist & Krebs, 2018). The purpose of the study described here was to describe the psychometric properties of the Administrator-Researcher Campus Climate Collaborative (ARC3) Campus Climate Survey (henceforth referred to as ARC3 survey) and to assess the approach of scoring of the ARC3 survey by module in a way that allows universities to quickly assess their progress or lack of progress in specific areas related to campus sexual assault. At most universities, the complex problem of campus sexual assault is addressed through a number of interventions and programming delivered by several departments across campus. Providing module scores allows university administrators to quickly assess progress in specific modules, such as bystander intervention effectiveness or peer responses.

An estimated one in five female college students is sexually assaulted during college (Krebs et al., 2016). Fewer than 20% of victims report the assault, citing reasons such as lack of proof, fear of retaliation by the perpetrator, lack of certainty about whether the event was serious enough, worry about hostile treatment by authorities, and not knowing how to report (Krebs et al., 2007). Seventy percent of women who are sexually assaulted know the perpetrator (DiJulio et al., 2015).

Forty-three percent of dating college women report experiencing violent and abusive dating behaviors, including physical, verbal, and emotional abuse (Know Your Title IX, n.d.). Thirteen percent of women report being stalked during their time in college, with 80% of those knowing the person stalking them (Know Your Title IX, n.d.). Hill and Silva (2005) reported that sexual harassment on college campuses is widespread, with 60% of female students experiencing some form of sexual harassment. Hill and Silva (2005) also reported that men are most often the perpetrators and that most victims do not report sexual harassment.

Campus Climate Surveys

The Office on Violence Against Women continues to recommend climate surveys as a best practice for the prevention of sexual assault (Lindquist & Krebs, 2018). Campus climate surveys typically examine student beliefs and experiences about sexual assault, dating violence, stalking, sexual harassment, and related campus prevention and response efforts. Many also include assessment of long- and short-term consequences of sexual assault and sexual misconduct on victims and survivors (Hart & Fellabaum, 2008). Campus climate surveys are an important tool in efforts to reduce sexual misconduct as they assess strengths and weaknesses around efforts to reduce sexual misconduct (Linquist & Krebs, 2018).

There are a number of campus climate surveys currently in use. The most commonly used instruments were described in detail by Wood et al. (2017). Some of the commonly used instruments include the following:

Association of American Universities Campus Climate Survey on Sexual Assault & Sexual Misconduct (abbreviation AAU)

University of Kentucky Campus Attitudes Toward Safety Survey (abbreviation CATS)

Higher Education Data Sharing Consortium Sexual Assault Campus Climate Survey (abbreviation HEDS)

Rutgers University Campus Climate Survey iSpeak (abbreviated iSpeak)

Johns Hopkins University, It’s on Us, Hopkins (abbreviated JHU)

University of Chicago Sexual Misconduct Survey (abbreviated UC)

University of Oregon Sexual Violence and Institution Behavior Campus Survey (abbreviated UO)

Variation in surveys is wide, largely because of lack of agreement about what behaviors constitute sexual assault and sexual misconduct and which outcome variables are most closely associated with sexual assault and sexual misconduct (Hart & Fellabaum, 2008; Wood et al., 2017). In 2016, the Bureau of Justice conducted a nine-school study to develop the Campus Climate Survey, (U.S. Department of Justice, 2017). The survey included assessments of prevalence and incidence of sexual assault, rape, and sexual battery. In addition to the goals of developing a valid survey instrument and survey methodology, the study aimed to create assessment reports that produce school-specific results to be compared across schools and are useful to participating schools. Latent class analysis validated the ability for sexual assault items to be aggregated for school comparison (Krebs et al., 2016). While the survey was methodologically sound and was successful in highlighting problems associated with sexual assault in college campuses, neither did it sample graduate students nor did it include other relevant components of campus safety such as alcohol use (Caamano-Isorna et al., 2018; Mellins et al., 2017; Parks et al., 2014; White & Hingson, 2013; Yeater et al., 2018), peer norms (Bountress et al., 2017), consent (Fedina et al., 2018), and bystander intervention (Kettrey et al., 2019).

Campus Climate Assessment

To expand upon school comparison of sexual assault data by the Bureau of Justice, the researchers of the present study examined module scores to allow university administrators and program leaders to quickly determine which areas related to sexual misconduct required the most immediate attention as well as identify areas of success. The ARC3 instrument was an ideal instrument for providing module scores because it was created with modules that “serve as a barometer of the success of policies, procedures, services, and prevention programs” (ARC3, 2015). The ARC3 survey addresses the experience of sexual misconduct from risk factors through outcomes and considers both perpetration and victimization. The modules in the ARC3 survey are as follows:

Possible Outcomes

Alcohol Use

Peer Norms

Perceptions of Campus Climate Regarding Sexual Misconduct

Sexual Harassment by Faculty/Staff

Sexual Harassment by Students

Stalking Victimization

Stalking Perpetration

Dating Violence Victimization

Dating Violence Perpetration

Sexual Violence Victimization

Sexual Violence Perpetration

Institutional Responses

Peer Responses

Consent

Bystander Intervention

Campus Safety

Additional Information

The Present Study

The primary purpose of this study was twofold. First, we examined the psychometric properties of the ARC3 survey. Once validity evidence was established to aggregate factor scores, the second purpose of this study was to examine the use of module scores as means of providing a brief but comprehensive view of campus safety status.

Method

The work described here was conducted as part of the Cultivating Safe College Campuses project, one of eight similar projects, funded by the U.S. Department of Health and Human (HHS) services. For this Cultivating Safe College Campuses study, 10 IHEs in Texas collaborated to improve efforts on their respective campuses to prevent and respond to campus sexual assault. One of the obligations of the project was for each IHE to conduct an annual campus climate survey. The IHEs were very diverse in size and student demographics. Public, private, urban, and rural universities were included. Two of the IHEs are designated Hispanic-serving institutions and one is a historically Black university. Student enrollment for these IHEs ranged from 1,074 to 37,979 students with an average of 11,160 students. More than half of the institutions were public schools (60.0%) and there were an equal number of rural and urban schools (50.0%). The IHE partners formed a consortium whose goal was to create a consistent, effective response to campus sexual assault. Approval from the institutional review board was obtained for the work described here.

Instrumentation

Several existing instruments were considered for use (Wood et al., 2017). The ARC3 survey was ultimately selected for a variety of reasons. The ARC3 survey is a free campus climate survey developed to assess perpetration and victimization of sexual misconduct on college campuses in the United States, and serves as a barometer of the success of policies, procedures, services, and prevention programs available on higher education campuses. The assessment is comprehensive and one of three instruments that measures both victimization and perpetration (Wood et al., 2017). It also gathers data on alcohol use, peer norms, institutional responses, peer responses, consent, bystander interventions, campus safety, and demographics. As many assessments were developed by universities themselves, the ARC3 survey was developed by a consortium of both professionals and practitioners—sexual assault researchers, social work professionals, psychologists, public health professors, association members, and representatives from higher education groups. It was founded on the principles of (a) inclusiveness, mutual respect, collaboration, (b) engaging in an iterative and transparent drafting process, (c) ensuring independence and integrity in research, (d) a commitment to use of the best scientific evidence as the foundation, (e) equal focus on surveying victimization and perpetration, (f) adoption of a civil rights approach grounded in Title IX, (g) framing efforts with the principles of the Belmont Report, and (h) sensitivity to the unique issues faced by various diverse populations and higher education institutional types (ARC3, 2015). The primary purpose of the Belmont report, one of the leading works concerning ethics in health care research, is to protect subjects in health research (U.S. Department of Health & Human Services, n.d.). The ARC3 developers framed the survey using the Belmont report principles of respect for persons, beneficence, and justice (ARC3, 2015).

The survey is appropriate for use with both graduate and undergraduate students (ARC3, 2015). The time estimated to complete the ARC3 survey is 30 minutes; most campus climate surveys require between 15 and 30 minutes to complete (Wood et al., 2017). The ARC3 survey is composed of 19 modules, which can be selected to provide IHEs with the flexibility in survey length and content. Most of these modules are composed of a series of interrelated items measuring the overall module concept, such as peer norms, alcohol use, and dating violence victimization. All individual modules are listed in Table 1. Though it is not required that all modules be administered, the developers recommend that the modules be presented in order (ARC3, 2015). Responses to items for many of the modules were given on a 5-point scale depending on the specific item. Table 1 contains the number of specific items and scale points contained within each module and submodule. For example, the first item in Module 1 stated “I would recommend attending [Institution] to others.” Responses were given on a 5-point scale that ranged from 1 = strongly disagree to 5 = strongly agree.

Table 1.

ARC3 Module Scale Information.

Modules	No. of items	Scale points	Scale endpoint descriptors
Module 1 (Student Well-being)	22	1-5
Module 1 Factor A (Academic Satisfaction)	2	1-5	Strongly disagree	Strongly agree
Module 1 Factor B (Academic Disengagement)	8	1-5	Almost always	Almost never
Module 1 Factor C (Satisfaction With Life)	5	1-5	Strongly disagree	Strongly agree
Module 1 Factor D (Mental Health)	5	1-5	Never	Always
Module 1 Factor E (General Well-being)	1	1-5	Poor	Excellent
Module 1 Factor F (General Safety)	1	1-5	Strongly disagree	Strongly agree
Module 3 (Peer Norms)	12	1-5
Module 3 Factor A (Peer Social Support Measures)	7	1-5	Strongly disagree	Strongly agree
Module 3 Factor B (Informational Peer Support)	5	1-5	Strongly disagree	Strongly agree
Module 4 (Perceptions of Campus Climate Regarding Sexual Misconduct)	23	1-5
Module 4 Factor A (Institutional Response)	12	1-5	Very unlikely	Very likely
Module 4 Factor B (Knowledge of Campus Sexual Misconduct Resources)	3	1-5	Strongly disagree	Strongly agree
Module 4 Factor C (Awareness of Community Sexual Misconduct Resources)	8	1-5	Not at all aware	Extremely aware
Module 5 (Sexual Harassment by Faculty/Staff)	16	1–5
Module 5 Factor A (Physical Sexual Harassment)	9	1-5	Never	Many times
Module 5 Factor B (Nonphysical Sexual Harassment)	7	1-5	Never	Many times
Module 6 (Sexual Harassment by Students)	12	1-5
Module 6 Factor A (Direct Sexual Harassment)	6	1-5	Never	Many times
Module 6 Factor B (Indirect Sexual Harassment)	6	1-5	Never	Many times
Module 7 (Stalking Victimization)	10	1-5
Module 7 Factor A (Invasive Stalking)	4	1-5	None	More than 8
Module 7 Factor B (Stalking from a Distance)	4	1-5	None	More than 8
Module 7 Factor C (Online Stalking)	2	1-5	None	More than 8
Module 8 (Stalking Perpetration)	10	1-5
Module 8 Factor A (Invasive Stalking)	4	1-5	None	More than 8
Module 8 Factor B (Stalking from a Distance)	4	1-5	None	More than 8
Module 8 Factor C (Online Stalking)	2	1-5	None	More than 8
Module 9 (Dating Violence Victimization)	6	1-5	Never	Many times
Module 10 (Dating Violence Perpetration)	6	1-5	Never	Many times
Module 11 (Sexual Violence Victimization)	25	0-3	0 Times	3+ Times
Module 11 Factor A (Private Area Victimization)	5	0-3	0 Times	3+ Times
Module 11 Factor B (Oral Victimization)	5	0-3	0 Times	3+ Times
Module 11 Factor C (Vaginal Victimization)	5	0-3	0 Times	3+ Times
Module 11 Factor D (Anal Victimization)	5	0-3	0 Times	3+ Times
Module 11 Factor E (Attempted Sexual Violence Victimization)	5	0-3	0 Times	3+ Times
Module 12 (Sexual Violence Perpetration)	25	0-3	0 Times	3+ Times
Module 12 Factor A (Private Area Perpetration)	5	0-3	0 Times	3+ Times
Module 12 Factor B (Oral Perpetration)	5	0-3	0 Times	3+ Times
Module 12 Factor C (Vaginal Perpetration)	5	0-3	0 Times	3+ Times
Module 12 Factor D (Anal Perpetration)	5	0-3	0 Times	3+ Times
Module 12 Factor E (Attempted Sexual Violence Perpetration)	5	0-3	0 Times	3+ Times
Module 13 (Institutional Responses)		1-5
Module 13 Factor A (Campus Response)	10	1-5	Not at all useful	Very useful
Module 13 Factor B (Family Response)	3	1-5	Not at all useful	Very useful
Module 13 Factor C (Confidant Response)	3	1-5	Not at all useful	Very useful
Module 14 (Anticipated Responses From Peers)	13	1-5
Module 14 Factor A (Anticipated Responses From Peers)	10	1-5	Never	Always
Module 14 Factor B (General Response)	3	1-5	Strongly disagree	Strongly agree
Module 15 (Consent)	7	1-5
Module 15 Factor A (Consent Signals)	5	1-5	Strongly disagree	Strongly agree
Module 15 Factor B (Withdrawal)	2	1-5	Strongly disagree	Strongly agree
Module 16 (Bystander Intervention)	7	1-5	Never	Always
Module 17 (Campus Safety)	7	1-5
Module 17 Factor A (Sense of Safety)	4	1-5	Strongly disagree	Strongly agree
Module 17 Factor B (Perception of Sexual Misconduct as Part of Campus Life)	3	1-5	Strongly disagree	Strongly agree

Note. Module 18 contains demographic questions, such as gender, class standing, and so on. Module 19 contains items related to distress level of the items, meaningfulness of completing the survey, and so on. ARC3 = Administrator-Researcher Campus Climate Collaborative.

Preliminary research utilizing a three-university sample has demonstrated evidence for the scale’s reliability (Swartout et al., 2018). Specifically, authors examined the internal consistency of modules within the ARC3 survey and determined that all but two had acceptable reliability with α > .70 (the remaining two modules (Modules 1 and 17) had internal consistency scores between .65 and .70. Although these results provide initial evidence of the scale’s reliability, additional psychometric evaluation is needed to provide additional validity and reliability support, as well as support for module items to be aggregated into statistically valid factors.

Data Collection

Each IHE administered the baseline ARC3 survey between February and May of 2017 using an online survey platform. Representative sample calculations were determined for each IHE based on the size of their student population, a 50% response distribution, a 95% confidence interval, and 5% margin of error. Representativeness of the samples was assessed by comparing the demographic makeup of the IHE student population on gender, age, and race. Three schools were removed from analysis due to very low data collection rates and nonrepresentative samples. On completion of the data collection, a validity check was conducted to ensure that valid participants were used in the data analyses. Participants who completed the survey in too short a time, incorrectly answered more than half of the attention trap questions, or dropped off before reaching the sexual harassment sections were removed from the data file. Response rates from the six included IHEs ranged from 16% to 23%. The average sample size collected was 967 respondents with a range of 361 to 2,637.

Sample and Data

The sample consisted of students from six¹ Texas Universities. These universities are diverse in terms of size, location, and student demographics. Data from each individual university were merged together to form a data set with 6,973 students. However, because each university had a set of optional modules from which they could select, the total number of observations differed per module. In order to cross-validate the proposed measures, the total sample of 6,973 was randomly split approximately into half (n = 3,389 for the calibration sample and n = 3,386 for the validation sample). To ensure that universities were equally represented in each subset, the random split was done within each school. The two samples did not significantly differ with regard to gender, χ²(1) = 0.001, p = .97; ethnic composition, χ²(5) = 3.65, p = .60; and class standing, χ²(5) = 2.33, p = .80. Frequencies and percentages for each group/sample are presented in Table 2.

Table 2.

Descriptive Statistics for Samples 1 and 2.

Demographic variable	Sample 1		Sample 2
Demographic variable	n	%	n	%
Gender
Female	1,434	70.0	1,445	70.0
Male	572	27.9	576	27.9
Other	44	2.1	33	2.0
Missing
Ethnicity
Black/African	141	6.9	137	6.6
White/Caucasian	1,195	58.3	1,170	56.6
Hispanic/Latino	369	18.0	385	18.6
Asian/Asian American	110	5.4	132	6.4
Multiple	191	9.3	190	5.6
Other	42	2.1	52	2.5
Missing
Classification
First year	364	17.8	337	18.2
Second year	312	9.2	333	16.1
Third year	448	13.2	468	22.7
Fourth year	356	10.5	344	16.7
Five+ years undergraduate	147	4.3	150	7.3
Graduate	391	11.5	371	18.0
Professional (e.g., law, medicine)	29	0.9	23	1.1
Missing

Note. Chi-square tests indicated the samples were not significantly different on any demographic factor (p > .05 for all analyses).

Data Analysis

Exploratory factor analysis (EFA) was used to examine the factor structure of each predesigned module of the ARC3 survey. Individual EFAs were conducted for each predesigned factor that made up the overall modules. For example, Module 1 contained five factors (Factors A through F; see Table 1). However, factors E and F contained only one item, thus, it was not appropriate to conduct EFA within those specific factors. The goal of EFA in scale development is to improve the psychometric properties, and ultimately the validity, of a measure. As defined by Reise et al. (2000), the concept of validity implies that a measure (a) has item content and a corresponding factor structure that is representative of what is known regarding a construct, (b) has a factor structure that is replicable and generalizable, and (c) has a clear interpretability.

As recommended by Henson and Roberts (2006), factor retention was determined using parallel analysis. Varimax rotation was used when more than one factor existed within a module to improve interpretation. Items with factor loadings less than .300 or items that did clearly identify with a single factor (i.e., items that “cross-loaded”) were considered a criterion for removal (Brown, 2006; Kline, 1994). The internal consistency in every module and the underlying subscales were examined using Cronbach’s alpha. A scale was considered to have adequate internal consistency when α > .70 (Field, 2013). The percent of variance explained by factors was reported. When more than one factor was extracted, variance explained based on the rotated sum of squares was reported. All EFAs were performed using IBM SPSS Version 25.

Once the EFA process was complete, the proposed factor structure was confirmed using CFA in the second subset of data (the validation sample). The robust maximum likelihood-based estimation to correct for nonnormality, comparative fit index (CFI), and standardized root mean square residual (SRMR) were examined to collectively test model fit. A general rule of thumb for acceptable model fit using these indexes is SRMR ≤ .08; and CFI ≥ .90 (Hooper et al., 2008; McDonald & Ho, 2002; Sun, 2005). While these values correspond with “acceptable” model fit, SRMR ≤ .5 and CFI ≥ .95 indicate “good” model fit. As noted by Clark and Watson (1995), scales will yield more information and have improved validity when they contain more differentiated items that are only moderately correlated. Thus, the goal of the current analyses was to demonstrate that items within each ARC3 subscale were consistent with one another but were not overly redundant (i.e., α > .95, or near-perfect model fit statistics). CFAs were performed using the lavaan package of R Version 3.1.2.

Standard EFA using SPSS was not appropriate for Modules 2, 11, 12, and 13 due to within-module items on difference scales (Module 2—alcohol use), occurrence of items with highly nonnormal distributions (Module 11—Sexual Violence Victimization and Module 12—Sexual Violence Perpetration) and items with dichotomous answer choices (Module 13—Institutional Responses); thus, factor analysis of mixed data was conducted on Sample 1 and confirmatory factor analysis was conducted on Sample 2 using R Version 3.1.2 (Pagès, 2004, 2014). Finally, Module 13 contained dichotomous items (i.e., yes/no responses) and was not appropriate for Cronbach’s alpha. To test the reliability of the binary items in Module 13, latent variable modeling was used (Raykov et al., 2010). The procedure calculates scale reliability by estimating true score variance and error variance among a set of binary items.

Results

Exploratory Factor Analyses

EFAs were conducted on items within each module of the ARC3 survey containing Likert-type scale items, specifically Modules 1, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, and 17. Module 1 contained four subscales that encompassed components of academic satisfaction, academic disengagement, satisfaction with life, and mental health. Results of the four EFAs indicated that items within each module loaded onto a single factor with each factor explaining 86%, 38%, 70%, and 61% of variance, respectively (see Table 3). Internal consistency for each subscale was α = .84, .76, .89, and .87, respectively.

Table 3.

Summary of Exploratory Factor Analysis and Reliability Results for Possible Outcomes (Module 1).

Factor name and item	Factor 1
Academic Satisfaction (α = .840)
M1.A1 I would recommend attending [Institution] to others.	.930
M1.A2 If I had it to do over again, I would still attend [Institution].	.930
Academic Disengagement (α = .755)
M1.B1 Missed class	.719
M1.B2 Made excuses to get out of class	.693
M1.B3 Been late for class	.585
M1.B4 Done poor work	.662
M1.B5 Attended class intoxicated or high	.430
M1.B6 Slept in class	.469
M1.B7 Thought about dropping a class	.662
M1.B8 Thought about quitting school	.627
Satisfaction with Life Scale (α = .889)
M1.C1 In most ways, my life is close to ideal.	.866
M1.C2 The conditions of my life are excellent.	.868
M1.C3 I am satisfied with life.	.878
M1.C4 So far, I have gotten the important things I want in life.	.830
M1.C5 If I could live my life over, I would change almost nothing.	.741
Mental Health (α = .869)
M1.D1 Felt calm and peaceful?	.733
M1.D2_rev. Been a very nervous person?	.692
M1.D3_rev. Felt so down in the dumps that nothing could cheer you up?	.846
M1.D4_rev. Felt downhearted and blue?	.852
M1.D5 Been a happy person?	.765

Module 3 contained two subscales: peer social support and informational peer support. The EFA for peer social support indicated a two-factor solution (see Table 4). The first factor was primarily made up of variance from Items 3 through 7 and explained 45% of variance in the items (based on the rotated sum of squares). Based on the items that loaded onto this factor, it appeared that the factor represented support of coerced/forced sexual contact. The second factor explained 26% of variance among the items. Based on pattern coefficients, this second factor was primarily composed of variance from the first two items, which inquired about peer support for having multiple sexual partners and telling stories about sexual experiences. Internal consistency for the two factors in the peer social support subscale was α = .83 and .78, respectively. Regarding the second subscale in Module 3 (informational peer support), results indicated a one-factor solution that explained 54% of variance among the items. Internal consistency was α = .72.

Table 4.

Summary of Exploratory Factor Analysis and Reliability Results for Peer Norms (Module 3).

Factor name and item	Factor 1	Factor 2
Peer Social Support (α = .732)
Coerced/Forced Sexual Contact (α = .831)
M3.A3 Getting someone drunk or high to have sex with them.	.733	.286
M3.A4 Lying to someone in order to have sex with them.	.733	.330
M3.A5 Forcing someone to have sex.	.912	−.052
M3.A6 Using physical force, such as hitting or beating, to resolve conflicts with dates.	.891	−.078
M3.A7 Insulting or swearing at dates.	.676	.203
Sexual Experiences (α = .784)
M3.A1 Having many sexual partners.	.148	.880
M3.A2 Telling stories about sexual experiences.	.070	.888
Informational Peer Support (α = .719)
M3.B1 It is alright for someone to hit a date in certain situations.	.625
M3.B2 Someone you are dating should have sex with you when you want.	.602
M3.B3 When you spend money on a date, the person should have sex with you in return.	.814
M3.B4 You should respond to a date’s challenges to your authority by insulting them or putting them down.	.823
M3.B5 It is alright to physically force a person to have sex under certain conditions.	.801

Module 4 contained four subscales: institutional response, knowledge of campus sexual misconduct resources, sexual misconduct discussion/information (Question 2 in Module 4-C), and awareness of services (Question 4 in Module 4-C), as shown in Table 5. The first EFA for institutional response items indicated a two-factor solution. The first factor explained 51% of variance among items and was primarily made up of variance from Items 1 through 9 (α = .95). Based on the item loadings, it was apparent that the first factor represented positive institutional behaviors, such as handling reports seriously, fairly, and privately. The second factor explained 23% of variance in the items and was primarily made up of variance from the final three questions in the subscale. Internal consistency for the three items was α = .88. The factor appeared to represent negative institutional behaviors such as “punish a person who made a report,” or “would have a hard time supporting someone who made a report.”

Table 5.

Summary of Exploratory Factor Analysis and Reliability Results for Perceptions of Campus Climate Regarding Sexual Misconduct (Module 4).

Factor name and item	Factor 1	Factor 2
Institutional Response (α = .924)
Positive Institutional Behaviors (α = .946)
M4.A1 The institution would take the report seriously.	.841	.226
M4.A2 The institution would maintain the privacy of the person making the report.	.771	.193
M4.A3 The institution would do its best to honor the request of the person about how to go forward with the case.	.861	.175
M4.A4 The institution would take steps to protect the safety of the person making the report.	.878	.200
M4.A5 The institution would support the persona making the report.	.864	.231
M4.A6 The institution would provide accommodations to support the person (e.g., academic, housing, safety).	.808	.138
M4.A7 The institution would take action to address the factors that may have led to the sexual misconduct.	.841	.166
M4.A9 The institution would handle the report fairly.	.796	.163
Negative Institutional Behaviors (α = .875)
M4.A10_rev The institution would label the person making the report a troublemaker.	.162	.886
M4.A11_rev The institution would have a hard time supporting the person who made the report.	.335	.838
M4.A12_rev The institution would punish the person who made the report.	.121	.884
Knowledge of Campus Sexual Misconduct Resources (α = .888)
M4.B1 If a friend or I experienced sexual misconduct, I know where to go to get help on campus.	.933
M4.B2 I understand what happens when a student reports a claim of sexual misconduct at [IHE].	.920
M4.B3 I would know where to go to make a report of sexual misconduct.	.856
Seen, Heard, or Discussed Sexual Misconduct (α = .913)
M4.C2.a Discusses sexual misconduct/rape in class	.724
M4.C2.b Discussed the topic of sexual misconduct with friends	.545
M4.C2.c Discussed sexual misconduct with a family member	.615
M4.C2.d Attended an event or program about what you can do as a bystander to stop sexual misconduct	.778
M4.C2.e Attended a rally or other campus event about sexual misconduct or sexual assault	.817
M4.C2.f Seen posters about sexual misconduct (e.g., raising awareness, preventing rape, defining sexual misconduct)	.467
M4.C2.g Seen or heard campus administrators or staff address sexual misconduct	.723
M4.C2.h Seen crime alerts about sexual misconduct	.578
M4.C2.i Read a report about sexual violence rates at [IHE]	.729
M4.C2.j Visited a [IHE] website with information on sexual misconduct	.779
M4.C2.k Volunteered or interned at an organization that addresses sexual misconduct	.792
M4.C2.l Seen or heard about sexual misconduct in a student publication or media outlet	.676
M4.C2.m Taken a class to learn more about sexual misconduct	.788
Awareness of Services (α = .865)
M4.C4.a Office for Violence Prevention and Victim Assistance	.779
M4.C4.b Office of Student Conduct	.795
M4.C4.c Title IX Compliance	.744
M4.C4.d Student Legal Services	.793
M4.C4.e Counseling Services	.678
M4.C4.f The Office of Employment Equity	.760
M4.C4.g Health Services	.662

Note. IHE = institution of higher education.

The EFA for the second, third, and fourth subscales in Module 4 (knowledge of campus sexual misconduct resources, sexual misconduct discussion/information, and awareness of services) all indicated a single-factor solution. Variance explained by each factor was 82%, 49%, and 56%, respectively. Likewise, internal consistency for these three subscales was α = .89, .91, and .87. Based on the large amount of variance explained and high alpha coefficients, it was apparent that a single factor represented all items in the second, third, and fourth subscales of Module 4.

Module 5 was composed of a single subscale that encompassed sexual harassment by faculty/staff. Results of EFA indicated a two-factor solution (see Table 6). The first factor was primarily made up of variance from Items 9 through 16 and explained 43% of variance in the items (α = .95). Based on the items, it appeared that the factor represented pressure to engage in sexual acts. The second factor was made up of variance from Items 1 through 5, and Item 7. This factor explained 31% of variance in the items (α = .91) and appeared to represent offensive comments or jokes. Items 6 and 8 were trimmed from the EFA because they did not clearly relate to one of the two factors (i.e., cross-loading).

Table 6.

Summary of Exploratory Factor Analysis and Reliability Results for Sexual Harassment by Faculty/Staff (Module 5).

Factor name and item	Factor 1	Factor 2
Physical Harassment (α = .952)
M5.A9 Made unwanted attempts to establish a romantic sexual relationship with you despite your efforts to discourage it?	.763	.412
M5.A10 Continued to ask you for dates, drinks, dinner, etc., even though you said “No”?	.788	.344
M5.A11 Touched you in a way that made you feel uncomfortable?	.780	.379
M5.A12 Made unwanted attempts to stroke, fondle, or kiss you?	.876	.246
M5.A13 Made you feel like you were being bribed with a reward to engage in sexual behavior?	.827	.152
M5.A14 Made you feel threatened with some sort of retaliation for not being sexually cooperative?	.852	.277
M5.A15 Treated you badly for refusing to have sex?	.878	.244
M5.A16 Implied better treatment if you were sexually cooperative?	.874	.180
Nonphysical Harassment (α = .904)
M5.A1 Treated you “differently” because of your sex?	.161	.843
M5.A2 Displayed, used, or distributed sexist or suggestive materials?	.298	.776
M5.A3 Made offensive sexist remarks?	.150	.847
M5.A4 Put you down or was condescending to you because of your sex?	.234	.825
M5.A5 Repeatedly told sexual stories or jokes that were offensive to you?	.391	.705
M5.A7 Made offensive remarks about your appearance, body, or sexual activities?	.489	.644

Note. Items 6 and 8 were trimmed from the Nonphysical Harassment subscale due to cross-loading.

For the sexual harassment by students items (Module 6), the EFA indicated a one-factor solution (see Table 7). Item 10 had a small pattern coefficient and was trimmed from the analysis. The updated model produced a single factor that accounted for 58% of the total variance. All items loaded well on the factor and internal consistency was α = .93. Based on the variance explained and alpha level, it appeared that a single factor was useful for representing the items in Module 6.

Table 7.

Summary of Exploratory Factor Analysis and Reliability Results for Sexual Harassment by Students (Module 6).

Factor name and item	Factor 1
M6.A1 Treated you “differently” because of your sex?	.761
M6.A2 Displayed, used, or distributed sexist or suggestive materials?	.773
M6.A3 Made offensive sexist remarks?	.827
M6.A4 Put you down or was condescending to you because of your sex?	.815
M6.A5 Repeatedly told sexual stories or jokes that were offensive to you?	.824
M6.A6 Made unwelcome attempts to draw you into a discussion of sexual matters?	.804
M6.A7 Made offensive remarks about your appearance, body, or sexual activities?	.821
M6.A8 Made gestures or used body language of a sexual nature which embarrassed or offended you?	.811
M6.A9 Made unwanted attempts to establish a romantic sexual relationship with you despite your efforts to discourage it?	.719
M6.A11 Sent or posted unwelcome sexual comments, jokes or pictures by text, email, Facebook, or other electronic means?	.729
M6.A12 Spread unwelcome sexual rumors about you by text, email, Facebook, or other electronic means?	.603
M6.A13 Called you gay or lesbian in a negative way by text, email, Facebook, or other electronic means?	.613

Note. Item 10 was trimmed due to low factor loading.

Results indicated that Module 7 (stalking victimization) was composed of two factors (see Table 8). The first (made up of Items 5, 6, 7, 9, and 10) explained 33% of variance in the items and had an internal consistency of α = .86. This factor could be conceptualized as online stalking. The second factor (made up of Items 1, 2, 3, 4, and 8) explained 31% of variance in the items and had an internal consistency of α = .80. Based on the items that loaded on this second factor, it appeared to represent physical stalking.

Table 8.

Summary of Exploratory Factor Analysis and Reliability Results for Stalking Victimization (Module 7).

Factor name and item	Factor 1	Factor 2
Online stalking (α = .859)
M7.A5 Left you unwanted messages (including text or voice messages)?	.811	.287
M7.A6 Made unwanted phone calls to you (including hang up calls)?	.764	.322
M7.A7 Sent you unwanted emails, instant messages, or sent messages through social media apps?	.819	.206
M7.A9 Made rude or mean comments to you online?	.775	.098
M7.A10 Spread rumors about you online, whether they were true or not?	.674	.200
Physical stalking (α = .801)
M7.A1 Watched or followed you from a distance, or spied on you with a listening device, camera, or GPS [global positioning system]?	.208	.644
M7.A2 Approached you or showed up in places, such as your home, workplace, or school when you didn’t want them to be there?	.381	.635
M7.A3 Left strange or potentially threatening items for you to find?	.136	.828
M7.A4 Sneaked into your home or car and did things to scare you by letting you know they had been there?	.129	.805
M7.A8 Left you cards, letters, flowers, or presents when they knew you didn’t want them to?	.298	.809

Factor analysis results for Modules 8 (stalking perpetration), 9 (dating violence victimization), and 10 (dating violence perpetration) all indicated that one factor could be used to represent each module. EFA results for the three modules are located in Tables 9, 10, and 11, respectively. Total variance accounted for by each factor was 53%, 69%, and 57% and internal consistency was α = .87, .89, and .83, respectively. Based on the variance explained and strong alpha coefficients, it was apparent that items within each module worked well together to capture the focus of the module. In this case, the items in Module 8 pertaining to stalking perpetration all worked well together to represent the concept of stalking perpetration. This was also the case with Module 9 for dating violence victimization and Module 10 dating violence perpetration.

Table 9.

Summary of Exploratory Factor Analysis and Reliability Results for Stalking Perpetration (Module 8).

Factor name and item	Factor 1
M8.A1 Watched or followed them from a distance, or spied on them with a listening device, camera, or GPS [global positioning system]?	.594
M8.A2 Approached them at places, such as their home, workplace, or school when they didn’t want you to be there?	.759
M8.A3 Left strange or potentially threatening items for them to find?	.797
M8.A4 Sneaked into their home or car and did things to scare them by letting them know you had been there?	.806
M8.A5 Left unwanted messages for them (including text or voice messages)?	.775
M8.A6 Made unwanted phone calls to them (including hang-up calls)?	.639
M8.A7 Sent them unwanted emails, instant messages, or messages through social media apps?	.731
M8.A8 Left cards, letters, flowers, or presents for them when you knew they didn’t want you to?	.875
M8.A9 Made rude or mean comments to them online?	.662
M8.A10 Spread rumors about them online, whether they were true or not?	.606

Table 10.

Summary of Exploratory Factor Analysis and Reliability Results for Dating Violence Victimization (Module 9).

Factor name and item	Factor 1
M9.A1 Not including horseplay or joking around, the person threatened to hurt me and I thought I might really get hurt.	.874
M9.A2 Not including horseplay or joking around, the person pushed, grabbed, or shook me.	.884
M9.A3 Not including horseplay or joking around, the person hit me.	.873
M9.A4 Not including horseplay or joking around, the person beat me up.	.739
M9.A5 Not including horseplay or joking around, the person stole or destroyed my property	.844
M9.A6 Not including horseplay or joking around, the person can scare me without laying a hand on me.	.758

Table 11.

Summary of Exploratory Factor Analysis and Reliability Results for Dating Violence Perpetration (Module 10).

Factor name and item	Factor 1
M10.C1 Not including horseplay or joking around, I threatened to hurt the person and I meant it.	.751
M10.C2 Not including horseplay or joking around, I pushed, grabbed, or shook the person.	.750
M10.C3 Not including horseplay or joking around, I hit the person.	.810
M10.C4 Not including horseplay or joking around, I beat up the person.	.815
M10.C5 Not including horseplay or joking around, I stole or destroyed the person’s property.	.762
M10.C6 Not including horseplay or joking around, I can scare this person without laying a hand on them.	.641

Module 14 (dating violence perpetration) was made up of two subscales: peer responses and general responses (see Table 12). EFA results for the peer responses subscale indicated that two factors could be used to represent the 10 items. The first factor, which explained 36% of variance in the items (α = .86), was primarily made up of variance from Items 1, 3, 5, 7, 8, and 10. Based on the items, it appeared the factor represented judgmental or critical comments from peers. The second factor was made up of variance from Items 2, 4, 6, and 9. This factor explained 31% of variance in the items (α = .90) and appeared to represent positive/supportive peer comments. The second subscale in Module 14 (general responses) was represented by a single factor that explained 75% of variance. The three items also had a strong internal consistency (α = .83).

Table 12.

Summary of Exploratory Factor Analysis and Reliability Results for Peer Response (Module 14).

Factor name and item	Factor 1	Factor 2
Anticipated Responses From Peers (α = .819)
Anticipated negative responses (α = .856)
M14.A1 Tell you that you were irresponsible or not cautious enough.	.710	.099
M14.A3 Treat you differently in some way than before you told them that made you uncomfortable.	.752	.031
M14.A5 Tell you that you could have done more to prevent this experience from occurring.	.722	.040
M14.A7 Avoid talking to you or spending time with you.	.784	.087
M14.A8 Treat you as if you were a child or somehow incompetent.	.812	.061
M14.A10 Make you feel like you didn’t know how to take care of yourself.	.843	.090
Anticipated positive responses (α = .897)
M14.A2_rev Reassure you that you are a good person.	.036	.842
M14.A4_rev Comfort you by telling you it would be all right or by holding you.	.066	.872
M14.A6_rev Provide information and discussed options.	.093	.879
M14.A9_rev Help you get information of any kind about coping with the experience.	.115	.893
General responses (α = .829)
M14.B1 Students would label the person making the report a troublemaker.	.900
M14.B2 Students would have a hard time supporting the person who made the report.	.911
M14.B3 The alleged offender(s) or their friends would try to get back at the person making the report.	.791

The EFA for Module 15 (consent) also indicated a two-factor solution, as shown in Table 13. The first factor was primarily made up of variance from Items 4 through 8. This factor appeared to represent interpretations of consent (e.g., “If someone invites you to their place, they are giving consent for sex”). There was 42% of variance explained by Factor 1 and internal consistency was α = .82. The second factor was primarily made up of variance from Items 1 and 3, which inquired about consent during a sexual encounter. This factor explained 20% of variance in the items and had an internal consistency of α = .56. It should be noted that the alpha statistic tends to underestimate reliability when the number of test items is low (Tavakol & Dennick, 2011).

Table 13.

Summary of Exploratory Factor Analysis and Reliability Results for Consent (Module 15).

Factor name and item	Factor 1	Factor 2
Automatic consent (α = .817)
M15.4_rev If a person doesn’t physically resist sex, they have given consent.	.740	.108
M15.5_rev Consent for sex one time is consent for future sex.	.761	.038
M15.6_rev If you and your sexual partner are both drunk, you don’t have to worry about consent.	.748	.129
M15.7_rev Mixed signals can sometimes mean consent.	.751	.172
M15.8_rev If someone invites you to their place, they are giving consent for sex.	.808	.120
Given consent (α = .555)
M15.1 Consent must be given at each step in a sexual encounter.	.188	.798
M15.3 If a person initiates sex, but during foreplay says they no longer want to, the person has not given consent to continue.	.056	.846

The results from Module 16 suggests only one factor that explained 72% of variance (α = .94). Based on high variance explained and strong alpha coefficients, all seven items within the module worked well together in defining bystander intervention. Specific EFA results are located in Table 14.

Table 14.

Summary of Exploratory Factor Analysis and Reliability Results for Bystander Intervention (Module 16).

Factor name and item	Factor 1
M16.1 Walked a friend who has had too much to drink home from a party, bar, or other social event.	.862
M16.2 Talked to the friends of a drunk person to make sure they don’t leave him/her behind at a party, bar, or other social event.	.859
M16.3 Spoke up against sexist jokes.	.670
M16.4 Tried to distract someone who was trying to take a drunk person to another room or trying to get them to do something sexual.	.879
M16.5 Ask someone who looks very upset at a party if they are okay or need help.	.863
M16.6 Intervene with a friend who was being physically abusive to another person.	.892
M16.7 Intervene with a friend who was being verbally abusive to another person.	.892

Module 17 (campus safety) was composed of two subscales: sense of safety and perception of sexual misconduct as part of campus life (see Table 15). The EFA for sense of safety indicated that the items worked well in forming a single factor that explained 82% of their variance (α = .93). Likewise, items in the second subscale were represented by a single factor that explained 52% of variance (α = .52). Again, the alpha coefficient was likely underestimated due to the small number of items used (Tavakol & Dennick, 2011).

Table 15.

Summary of Exploratory Factor Analysis and Reliability Results for Campus Safety Items (Module 17).

Factor name and item	Factor 1
Sense of safety (α = .929)
M17.A1 On or around this campus, I feel safe from sexual harassment.	.914
M17.A2 On or around this campus, I feel safe from dating violence.	.895
M17.A3 On or around this campus, I feel safe from sexual violence.	.944
M17.A4 On or around this campus, I feel safe from stalking.	.883
Perception of sexual misconduct as part of campus life (α = .522)
M17.B1 I don’t think sexual misconduct is a problem at IHE.	.730
M17.B2 I don’t think there is much I can do about sexual misconduct on this campus.	.565
M17.B4 There isn’t much need for me to think about sexual misconduct while at college.	.834

Note. IHE = institution of higher education.

Confirmatory Factor Analyses

Confirmatory factor analyses were conducted to verify the factor structure that resulted from individual EFAs. See Table 16 for a summary of the fit indices from the confirmatory factor analyses and latent variable correlations where applicable. Mean scores were calculated for the items in each factor and subfactor. See Table 17 for a summary of descriptive statistics for each factor and subfactor.

Table 16.

Summary of Confirmatory Factor Analysis Fit Indices and Latent Variable Correlations.

Module		SRMR	CFI	Latent variable correlations
M1	Possible Outcomes	.045	.933	.145-.705
M3	Peers Norm	.049	.906	.177-.763
M4	Perceptions of Campus Climate Regarding Sexual Misconduct	.052	.915	.147-.625
M5	Sexual Harassment by Faculty/Staff	.075	.931	.525
M6	Sexual Harassment by Students	.051	.914	NA
M7	Stalking Victimization	.061	.911	.724
M8	Stalking Perpetration	.099	.989	NA
M9	Dating Violence Victimization	.036	.973	NA
M10	Dating Violence Perpetration	.025	.993	NA
M13	Institutional Responses	.078	.903	NA
M14	Peer Responses	.029	.977	.194-.365
M15	Consent	.023	.959	.437
M16	Bystander Intervention	.018	.981	NA
M17	Campus Safety	.028	.993	.598

Note. CFI = comparative fit index; SRMR = standardized root mean square residual; NA = not applicable.

Table 17.

Descriptive Statistics of Module and Subfactor on Full Sample.

Module and subfactor	n	M	SD	Minimum	Maximum
M1. Possible Outcomes	4,267	2.76	0.39	1.15	4.85
Academic Satisfaction	4,266	4.15	0.93	1.00	5.00
Academic Disengagement	4,266	1.56	0.53	1.00	5.00
Satisfaction with Life Scale	4,263	3.41	0.92	1.00	5.00
Mental Health	4,257	3.49	0.79	1.00	5.00
M3. Peers Social Support (PSS)	6,657	1.42	0.38	1.00	5.00
PSS Coerced/Forced Sexual Contact	6,656	1.17	0.39	1.00	5.00
PSS Sexual Experiences	6,656	2.60	1.11	1.00	5.00
Informational Peer Support	6,526	1.17	0.37	1.00	5.00
M4. Institutional Response	6,106	3.20	0.77	1.00	6.00
Positive Institutional Behaviors	6,090	3.93	0.86	1.00	5.00
Negative Institutional Behaviors	6,082	3.92	1.00	1.00	5.00
Knowledge of Campus Sexual Misconduct Resources	6,018	3.10	1.18	1.00	5.00
Seen, Heard, or Discussed Sexual Misconduct	5,613	2.80	1.32	1.00	6.00
Awareness of Services	5,308	2.55	0.94	1.00	5.00
M5. Sexual Harassment by Faculty/Staff	3,127	1.11	0.33	1.00	5.00
Physical	3,126	1.05	0.27	1.00	5.00
Nonphysical	3,125	1.20	0.50	1.00	5.00
M6. Sexual Harassment by Students	3,684	1.33	0.60	1.00	5.00
M7. Stalking Victimization	2,941	1.11	0.32	1.00	5.00
Online Stalking	2,937	1.17	0.48	1.00	5.00
Physical Stalking	2,939	1.06	0.24	1.00	5.00
M8. Stalking Perpetration	2,884	1.02	0.11	1.00	4.00
M9. Dating Violence Victimization	4,023	1.12	0.39	1.00	5.00
M10. Dating Violence Perpetration	2,994	1.03	0.15	1.00	5.00
M14. Peer Responses	4,136	1.91	0.63	1.00	5.00
Anticipated Negative Responses	4,116	1.53	0.68	1.00	5.00
Anticipated Positive Responses	4,114	2.31	1.19	1.00	5.00
General Responses	4,126	2.14	0.96	1.00	5.00
M15. Consent	4,135	4.66	0.49	1.00	5.00
Automatic Consent	4,134	4.66	0.55	1.00	5.00
Given Consent	4,135	4.66	0.68	1.00	5.00
M16. Bystander Intervention	3,634	2.74	1.33	0.00	5.00
M17. Campus Safety	4,092	3.35	0.71	1.00	5.00
Sense of Safety	4,090	3.95	0.93	1.00	5.00
Perception of Sexual Misconduct as Part of Campus Life	4,089	2.56	0.79	1.00	5.00

According to the predesignated structures, the four-factor model for possible outcomes fit the data satisfactorily, SRMR = .045 and CFI = .933. These four factors were positively correlated with each other (latent variable correlation ≥ .145, ps < .001). The three-factor model for peer norms fit the data satisfactorily, SRMR = .049 and CFI = .938. They were positively related to each other (latent variable correlation ≥ .177, ps < .001). The factors in the perceptions of campus climate regarding sexual misconduct also fit the model satisfactorily, SRMR = .052 and CFI = .915. These five factors in this module were positively correlated with each other (latent variable correlation ≥ .147, ps < .001).

Regarding the sexual harassment module, the fit for the two-factor model was acceptable (SRMR = .075 and CFI = .931). The two factors were significantly associated with each other (latent variable correlation = .525, p < .001). Also, the one factor in the sexual harassment by students demonstrated adequate model fit (SRMR = .051 and CFI = .914). For the stalking victimization, the model fit for the underlying two factors was satisfactory (SRMR = .061 and CFI = .911). Physical stalking and online stalking factors were significantly related to each other (latent variable correlation = .724, p < .001). The one-factor model fit in the stalking perpetration was considered acceptable (SRMR = .099 and CFI = .989). With respect to dating violence victimization and perpetration, only one factor was detected in the EFA separately. Both performed good model fits in dating violence victimization (SRMR = .036 and CFI = .973) and dating violence perpetration (SRMR = .025 and CFI = .993).

Peer responses was composed of three factors and the model fit was good (SRMR = .029 and CFI = .977). After reverse coding anticipated positive responses, the three factors were positively correlated with each other (latent variable correlation ≥ .194, ps < .001). As for the consent items, the two factors generated from EFA demonstrated good model fit (SRMR = .023 and CFI = .959). These two factors were positively associated with each other (latent variable correlation = .437, p < .001). In the bystander intervention, only one factor was extracted, which showed good model fit as well (SRMR = .018 and CFI = .981). Last, the predesignated two-factor model in the campus safety fit the data satisfactorily (SRMR = .028 and CFI = .993). Sense of safety was positively correlated with perception of sexual misconduct as part of campus life (latent variable correlation = .598, p < .001).

Factor Analysis of Mixed Data

Factor analysis of mixed data and confirmatory factor analyses were conducted to verify the factor structure for Modules 2, 11, 12, and 13. See Table 18 for a summary of the fit indices and reliability statistics. Mean and summed scores were calculated for the items in each factor. Due to the unique answer scales on each item, Module 2 items were standardized before creating a mean score. See Table 19 for a summary of descriptive statistics for each factor and subfactor.

Table 18.

Summary of Fit Indices From Mixed Data and Confirmatory Factor Analyses.

Module		Sample 1		Sample 2
Module		SRMR	CFI	SRMR	CFI
M2	Alcohol Use (α = .829)	.002	.995	.004	.990
M11	Sexual Violence Victimization (α = .771)	.023	.997	.022	.989
M12	Sexual Violence Perpetration (α = .764)	.009	.995	.021	.981
M13	Institutional Responses (latent reliability = .904)^a	.078	.903	.072	.907

Note. CFI = comparative fit index; SRMR = standardized root mean square residual.

Reliability calculated using latent variable modeling due to dichotomous items.

Table 19.

Descriptive Statistics of Modules 2, 11, 12, and 13.

Module and subfactor	n	M	SD	Minimum	Maximum
M2. Alcohol Use	6,769	0.20	0.92	−3.40	1.46
M2.1a how often did you usually have any kind of drink containing alcohol?	6,769	7.00	2.74	1.00	11.00
M2.2 how many alcoholic drinks did you have on a typical day when you drank alcohol?	5,263	8.51	1.45	1.00	10.00
M2.3 how often did you have 5 or more (males) or 4 or more (females) drinks containing any kind of alcohol within a 2-hour period?	5,320	8.14	2.03	1.00	10.00
M2.4 what is the maximum number of drinks containing alcohol that you drank within a 24-hour period?	5,622	6.92	2.18	1.00	10.00
Zscore: M2.1	6,769	0.00	1.00	−2.19	1.46
Zscore: M2.2	5,263	0.00	1.00	−5.17	1.02
Zscore: M2.3	5,320	0.00	1.00	−3.51	0.91
Zscore: M2.4	5,622	0.00	1.00	−2.72	1.42
M11. Sexual Violence Victimization (Sum score)	4,671	0.52	1.63	0.00	15.00
M12. Sexual Violence Perpetration (Sum score)	4,310	0.02	0.27	0.00	9.00
M13. Institutional Responses (Sum score)	6,775	0.58	2.92	0.00	25.00

Module 2 (Alcohol Use) asked about having ever consumed alcohol, typical number of drinks, frequency of episodic drinking, and maximum number of drinks in a 24-hour period. The model for alcohol use (Module 2) was composed of four items and results confirmed very good model fit for this predesignated one factor (Sample 1: SRMR = .002 and CFI = .995; Sample 2: SRMR = .004 and CFI = .990). Reliability of these four items was also good (α = .829). Similarly, Module 11 (Sexual Violence Victimization), Module 12 (Sexual Violence Perpetration), and Module 13 (Institutional Responses) each also had good one-factor solutions (see Table 18).

Discussion

The ARC3 survey is a reliable and valid instrument to measure campus climate around sexual misconduct and related concepts. Taken together, predesignated and generated factors in each module demonstrated strong internal consistency across their respective items. Only Modules 5 (Sexual Harassment by Faculty/Staff) and 6 (Sexual Harassment by Students) had items that did not appear to have strong consistency with the other items. CFA further confirmed the satisfactory factor structure in the predesignated modules. Thus, items can be aggregated into a subscale score composed of the mean of the items within that factor. For example, Module 5 is composed of two separate factors (Physical Sexual Harassment and Nonphysical Sexual Harassment). Based on the factorial validity evidence presented above, individual mean scores could be calculated to represent each factor. These mean scores can be used by IHE administrators and researchers to attain a general idea about campus climate and the prevalence of sexual abuse and misconduct, as well as areas to prioritize for intervention.

Using factor scores from the ARC3 instrument allowed evaluators and IHE leadership to take a strategic, diagnostic, and operational approach to the data, identify areas of weakness on their particular campus, and develop programs to facilitate a culture shift, as well as incorporate regular campus climate surveys to assess potential change. Strategies for action and change were selected on an institution-by-institution basis based on the scores they received at the baseline measurement, which served as a diagnostic measure. Each IHE and the project office worked together to strategically determine the prevention approaches best fit for addressing the areas in need of improvement based on the module scores the IHE it received. Posttest scores will reflect whether their efforts are working and offer other areas for further improvement. Utilizing a tailored approach to targeted efforts for each IHE allows for effective allocation of resources, as there is not a one-size-fits-all approach to addressing campus climate. Module scores allowed IHEs to focus their attention on increasing or decreasing scores for targeted areas specific to their institutions. Whereas one IHE may need to address bystander interventions more thoroughly, another may need to address the perceptions of risk on their campus.

Limitations and Directions for Future Research

Several limitations existed in the current study. First, while the sample size was large, participants in the samples were primarily White (57%) and female (70%). Future studies could examine whether the psychometric properties of the ARC3 survey are upheld in more diverse samples. Second, participants self-selected to complete the survey; individual interest in the topic may have represented a source of bias. Further, many students with incomplete data or who did not respond correctly to the validity questions were removed from analysis, therefore, the sample may represent only conscientious students. Third, the ARC3 survey is a comprehensive survey that requires approximately 30 minutes to complete (Swartout et al., 2018). The time commitment may have dissuaded some participants from participating. Finally, the ARC3 survey does not include an assessment of social desirability response bias. This is of particular note given the sensitive nature of many of the items.

Conclusion

Campus climate surveys are an important tool in helping universities understand the nature and extent of sexual assault on their campuses. The ARC3 survey is an effective instrument to collect university data on campus climate related to sexual assault and sexual misconduct. This psychometric analysis of the ARC3 survey indicated that it is a valid self-report measure that provides important data about the problem of sexual assault and associated variables. This analysis also indicated that generally factors and subfactors can be aggregated from the items predesignated within the ARC3 survey to allow academic leadership to develop strategies for action from the module scores report. Utilizing a tailored approach to targeted efforts for each IHE means that resources can be utilized more effectively, as there is not a one-size-fits-all approach to addressing campus climate. Module scores allow IHEs to focus their attention on increasing or decreasing scores for targeted areas. This practical and operational approach to managing outcomes has the potential to help other campus initiatives more fully address the issues specific to their institutions.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by a grant (Cultivating Safe College Campuses) from the Department of Health and Human Services, Office on Women’s Health. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Office on Women’s Health or the U.S. Department of Health and Human Services.

ORCID iDs

Donna Scott Tilley

Ann Kolodetsky

Notes

Supplement Note

This article is part of the Health Education & Behavior supplement issue, “Campus Sexual Assault Policy and Prevention.” Thank you to the following organizations which have generously supported the open access publication of this special supplement: Johns Hopkins Bloomberg School of Public Health, Center for Injury Research and Policy; Johns Hopkins University School of Nursing; Futures Without Violence; NORC at the University of Chicago; Texas Woman’s University; University of Kansas Department of Women, Gender & Sexuality Studies; University of Kansas School of Social Welfare; University of Kansas School of Public Affairs & Administration; and University of New Hampshire’s Prevention Innovations Research Center (PIRC). The entire supplement is available open access at .

References

Administrator-Researcher Campus Climate Collaborative. (2015, September 1). ARC3 Campus Climate Survey. https://campusclimate.gsu.edu/arc3-campus-climate-survey/

Bountress

K. E.

Metzger

I. W.

Maples-Keller

J. L.

Gilmore

A. K.

(2017). Reducing sexual risk behaviors: Secondary analyses from a randomized controlled trial of a brief web-based alcohol intervention for underage, heavy episodic drinking college women. Addiction Research & Theory, 25(4), 302–309. https://doi.org/10.1080/16066359.2016.1271416

Breiding

M. J.

Smith

S. G.

Basile

K. C.

Walters

M. L.

Chen

Merrick

M. T.

(2014). Prevalence and characteristics of sexual violence, stalking, and intimate partner violence victimization—National Intimate Partner and Sexual Violence Survey, United States, 2011. Morbidity and Mortality Weekly Report. Surveillance Summaries, 63(8), 1–18. https://www.cdc.gov/mmwr/preview/mmwrhtml/ss6308a1.htm?

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. Guilford Press.

Caamano-Isorna

Adkins

Moure-Rodriguez

Conley

A. H.

Dick

(2018). Alcohol use and sexual and physical assault victimization among university students: Three years of follow-up. Journal of Interpersonal Violence. Advance online publication. https://doi.org/10.1177/0886260518780413

Clark

L. A.

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. https://doi.org/10.1037/1040-3590.7.3.309

DiJulio

Norton

Craighill

Clement

Brodie

(2015, June 12). Survey of current and recent college students on sexual assault. Kaiser Family Foundation. https://www.kff.org/other/poll-finding/survey-of-current-and-recent-college-students-on-sexual-assault/

Fedina

Holmes

J. L.

Backes

B. L.

(2018). Campus sexual assault: A systematic review of prevalence research from 2000 to 2015. Trauma, Violence, & Abuse, 19(1), 76–93. https://doi.org/10.1177/1524838016631129

Field

(2013). Discovering statistics using IBM SPSS. Sage.

10.

Hart

Fellabaum

(2008). Analyzing campus climate studies: Seeking to define and understand. Journal of Diversity in Higher Education, 1(4), 222–234. https://doi.org/10.1037/a0013627

11.

Henson

R. K.

Roberts

J. K.

(2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological measurement, 66(3), 393–416. https://doi.org/10.1177/0013164405282485

12.

Hill

Silva

(2005). Drawing the line: Sexual harassment on campus. American Association of University Women.

13.

Hooper

Coughlan

Mullen

(2008). Structural equation modeling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60. https://arrow.dit.ie/cgi/viewcontent.cgi?referer=https://scholar.google.com/&httpsredir=1&article=1001&context=buschmanart

14.

Kettrey

H. H.

Marx

R. A.

Tanner-Smith

E. E.

(2019). Effects of bystander programs on the prevention of sexual assault among adolescents and college students: A systematic review. Campbell Systematic Reviews, 15(1–2), e1013. https://doi.org/10.4073/csr.2019.1

15.

Kline

(1994). An easy guide to factor analysis. Routledge.

16.

Know your title IX. (n.d.). Statistics. https://www.knowyourix.org/issues/statistics/

17.

Krebs

Lindquist

Berzofsky

Shook-Sa

Peterson

(2016, January). Campus climate survey validation study final technical report. Bureau of Justice. https://www.bjs.gov/content/pub/pdf/ccsvsftr.pdf

18.

Krebs

Lindquist

Warner

Fisher

Martin

(2007, October). The campus sexual assault (CSA) study. U.S. Department of Justice. https://www.ncjrs.gov/pdffiles1/nij/grants/221153.pdf

19.

Lindquist

Krebs

(2018, April 9). Sexual assault on college campuses: Are climate surveys still relevant? RTI International. https://www.rti.org/insights/sexual-assault-college-campuses-are-climate-surveys-still-relevant

20.

McDonald

R. P.

M. R.

(2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7(1), 64–82. https://doi.org/10.1037//1082-989X.7.1.64

21.

Mellins

C. A.

Walsh

Sarvet

A. L.

Wall

Gilbert

Santelli

J. S.

Thompson

Wilson

P. A.

Khan

Benson

Bah

Kaufman

K. A.

Reardon

Hirsch

J. S.

(2017). Sexual assault incidents among college undergraduates: Prevalence and factors associated with risk. PLOS ONE, 12(11), Article e0186741. https://doi.org/10.1371/journal.pone.0186471

22.

Pagès

(2004). Analyse Factorielle de Donnees Mixtes [Factorial analysis of mixed data]. Revue Statistique Appliquee, 4, 93–111.

23.

Pagès

(2014). Multiple factor analysis by example using R. Chapman & Hall/CRC Press. https://doi.org/10.1201/b17700

24.

Parks

K. A.

Hsieh

Taggart

Bradizza

C. M.

(2014). A longitudinal analysis of drinking and victimization in college women: Is there a reciprocal relationship? Psychology of Addictive Behaviors, 28(4), 943–951. https://doi.org/10.1037/a0036283

25.

Raykov

Dimitrov

D. M.

Asparouhov

(2010). Evaluation of scale reliability with binary measures using latent variable modeling. Structural Equation Modeling, 17(2), 265–279. https://doi.org/10.1080/10705511003659417

26.

Reise

S. P.

Waller

N. G.

Comrey

A. L.

(2000). Factor analysis and scale revision. Psychological Assessment, 12(3), 287–297. https://doi.org/10.1037//1040-3590.12.3.287

27.

Sun

(2005). Assessing goodness of fit in confirmatory factor analysis. Measurement and Evaluation in Counseling and Development, 37(4), 240–256. https://doi.org/10.1080/07481756.2005.11909764

28.

Swartout

K. M.

Flack

W. F.

Jr. Cook

S. L.

Olson

L. N.

Smith

P. H.

White

J. W.

(2019). Measuring campus sexual misconduct and its context: The Administrator Researcher Campus Climate Consortium (ARC3) Survey. Psychological Trauma: Theory, Research, Practice, and Policy, 11(5), 495–504. https://doi.org/10.1037/tra0000395

29.

Tavakol

Dennick

(2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

30.

U.S. Department of Health & Human Services. (n.d). The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html/

31.

U.S. Department of Justice. (2017, January). Administrator information packet on campus climate surveys. https://www.justice.gov/ovw/page/file/929846/download

32.

White

Hingson

(2013). The burden of alcohol use: Excessive alcohol consumption and related consequences among college students. Alcohol Research, 35(2), Article 201–18. https://www.ncbi.nlm.nih.gov/pubmed/24881329

33.

Wood

Sulley

Kammer-Kerwich

Follingstad

Busch-Armendariz

(2017). Climate surveys: An inventory of understanding sexual assault and other crimes of interpersonal violence at institutions of higher education. Violence Against Women, 23(10), 1249-1267. https://doi.org/10.1177/1077801216657897

34.

Yeater

E. A.

Witkiewitz

Lopez

Ross

R. S.

Vitek

Bryan

(2018). Latent profile analysis of alcohol consumption and sexual attitudes among college women: Associations with sexual victimization risk. Violence Against Women, 24(11), 1279–1298. https://doi.org/10.1177/1077801218787926