Differences in Nonresponse Bias and Victimization Reports Across Self-Administered Web-Based and Paper-and-Pencil Versions of a Campus Climate Survey

Abstract

Using a standardized campus climate survey that was disseminated across three modes of administration (N = 5,137), this study assesses the nonresponse bias of two web-based versions to a self-administered paper-and-pencil version conducted at a Southeastern 4-year university. Significant differences emerged across all three modes of administration and victimization measures (bullying, sexual assault, rape, emotional abuse, and intimate partner violence [IPV]). Respondents were more likely to report victimization in the web-based surveys administered to online-only classes and via mass email compared to the paper survey. Policy implications, especially as it relates to survey administration, are discussed.

Keywords

campus climate survey victimization sexual victimization nonresponse bias self-administered surveys

Recent years have seen a sharp increase in scholarship, policy reports, activism, and legal reforms directed at campus safety, with specific attention to interpersonal and sexual violence. These highly publicized contemporary efforts directed at campus climate have mistakenly communicated to the public that victimization on college campuses is a more recent issue. However, interpersonal victimization, including sexual aggression and harassment against college women, has long been a serious problem (Fisher et al., 2009; Kirkpatrick & Kanin, 1957; Koss, 1989; Krebs et al., 2007). Several influential studies suggest that between 15% and 25% of women experience some form of sexual victimization during their college career (Fisher et al., 2000; Krebs et al., 2007).

The past decade has witnessed widespread efforts to highlight, respond to, and address campus interpersonal and sexual violence. More than 35 years after the passing of formative legislation, Title IX, and nearly 20 years after the passing of the Clery Act, recent legal reforms have again directed attention to the importance of safety on university campuses. In 2011, the United States Department of Education, Office for Civil Rights released a “Dear Colleague” letter aimed at guiding institutions of higher education (IHEs) in effective steps to respond to sexual violence (Office of the Assistant Secretary, 2011). Soon thereafter, in 2013, the Violence Against Women Act was reauthorized and the Campus Sexual Violence Elimination (Campus SaVE) Act was enacted. Finally, in 2014, the White House Task Force to Protect Students from Sexual Assault was formed, cementing the importance of addressing campus climate. The Task Force encouraged universities to collect self-reported data on student perceptions of “campus climate” as well as their victimization experiences (Lindquist & Krebs, 2017), underscoring the necessity of understanding students’ institutional experiences. These efforts continue to develop, as legislation and amendments have been proposed with aims at increasing the scientific rigor of surveys conducted on college campuses with student populations (Cantalupo, 2014).

Even with unprecedented numbers of colleges conducting climate surveys, there is not a standardized instrument (L. Wood et al., 2017). Web-based and email surveys provide viable administrative approaches for collecting data and internet-based administrations appear particularly well-suited for studying college student populations (Wells et al., 2012). With that said, best practices for the administration and dissemination of climate surveys remain unknown. This is problematic as research suggests that computerized self-administration leads to greater reporting of socially “undesirable” behavior, decreasing item nonresponse and improving accuracy (Kreuter et al., 2008; Tourangeau & Yan, 2007). Using a climate survey that was disseminated across three modes of administration, this study assesses the nonresponse bias of two self-administered web versions to a self-administered in-person paper version of a climate survey conducted at a Southeastern 4-year university (i.e., SE4YU). The web-based forms include an electronic distribution to a random sample of online courses and a mass-email distribution to the entire student body. This comparison will reveal whether traditional paper surveys are necessary to adequately capture campus climate or if more cost-effective and time-efficient internet-based modes of dissemination can be used.

Measurement of Violence on College Campuses

Documenting the prevalence and incidence of violence on university campuses is challenging (L. Wood et al., 2017). Sexual assault that occurs on college campuses suffers from underreporting (Fisher et al., 2003). In addition, administrators and donors may be concerned that high crime rates reported in university statistics will affect their institution’s reputation (Luther, 2016). Although assessing the prevalence of these crimes can be challenging, IHEs have attempted to capture data using two mechanisms—the Clery Act and campus climate surveys.

To enhance transparency surrounding campus crime and associated policies, the Clery Act mandates that all IHEs eligible for student aid meet three requirements (Fisher et al., 2002). First, within 2 days of being notified of an offense, police or security must include the criminal incident in a public crime log that must be available during typical business hours and be kept open for 60 days. Second, both respondents and complainants involved in university-related disciplinary charges are guaranteed the right to have people present at the hearing and to learn the outcome of the hearing. The final requirement is most directly related to the reporting of campus crime statistics; IHEs must publish an annual report on crime statistics and security policies aimed at improving campus safety by October 1. This annual report must contain crime data from the three most recent calendar years.

Research has demonstrated limitations associated with reporting requirements under the Clery Act (Cantalupo, 2011). The biggest critique of the Clery Act, as it relates to the collection of campus crime statistics, involves the underreporting of sexual assault. This critique is demonstrated in Gardella and colleagues’ (2015) work that compared reports of sexual violence that were reported under the Clery Act to survey responses from anonymous students. Findings indicated that reports of sexual violence collected via an anonymous survey were six times greater than those found in the reports under the Clery Act (Gardella et al., 2015). Scholarship demonstrating the limitations of reporting under the Clery Act has resulted in the development of campus climate surveys. These surveys were designed to address these limitations and illuminate the prevalence of campus violence, including intimate partner violence (IPV) and sexual victimization, which are often not reported to campus authorities.

In response to federal efforts and Clery Act limitations, universities have implemented various strategies to address campus safety generally and sexual assault specifically. Under federal protections associated with Title IX and clarified in the 2011 “Dear Colleague” letter, colleges and universities are responsible for addressing sexual assault, IPV, sexual harassment, and stalking (Hughes-Miller, 2017). Therefore, university administrators and scholars at IHEs have made serious attempts to collect data on the rates of these offenses through campus climate surveys. Goals of these student surveys include identifying the amount of unreported campus crimes not captured by the Clery Act, assessing student perceptions of campus safety, and establishing student knowledge of on- and off-campus resources. These surveys also assess a wide range of behaviors and factors beyond victimization that potentially affect student safety or one’s ability to obtain an education (Henry et al., 2011). The importance of these surveys as they relate to campus safety were reinforced through the endorsement by the Obama administration and strongly encouraged under the Title IX and Clery Act compliance program.

In 2015, the Association of American Universities conducted a standardized cross-university climate survey at 27 IHEs to identify the “incidence, prevalence, and characteristics of sexual assault and misconduct” (Cantor et al., 2015, p. iii). Overall, 11.7% of students at the 27 IHEs reported they had experienced nonconsensual penetration or sexual touching by force or incapacitation, and that the majority of these individuals did not report the incident to an organization or agency (Cantor et al., 2015). At the same time, the Campus Climate Survey Validation Study was ongoing at nine IHEs, with the goal of improving the methodological rigor of climate surveys (Krebs et al., 2016; Lindquist & Krebs, 2017). The findings from these cross-university efforts can inform best practices for university-specific climate surveys.

Web-Based Versus Paper-and-Pencil Survey Administration

Over the past four decades, technology has fundamentally changed the ways survey research is conducted (Evans & Mathur, 2005). The end of the 20th century brought tremendous innovation, beginning with the development of email surveys in the 1980s and web-facilitated surveys in the 1990s (Schonlau et al., 2001). Moreover, in response to the increased use of technology and internet-facilitated research methodologies, several websites (e.g., Survey Monkey, Qualtrics) now serve as platforms for the administration of web-based surveys.

There are distinct advantages and disadvantages associated with both web-based and paper-and-pencil face-to-face survey development, management, and dissemination. Regarding advantages, web-based survey methodologies are typically associated with affordability, greater dissemination, and enhanced participant anonymity and confidentiality (Couper, 2000; Lewis et al., 2009; Sue & Ritter, 2007). Moreover, web-based survey administration is often considered more convenient for both researchers and participants, in addition to being less time-consuming overall (Davidov & Depner, 2011; Lewis et al., 2009). Paper-and-pencil face-to-face surveys, in contrast, do not have complications associated with the “digital divide”—the gap that differentiates those who have access to technology and the internet and those who have access to technology and the internet and thos do not (Warschauer, 2004). Finally, paper-and-pencil face-to-face survey methods allow the research team to control the environment in which the survey is taken and to answer questions raised by respondents during administration (E. Wood et al., 2006).

There are important considerations that must be taken into account when administering an electronic survey. For instance, McMahon and colleagues (2018) elected to use a census sampling strategy when administering a campus climate survey. This strategy allowed every student the opportunity to have a voice. A census strategy must be weighed against a random sample strategy, which allows for generalizability claims across the entire population—assuming there is no nonresponse bias (Maxfield & Babbie, 2016)—but relies on fewer participants.

Nonresponse in Self-Administrated Surveys

Surveys investigating sensitive topics often rely on self-administration, a strategy that increases reporting of sensitive information (Lind et al., 2013). More specifically, research suggests that computerized self-administration leads to greater reporting of socially “undesirable” behavior, decreasing item nonresponse and improving accuracy (Kreuter et al., 2008; Tourangeau & Yan, 2007). Although web-based survey administration is associated with a multitude of advantages, one potential limitation includes nonresponse. Nonresponse occurs when a subset of the sampled population does not participate in the survey (Groves, 2004). Nonresponse may lead to incorrect survey results if systematic differences between respondents who elected to complete the survey and sampled individuals who did not complete the survey arise (Couper, 2000; Groves, 2004).

Limited research exists regarding whether the mode of self-administration has an impact on item nonresponse and/or responding to questions that are perceived to be intrusive. Survey outcomes regarding sensitive questions can be affected in three important ways: (a) overall response rates, (b) item nonresponse rates, and (c) response accuracy (Tourangeau et al., 2013; Tourangeau & Yan, 2007). In terms of overall response rates, participants may have confidentiality concerns. The mode of administration may affect the respondent’s understanding of confidentiality concerns. Misreporting about sensitive topics on surveys is frequent and may be driven by the respondent’s embarrassment or features of the survey itself (Tourangeau & Yan, 2007). In terms of item nonresponse, research has demonstrated that participants may be embarrassed or afraid to disclose their victimization experiences (Jordan, 2001; Kahn et al., 2003). In this study, the data collection setting differed across the modes of administration and may therefore affect the total amount of missing data. Taken together, the mode of administration may have an effect on overall response rates as well as item nonresponse.

Current Study

The goal of this study was to examine nonresponse bias across three different modes of a campus climate survey administration. We examine if type of survey administration is associated with five types of victimization experiences to identify if any of the sampling approaches are more apt at collecting information on victimization experiences. More specifically, we evaluate differences in victimization experiences between students who completed the survey during an in-person course compared to students from randomly selected online courses and those who completed the survey in response to a mass-email solicitation.

Materials and Methods

Campus Climate Survey at Southeastern 4-Year University

The campus climate survey that was administered for this project was adapted from the University of Kentucky’s (2015) Campus Attitudes Toward Safety. Questions on the survey included perceptions of safety, knowledge of campus resources, experiences on campus, social attitudes, and use of campus and local social services. At SE4YU, the climate survey was administered in three different formats: (a) an in-person paper survey administered in randomly selected classes across all campuses, (b) an electronic survey sent to students enrolled in randomly selected online-only courses, and (c) a mass email sent to every SE4YU student.

In-person paper survey to randomly selected courses

To identify the target population of in-person classes, we created a list of all course offerings at SE4YU for spring 2017. Laboratories, independent studies, and duplicate sections were removed from this list. We elected to retain courses offered at the University’s satellite campus, which is about a 45-minute drive from the main campus. With this complete list of course offerings, we randomly selected approximately 10% of classes offered at SE4YU during the spring of 2017 (N = 198).

Of the 198 selected courses, the instructor on record was emailed in mid-December 2016 to request that they allow their students to participate in the survey during the first 2 weeks of classes. The email to the instructor highlighted the survey was being conducted in coordination with the Title IX office on campus along with a brief description of the project. Within the body of the email, we suggested allowing the research team to administer the survey on the first day of class. If that date was not convenient for the instructor, the research team worked to find a date or time that was suitable for the instructor. At the discretion of the instructor, students were offered extra credit for their participation. In cases where the instructor did not respond to our initial inquiry, we sent two additional follow-up emails in January of 2017. In total, 96 instructors or half of the classes (48.24%) solicited for participation allowed the research team to administer the survey to their class. While some instructors simply did not write back to the email solicitation, the most common reason given by instructors for electing not to have their students participate was that they could not give that much class time. The surveys cost roughly $1,800 to print, and 2,265 individuals answered at least one item on the survey.

Electronic survey to online-only classes

In a similar process as mentioned above, a random sample of online classes was also selected. A list of all online course offerings for spring 2017 at the SE4YU was generated in December 2016. As there were a smaller total number of online classes offered at SE4YU (N = 639), we randomly selected 20% of online classes (N = 130). One data collection obstacle encountered with recruiting online-only classes included some instructors having collapsed multiple sections of their course into one online section. In these few cases, the instructors agreed to allow us to survey their students, but requested that we survey all their sections. In these instances, we agreed to survey all of their sections.

Instructors in these classes were initially emailed and asked if they would allow their students to participate and consider offering extra credit for participation. Of the 130 randomly selected online-only courses, 78 (60.00%) instructors allowed us to survey their course. The most common reasons given for not allowing students to participate was survey fatigue and lack of time. For the courses where the instructor agreed, a personalized email was drafted and sent to the instructor for dissemination to their students through the online course system. Overall, 1,325 individuals consented to participate in the survey administered to online-only classes.

Electronic survey via mass email to SE4YU student body

To recruit respondents from the entire student body, an electronic version of the survey was created in Qualtrics. This electronic version presented questions in the same order as the paper survey but had skip-patterns automatically built into the survey (as opposed to the paper version where students had to [correctly] interpret and execute the skip-patterns themselves). An email, with the survey link embedded in it, was sent to every SE4YU student—both graduate and undergraduate students—on three separate occasions. The first email was sent on March 21, 2017. Two follow-up emails were sent on March 30, 2017, and April 10, 2017. The timing of these emails was important, as incentives (i.e., gift cards) were distributed based on when the student completed the survey. For the first mass email, the first 25 students who completed the survey received a $10 Starbucks gift card. For the two follow-up emails, 25 students who completed the survey within 48 hours of the email being sent were randomly selected to receive a $10 Starbucks gift card. All students who completed the survey were eligible to win one of eight $100 Amazon gift cards. A total of $1,300 in incentives was distributed.

Consistent with Pickett and colleagues’ (2018) recommendation, we provide information on the survey process to help readers understand how nonresponse may (or may not) be associated with other items in the survey. The subject line of the email sent to the entire SE4YU student body reflected the name of the broader project and read “Investigating [School Mascot Removed] Attitudes Toward Safety.” Although the email was sent through an automated university system, the first author’s name and email was listed as the sender. The body of the email invited the student to participate in an anonymous study, which had been approved by the University’s Institutional Review Board. Additional details on the topic of the survey were not provided in the email. The email emphasized that if the student took the survey for an in-person or online course they could still complete the mass-email version. Once students completed the survey, they were directed to a final link if they wanted to be considered for an incentive. Once data collection ended, the students who received incentives were selected, contacted, and all email addresses were destroyed. No further identifying information was collected in the survey. In total, 1,583 individuals consented to take the survey.

Measures

Dependent Variables

We examine different types of victimization—instead of collapsing measures to represent general campus victimization—because students may feel less (or more) comfortable disclosing certain experiences. For example, disparity in the actual occurrence and official reporting of rape and sexual victimization is often associated with victim-blaming responses and the overall discouragement victims experience after disclosure (Kahn et al., 2003), in addition to fears that they will not be believed (Jordan, 2001). Overall, differences across victimization experiences as they relate to stigma, public awareness, and societal and institutional responses may result in differences across reporting/disclosure practices.

Bullying

Respondent’s experience with bullying was a binary indictor created from three measures. The header for this question series stated, “During the past 12 months, how many times has another SE4YU student (NOT someone you are dating or a spouse/partner)” and listed a series of behaviors. Bullying behaviors included: (a) “spreading rumors, trying to ruin reputation, calling names, or telling mean jokes to the respondent’s face or near where they could hear”; (b) “spreading rumors, trying to ruin reputation, calling names, or telling mean jokes on social media”; or (c) “tripping, shoving, hitting, breaking something, or threatening physical violence.” Respondents could report if the behavior happened never, once, sometimes (2–5 times), or often (6+ times). Responses were collapsed across all measures so that if the respondent experienced at least one behavior at least once over the last 12 months they were coded “1” for having experienced bullying victimization. If the respondent did not report experiencing any of the behaviors over the last 12 months, they were coded “0.” The bullying victimization measure was then disaggregated into two binary measures to reflect social bullying (i.e., spreading rumors etc. in person or online) and physical bullying.

Sexual Victimization

Rape victimization

The dichotomous measure of rape victimization was created from five behaviorally specific items. Respondents were asked if during the past 12 months they were forced to have sex (intercourse, oral sex, anal sex, or penetration with an object) against their will; they were threatened with harm if they did not have sex; they were unable to consent because of drugs or extra alcohol that was slipped to them; they were unable to consent because of alcohol or drugs they took voluntarily; or if they managed to escape a situation where they were in the process of being physically forced to have sex. If the respondent responded affirmatively to at least one of the five questions, they were coded “1”—experienced rape victimization during the last 12 months. Respondents who answered no to all five questions were coded “0.”

Sexual victimization

Respondents were also asked if someone said sexual things, sent sexual messages, pressured them for a date/sexual favors, made unwanted sexual gestures, touched the respondent sexually, or exposed themselves. In an effort to capture the range of sexual victimization, respondents who indicated they experienced any of these behaviors or experienced rape were combined into a binary indicator where “1” = experienced sexual victimization during the last 12 months.

Emotional Abuse

Respondents were asked if during the past 12 months someone you were dating or a spouse/partner engaged in the following behaviors (not in a playful or joking manner): (a) “Checked up on you by following you, invaded your privacy by reading private messages, or listened in on calls that were NOT done in a joking or playful manner”; (b) “Threatened or intimidated you by destroying something”; (c) “Threatened to harm you or others”; (d) “Made you do what they wanted by threatening to end the relationship”; (e) “Made you do what they wanted by threatening to commit suicide”; (f) “Tried to make your personal decisions for you”; or, (g) “In front of others, insulted you, acted like they hated you, flirted with others.” Respondents could report if the behavior never happened, happened once, sometimes (2–5 times), or often (6+ times). Responses were collapsed across all measures so if the respondent experienced at least one behavior once over the past year they were coded “1.” Respondents who did not report experiencing any of the emotionally abusive behaviors over the last 12 months were coded “0.”

IPV

Physically abusive behaviors included being shoved, slapped, hit, burned, strangled, or threatened with a weapon over the past year by someone the respondent was dating or a spouse/partner. Responses across these 10 items were combined into a binary measure where “1” = respondent experienced IPV in his or her relationship in the past year.

Independent Variables

Mode of administration

Three dummy variables were created to capture the format in which the respondent completed the survey. The three indicators included (a) in-person paper survey, (b) online class, and (c) mass email.

Control Measures¹

Rape myth acceptance

Research suggests that high rape myth acceptance is associated with sexual assault (Abbey et al., 2001). Rape myth acceptance was measured with a 15-item scale that ascertains the respondent’s attitudes toward sexual violence (e.g., An incident can only be sexual assault or rape if the person says “no”). The responses for the items were measured with a Likert-type scale ranging from “0” = Strongly disagree to “3” = Strongly agree. Two items reflect a nonadversarial relationship and were reverse coded. The 15 items were summed into a scale and averaged, where a higher score indicated greater rape myth acceptance (mean = 0.92; SD = 0.54; Cronbach’s α = 0.87; range = 0.00–2.66).

Binge drinking

Prior research has shown that alcohol use, especially binge drinking, is associated with victimization risk on college campuses, including sexual victimization (McCauley et al., 2010). A binary measure captured if the respondent reported they had five or more drinks in a row during a 2-hour period over the last 2 weeks. Respondents who answered yes were coded “1.” Respondents who answered “No,” “Not in the last 2 weeks,” or that they have never drank were coded “0.”

Other control measures

A series of variables previously shown to be associated with victimization risk on college campuses were also included in analyses (Cantor et al., 2015; Franklin, 2016; Franklin et al., 2012). Days on campus was a continuous measure (mean = 3.97; SD = 1.80; range = 1–7 days). Gender was a binary measure where “1” = Female and “0” = Male. If the respondent lived on campus was a binary measure where “1” = Lived on campus and “0” included living in off-campus apartments, off campus with family, or off campus with friends. Race was included as a binary indicator where “1” = People of color and “0” = White. Owing to the digital divide (Warschauer, 2004), we included age as a dichotomous measure where “1” = Respondent was of traditional college age (i.e., 18–22 years old) and “0” = Respondent was 23 years or older. Finally, we included three measures that captured the respondent’s involvement with on-campus organizations where “1” = Respondent was a member of that organization. These measures included if the respondent was a member of an on-campus athletic team, an intramural athletic team, or Greek life.² Descriptive statistics by mode of survey administration as well as for the overall sample can be found in Table 1.

Table 1.

Descriptive Statistics.

	Whole sample (N = 5,173)		In-person survey (n = 2,265)		Online survey (n = 1,325)		Mass-email survey (n = 1,583)		Student body (N = 19,278)
Variables	N	%	n	%	n	%	N	%	n	%
Dependent variables
Bullying	1,074.2	20.77	410.5	18.12	237.3	17.91	426.4	26.94
No bullying	4,098.8	79.23	1,854.5	81.88	1,087.7	82.09	1,156.6	73.06
Physical bullying	292.8	5.66	87.1	3.85	74.9	5.65	130.8	8.26
No physical bullying	4,880.2	94.34	2,177.9	96.15	1,250.1	94.35	1,452.2	91.74
Social bullying	992.5	19.19	385	17.00	216.2	16.32	391.3	24.72
No social bullying	4,180.5	80.81	1,880	83.00	1,108.8	83.68	1,191.7	75.28
Rape	827.4	15.99	195.4	8.63	199.8	15.08	432.2	27.30
No rape	4,345.6	84.01	2,069.6	91.37	1,125.2	84.92	1,150.8	72.70
Sexual victimization	2,168.7	41.92	777.9	34.34	533.9	40.29	856.9	54.13
No sexual victimization	3,004.3	58.08	1,487.1	65.66	791.1	59.71	726.1	45.87
Emotional abuse	1,716.5	33.18	496.8	21.93	437.3	33.00	782.4	49.43
No emotional abuse	3,456.5	66.82	1,768.2	78.07	887.7	67.00	800.6	50.57
IPV	1,801.4	34.82	340.3	15.02	544.6	41.10	916.5	57.90
No IPV	3,371.6	65.18	1,924.7	84.98	780.4	58.90	666.5	42.10
Independent variables
In-person survey	2,265	43.79	—	—	—	—	—	—
Online survey	1,325	25.61	—	—	—	—	—	—
Mass-email survey	1,583	30.50	—	—	—	—	—	—
Control variables
Rape myth acceptance^a	0.92	0.54	0.87	0.48	0.89	0.56	1.01	0.59
Days on campus^a	3.97	1.80	4.45	1.48	2.93	1.83	4.15	1.83
Gender
Female	3,592.3	69.44	1,452.7	64.14	980.2	73.98	1,159.4	73.24	1,2031	62.41
Male	1,580.7	30.56	812.3	35.86	344.8	26.02	423.6	26.76	7,247	37.59
Lives on or off campus
On campus	1,137.2	21.98	577	25.47	185.8	14.02	374.4	23.65
Off campus	4,035.8	78.02	1,688	74.53	1,139.2	85.98	1,208.6	76.35
Race
White	3,093.4	59.80	1,343.4	59.31	792.9	59.84	957.1	60.46	1,0037	52.06
Students of color	2,079.6	40.20	921.6	40.69	532.1	40.16	625.9	39.54	9,241	47.94
Traditional college age (18–22 years old)
Traditional college age	3,366.8	65.08	1,699.6	75.04	711.3	53.68	955.9	60.39	1,1912	61.79
23+ years old	1,806.2	34.92	565.4	24.96	613.7	46.32	627.1	39.61	7,366	38.21
Member of athletic team, intramural team, or Greek life
Athletic team	239	4.62	158	6.98	60	4.53	21	1.33
Not a member	4,934	95.38	2,107	93.02	1,265	95.47	1,562	98.67
Intramural team	486	9.39	313	13.82	80	6.04	93	5.87
Not a member	4,687	90.61	1,952	86.18	1,245	93.96	1,490	94.13
Member of Greek life	375	7.25	216	9.54	83	6.26	76	4.80
Not a member	4,798	92.75	2,049	90.46	1,242	93.74	1,507	95.20
Binge drinking
Binge drank last 2 weeks	693.6	13.41	360.2	15.90	153.9	11.62	179.5	11.34
Did not binge drink last 2 weeks	4,479.4	86.59	1,904.8	84.10	1,171.1	88.38	1,403.5	88.66

Note. IPV = intimate partner violence. Mean and S.D. are reported instead of count (n) and percentage (%). Total N by item contains decimals for some binary items. This is a function of pooling estimates across 10 imputed data sets.

Indicates continuous variables.

Analyses

To begin, we examine cooperation rates and missing data across the three modes of administration for the dependent variables. Scholars have questioned whether low response rates result in meaningful bias, suggesting that a nonresponse rate alone does not fully demonstrate the extent of bias (Curtin et al., 2000; Groves, 2006). On the contrary, other scholars have suggested that the fixation of evaluating research on response rates is tied to disciplinary norms, including within criminology specifically (Pickett et al., 2018). In an effort to capture the degree (if any) of bias present, we compared the respondents’ demographic characteristics of the three modes of administration to the demographics (i.e., traditional college age [or not], gender, and race) of the SE4YU student body with chi-square analyses. Next, differences across the forms of survey administration on the victimization measures were examined with chi-square analyses and independent samples t-tests. Finally, binary logistic regression models were estimated to examine differences in victimization experiences across the three different methods of survey administration.³ Mode of administration was captured with three dummy variables—in-person survey, online survey, and mass-email survey—where the in-person survey served as the reference category.

Little’s MCAR test was significant (Little’s MCAR χ² = 25.74, p < .001), which suggests that data were not missing completely at random. Accordingly, we imputed the data using multiple imputations and created 10-stacked data sets. Although five imputations have historically been considered the standard (Rubin, 1987), we increased the number of imputations to 10 to ensure the stability of standard error estimates, confidence intervals, and p values (see Bodner, 2008). Research has established the validity of multiple imputation techniques (Van der Heijden et al., 2006). Pooled results are presented.

Similar to the work of Wells and colleagues (2012), it was possible for students to have completed the survey three times—in their in-person courses, through their online courses, and via the mass email. In total, 159 students who completed the paper survey stated they took an online survey. Of the students who completed the mass-email version, 197 reported they took the paper survey. Finally, of the students who completed the survey through their online classes, 153 indicated they took the paper survey. The goal of the current analyses is to examine differences across survey methods and excluding these respondents may discount their responses (Wells et al., 2012). Therefore, all respondents were retained for analyses.

Results

Cooperation Rates and Respondent Characteristics

Cooperation rates varied across the three modes of administration. For the in-person administrations, all students who attended class on the survey date were asked to participate. Within these classes, cooperation rates (i.e., proportion of cases surveyed to eligible units that were contacted, American Association for Public Opinion Research, 2016) were high. Across all classes, 89.65% of the students who attended class the day the survey was administered participated in the survey. These numbers indicate that once the research team was granted access, the majority of present students voluntarily took part in the survey.

For the online random sample of classes, 1,887 eligible students were enrolled across the 78 classes. In total, 1,325 students began the online survey distributed to the online classes. This equates to a 70.22% cooperation rate. It is important to note that students could have been enrolled in more than one course, something we were unable to determine to ensure the confidentiality of the respondents and which may affect the calculation of response rates.

Regarding the mass-email survey, more than 19,000 students were enrolled at the SE4YU in the spring of 2017. In total, 1,583 students began the survey (7.74% of the SE4YU student body). In the end, 990 students completed the final question on the survey.

Next, we discuss missing data points for each dependent variable across the three different modes of administration. Bullying victimization, which was the first series of victimization questions in the survey and may be interpreted as the “least intrusive” of the victimization questions, had the least amount of missing data across modes of administration. More specifically, the paper survey had the lowest amount of missing data on the bullying questions (5.5% of the sample) compared to the mass-email sample (34.9%). In comparison, a much larger percent of the in-person sample was missing data on the rape victimization (19.2%) and sexual victimization (20.4%) measures. Again, roughly a third of the mass-email sample was missing data on the rape victimization (38.0%) and sexual victimization (38.5%) measures. Estimates of missing data for the online class administration fell between these two estimates (15.7% for bullying; 16.9% for rape victimization, and 17.9% for sexual victimization).

In an effort to capture the degree (if any) of bias present, we compare the respondents’ characteristics of the three modes of administration to the characteristics (i.e., gender, race, and traditional college age) of the SE4YU study body in general. Regarding gender, the in-person sample was representative of the student body. However, female-identifying respondents were more likely to participate in the online class survey (73.98%) and the mass email (73.24%) compared to their representation in the SE4YU student body (62.41%; χ² = 71.13, p < .001; χ² = 73.49, p < .001, respectively). People of color were significantly underrepresented across all three samples (in-person = 40.69%; online = 40.16%; mass email = 39.54%) compared to the SE4YU student body (47.94%; χ² = 42.50, p < .001; χ² = 30.13, p < .001; χ² = 41.31, p < .001, respectively). Respondents who completed the in-person survey were significantly more likely to be of traditional college age (75.04%) compared to the SE4YU student body (61.79%; χ² = 153.31, p < .001), while the online-class sample were significantly less likely to be of traditional college age (53.68%) compared to the SE4YU student body (61.79%; χ² = 34.53, p < .001).

Victimization Across Three Sampling Approaches

Chi-square analyses indicate significant differences regarding all victimization items across the three modes of survey administration (bullying χ² = 53.873, p ≤ .001; social bullying χ² = 46.41, p ≤ .001; physical bullying χ² = 36.35, p ≤ .001; rape victimization χ² = 245.51, p ≤ .001; sexual victimization χ² = 147.78, p ≤ .001; emotional abuse χ² = 320.77, p ≤ .001; IPV χ² = 7.86.13, p ≤ .001). Results from independent samples t-tests examining within-group comparisons are presented in Table 2. Significant differences emerged across the different modes of administration (I = in-person; O = online; M = mass email) and the experience of (a) rape victimization (I vs. O t = 3.99; p = .001; I vs. M t = −5.29; p ≤ .001; O vs. M t = −4.33; p = .001), (b) sexual victimization (I vs. O t = 3.45; p = .001; I vs. M t = −8.76; p ≤ .001; O vs. M t = −5.71; p ≤ .001), (c) emotional abuse (I vs. O t = 5.50; p < .001; I vs. M t = −8.61; p ≤ .001; O vs. M t = −4.73; p < .001), and (d) IPV (I vs. O t = 14.69; p = .001; I vs. M t = −23.67; p ≤ .001; O vs. M t = −7.60; p < .001).

Table 2.

Mean Differences in Victimization Experiences Across Mode of Administration.

Type of victimization	Mode of administration	N	%	t value	p value	Type of victimization	Mode of administration	N	%	t value	p value
Bullying	In-person	2,265	18.12	−0.15	.88	Sexual victimization	In-person	2,265	39.36	3.45	.001
	Online	1,325	17.91				Online	1,325	45.57
	In-person	2,265	18.12	−4.20	.001		In-person	2,265	39.36	−8.76	<.001
	Mass email	1,583	26.94				Mass email	1,583	59.08
	Online	1,325	17.91	−4.65	<.001		Online	1,325	45.57	−5.71	<.001
	Mass email	1,583	26.94				Mass email	1,583	59.08
Social bullying	In-person	2,265	17.00	−0.48	.63	Emotional abuse	In-person	2,265	21.93	5.50	<.001
	Online	1,325	16.32				Online	1,325	33.00
	In-person	2,265	17.00	−3.75	.002		In-person	2,265	21.93	−8.61	<.001
	Mass email	1,583	24.72				Mass email	1,583	49.43
	Online	1,325	16.32	−4.27	<.001		Online	1,325	33.00	−4.73	<.001
	Mass email	1,583	24.72				Mass email	1,583	49.43
Physical bullying	In-person	2,265	3.85	1.96	.06	IPV	In-person	2,265	15.02	14.69	<.001
	Online	1,325	5.65				Online	1,325	41.10
	In-person	2,265	3.85	−2.46	.03		In-person	2,265	15.02	−23.67	<.001
	Mass email	1,583	8.26				Mass email	1,583	57.90
	Online	1,325	5.65	−1.69	.11		Online	1,325	41.10	−7.60	<.001
	Mass email	1,583	8.26				Mass email	1,583	57.90
Rape victimization	In-person	2,265	8.63	3.99	.001
	Online	1,325	15.08
	In-person	2,265	8.63	−5.29	<.001
	Mass email	1,583	27.30
	Online	1,325	15.08	−4.33	.001
	Mass email	1,583	27.30

Note. IPV = intimate partner violence.

Significant differences emerged between the (a) in-person sample and the mass-email sample, and (b) the online sample and the mass-email sample and the experience of bullying in general (t = −4.20; p = .001; t = −4.65; p < .001, respectively). Significant differences emerged between the (a) in-person sample and the mass-email sample and (b) the online sample and the mass-email sample and the experience of social bullying (t = −3.75; p = .002; t = −4.27; p < .001, respectively). Significant differences did not emerge between the in-person sample and the online class sample and the experience of both bullying in general (t = −0.15, p = .88) and social bullying (t = −0.48; p =.63). There were not significant differences between the (a) in-person sample and the online sample, and (b) the online sample and the mass-email sample and the experience of physical bullying (t = 1.96; p = .06; t = −1.69; p = .11, respectively). Significant differences emerged between the in-person sample and mass-email sample and the experience of physical bullying (t = −2.46; p = .03).

Binary Logistic Regression Results

Tables 3 and 4 present the findings from the binary logistic regressions. Chi-square values for model fit across all models for all outcome measures produced statistically significant results (p ≤ .001). Overall, these results suggest that aspects of student life such as time spent on campus, involvement in college sports and Greek life, and alcohol consumption all influence the likelihood of victimization over the past year. The more days the respondent spent on campus, the more likely they were to have been bullied—both in general or social bullying—over the last 12 months. Female-identifying respondents were more likely to report they had been bullied, both in general and social bullying; raped; or experienced sexual assault over the last 12 months compared to male-identifying respondents. Students of color, compared to white students, were more likely to report they had experienced IPV over the last 12 months. Respondents who were members of an athletic team were less likely to have been raped over the last 12 months compared to respondents who were not members of an athletic team. Respondents who were members of an intramural team were more likely to report they had been bullied—both in general and social bullying—or experienced IPV over the last 12 months than respondents who were not members of an intramural team. Students affiliated with Greek life were more likely to report they had experienced bullying or sexual victimization over the last 12 months than respondents who were not members of Greek life. Respondents who reported binge drinking in the last 2 weeks were more likely to report experiencing bullying, both in general and social bullying; rape; sexual victimization; and emotional abuse over the last 12 months than respondents who did not report binge drinking in the last 2 weeks. Finally, respondents who reported greater acceptance of rape myths were more likely to report all forms of victimization over the last 12 months.

Table 3.

Differences in Victimization Experiences Across the Three Methods of Survey Administrations: Findings from the Binary Logistic Regression by Victimization Type (N = 5,173).

	Bullying			Social bullying			Physical bullying
Variables	B	SE	Exp(b)	B	SE	Exp(b)	B	SE	Exp(b)
Independent variables
Online survey to online classes^a	0.23	0.11	1.26	0.20	0.12	1.22	0.57	0.21	1.76*
Mass-email survey^a	0.63	0.12	1.87***	0.60	0.12	1.82***	0.82	0.25	2.28***
Control measures
Rape myth acceptance	0.45	0.09	1.57***	0.41	0.09	1.50***	0.82	0.23	2.26***
Days on campus	0.11	0.03	1.12***	0.12	0.03	1.13***	0.05	0.05	1.05
Female^b	0.33	0.10	1.39***	0.36	0.10	1.44***	0.12	0.21	1.13
Lived on campus^c	−0.09	0.12	0.92	−0.11	0.12	0.90	0.27	0.23	1.31
Students of color^d	0.02	0.08	1.02	−0.01	0.08	0.99	0.12	0.17	1.13
Traditional college age^e	0.20	0.12	1.22	0.19	0.13	1.21	0.18	0.22	1.19
Athletic team^f	0.28	0.17	1.33	0.36	0.17	1.43	0.23	0.30	1.25
Intramural team^g	0.37	0.13	1.45***	0.35	0.13	1.42**	0.23	0.22	1.26
Member of Greek life^h	0.92	0.12	2.51***	0.94	0.13	2.55***	0.78	0.21	2.18***
Binge drank last 2 weeksⁱ	0.52	0.11	1.68***	0.52	0.11	1.68***	0.45	0.21	1.57
Nagelkerke R²			0.09			0.08			0.09

Reference group is the in-person paper survey. ^bReference group is male (0 = male, 1 = female). ^cReference group is lived off campus. ^dReference group is white students. ^eReference group is students not who are not traditional college age (23 years or older). ^fReference group is student is not member of athletic team. ^gReference group is student is not a member of intramural team. ^hReference group is student is not a member of Greek life. ⁱReference group is student did not binge drink in the last 2 weeks.

p ≤ .05. **p ≤ .01. ***p ≤ .001.

Table 4.

Differences in Victimization Experiences Across the Three Methods of Survey Administrations: Findings from the Binary Logistic Regression by Victimization Type (N = 5,173).

	Rape victimization			Sexual victimization			Emotional abuse			IPV
Variables	B	SE	Exp(b)	B	SE	Exp(b)	B	SE	Exp(b)	B	SE	Exp(b)
Independent variables
Online survey to online classes^a	0.53	0.18	1.71**	0.30	0.09	1.35***	0.66	0.10	1.94***	1.54	0.11	4.68***
Mass-email survey^a	1.13	0.21	3.10***	0.75	0.10	2.11***	1.26	0.12	3.52***	2.05	0.09	7.73***
Control measures
Rape myth acceptance	1.83	0.29	6.24***	0.64	0.13	1.89***	0.57	0.13	1.76***	1.10	0.09	3.01***
Days on campus	0.01	0.08	1.01	0.06	0.03	1.07	0.05	0.03	1.05	0.09	0.04	1.09
Female^b	0.70	0.24	2.01**	1.14	0.11	3.13***	0.15	0.12	1.16	−0.02	0.14	0.98
Lived on campus^c	0.19	0.22	1.21	−0.12	0.10	0.89	−0.08	0.11	0.93	0.05	0.13	1.05
Students of color^d	0.05	0.19	1.05	−0.04	0.09	0.97	0.05	0.10	1.05	0.33	0.12	1.39**
Traditional college age^e	0.04	0.26	1.04	0.21	0.12	1.23	0.12	0.10	1.13	0.04	0.12	1.04
Athletic team^f	−0.91	0.31	0.40***	−0.27	0.15	0.76	−0.05	0.16	0.96	−0.41	0.18	0.67
Intramural team^g	−0.33	0.21	0.72	−0.02	0.12	0.98	0.17	0.13	1.18	−0.45	0.16	0.64**
Member of greek life^h	0.14	0.20	1.15	0.30	0.12	1.36**	0.24	0.14	1.27	−0.34	0.15	0.71
Binge drank last 2 weeksⁱ	0.79	0.14	2.20***	0.66	0.10	1.94***	0.40	0.13	1.49**	0.19	0.13	1.21
Nagelkerke R²			0.29			0.14			0.12			0.30

Note. IPV = intimate partner violence.

p ≤ .01. ***p ≤ .001.

Respondents who took the survey in an online course were more likely than respondents of the paper survey to report rape victimization, sexual victimization, emotional abuse, and IPV over the last 12 months. In addition, respondents who completed the survey in an online class were more likely than respondents of the paper survey to report physical bullying but not social bullying or bullying in general. Across all five types of victimization, respondents who completed the survey via mass email were more likely than respondents who completed the paper version to report that each type of victimization occurred during the last 12 months.

Discussion

In this study, three different modes of self-administrated surveys were assessed. Cooperation rates remained relatively high and item nonresponse remained relatively low among students who completed the paper version of the survey in randomly selected classes. Cooperation rates may have been higher among the in-person sample because they could be assured confidentiality by one of the research team members (Tourangeau & Yan, 2007). In comparison, cooperation rates were much lower among the mass-email sample and item nonresponse was much higher, but a greater percent of respondents reported victimization experiences. Binary logistic regression analyses revealed significant differences emerged across all three modes of survey administration and the victimization measures except for one—respondents who completed the survey in an online course and their experience of bullying and, in particular, social bullying—during the last 12 months compared to respondents who completed the paper survey. It is possible that students who have been victimized are self-selecting to complete the mass-email version of the survey because the topic is of particular interest to them (Dillman et al., 2009; Lavrakas, 2008). The SE4YU where the study was conducted has multiple student organizations actively involved in violence prevention and a large body of students who are criminal justice majors. It might also be that these students perceived the topic to be important and were more likely to participate (Dillman et al., 2009).

Also relevant to cooperation rates, recall that the in-person survey had higher rates of missing data on the rape and sexual victimization questions. This finding may be a result of survey fatigue (Porter et al., 2004). Rape and sexual victimization items appeared later in the survey, and it is possible that respondents began to lose interest during the completion of the survey. Conversely, the mass-email and online samples did not show the same degree of survey break-off. There are a couple of explanations for this finding. It is possible that respondents answered the beginning questions and then skipped quickly through the rest of the survey, neglecting to provide responses on later items. Another explanation may be student familiarity with and preference for technological administrations. Indeed, technology and the internet are now typical features of college life and studenthood.

Nevertheless, it is important to consider the overall objectives of the campus climate survey. Campus climate surveys were originally designed to identify the prevalence of unreported rape on college campuses (e.g., Cantor et al., 2015). However, scholars have consistently noted that sexual victimization includes a continuum of behaviors and that these other forms of sexual victimization are typically more common than the experience of rape (Fisher et al., 2009). The mass-email sample and online sample had a much higher rate of missing data despite an increased likelihood of reported victimization when compared to the in-person sample. The scope of campus climates surveys has recently become more expansive, with Title IX extending protections to the experience of IPV, sexual harassment, and stalking (Hughes-Miller, 2017). Questions remain, then, if in-person self-administration can capture the nature and extent of victimization on college campuses despite having the highest overall cooperation rate. IHEs needs to clearly identify the purpose of the climate survey before designing and selecting the method of administration, challenging the “more is better assumption” (Tourangeau & Yan, 2007, p. 663). After all, the findings of this study suggest that paper campus climate survey administrations may undercount the occurrence of various types of victimization.

Some IHEs have elected to conduct a census campus climate survey (McMahon et al., 2018). However, this is a time-consuming endeavor that requires significant coordination among various university personnel. An in-person random sample or survey sent to all students may be a more accessible route for researchers and university offices, like Title IX, to capture these data. Each method can provide estimates that can be used to develop policy and procedures to address victimization on campus. However, criticisms surrounding the increased costs and person-power needed to administer a paper survey can deter scholars from taking on such a task. One way for a university to preserve resources is to administer an electronic survey to a random sample of in-person classes. That is, the research team visits each class to recruit participants and provides a survey link for completion on a smart phone or laptop. Researchers can assuage respondent confidentially concerns, hopefully increasing overall cooperation rates (Tourangeau & Yan, 2007). This study found higher cooperation rates for the in-person survey and less nonresponse in face-to-face research settings; however, students who took the survey in this mode reported fewer victimization experiences compared to the online formats. Therefore, the administration approach mentioned above—where researchers enter classrooms and provide a link to the survey—may help remedy concerns about undercounting victimization experiences while enhancing cooperation rates and item response (Kreuter et al., 2008). If students are concerned about the presence of others when completing questions, they can then elect to complete the survey in privacy at home. This type of survey administration would combine the benefits of a random sample with the ease of electronic administration while improving overall response rates, item nonresponse, and response accuracy (Tourangeau & Yan, 2007). Below, we identify the advantages and limitations of each type of administration to highlight considerations that must be addressed before a researcher undertakes that particular mode of administration.

Advantages and Limitations of in Person Paper Survey

The in-person survey to randomly selected classes has several methodological advantages related to the logistics and protocol of administering a survey this way. For example, the research team was able to answer respondents’ questions when they arose during an in-person administration as well as assure respondent anonymity. Also, in these face-to-face classes, cooperation rates were much higher when the instructor offered extra credit.

There were also disadvantages to administering the survey in-person that merit mention. Over 2,000 copies of the survey had to be printed. The total cost for this version of the survey was $500 more than the incentives associated with the mass-email survey. Finally, the survey was randomized at the class level—not the student level—and therefore may not be reflective of the student body. Randomization at the class level was used because it provided a sampling frame. As suggested in Table 1, the sample ultimately drawn may not have been representative.

Advantages and Limitations of Random Sample of Online Classes

Despite being an electronic survey that automatically generated the data, the administration of the survey to online classes had more challenges than the other two forms of administration. First, the survey needed a way to link the student to a particular course to receive extra credit. Given the sensitive nature of the questions included in the survey, the research team wanted to design the survey in such a way that students did not provide their full name in conjunction with their course section. Instead, students provided the course number and their SE4YU email. At the SE4YU, email addresses are a mixture of three letters and three numbers that make it difficult to identify a student based on sight alone. At the end of the semester, the research team then had to compile a list—by class section—of student emails who completed the extra credit. In addition, students could have been enrolled in multiple courses that offered extra credit. This resulted in follow-up emails from instructors to check if the student had completed the survey for another section. Another unexpected limitation was that many of the instructors had multiple sections of the online course that were combined once the semester began in order for the class to make. In these cases, we allowed the instructor to email all students in all sections. A disadvantage of this method is also sampling error.

Advantages and Limitations of Mass-Email Survey

The distribution of the survey via a mass email to the entire student body had several advantages. As the survey was electronic, the data were automatically generated and did not require entry like the paper version. In addition, the survey program that was used for survey construction (i.e., Qualtrics) calculated the time a respondent spent on the survey. This allows the researcher to determine if the respondent was rushing through the survey to be eligible for the incentives. This approach was also more cost-effective.

Given the high rate of missing data for the mass-email survey, break-off likely occurred whereby respondents began the survey but did not complete it (Tourangeau et al., 2013). Break-offs are less likely to occur in face-to-face administrations, as evidenced by the lower rate of missing data. Another methodological limitation of the mass-email distribution of the survey is that findings cannot be generalized to the entire SE4YU student body and there is sampling error. Female-identifying and white respondents were more likely to complete the survey compared to their representation in the entire SE4YU student body. Students of traditional college age were also more likely to take the survey when compared to students of nontraditional college age. There may be a “digital divide” between younger and older individuals (Warschauer, 2004), which may influence whether or not they completed the survey. Also, there is no way to tell how many students opened the email and elected not to participate, deleted the email because it was from someone they did not know, or simply never checked their SE4YU email account.

Other Considerations

Requiring students to open the survey (and opting in or out of each question) as a course registration requirement may enhance representativeness. A census strategy provides every student the opportunity to have a voice, as the entire population is targeted for sampling. However, simply making the survey available to all students will not necessarily result in representativeness. In this study, people of color were significantly underrepresented across all three administration techniques—including the mass-email—compared to the student body. This demonstrates that census approaches alone may not encourage participation from all student groups. Indeed, existing instruments—including the present—may not adequately consider important elements of culture and context, potentially affecting the data collection process (Association of American Universities, 2017; L. Wood et al., 2017). Therefore, improving instruments in this way and requiring that students open the survey before registering for classes may improve response rates among historically underrepresented groups. As stated above, census approaches require commitment from various university bodies; it is quite possible that some IHEs do not have the infrastructure to implement a census-wide campus climate survey.

Items related to bullying, emotional abuse, IPV, and sexual victimization were not exhaustive and also included behaviors that could occur electronically. Respondents who have been victimized via technological or internet-based avenues may be more likely to respond to email solicitations and participate in web-based surveys. As individuals become more reliant on technology, and IPV and stalking behaviors continue to transition online (Woodlock, 2016), researchers will need to critically consider the inclusion of these measures.

Conclusion

The administration of climate surveys has grown exponentially over the past decade. Yet, a standardized instrument has yet to be developed. Findings from the current analyses indicate that if IHEs are interested in prevalence across multiple forms of victimization, nonrandom-sample electronic surveys may result in higher prevalence estimates than a paper survey administered to a random sample of classes. Stated differently, prevalence estimates from paper surveys may underestimate the prevalence of victimization among students enrolled at that university. It is therefore vital that IHEs clearly identify the scope and purpose of the climate survey before survey design and administration.

Footnotes

Authors’ Note

Eryn Nicole O’Neal is no longer affiliated with Sam Houston State University.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Law Enforcement Management Institute of Texas and the Department of Criminal Justice and Criminology at Sam Houston State University.

ORCID iDs

Brittany E. Hayes

Eryn Nicole O’Neal

Notes

Author Biographies

Brittany E. Hayes, PhD, (John Jay College of Criminal Justice, City University of New York) is an assistant professor in the School of Criminal Justice at the University of Cincinnati. Her research focuses on the correlates and consequences of victimization and has been published in the American Sociological Review and the Journal of Research in Crime and Delinquency. She is the 2019 recipient of the Division on Women and Crime’s New Scholar Award from the American Society of Criminology.

Eryn Nicole O’Neal, PhD, is an activist-scholar, educator, and critical criminologist. She’s an executive counselor for the ASC Division on Queer Criminology and has received seven national awards for her research/teaching. She applies a care/love ethic to her teaching, mentoring, advising, student advocacy, and social justice activism. She believes that pedagogies of love are crucial in the co-liberation of marginalized communities. Her research focuses on critical/feminist criminological perspectives. She combines community/campus activism, teaching, and scholarship to speak truth to power and dismantle systems of oppression that impact the criminal legal system and academe. Black Lives Matter.

References

Abbey

McAuslan

Zawacki

Clinton

A. M.

Buck

P. O.

(2001). Attitudinal, experiential, and situational prediction of sexual assault perpetration. Journal of Interpersonal Violence, 16(8), 784–807. https://doi.org/10.1177/088626001016008004

American Association for Public Opinion Research. (2016). Standard definitions: Final dispositions of case codes and outcome rates for surveys. https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf

Association of American Universities. (2017). AAU Campus Survey on Sexual Assault and Misconduct: Key findings. https://www.aau.edu/sites/default/files/AAU-Files/Key-Issues/Campus-Safety/AAU-Campus-Climate-Survey-FINAL-10-20-17.pdf

Bodner

T. E.

(2008). What improves with increased missing data imputations? Structural Equation Modeling: A Multidisciplinary Journal, 15, 651–675. https://doi.org/10.1080/10705510802339072

Cantalupo

N. C.

(2011). Burying our heads in the sand: Lack of knowledge, knowledge avoidance, and the persistent problem of campus peer sexual violence. Loyola University Chicago Law Journal, 43(6), 205–266.

Cantalupo

N. C.

(2014). Institution-Specific Victimization Surveys: Addressing legal and practical disincentives to gender-based violence reporting on college campuses. Trauma, Violence, & Abuse, 15(3), 227–241. https://doi.org/10.1177/1524838014521323

Cantor

Fisher

Chibnall

S. H.

Townsend

Lee

Bruce

Thomas

(2015). Report on the AAU Campus Climate Survey on Sexual Assault and Sexual Misconduct. https://www.aau.edu/sites/default/files/%40%20Files/Climate%20Survey/AAU_Campus_Climate_Survey_12_14_15.pdf

Couper

M. P.

(2000). Web Surveys: A review of issues and approaches. Public Opinion Quarterly, 64(4), 464–494. https://doi.org/10.1086/318641

Curtin

Presser

Singer

(2000). The effects of response rate changes on the Index of Consumer Sentiment. Public Opinion Quarterly, 64(4), 413–428. https://doi.org/10.1086/318638

10.

Davidov

Depner

(2011). Testing for measurement equivalence of human values across online and Paper-and-Pencil Surveys. Quality & Quantity, 45(2), 375–390. https://doi.org/10.1007/s11135-009-9297-9

11.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2009). Internet, mail, and mixed-mode surveys: The tailored method. John Wiley.

12.

Evans

J. R.

Mathur

(2005). The value of online surveys. Internet Research, 15(2), 195–219. https://doi.org/10.1108/10662240510590360

13.

Fisher

B. S.

Cullen

F. T.

Turner

M. G.

(2000). The sexual victimization of college women (Report No. NCJ 182369). National Institute of Justice, Bureau of Justice Statistics.

14.

Fisher

B. S.

Daigle

L. E.

Cullen

F. T.

(2009). Unsafe in the ivory tower: The sexual victimization of college women. Sage.

15.

Fisher

B. S.

Daigle

L. E.

Cullen

F. T.

Turner

M. G.

(2003). Reporting sexual victimization to the police and others: Results from a national-level study of college women. Criminal Justice and Behavior, 30(1), 6–38. https://doi.org/10.1177/0093854802239161

16.

Fisher

B. S.

Hartman

J. L.

Cullen

F. T.

Turner

M. G.

(2002). Making campuses safer for students: The Clery Act as a symbolic legal reform. Stetson Law Review, 32, 61–89.

17.

Franklin

C. A.

(2016). Sorority affiliation and sexual assault victimization: Assessing vulnerability using path analysis. Violence Against Women, 22(8), 895–922. https://doi.org/10.1177/1077801215614971

18.

Franklin

C. A.

Franklin

T. W.

Nobles

M. R.

Kercher

G. A.

(2012). Assessing the effect of routine activity theory and self-control on property, personal, and sexual assault victimization. Criminal Justice and Behavior, 39(10), 1296–1315. https://doi.org/10.1177/0093854812453673

19.

Gardella

J. H.

Nichols-Hadeed

C. A.

Mastrocinque

J. M.

Stone

J. T.

Coates

C. A.

Sly

C. J.

Cerulli

(2015). Beyond Clery Act statistics: A closer look at college victimization based on self-report data. Journal of Interpersonal Violence, 30(4), 640–658. https://doi.org/10.1177/0886260514535257

20.

Groves

R. M.

(2004). Survey errors and survey costs. Wiley.

21.

Groves

R. M.

(2006). Nonresponse rates and nonresponse bias in Household Surveys. Public Opinion Quarterly, 70(5), 646–675. https://doi.org/10.1093/poq/nfl033

22.

Henry

W. J.

Fowler

S. R.

West

N. M.

(2011). Campus climate: An assessment of students’ perceptions in a college of education. Urban Education, 46(4), 689–718. https://doi.org/10.1177/0042085911399791

23.

Hughes-Miller

(2017). From sexual harassment to sexual violence: The evolution of Title IX’s response to sexual victimization. In Kaukinen

Miller

M. H.

Powers

(Eds.), Addressing violence against women on college campuses (pp. 97–112). Temple University Press.

24.

Jordan

(2001). True “lies” and false “truths”: Women, rape, and the police [Doctoral dissertation]. Victoria University of Wellington.

25.

Kahn

A. S.

Jackson

Kully

Badger

Halvorsen

(2003). Calling it rape: Differences in the experience of women who do or do not label their sexual victimization as rape. Psychology of Women Quarterly, 27(3), 233–242. https://doi.org/10.1111/1471-6402.00103

26.

Kirkpatrick

Kanin

(1957). Male sexual aggression on a university campus. American Sociological Review, 22(1), 52–58. https://doi.org/10.2307/2088765

27.

Koss

M. P.

(1989). Hidden rape: Sexual aggression and victimization in a national sample of students in higher education. In Burgess

A. W.

(Ed.), Rape and sexual assault II (pp. 3–25). Garland.

28.

Krebs

C. P.

Lindquist

C. H.

Berzofksy

Shook-Sa

Peterson

(2016). Campus Climate Survey Validation Study: Final technical report (Report No. NCJ 249545). Bureau of Justice Statistics, U.S. Department of Justice.

29.

Krebs

C. P.

Lindquist

C. H.

Warner

T. D.

Fisher

B. S.

Martin

S. L.

(2007). The Campus Sexual Assault (CSA) Study: Final report (Report No. 221153). National Institute of Justice, US Department of Justice.

30.

Kreuter

Presser

Tourangeau

(2008). Social desirability bias in CATI, IVR, and Web Surveys: The effects of mode and question sensitivity. Public Opinion Quarterly, 72(5), 847–865. https://doi.org/10.1093/poq/nfn063

31.

Lavrakas

P. J.

(2008). Encyclopedia of survey research methods. Sage.

32.

Lewis

I. M.

Watson

B. C.

White

K. A.

(2009). Internet versus Paper-and-Pencil Survey methods in psychological experiments: Equivalence testing of participant responses to health-related messages. Australian Journal of Psychology, 61(2), 107–116. https://doi.org/10.1080/00049530802105865

33.

Lind

L. H.

Schober

M. F.

Conrad

F. G.

Reichert

(2013). Why do survey respondents disclose more when computers ask the questions? Public Opinion Quarterly, 77(4), 888–935. https://doi.org/10.1093/poq/nft038

34.

Lindquist

Krebs

C. P.

(2017). Campus Climate Surveys. In Kaukinen

Hughes-Miller

Powers

(Eds.), Addressing violence against women on college campuses (pp. 217–229). Temple University Press.

35.

Luther

(2016). Unsportsmanlike conduct: College football and the politics of rape. Akashic Books.

36.

Maxfield

M. G.

Babbie

E. R.

(2016). Basic of research methods for criminal justice and criminology. Cengage Learning.

37.

McCauley

J. L.

Calhoun

K. S.

Gidycz

C. A.

(2010). Binge drinking and rape: A prospective examination of college women with a history of previous sexual victimization. Journal of Interpersonal Violence, 25(9), 1655–1668. https://doi.org/10.1177/0886260509354580

38.

McMahon

Stepleton

Cusano

O’Connor

Gandhi

McGinty

(2018). Beyond Sexual Assault Surveys: A model for comprehensive campus climate assessments. Journal of Student Affairs Research and Practice, 55(1), 78–90. https://doi.org/10.1080/19496591.2017.1358629

39.

Office of the Assistant Secretary. (2011). Dear colleague letter. Office for Civil Rights, United States Department of Education.

40.

Pickett

Cullen

Bushway

S. D.

Chiricos

Alpert

(2018). The response rate test: Nonresponse bias and the future of survey research in criminology and criminal justice (SSRN Scholarly Paper ID 3103018). Social Science Research Network. https://papers.ssrn.com/abstract=3103018

41.

Porter

S. R.

Whitcomb

M. E.

Weitzer

W. H.

(2004). Multiple surveys of students and survey fatigue. New Directions for Institutional Research, 2004(121), 63–73. https://doi.org/10.1002/ir.101

42.

Rubin

D. B.

(1987). Multiple imputation for nonresponse in surveys. John Wiley.

43.

Schonlau

Fricker

R. D.

Elliott

M. N.

(2001). Conducting Research Surveys via e-mail and the web. Rand Corporation.

44.

Sue

V. M.

Ritter

L. A.

(2007). Conducting online surveys. Sage.

45.

Tourangeau

Conrad

F. G.

Couper

M. P.

(2013). The science of web surveys. Oxford University Press.

46.

Tourangeau

Yan

(2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859–883. https://doi.org/10.1037/0033-2909.133.5.859

47.

University of Kentucky. (2015). C.A.T.S—Campus attitudes toward safety, 2016. Center for Research on Violence against Women, University of Kentucky. https://www.uky.edu/CATSseesafety/

48.

Van der Heijden

G. J.

Donders

A. R. T.

Stijnen

Moons

K. G.

(2006). Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: A clinical example. Journal of Clinical Epidemiology, 59(10), 1102–1109. https://doi.org/10.1016/j.jclinepi.2006.01.015

49.

Warschauer

(2004). Technology and social inclusion: Rethinking the digital divide. MIT Press.

50.

Wells

Cavanaugh

M. R.

Bouffard

J. A.

Nobles

M. R.

(2012). Non-response bias with a Web-Based Survey of college students: Differences from a classroom Survey About Carrying Concealed Handguns. Journal of Quantitative Criminology, 28(3), 455–476. https://doi.org/10.1007/s10940-011-9148-4

51.

Wood

Nosko

Desmarais

Ross

Irvine

(2006). Online and traditional paper-and-pencil survey administration: Examining experimenter presence, sensitive material, and long surveys. Canadian Journal of Human Sexuality, 15(3–4), 147–155.

52.

Wood

Sulley

Kammer-Kerwick

Follingstad

Busch-Armendariz

(2017). Climate Surveys: An inventory of understanding sexual assault and other crimes of interpersonal violence at institutions of higher education. Violence Against Women, 23(10), 1249–1267. https://doi.org/10.1177/1077801216657897

53.

Woodlock

(2016). The abuse of technology in domestic violence and stalking. Violence Against Women, 23(5), 584–602. https://doi.org/10.1177/1077801216646277

Differences in Nonresponse Bias and Victimization Reports Across Self-Administered Web-Based and Paper-and-Pencil Versions of a Campus Climate Survey

Abstract

Keywords

Measurement of Violence on College Campuses

Web-Based Versus Paper-and-Pencil Survey Administration

Nonresponse in Self-Administrated Surveys

Current Study

Materials and Methods

Campus Climate Survey at Southeastern 4-Year University

In-person paper survey to randomly selected courses

Electronic survey to online-only classes

Electronic survey via mass email to SE4YU student body

Measures

Dependent Variables

Bullying

Sexual Victimization

Rape victimization

Sexual victimization

Emotional Abuse

IPV

Independent Variables

Mode of administration

Control Measures 1

Rape myth acceptance

Binge drinking

Other control measures

Analyses

Results

Cooperation Rates and Respondent Characteristics

Victimization Across Three Sampling Approaches

Binary Logistic Regression Results

Discussion

Advantages and Limitations of in Person Paper Survey

Advantages and Limitations of Random Sample of Online Classes

Advantages and Limitations of Mass-Email Survey

Other Considerations

Conclusion

Footnotes

Authors’ Note

Declaration of Conflicting Interests

Funding

ORCID iDs

Notes

Author Biographies

References

Control Measures¹