Misunderstanding Law: Undergraduates’ Analysis of Campus Title IX Policies

Abstract

Colleges and universities are legally required to attempt to prevent and redress sexual violations on campus. Neo-institutional theory suggests that the implementation of law by compliance professionals rarely achieves law’s goals. It is critical in claims-based systems that those who are potential claimants understand the law. This article demonstrates that (a) intended subjects of the law (colleges and universities) interpret and frame the law in very similar ways; (b) resultant policies are complex and difficult to navigate; and (c) university undergraduates in an experimental setting are not able to comprehend the Title IX policies designed to protect them. These findings suggest that current implementations of Title IX policies leave them structurally ineffective to combat sexual assaults on campus.

Keywords

colleges educational policy law/legal policy violence statistics quasi-experimental analysis measurements

Sexual assault on college campuses is an old problem that has gained increased media and academic attention due to new federal guidance on campus requirements (Gersen et al., 2019; Green, 2018); major demonstrations (Dastagir, 2019; Hartocollis, 2019); the HBO documentary, The Hunting Ground (Lady Gaga et al., 2015); high-profile cases like the Brock Turner rape case on the Stanford University campus (Dastagir, 2019; Hauser, 2018; Hayes & Bacon, 2018); and the formation of new social movement campaigns such as “End Rape on Campus” and “Know Your IX” (EROC, 2020; Know Your IX, 2020). Colleges and universities that receive federal funding are required to ensure equal educational opportunity, which includes preventing and addressing sexual assault, harassment, and discrimination (Federal Register, 2018). Currently, some states are considering or have passed laws that detail how schools in their state must define consent (Know Your IX, 2020).

The relevant pieces of federal legislation about campus sexual assault are Title IX of the Civil Rights Act of 1972 and the Clery Act of 1991 (including the amendments to it in the Campus SAVE Act).^1,2 Title IX is a comprehensive federal law that prohibits discrimination on the basis of sex in any federally funded education program or activity on college and university campuses that receive federal funds. In its early days, Title IX was most known for equalizing opportunity in women’s sports programs on campuses, but it is increasingly the basis for addressing the role of colleges and universities in attempting to prevent and mitigate the harms of sexual assault. Like other civil rights laws designed to disrupt hierarchies of unearned privilege, the federal law is administered by a regulatory agency for rule-making (the Department of Education [ED]) and then interpreted and implemented by elites in organizations (lawyers, deans, and administrators). The policies derived from federal law and administrative agency guidance in the ED are then communicated to the people who are the intended beneficiaries of the law (students) via written policies, educational online trainings, and through informal processes.

Sociolegal scholars, especially those who study law in everyday life or legal consciousness, are rightly skeptical about whether the text of these policies (the “law on the books”) matters for ordinary people in their decision-making process. And yet, we know that analyzing formal policies can be instructive for understanding organizational response to and diffusion of law, organizational signaling inside and outside an institutional field, and as artifacts of legal processes. Accordingly, there is no shortage of analysis of policies in general or of Title IX policies in particular.

Despite the analysis of Title IX policies, as of yet, there has been little research about how undergraduate students understand these policies. For funding men’s and women’s athletics, undergraduates may not need to understand the full picture of finances of the college, but when it comes to sexual assault on college campuses, the policies include much information that undergraduates need to understand both for adequately (at least under the rules of their school) obtaining sexual consent and for understanding what is to happen should an undergraduate initiate a Title IX complaint. Importantly, Title IX complaints are often student-initiated, which requires substantial knowledge on the part of a student who wishes to initiate a complaint. This process can be made more difficult because crucial decisions like deciding to talk to a mandated reporter or saving physical evidence cannot be undone if a student did not understand the process when making such decisions.

The final reason we seek to understand whether and how undergraduates understand campus sexual assault policy comes from the larger project in which this experiment is embedded. We do not systematically present these data in this article, but one author has completed 150 qualitative, in-depth interviews of undergraduate students on seven campuses in three states nationwide. That research demonstrates that the vast majority of undergraduates know remarkably little about their institutions’ sexual harassment and assault policies. Nevertheless, students are confident that “if anything happened” they would be able to both locate and understand the policies with a quick search of the school’s website.³

For these reasons, we seek to measure whether and how students understand these policies at all. The article describes the process of testing whether undergraduates are able to (a) locate important provisions in written Title IX policies and (b) comprehend some of the more complex concepts associated with the complaint process should an undergraduate believe they were sexually harassed or assaulted.

We find that (a) undergraduate students reliably are able to locate important concepts in these policies, but (b) the more complex comprehension of various terms and concepts is almost universally misunderstood. Finally, in an attempt to diagnose the problem of policy legibility, we performed a Lexile analysis of a sample of the policies. This analysis reveals that the average Lexile score for a Title IX policy (that a 17- or 18-year-old undergraduate might confront if they had been sexually assaulted and looked to campus resources for help) is 30.08—around Harvard Law Review level of reading ability and vocabulary.

Our analysis demonstrates that this level of difficulty is insurmountable for college undergraduates when asked to locate and comprehend basic elements of a typical Title IX policy in an online experimental study. This means that even in the most ideal conditions, conditions that certainly do not exist after a student is sexually assaulted, students cannot activate the very Title IX policy that is meant to protect them.

This article proceeds in six sections. Section “How Campus Sexual Assault Contributes to Educational Inequality” elaborates on how campus sexual assault contributes to educational inequality. It is easy to take for granted that campus sexual assault is harmful to its victims in many ways, but we include some of the research demonstrating the magnitude of the impact of campus sexual assault on college performance. It is also instructive as we consider the capacity of victims of sexual assault and their thought processes in the wake of it. Section “From Law to Policy to Legal Consciousness” provides the literature review to make the case for studying how undergraduates themselves understand these written policies. It also elaborates on our research questions. Section “Method” conveys our experimental design methodology. Section “Results” analyzes the results, and we conclude briefly in section “Conclusions and Implications.”

How Campus Sexual Assault Contributes to Educational Inequality

Sexual assault at colleges and universities has a profound impact on educational opportunity for undergraduate students. Quantifying the true impact of sexual assault on the undergraduate student population is very difficult due to substantial under-reporting of intimate partner violence, broadly speaking by persons of all genders and sexualities, although scholars postulate differing patterns or reasons for nonreporting across different populations (Turchik et al., 2016). While the true scope of the problem of sexual assault on campus remains unknown, recent work has found that 6% to 8% of males and around 20% of females have experienced sexual abuse or sexual misconduct during their college career (Anderson et al., 2015; Muehlenhard et al., 2015). Relatedly, there has been relatively recent recognition by higher education institutions and legislators that educational equality requires attention to student safety and in ensuring that campuses are as safe as they can be from sexual assault. And, where sexual assault does occur, that colleges and universities have adequate processes in place to deal with it.

Being sexually assaulted during college takes enormous psychological, physical, social, and academic tolls on a victim. While the trauma of sexual abuse is taken for granted in these discussions, scholars often fail to consider exactly how being the target of sexual assault or rape in college perpetuates and magnifies gender and other forms of inequality.

A history of sexual assault is associated with an increased prevalence of a lifetime suicide attempt after controlling for sex, age, education, posttraumatic stress disorder (PTSD) symptoms, and psychiatric disorder (Davidson et al., 1996; Davidson & Foa, 1993). Some 33% of rape victims report having ever thought seriously about committing suicide versus 8% of nonvictims, meaning that rape victims are 4.1 times more likely to have contemplated suicide (National Victim Center, 1992). Similarly, elevated likelihoods of PTSD and depressive episodes are reported by rape victims:

Almost one third (31%) of all rape victims developed PTSD sometime during their lifetimes, and more than one in ten rape victims (11%) still has PTSD at the present time. 30% of rape victims experience at least one major depressive episode in their lifetime and 21% experience it at the time of the rape. (National Victim Center, 1992)

Worst of all, rape victims are 13 times more likely than noncrime victims to have actually made a suicide attempt (13% vs. 1%; National Victim Center, 1992).

Given the psychological, physical, and social toll that sexual violence has on its victims, it is perhaps not surprising that academic performance suffers dramatically. To be sure, students who experience any kind of violence while at school have higher than average negative academic outcomes, but targets of sexualized violence face the most negative outcomes. When students who have been the victim of violence remain in college, those who suffered nonsexualized violence incurred, on average, a Grade Point Average (GPA) decline of 0.35, whereas targets of sexual violence see something closer to a full letter grade drop (Mengo & Black, 2016). This is likely related to the fact that victims of sexual violence among college students have decreased class attendance and decreased quantity and quality of work (Smith et al., 2003).

This decline in performance inevitably leads to leaving college altogether. Students who are victims of any kind of violence while at college have an increased risk of dropping out, but victims of sexual violence are dramatically more likely than victims of other types of violence to do so. Nationally, the average the rate of dropout for victims of violence is 24.3%, but victims of sexual violence have a dropout rate of 34.1%, whereas victims of verbal or physical violence have a 12.1% dropout rate (Mengo & Black, 2016). This research is compelling evidence of a link between sexual violence and underperformance in college not just vis-à-vis the general student population, but even as compared with victims of other kinds of violence. Sexual assault is a particularly insidious way that gendered academic and social inequality is maintained and reproduced.

In partial response to this problem, Title IX requires campuses that receive federal funding to attempt to prevent sexual assault and to mitigate these damaging consequences when it occurs. In the section that follows, we place our research questions in their broader theoretical location.

From Law to Policy to Legal Consciousness

A civil rights law is what gives individuals who think their rights have been violated a cause of action to sue. This is a primary tool to address the problem of unequal educational opportunity for women in colleges and universities. This litigious policy (Barnes & Burke, 2014; Epp, 2009; Kagan, 2003) places the burden for enforcing the law on an individual (Albiston, 2010; Berrey et al., 2017; Edelman, 2016; Galanter, 1974) who must complain within the organization’s structure (Bumiller, 1992; Felstiner et al., 1980). As such, it is crucial to understand the processes by which the law is transformed into organizational practices, conveyed to people that the law is intended to benefit, and how those individuals understand the policies enacted by the organizations in which they may want or need to assert their rights. Next, we describe the “law on the books” designed to incentivize colleges and universities to provide help for students who have experienced sexual assault or rape on campus. We go on to explain the hypothesis that the study of managerialized law would lead us to grim predictions about the implementations of Title IX policies in colleges and universities. We conclude this section with our research questions.

Law on the Books

As mentioned in the introduction, at the Federal level, two primary sources of law frame questions about campus sexual assault: Title IX of the Civil Rights Act of 1972 and the Clery Act of 1991 (including the amendments to it in the Campus SAVE Act).^4,5 Title IX is a comprehensive federal law that prohibits discrimination on the basis of sex in any federally funded education program or activity. The principal objective is to avoid the use of federal funds to support schools that engage in sex discrimination and to provide individuals affected by discriminatory policies a mechanism to challenge the policies and practices of these institutions. Perhaps most widely known for the provisions and litigation around equality in campus athletics, Title IX guarantees equal educational opportunity, which includes an educational environment free from—or at least responsive to—sexual assault. Administered by the ED, these federal laws are implemented differently under different administrations and are influenced by practices at the ED and quasi-legal directives.

Since 1980, courts have interpreted Title IX to include protections from sexual harassment and sexual assault.⁶ In 2011, the Office for Civil Rights (OCR) within the ED issued a “Dear Colleague” letter (DCL) advising universities that they must provide a learning environment free from sexual violence. A DCL does not have the status of law, but this one was determined to be a “significant guidance document” which spurred university revisions to sexual assault policies. The Clery Act requires all colleges and universities to have comprehensive crime prevention strategies in place and to compile and publish crime data.

The Obama administration’s DCL met with considerable resistance. The Harvard Law School Faculty asserted that defining sexual assault and rape was difficult and that accused students seemed to lack due process protections (Ryan, 2018). Obama-era rules instructed colleges and universities to employ the lower “preponderance of the evidence” standard for determining whether or not an incident occurred. The Trump-era rules favor the higher burden of “clear and convincing evidence” for a finding of sexual assault. The standard of proof that universities and colleges must use in a disciplinary proceeding is much debated and the lower, Obama-era standard means the threshold for finding a perpetrator responsible favors the complainant/victim. Along with other provisions, some scholars and men’s rights activists critiqued the burden of proof as undervaluing due process for the accused.

The Obama-era DCL has undergone substantial changes during the Trump administration. In February 2017, the U.S. Department of Justice and the ED released a new letter that rescinded parts of the previous letter. This included redacting application of the Title IX guidelines to transgender students.⁷ In May 2020, the ED released a 2,033-page document defining a new set of regulations on how schools should manage sexual misconduct. In this document, sexual misconduct is redefined at a higher bar of “severe, pervasive, and objectively offensive that it effectively denies a person equal access,” which limits the types of conduct under the purview of the Title IX office.⁸ The new rule also requires survivors seeking formal resolution to undergo cross-examination (Bedera, 2020). The rule also sets a new standard of “deliberate indifference” rejecting the previous “should have known” standard to the effect of reducing the responsibility of schools. The sum of these changes is generally considered to make it harder or less likely for survivors to seek resolutions while loosening the responsibility of schools (Bedera, 2020; Downey, 2020). As with the previous DCL, this rule constitutes a procedural floor and suggests guidelines rather than limiting the ceiling of Universities’ Title IX response.

In addition to federal law, some states have taken measures to specify college and university obligations about ensuring equal educational opportunity by preventing sexual assault on campus. Most notably, California became the first state in the country to adopt a “Yes Means Yes” position on consent with SB-967, a statute that was approved by the governor and signed into law on September 28, 2014.^9,10 Title IX hearings and investigations continue to be litigated in state courts, demonstrating the continued need for improvement to Title IX procedures. As recently as July 2021 and clarified in August 2021, a Massachusetts district court ruled to vacate a section of the DE 2020 Title IX Final Rule that prevented schools from considering statements from parties who declined to participate in cross-examination at live hearings (Victim’s Rights Law Center v. Cardona, 2021). This rule change might feasibly change the propensity of witness and victim statements to be included as evidence and change the way witnesses and victims are willing/able to participate. Soon after the ruling, institutions appeared to be contemplating removing relevant provisions from their own policies (Zaccheo, 2021).

State law and federal law are not self-enforcing, however. The existence of and disputes over the language of these laws and directives make the issue salient for everyone involved in higher education. Yet how these laws are understood, interpreted, and presented to the students who will actually be trying to engage in consensual sex practices is understudied. Of course, rights like those conferred by Title IX of the Civil Rights Act are not self-enforcing either. Sociolegal scholarship about law provides a number of insights into how the law, in the form of rights, will operate in action. In what follows, we take up neoinstitutional theory, which provides us some hypotheses about how these laws will operate in practice.

Law in Action: Managerialization/Neoinstitutional Theory

Analyzing organizational policies about legal compliance over time and across institutions is useful for many purposes. First, it allows for the identification of multiple audiences that organizational leaders want to signal to or be in conversation with (Gould, 2005). This in turn can tell us something about an organization’s prestige within a field (Armstrong et al., 2017). They can also serve as historical artifacts of legal processes (Riles, 2006). Neoinstitutional theory suggests that the interests of these various parties are orthogonal or even contradictory. In addition, bureaucratic implementation or “managerialization” of law is likely to have its intended effects only performatively, if it has any effects at all (Barnes & Burke, 2014; Berrey, 2015; Berrey et al., 2017; Burke, 2004; Dobbin and Kalev, 2016, 2017; Edelman, 1992, 2016). These studies provide a number of hypotheses about how the managerialized version of Title IX might work and the memetic processes that produce policies in a particular order and with a certain likeness.

This research builds on the scholarship that demonstrates that, in the United States, we use rights—often conferred as individual rights—to achieve important social goals (Burke, 2004; Epp, 2009). Rights are significant sources of power because we construe them to (at least formally) be available equally to everyone, neutral, and backed by the legitimate authority of the state. In courts, social actors are expected to take notice and implement changes (in this case, university administration) to achieve social change (McCann, 1994; Nielsen, 2004; Scheingold, 1974; Williams, 1991). Yet both empirical social scientists and theoretically minded critical legal scholars note many problems in relying upon rights-based litigation to effect social change: Many people whose rights have been violated do nothing to vindicate them (Berrey et al., 2017; Bumiller, 1992; Engel & Munger, 2003; Felstiner et al., 1980; Nielsen & Nelson, 2005); rights are not self-enforcing; rights are enforced differently based on the relationships of the parties involved (Albiston, 2010; Engel, 1984; Yngvesson, 1988); and rights may inappropriately introduce politics into law (Galanter, 1974; Kairys, 1998; Miller & Sarat, 1980; Nielsen, 2004).

Sociolegal models of legal compliance demonstrate the myriad factors that influence the possibility of law influencing behavior. Laws must first be established, then translated into organizational practices (Edelman, 1992), reinterpreted by courts (Edelman, 2016), taught to ordinary people (Edelman, 2016; Edelman et al., 1992), and then compete with other institutional, organizational, and individual factors that shape behavior (Heimer & Staffen, 1998; McElhattan et al., 2017).

Civil rights that are policed and enforced by large organizations, colleges, and universities, in this case, require careful attention because the interests of the person whose rights have been violated and the organization often are at odds. This phenomenon is well documented in the workplace (Berrey et al., 2017; Edelman et al., 2001) and it is easy to see how it would work in the university setting. Just as workplaces do not want to be accused of discrimination, no university wants to have large numbers of sexual assaults or rapes.

There is much to analyze in terms of how “law” is interpreted by university counsel, the field of professional consultants that advise universities on how they should enact these policies, institutional “best practices,” and what will constitute a defensible strategy should a student complain to the ED OCR or file a Title IX lawsuit.

Because rights are defined by law and shaped by the features of the bureaucratic, regulatory, and judicial systems through which rights are asserted, considerable recent research examines campus compliance with ED regulations about Title IX (Armstrong et al., 2017; DeMatteo et al., 2015; Graham et al., 2017). It finds that elite universities—including Ivy League schools, flagship state universities, and other selective institutions—are more likely to have gone beyond simply complying with the DCL and other federal recommendations regarding campus sexual consent policies (Armstrong et al., 2017). Smaller schools, those with higher male enrollment, and private schools have more room for improvement (Graham et al., 2017). And, all campus sexual assault policies must comply with more than 400 state statutes or regulations about sexual misconduct (DeMatteo et al., 2015).

Although there are differences among policies, there is significant homogeneity in many aspects of these policies at colleges and universities, which likely is attributable to “compliance entrepreneurs” that produce and market sexual assault (and HR and alcohol) policies to colleges and universities in the name of legal compliance and best practices.¹¹

Like the employment context, we see a pattern of law intending to disrupt hierarchies of inequality by requiring institutions whose primary purpose is not civil rights enforcement to create policies and implement practices to ensure the rights of those in the organization. Those institutions turn to compliance professionals inside and outside the organization to create a set of best practices for ensuring civil rights—one of which is the written policies we analyze here.

Title IX policies and the process of managerialized rights play a particularly important role, compared with other types of crime and safety policies. When an individual is assaulted, the onus is on them to navigate the specific processes of reporting in a way that differs substantially from other forms of victimization. That is, victims are required to determine who confidential reporters are, how to save and store evidence, and what channels of authority to contact at the university. These processes are time-sensitive and are also imbued with stigma that other policy protective conditions do not have. For example, consider something like theft of a laptop from the campus library. While a student would be required to initiate the complaint, they would not be required to procure forensic evidence, produce suspects, and identify confidential reporters. In addition, there are also specific barriers to reporting sexual assaults like shame, secrecy, concerns about confidentiality, and fear of not being believed (Sable et al., 2006). These barriers may uniquely drive students to Title IX policies before making reports, making the content of those policies uniquely essential for the victim to enact their rights.

Rarely do we analyze whether or not individuals understand the policies intended to benefit/protect them. In other words, what do the policies look like across different college campuses? Can undergraduates reliably navigate these policies to locate various aspects of these policies? Finally, can undergraduates comprehend crucial aspects of the policies?

Method

To understand whether and how undergraduates understand written Title IX policies, we developed and fielded an experiment focusing on five representative campus policies drawn from a large, representative database of coded Title IX policies. In this section of the article, we describe the process of collecting and analyzing the sample of Title IX policies. We also describe the process of choosing the five representative policies and the design of the experiment we used to test undergraduates’ ability to (a) navigate the policy; (b) locate particular provisions of the policy; and (c) understand various aspects of the complexity of the policy.

Constructing the Sample of Policies

In our attempt to understand the range of Title IX policies (the “law on the books”), we used the Carnegie classifications of U.S. colleges and universities to identify three types of schools by the highest degree offered: colleges that grant bachelor’s degrees only; colleges that grant bachelor’s and master’s degrees; and colleges that grant bachelor’s, master’s, and doctoral degrees. We then randomly selected 100 schools from each category to construct a 300-school database of Title IX policies.

Like other scholars of these policies, we initially used the framework of the 2011 DCL to develop a detailed coding protocol to code each policy for analysis of what we call “adherence to the law.” This coding protocol included questions designed to interrogate definitions of consent, incapacitation, information about reporting sexual assault, standards of proof, resources for victims, information about the appeals process, and potential sanctions for offenders.

Our original intention was to have a large class of undergraduate students at an elite private university code these policies as part of a class exercise about doing social science research in an introductory course for legal studies majors. We trained the students in the coding procedure and multiple students coded each policy along a variety of dimensions inputting the data into a Google Form. Some of the questions were dichotomous (e.g., “Does the policy specify a standard of proof for determining whether the accused is subject to a hearing?” And, “Is there a 24-hour hotline listed anywhere in the policy?”). Other questions were more substantive (e.g., “What is the standard of proof used in a disciplinary hearing if a student is accused of sexual assault?” And, “Is the complainant/victim allowed a lawyer in the disciplinary hearing?”).

When we set out to analyze the data, the preliminary analysis of inter-rater reliability indicated that the data were unusable for determining what each policy actually contained.¹² The undergraduate students were unable to identify the same information in the same policy, with common patterns of data coding being the inclusions of large swaths of irrelevant information and incorrect specific factual assertions. There are multiple possible explanations for why students generated such different results including: differing capacity to comprehend text; prior experience reading a sexual assault policy (or other institutionally generated legal compliance documents); and level of computer skills (using “find” commands to locate sections of the policies). Within the elite university where this experiment began, the range of these capabilities should be relatively limited by admission to the university.

Fundamentally, however, the lack of uniformity in the errors led the authors to predict that the explanation for their disagreement may not be as simple as systematic error in training the students. This coding failure, across legally interested undergraduate students who are part of the population ostensibly protected by Title IX policies, led the researchers to conduct more rigorous coding procedures and tests of the policies themselves in an attempt to lessen the impact of possibly competing explanations.

We next attempted to code the polices with a smaller and more well-trained team of three researchers, two of whom are legally trained social scientists and the third an advanced undergraduate, all of whom are authors of this article. The undergraduate (now graduate) worked on this Title IX research team for 2 years and was trained by the first two authors in legal jargon and interpretation in Title IX policies over the course of several months. Following this much more stringent procedure, we ultimately were able to analyze the national sample with very good interrater reliability, but the question raised by the preliminary coding failure continued to trouble the research team. What did it mean that undergraduates could not understand these policies? If students interested in “law and society” at an elite private university were unable to reliably understand these policies, what would that mean for students at other colleges, at universities in different programs of study, and for undergraduate students overall?

Once the team reliably rated the policies, we drew a subsample for the experiment reported here to determine whether the results we saw in the undergraduate coding exercise with our undergraduates would hold up if tested.

Experiment Procedures

To analyze whether or not undergraduate students can make sense of Title IX policies, we randomly selected five undergraduate schools from the larger coded data set to serve as the stimuli for the experiment. Before simply using them, however, we compared various aspects of the five policies to the larger data set to ensure that the policies were representative of the sample. Of the five undergraduate degree schools they were asked to code, two were religiously affiliated, two were for-profit, two were private, and one was public. Four were in the south, and one was in the northeast. To ensure that the policies were representative of the larger sample, we compared these five with the larger sample and found that the selected schools ranged from including 52% to 86% of the recommended components from the Obama-era DCL which is representative of the larger sample of policies. While the policies have differences, none of them were uniquely or dramatically different despite the fact that the random draw included religious and secular schools, public and private schools, and schools across various regions of the United States.

We then recruited 200 unique undergraduate student-participants using Amazon Mechanical Turk (MTurk) to code each of the five policies. The U.S. worker population of MTurk varies from the demographics of the general U.S. population. Research finds that U.S. MTurk workers are a slight majority female (55%), are about equally likely to be single or married, have a lower income on average than the general population, and are majority White (71.8%; Difallah et al., 2018; Levay et al., 2016). However, scholars also assert that MTurk populations do not seem to vary from the general population in unmeasurable ways and advocate for use of MTurk samples when precautions to ensure data quality are correctly taken (Levay et al., 2016). As such, we took a number of precautions to ensure that our sample was of high quality. First, we composed the study using a prescreening survey that did not indicate to potential participants that we were only looking for current undergraduate students. This helped guard against potential participants falsely indicating student status to complete the survey. We also required that all participants be located in the United States and have lifetime MTurk approval ratings (given by the platform) of 95% or higher. In addition, we included attention checks in each policy requiring participants to list the page numbers in the policies to ensure that participants were using the included downloadable policy for their coding. We manually evaluated the resultant data, carefully screening for signs of “botting” or automatic question completion, usually identifiable using nonsensical content. We identified and removed the only suspected incident of botting. We also timed the length of time each participant evaluated each policy to ensure that average completion times were realistic to the task. We determined that participants spent an average of 21.44 minutes per policy, which we deemed plausible for question completion.

We report some demographics of the student-participants in the appendix, but some merit mentioning here. Male student-participants far outnumbered female student-participants (66.83% vs. 32.66%). We also asked student-participants what region they were from and what college or university they attend to verify such a school exists, but redact this for student privacy. Student-participants came from all across the United States, with slightly fewer hailing from the Midwest and the noncontinental United States (represented in the “Other” category). We present only this very limited set of demographic variables because our focus in this study was on the policy as a unit of analysis, rather than a composed power analysis that would accurately compare subgroups or evaluate differences in policy reading. Such a study would be extremely valuable in its own right, but would require different sampling methodology than what we use here.

Student-subjects were asked to answer 10 questions about the policy they were randomly assigned. The questions consisted of what we are calling “locating” questions and “comprehension” questions. “Locating” questions required the student-participants to find a specific piece of information, but not to interpret or understand what it means. “Comprehension” questions required some type of meaning-making to retrieve the correct answer.

Student-subjects could answer the locating questions correctly with diligence (by counting or conducting a word search using “control-f”). For example, to know whether any information about incapacitation is contained in the policy, a student coder does not necessarily need to know how the policy defines incapacitation. Instead, they could use the control+f function to search for the word “incapacitation” and report that the policy does or does not include something about “incapacitation.” All the locating questions contain terms of art used in most policies and do not require any interpretation. Student-subjects were simply asked, “Does the policy include information about “incapacitation” to which the student-subject only needs to answer yes or no. The locating questions should be relatively easy to reliably answer.¹³

The second block of questions (the “comprehension” questions) are more difficult but contain questions central to definitions of consent, sexual assault, and the schools’ process for managing these complaints. These questions represent crucial aspects of the schools’ disciplinary processes that should matter to someone contemplating making a report of sexual violence. For example, we asked student-participants to report the “standard of proof” for proving sexual assault in the disciplinary process. But many policies do not use the words, “standard of proof.” Sometimes the answer is clear and something the students have heard before, like “beyond a reasonable doubt.” Other times the standard of proof information represents a legal standard that lay-people are less familiar with, such as “by a preponderance of the evidence” or “clear and convincing evidence.”^14,15 Standards of proof are also a useful example of exactly why the language in policies is so important for the enactment of managerialized rights like those protected under Title IX. In sexual assault cases, it is up to the victim of the assault to save forensic evidence relating to their victimization immediately after their assault. Therefore, if they were to read the Title Policy after being assaulted and misunderstand the standard of evidence required by their university, they may erroneously believe the evidence they have is insufficient to pursue a Title IX claim. These evidentiary standards are a source of much political debate, but they only matter if a report is made, a disciplinary process begun, and disciplinary procedures implemented. Nonetheless, the issue of standard of proof is a high-profile point of political contest.¹⁶

All five of the comprehension questions require student-participants to understand what is permitted or required at a particular point in the Title IX complaint process and to make sense of legal terms and standards. We hypothesize that there will be far less intercoder agreement for the comprehension questions.

Specifying “Understanding”

To measure if and how well students understand the Title IX policies, we asked multiple students the same questions about each policy (40 student-participants coded each of the five policies) and then measure the student-participants’ agreement on their answers to the 10 questions (five locating and five comprehension). The more understandable the policy, the higher agreement we should see among the student-participants. We choose to use interrater reliability scores over correct answers because it is a more flexible measure of understanding. Interrater reliability scores allow us to study a larger universe of potential outcomes that clarifies whether the problem with the document is that it is similarly misleading (generating consensus in the same wrong direction) or generally not legible (generating more varied responses with less consensus). That is, theoretically all of the students might come to the same wrong conclusion or different wrong conclusions that if graded purely on correctness it would not be apparent in the results. Because we are interested in both students’ ability to come to consensus and ability to understand the policies as a population, we employ interrater reliability. We use Fleiss’s kappa scores to measure interrater reliability. Fleiss’s kappa is a variation of Cohen’s kappa that allows comparisons across a larger number of responders with discrete categorical questions than Cohen’s kappa which only measures agreement across two responders. In this work, each question has as many as 40 answers necessitating using Fleiss’s kappa (further discussion in the following section). We therefore measure “understanding” or rather “misunderstanding,” as whether or not multiple undergraduate readers of the same policy give the same answers to the 10 questions.

We include Tables 1 and 2 to demonstrate the kind of analysis expected of subjects with the following caveat. The answers provided in Tables 1 and 2 represent our coding of the policies used as stimuli for the subjects. In a way, our analysis of the contents of the policy is less relevant than whether or not student-respondents analyzed the questions similarly. We provide this analysis so that the reader can understand the differences across policies and get a sense of the breadth of the documents student-respondents were working with. To protect the confidentiality of the schools whose policies we used, we have made subtle changes in the wording drawn from their policies without changing meaning. We also give page ranges rather than actual page numbers and have taken other insignificant licenses to cloak identity, but these are the passages the student-respondents needed to find and code.

Table 1

Locating Questions

Campus	Pages	Mentions incapacitation?	Appeals process?	Contact no. for Title IX officer?	Does consent require consciousness?
1	<20	Yes: Consent may never be obtained through the use of force, coercion, or intimidation or if the victim is mentally or physically disabled or incapacitated, including through the use of drugs or alcohol	No or Not mentioned	Yes: Midway through policy	Yes: Consent is not possible with a person who is incapable of consent by reason of sleep, drunkenness, stupefaction, or unconsciousness . . .
2	40–50	No. Closest is: No consent is considered: where one participant is incapable of consenting to the activity	Yes: The decision of the campus Director of Education/Dean may be appealed by petitioning the Campus President’s office . . . within 20 days of receipt of the determination letter	No: Just the police	No: (word does not appear, cf.) Sexual acts perpetrated against a person’s will or where a person is incapable of giving consent to the victim’s use of drugs or alcohol
3	40–50	No. Closest is: Sexual Exploitation means any act of taking nonconsensual, unjust, or abusive sexual advantage of another person by: Causing or attempting to cause the incapacitation of another person to gain a sexual advantage over such person	Yes: The Complainant and/or the Respondent may appeal the Assigned Title IX Coordinator’s decision in writing to the Appropriate Divisional Leader and provide a copy of the appeal to the Assigned Title IX Coordinator within 10 days of receipt of the notice of closure	Yes: Last pages of policy	Yes: Persons who are incapacitated (whether as a result of drugs, alcohol, or otherwise), unconscious, asleep, or otherwise physically helpless or mentally or physically unable to make informed, rational judgments
4	30–40	Yes: Incapacitation due to physical condition includes the inability, temporarily or permanently, to give consent because the individual is mentally and/or physically helpless due to drug or alcohol consumption . . .	Yes: The Respondent and Complainant may request an appeal of the decision rendered by the Adjudicator. Disagreement with the finding or corrective action is not, by itself, grounds for appeals	Yes: Midway through policy	Yes: Consent is not present when an individual is voluntarily or involuntarily incapacitated, voluntarily or involuntarily, due to . . . lack of consciousness . . . that impairs the individual’s ability to provide consent
5	50–60	Yes: An incapacitated individual is unable to make rational, reasonable decisions . . .	Yes: A respondent or complainant has up to three business days (or by 8 a.m. on the next university business day if the deadline occurs when university offices are closed) from the date of the decision notification to submit an appeal in writing to the Office of Student Conduct	Yes: Midway through policy	Yes: Incapacitation can occur as a result of mental disability, sleep, involuntary physical restraint, unconsciousness, voluntary (or involuntary) use of alcohol and/or drugs, or when a person is otherwise physically helpless

Table 2

Comprehension Questions

Campus	What is the standard of proof for a finding of sexual assault?	Does the policy mention retaining evidence?	Is the victim allowed to bring an attorney to a hearing?	Is there a time frame for the investigation and proceedings?	Are interim measures against the accused allowed?
1	No: but refers to a different policy	No	No: Attorney is not specifically mentioned but accused and accuser get the same	No	Yes: Following an alleged sex offense, if so requested by the victim and if such changes are reasonably available, assistance in changing residential and/or academic situations is provided by the Dean of Students
2	No mention	Yes: This can be done by not bathing, showering, or using toothpaste or mouthwash after an incident of sexual assault. Do not wash clothing, bed sheets, pillows, or other potential evidence	No or ambiguous: The student may be accompanied during investigation meetings and discussions by one person (family member, friend, etc.) who can act as an observer. . . and may be removed at the discretion of the Dean	Yes: The student who made the complaint and the accused shall be informed promptly in writing when the investigation is completed, no later than 45 calendar days from the date the complaint was filed	Yes: but only for victim once a preliminary determination is made and then applied to perpetrator as well. And, refers victim to “law enforcement” for orders of protection
3	Yes: The Hearing Panel will confer and by majority vote determine whether the evidence establishes that it is more likely than not that the Respondent committed Actionable Sexual Misconduct	Yes: To best preserve evidence for an evidence collection kit, it may be advisable to avoid showering, bathing, going to the bathroom, or brushing your teeth before the kit is completed. You should also wear (or take with you in a paper—not plastic—bag) to the hospital the same clothing that you were wearing during the assault	No: Under no circumstances may legal counsel be present at the mediation on behalf of the Complainant or the Respondent. The University, however, may seek advice from the University’s in-house or outside counsel on questions of law and procedure throughout the mediation process	Yes: Multiple and at every step 72 hours, 21 days, etc.	Yes: If at any point during the complaint, investigative, or disciplinary processes, the Assigned Title IX Coordinator deems it necessary for the protection of any member of the University community . . . the University will take prompt action to limit the effects of the alleged Sexual Misconduct and to prevent its recurrence including a “no-contact” order or take other appropriate interim measures to ensure an individual’s safety even in the absence of a formal proceeding
4	Yes: If a complaint is found to be supported by a preponderance (50.1%) of the evidence, appropriate corrective action will follow, up to and including separation of the offending party from the College, consistent with College procedure	Yes: If possible, an individual who has been sexually assaulted should not shower, bathe, douche, or change clothes or bedding before going to the hospital or seeking medical attention. If the individual decides to change clothes, he or she should not wash the clothes worn during the assault and should bring them to the hospital or medical facility	No: Each party, the accuser and the accused, may select her/his own liaison who is a member of the college faculty/staff, but not a member of the Sexual Misconduct Committee. The accuser and the accused are entitled to the same opportunities to have others present during a disciplinary proceeding	No: Investigations will be conducted as expeditiously as possible and are usually completed within 60 days, although this may vary based on the availability of witnesses, the scope of the investigation, or unforeseen circumstances	Yes: In addition, the College may elect to suspend the accused during the investigation
5	Yes: Preponderance of evidence is the required standard for determining a policy violation. Administrators . . . must be convinced based on the information provided that a policy violation was more likely to have occurred than to not have occurred to find a respondent responsible for violating a policy	No	Yes: Complainants and Respondents who wish to consult with an attorney may do so at their own expense; the attorney may act as the student’s advisor and accompany the student to any investigation meeting and/or student conduct hearing. An advisor may be present for any investigation or student conduct meeting	No: The University seeks to resolve all reports within 60 calendar days of the initial report. All time frames expressed in this Policy are meant to be guidelines rather than rigid requirements	Yes: Upon receipt of a report, the University will impose reasonable and appropriate interim measures designed to eliminate the hostile environment and protect the parties involved. . . including changing academic, living, transportation, and working situations or protective measures, if such accommodation is reasonably available

Results

With this in mind, we turn to the analysis of the locating and understanding questions answered by student-coders on MTurk. To measure whether or not Title IX policies are understood by student-participants, we evaluated intercoder reliability using Fleiss’s kappa. The general principle is illustrated in Equation 1.¹⁷ We employ Landis and Koch’s (1977) framework for interpreting Fleiss’s kappa coefficients. The full framework appears in Table 3, but for our purposes, 0.41–0.60 represent moderate agreement; 0.61–0.80 indicates substantial agreement; and 0.81–1.0 represent almost perfect agreement. We would hope that multiple undergraduates reading the same policy would be able to achieve at least substantial agreement.

Table 3

Interpreting Fleiss’s Kappa Coefficients

Kappa coefficient	Interpretation
<0	Poor agreement
0.01–0.20	Slight agreement
0.21–0.40	Fair agreement
0.41–0.60	Moderate agreement
0.61–0.80	Substantial agreement
0.81–1.00	Almost perfect agreement

Table 4 shows Fleiss’s kappa scores for each of the five schools’ policies by locating and comprehension questions. We include the adherence score of each school for reference.¹⁸ The easier, locating questions show a relatively high level of understandability as measured by agreement among student-participants. They achieved moderate to substantial agreement on four of the five policies analyzed. In other words, for 80% of the policies analyzed, students agreed enough on the locating questions for us to conclude that the policy is moderately or substantially understandable at least regarding the locating questions.

Table 4

Fleiss’s Kappa Interrater Reliability by School and Question Type

School	Locating	Comprehension	Adherence score (%)
School 1	0.60*	0.23*	77.68
School 1	(0.18)	(0.08)
School 2	0.50	0.17	52.08
School 2	(0.22)	(0.08)
School 3	0.70*	0.06	79.46
School 3	(0.22)	(0.04)
School 4	0.71*	0.07	85.71
School 4	(0.21)	(0.03)
School 5	0.70*	0.12	76.19
School 5	(0.22)	(0.05)

Note. N = 200, SEs in parentheses.

p < .05.

The results are not as inspiring for the more difficult comprehension questions. For four of the five schools’ policies analyzed, student-participants could not reach even slight agreement about the comprehension questions. For the one school’s policy that student-participants could reach agreement on comprehension questions, the agreement barely reached “slight” agreement.

In sum, in four of five cases student-coders produced significantly similar coded results for locating questions (p < .05), but in only one of the five cases were student-coders able to produce significantly better than random results for comprehension questions (p < .05). In the case of Fleiss’s kappa, p values alone are not sufficient to understand the strength of agreement, only that is better than random chance.

According to this framework, students were always able to achieve moderate-substantial agreement in answering the locating questions. This implies that students are putting in effort as coders and hints at a different explanation for a lack of reliability. The extremely low coefficients in the comprehension conditions, only once rising just over the threshold of slight agreement, lend further support to the idea that certain parts of Title IX policies are simply not understandable by the students who are meant to initiate these protective measures.

In a final attempt to assess the understanding of the policies, we sought to measure the functional understandability of these policies computationally. To do so, we used a sample of 160 of the 300 coded policies to estimate the Flesch reading-ease score (FRES) of the actual text. The Flesch reading-ease test is designed to measure how difficult English sentences are to understand and is part of the textual evaluation protocol for a number of U.S. Military and state technical and legal documents (McClure, 1987; Si & Callan, 2001). A higher score indicates that a passage is easier to read, whereas a lower score indicates a passage that is more difficult to read. The formula for calculating FRES is given in Flesch (1979).

Equation 2: FRES

\begin{array}{l} 206.835 - 1.015 (\frac{T o t a l w o r d s}{T o t a l s e n t e n c e s}) \\ - 84.6 (\frac{T o t a l s y l l a b l e s}{T o t a l w o r d s}) \end{array}

Flesch (1979) also provides a guide for interpreting the resultant scores as follows (see Table 5) but notes that particularly arduous passages can generate negative scores.

Table 5

Interpreting Flesch Reading-Ease Score

Score	School level	Interpretation
100.00–90.00	5th grade	Very easy to read. Easily understood by an average 11-year-old student
90.0–80.0	6th grade	Easy to read. Conversational English for consumers
80.0–70.0	7th grade	Fairly easy to read
70.0–60.0	8th and 9th grade	Plain English. Easily understood by 13- to 15-year-old students
60.0–50.0	10th to 12th grade	Fairly difficult to read
50.0–30.0	College	Difficult to read
30.0–10.0	College graduate	Very difficult to read. Best understood by university graduates
10.0–0.0	Professional	Extremely difficult to read. Best understood by university graduates

The 160 policies coded here (via computational software) averaged an FRES of 30.08. The policies ranged from just −0.62 to a maximum of 62.58 and are plotted in Figure 1.¹⁹

Figure 1.

Flesch reading-ease scores.

According to Flesch’s (1979) scale, this confirms that Title IX policies are generally difficult or very difficult to read. For comparison, Time Magazine has an FRES of around 57, The New York Times an FRES of around 48, and the Harvard Law Review an FRES of around 34 (Flesch, 1979; Stewart, 2017). This means that the average Title IX policy, a policy that may be used by a recently assaulted student to understand their rights in a student-initiated process, is more difficult to understand than the average Harvard Law Review article. A more recent audit and translation of FRES argues that scores from 20 to 40 (remember the average Title IX policy scored a 30.08) are most appropriately read and understood by lawyers and physicians. This paints a grim picture of the protective capacities of Title IX policies.

We also note that FRES is not purely a function of policy length. The correlation between FRES is modest at .11. Importantly, the fact that some FRES are closer to 60 suggests that it is possible to fashion Title IX policies that are substantially more readable, with Flesch’s (1979) scale describing such policies as “Plain English. Easily understood by 13-15-year-old students.”

Conclusions and Implications

College and university Title IX policies serve many purposes. Most obviously, policies like the ones we analyze here would be (and are) the backbone of a defense strategy should a student make a Title IX claim to the ED civil rights division or in federal court. But the policies also signal important messages to multiple constituencies. For prospective students and their parents, the existence of such policies shows that schools take campus sexual assault seriously. For faculty and staff, such policies demonstrate a well-ordered workplace in which one can feel (relatively) safe. For potential perpetrators of sexual assault, they may serve as a deterrent. To peer institutions, an up-to-date Title IX policy signals the school’s legitimacy. These are all important functions. And yet, the research about Title IX policy—voluminous though it is—fails to empirically examine how these policies are understood by perhaps the most important consumers of them—young adults who may, tragically, need to utilize the policy because they, or a friend, have been sexually assaulted and are considering their options for reporting, seeking counseling, or even immediate medical care.

Our research demonstrates that written Title IX policies are for the most part available and yet were inaccessible to undergraduate students in the active study. In other words, while we anticipate that undergraduates may be able to locate the policies online, we predict that the students are only somewhat likely to be able to locate specific provisions of the policy and are unable to reliably understand the content of the policies. One reason may be, as our analysis shows, that the median Lexile score of university Title IX policies is roughly equivalent to an article appearing in the Harvard Law Review. As our experiment demonstrates, undergraduates are unable to make heads or tails of the policy even in ideal experimental conditions.

These findings reveal the troubling implications of managerialized legal process in colleges and universities. Like research on employment civil rights (Barnes & Burke, 2006; Berrey et al., 2017; Edelman et al., 2001; Kelly & Dobbin, 1998), we see a Title IX process that identifies a social problem (campus sexual assault), entitles individuals to make a claim if they think their rights have been violated, and refers law-making to an agency to promulgate rules. Those rules are then interpreted by compliance entrepreneurs, lawyers, and other bureaucratic processes, which ultimately produces a written text that should ostensibly help undergraduate students. And yet, in its institutionalized form, law fails undergraduates who might need to access and mobilize it. At best, these policies are important signals to important constituencies that inadvertently fail students. At worst, they represent symbolic compliance designed to protect colleges and universities from civil rights litigation initiated by victims of sexual assault on college campuses.

Our research has limitations. First, this research was conducted on MTurk with a limited set of demographic factors. While we believe the study is sufficiently rigorous for the present object of analysis, we also urge future researchers to take up the question of how different groups of students may differently read and understand Title IX policies. This will allow researchers and policy makers to better understand how factors like gender demographic composition may affect the results. Second, the research examines only the written Title IX policies. College students are oriented to campus sexual assault rules and processes in a variety of ways, including orientation videos, campus events, and, in colleges with residential populations, through residential life programs. And, of course, students learn about Title IX processes and policies informally through gossip, campus news, and the like. Finally, our experiment uses typical campus policies and was tested on undergraduates from many universities. One might imagine that students are more familiar with their own university policy (although, as the analysis of these policies compared with the larger sample and as the excerpts in Tables 1 and 2 show, there is homogeneity across policies).

This research provides important insight about what institutions can do differently in drafting their Title IX policies. Institutions should use this work to motivate their own thorough adaptation and testing of the understanding of their own policies. To make these policies more accessible to the student constituents who must enact them, universities should take deliberate care to eliminate legal jargon and focus on reader comprehension over mitigating legal liability. If one policy cannot meet the needs of the various constituencies mentioned above, colleges and universities should develop student-friendly policies that might even be separate and apart from the voluminous policies. Such policies would need to be simple but comprehensive, refer back to the formal policy, and presented on college and university websites alongside the formal policies. These could be simple flowcharts or info graphics that start with basic definitions and explanations of students’ options. To be sure, many colleges and universities employ easier to read pages in different places on the website (counseling and psychological services, the “women’s” center, and other related units in the school), but ensuring the connection to the more formalistic and complicated policies like the ones we analyzed.

Moreover, federal and state compliance officers who work with institutions to develop the complex policies might recommend to colleges and universities that their website include empirically tested student accessible policy documents and websites. Consultants and attorneys that practice in this area could develop templates that would allow colleges and universities to customize policies to be consistent with their own. At the highest level, the ED Office of Civil Rights could recommend (using a DCL or some other mechanism) that colleges and universities have a policy that is legible for their students.

Individual rights, like the ability to have an attorney present, and institutional limits should be clearly spelled out and tested on undergraduate students to ensure they are comprehensible. State education agencies can serve as valuable intermediaries in this process, conducting testing on Title IX policies to ensure that universities are following best practices in producing actionable right-protective documents for students.

These limitations may bias our results to make it appear as though students have less of an understanding of these policies than students may actually have on the campus of which they are a part. However, our undergraduate research subjects had not been (to our knowledge) sexually assaulted just prior to participating in the study, which surely would affect anyone’s ability to read and comprehend a policy document, the likes of which we analyze here. This bias may work the other way as well. In other words, unlike an undergraduate recently sexually assaulted consulting their university’s policy for the first time, our student-participants’ lack of recent sexual trauma may have allowed them more latitude to code and analyze these policies than would have been the case otherwise.

We have begun, as part of a larger project, to conduct more situationally located research about undergraduate understanding and utilization of Title IX policies that takes the limitations of this study into account. Nevertheless, this research raises troubling questions about the rights-based, managerialized legal version of these policies and the implications for victims of sexual assault.

Footnotes

Appendix

Amazon Mechanical Turk Respondent Demographics

Demographic Variable	Schools					Total N	Total (%)
Demographic Variable	1	2	3	4	5	Total N	Total (%)
Sex
Female	10	13	12	12	17	65	32.66
Male	29	26	28	28	23	133	66.83
Other	1	0	0	0	0	1	0.50
Home region
Midwest	5	11	5	4	10	35	17.59
Northeast	11	13	11	15	8	58	29.15
South	10	7	10	8	10	45	22.61
West	13	9	11	13	12	58	29.15
Other	1	0	2	0	0	3	1.51
N = 199, 1 missing

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Kat Albrecht

Notes

Authors

KAT ALBRECHT is an assistant professor of criminal justice and criminology in the Andrew Young School of Policy Studies at Georgia State University. Her research lies at the intersection of computational social science, empirical legal studies, and criminal law. She is particularly interested in innovative survey methods to study fear and decision-making.

LAURA BETH NIELSEN is professor and chair, Department of Sociology, Northwestern University; research professor, American Bar Foundation; and president of the Law and Society Association. Her research interests are in law’s capacity for social change. Her primary field is the sociology of law, with particular interests in legal consciousness (how ordinary people understand the law) and the relationship between law and inequalities of race, gender, and class.

LYDIA WUORINEN is a JD candidate at the University of Pennsylvania Carey Law School. Her primary research interest is data-driven legal studies, specifically the ordinary meaning and comprehension of legal terms of art and accessibility within the legal system.

References

Albiston

C. R.

(2010). Institutional inequality and the mobilization of the family and medical leave act: Rights on leave. Cambridge University Press.

Anderson

Svriuga

Clement

(2015, September 21). Survey: More than 1 in 5 female undergrads at top schools suffer sexual attacks. The Washington Post. https://www.washingtonpost.com/local/education/survey-more-than-1-in-5-female-undergrads-at-top-schools-suffer-sexual-attacks/2015/09/19/c6c80be2-5e29-11e5-b38e-06883aacba64_story.html

Armstrong

E. A.

Levisky

Garrick

Porter

Badke

(2017). Leaders and laggards: Explaining variation in university responses to sexual violence in a changing legal environment. https://www.psc.isr.umich.edu/research/project-detail/37345.html

Barnes

J. E.

Burke

T. F.

(2006). The diffusion of rights: From law on the books to organizational rights practices. Law & Society Review, 40, 493–524.

Barnes

J. E.

Burke

T. F.

(2014). How policy shapes politics: Rights, courts, litigation, and the struggle over injury compensation. Oxford University Press.

Bedera

(2020, May 14). Trump’s new rule governing college sex assault is nearly impossible for survivors to use: That’s the point. Time. https://time.com/5836774/trump-new-title-ix-rules/

Berrey

(2015). The enigma of diversity: The language of race and the limits of racial justice. The University of Chicago Press.

Berrey

Nelson

R. L.

Nielsen

L. B.

(2017). Rights on trial: How workplace discrimination law perpetuates inequality. The University of Chicago Press.

Bumiller

(1992). The civil rights society: The social construction of victims (Revised ed,). Johns Hopkins University Press.

10.

Burke

T. F.

(2004). Lawyers, lawsuits, and legal rights. University of California Press. https://www.ucpress.edu/book/9780520243231/lawyers-lawsuits-and-legal-rights

11.

Culture of Respect. (2020). Culture of respect: Ending campus sexual violence. https://cultureofrespect.org/

12.

Dastagir

A. E.

(2019, June 5). Emily doe writing memoir to “reclaim the story” of sexual assault by Brock Turner. USA Today. https://www.usatoday.com/story/news/nation/2019/06/05/emily-doe-survivor-brock-turner-case-writing-memoir/1361247001/

13.

Davidson

J. R. T.

Foa

E. B.

(1993). Posttraumatic stress disorder: DSM-IV and beyond. American Psychiatric Press.

14.

Davidson

J. R. T.

Hughes

D. C.

George

L. K.

Blazer

D. G.

(1996). The association of sexual assault and attempted suicide within the community. Archives of General Psychiatry, 53(6), 550–555. https://doi.org/10.1001/archpsyc.1996.01830060096013

15.

DeMatteo

Galloway

Arnold

Patel

(2015). Sexual assault on college campuses: A 50-state survey of criminal sexual assault statutes and their relevance to campus sexual assault. Psychology, Public Policy, and Law, 21(3), 227–238. https://doi.org/10.1037/law0000055

16.

Difallah

Filatova

Ipeirotis

(2018, February). Demographics and dynamics of mechanical Turk workers. In Proceedings of the eleventh ACM international conference on web search and data mining (pp. 135–143). Association for Computing Machinery.

17.

Dobbin

Kalev

(2016). Why diversity programs fail. Harvard Business Review, 94(7). https://hbr.org/2016/07/why-diversity-programs-fail

18.

Dobbin

Kalev

(2017). Training programs and reporting systems won’t end sexual harassment —Promoting more women will. Harvard Business Review. https://hbr.org/2017/11/training-programs-and-reporting-systems-wont-end-sexual-harassment-promoting-more-women-will

19.

Downey

(2020, May 20). The Trump administration’s new title IX Rule. The Regulatory Review. https://www.theregreview.org/2020/05/20/downey-trump-administration-title-ix-rule/

20.

Edelman

L. B.

(1992). Legal ambiguity and symbolic structures: Organizational mediation of civil rights law. American Journal of Sociology, 97(6), 1531–1576. https://www-jstor-org.web.bisu.edu.cn/stable/2781548

21.

Edelman

L. B.

(2016). Working law: Courts, corporations, and symbolic civil rights. The University of Chicago Press.

22.

Edelman

L. B.

Abraham

S. E.

Erlanger

H. S.

(1992). Professional construction of law: The inflated threat of wrongful discharge articles on legal culture and legal discourse. Law & Society Review, 26, 47–84.

23.

Edelman

L. B.

Fuller

S. R.

Mara-Drita

(2001). Diversity rhetoric and the managerialization of law. American Journal of Sociology, 106(6), 1589–1641. https://doi.org/10.1086/321303

24.

End Rape on Campus (EROC). (2020). End rape on campus. https://endrapeoncampus.org

25.

Engel

D. M.

(1984). The Oven Bird’s song: Insiders, outsiders, and personal injuries in an American community. Law & Society Review, 18(4), 551–582. https://doi.org/10.2307/3053447

26.

Engel

D. M.

Munger

(2003). Rights of inclusion. The University of Chicago Press.

27.

Epp

C. R.

(2009). Making rights real: Activists, bureaucrats, and the creation of the legalistic state—Chicago series in law and society. The University of Chicago Press.

28.

Everfi. (2020). Higher education: Sexual assault prevention training for college sports teams. https://everfi.com/courses/higher-education/sexual-assault-prevention-college-sports/

29.

Federal Register. (2018, November 29). Nondiscrimination on the basis of sex in education programs or activities receiving federal financial assistance. https://www.federalregister.gov/documents/2018/11/29/2018-25314/nondiscrimination-on-the-basis-of-sex-in-education-programs-or-activities-receiving-federal

30.

Felstiner

W. L. F.

Abel

R. L.

Sarat

(1980). The emergence and transformation of disputes: Naming, blaming, claiming. . . . Law & Society Review, 15(3–4), 631–654. https://doi.org/10.2307/3053505

31.

Flesch

(1979). How to write plain English: A book for lawyers and consumers. Harper & Row.

32.

Galanter

(1974). Why the “haves” come out ahead: Speculations on the limits of legal change. Law & Society Review, 9(1), 95–160. https://doi.org/10.2307/3053023

33.

Gersen

J. S.

Gertner

Halley

J. E.

(2019). Comment on proposed title IX rulemaking (SSRN scholarly paper). Social Science Research Network. https://papers.ssrn.com/abstract=3325948

34.

Gould

(2005). Speak no evil: The triumph of hate speech regulation. The University of Chicago Press.

35.

Graham

L. M.

Treves-Kagan

Magee

E. P.

DeLong

S. M.

Ashley

O. S.

Macy

R. J.

Martin

S. L.

Moracco

K. E.

Bowling

M. J.

(2017). Sexual assault policies and consent definitions: A nationally representative investigation of U.S. colleges and universities. Journal of School Violence, 16(3), 243–258. https://doi.org/10.1080/15388220.2017.1318572

36.

Green

E. L.

(2018, August 29). New U.S. Sexual misconduct rules bolster rights of accused and protect colleges. The New York Times. https://www.nytimes.com/2018/08/29/us/politics/devos-campus-sexual-assault.html

37.

Hartocollis

(2019, June 8). New wave of student activism presses colleges on sexual assault. The New York Times. https://www.nytimes.com/2019/06/08/us/college-protests-dobetter.html

38.

Hauser

(2018, August 9). Brock Turner loses appeal to overturn sexual assault conviction. The New York Times. https://www.nytimes.com/2018/08/09/us/brock-turner-appeal.html

39.

Hayes

Bacon

(2018, June 6). Judge Aaron Persky, who gave Brock Turner lenient sentence in rape case, recalled from office. USA Today. http://www.usatoday.com/story/news/2018/06/06/judge-aaron-persky-who-gave-brock-turners-lenient-sentence-sanford-rape-case-recalled/674551002/

40.

Heimer

C. A.

Staffen

L. R.

(1998). For the sake of the children: The social organization of responsibility in the hospital and the home. The University of Chicago Press.

41.

Kagan

R. A.

(2003). Adversarial legalism: The American way of law. Harvard University Press.

42.

Kairys

(1998). The politics of law: A progressive critique. Hachette.

43.

Kelly

Dobbin

(1998). How affirmative action became diversity management: Employer response to antidiscrimination law, 1961 to 1996. American Behavioral Scientist, 7(41), 960–984.

44.

Know Your IX. (2020). Know Your IX: Empowering students to stop sexual violence. https://www.knowyourix.org/

45.

Lady Gaga

S. R. K.

Blavin

Ziering

Dick

Wadleigh

Kopp

. (2015). The hunting ground. Chain Camera Pictures.

46.

Landis

J. R.

Koch

G. G.

(1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310

47.

Levay

K. E.

Freese

Druckman

J. N.

(2016). The demographic and political composition of Mechanical Turk samples. SAGE Open, 6, Article 636466.

48.

McCann

M. W.

(1994). Rights at work: Pay equity reform and the politics of legal mobilization. The University of Chicago Press.

49.

McClure

G. M.

(1987). Readability formulas: Useful or useless? IEEE Transactions on Professional Communication, 1, 12–15.

50.

McElhattan

Nielsen

L. B.

Weinberg

J. D.

(2017). Race and determinations of discrimination: Vigilance, cynicism, skepticism, and attitudes about legal mobilization in employment civil rights—Race and determinations of discrimination. Law & Society Review, 51(3), 669–703. https://doi.org/10.1111/lasr.12276

51.

Mengo

C. M.

Black

B. M.

(2016). Violence victimization on a college campus: Impact on GPA and school dropout. Journal of College Student Retention: Research, Theory & Practice, 18(2), 234–248. https://doi.org/10.1177/1521025115584750

52.

Miller

R. E.

Sarat

(1980). Grievances, claims, and disputes: Assessing the adversary culture. Law & Society Review, 15(3-4), 525–566. https://doi.org/10.2307/3053502

53.

Muehlenhard

C. L.

Peterson

Z. D.

Humphreys

T. P.

Jozkowski

K. N.

(2015). Evaluating the one-in-five statistic: The prevalence of sexual assault among college students. Journal of Sex Roles, 54(4-5), 549–576.

54.

“National Association of College and University Attorneys Online Course.” (2020). https://www.nacua.org/program-events/online-courses/title-ix-coordinator-training/home

55.

National Victim Center. (1992). Rape in America: A report to the nation. https://books.google.com/books?id=G_InAQAAMAAJ

56.

Nielsen

L. B.

(2004). License to harass: Law, hierarchy, and offensive public speech—The cultural lives of law. Princeton University Press.

57.

Nielsen

L. B.

Nelson

R. L.

(2005). Rights realized: An empirical analysis of employment discrimination litigation as a claiming system new legal realism symposium: Is it time for a new legal realism: II—New legal realism in action: Exemplary research—A law, poverty, and land. Wisconsin Law Review, 2005, 663–712. https://heinonline.org/HOL/P?h=hein.journals/wlr2005&i=677

58.

Proust

(1913). Swann’s way ( Scott Moncrieff

, Trans.). Modern.

59.

Riles

(2006). Documents: Artifacts of modern knowledge. University of Michigan Press.

60.

Ryan

(2018, February 28). HLS profs sign letter slamming “victim-centered” sexual harassment policies. The Harvard Crimson. https://www.thecrimson.com/article/2018/2/28/law-profs-sign-sexual-assault-letter/#:~:text=HLS%20Profs%20Sign%20Letter%20Slamming%20’Victim%2DCentered’%20Sexual%20Harassment%20Policies,-Law%20Professor%20Janet&text=The%20letter%20culminates%20in%20a,and%20believe%20the%20victim%20practices.%E2%80%9D

61.

Sable

M. R.

Danis

Mauzy

D. L.

Gallagher

S. K.

(2006). Barriers to reporting sexual assault for women and men: Perspectives of college students. Journal of American College Health, 55(3), 157–162.

62.

Scheingold

S. A.

(1974). The politics of rights: Lawyers, public policy, and social change. University of Michigan Press.

63.

Callan

(2001). A statistical model for scientific readability. In Proceedings of the tenth international conference on information and knowledge management (pp. 574–576). https://www.thecrimson.com/article/2018/2/28/law-profs-sign-sexual-assault-letter/#:~:text=HLS%20Profs%20Sign%20Letter%20Slamming%20’Victim%2DCentered’%20Sexual%20Harassment%20Policies,-Law%20Professor%20Janet&text=The%20letter%20culminates%20in%20a,and%2

64.

Smith

P. H.

White

J. W. J.

Holland

L. J.

(2003). A longitudinal perspective on dating violence among adolescent and college-age women. American Journal of Public Health, 93(7), 1104–1109. https://doi.org/10.2105/AJPH.93.7.1104

65.

Stewart

J. E. J.

(2017). Reinterpreting the Flesch Readability Scale for the information age.

66.

Turchik

J. A.

Hebenstreit

C. L.

Judson

S. S.

(2016). An examination of the gender inclusiveness of current theories of sexual violence in adulthood: Recognizing male victims, female perpetrators, and same-sex violence. Trauma, Violence, & Abuse, 17(2), 133–148.

67.

Williams

P. J.

(1991). The alchemy of race and rights: Diary of a law professor. Harvard University Press.

68.

Yngvesson

(1988). Making law at the doorway: The clerk, the court, and the construction of community in a new England town. Law & Society Review, 22(3), 409–448. https://doi.org/10.2307/3053624

69.

Zaccheo

(2021). Assessing the evolving impact of victim rights law center et al v. Cardona on College and University Title IX procedures. High Education Law Report. https://www.bsk.com/higher-education-law-report/assessing-the-evolving-impact-of-victim-rights-law-center-et-al-v-cardona-on-college-and-university-title-ix-procedures