Assessing Survey Items on Social Norms Relating to Dating and Relationship Violence and to Gender: Cognitive Interviews with Young People in England

Abstract

Widespread among adolescents in England, dating and relationship violence (DRV) is associated with subsequent injuries and serious mental health problems. While DRV prevention interventions often aim to shift harmful social norms, no established measures exist to assess relevant norms and their role in mediating DRV outcomes. We conducted cognitive interviews exploring the understandability and answerability of candidate measures of social norms relating to DRV and gender roles, informing measure refinement. In all, 11 participants aged 13 to 15 from one school in England participated. Cognitive interviews tested two items assessing descriptive norms (beliefs about what behaviors are typical), three assessing injunctive norms (beliefs about what is socially acceptable), and (for comparison) one assessing personal attitudes. Findings were summarized by drawing on interview notes. Summaries and interview notes were subjected to thematic analysis. For some participants, injunctive norms items required further explanation to clarify that items asked about others’ views, not their own. Lack of certainty about, and perceived heterogeneity of, behaviors and views among a broad reference group detracted from answerability. Participants were better able to answer items for which they could draw on concrete experiences of observing or discussing relevant behaviors or social sanctions. Data suggest that a narrowed reference group could improve answerability for items assessing salient norms. Findings informed refinements to social norms measures. It is possible to develop social norms measures that are understandable and answerable for adolescents in England. Measures should assess norms that are salient and publicly manifest among a cohesive and influential reference group.

Keywords

dating and relationship violence adolescents social norms measurement qualitative

Introduction

Background

“Dating and relationship violence” (DRV) among young people refers to physical, sexual, or psychological abuse by a current or former intimate partner (Barter & Stanley, 2016; Young et al., 2017). DRV is widespread in England, with 49.1% of young people with a mean age of 13.4 years (SD = 0.6) reporting psychological victimization and 39.5% reporting physical victimization (Meiksin et al., 2020). Among those aged 14 to 17 years in England who have ever been in a relationship, 41% of girls and 14% of boys report experiencing sexual DRV, assessed as having been pressured or physically forced into intimate touching or sexual intercourse (Barter et al., 2014). In addition to causing injuries (Foshee et al., 1996), DRV victimization is associated with subsequent antisocial behavior, substance misuse (Exner-Cortens et al., 2013; Roberts et al., 2003), and mental health problems (Castellví et al., 2017; Exner-Cortens et al., 2013; Roberts et al., 2003), including suicidal ideation (Exner-Cortens et al., 2013) and suicide attempts (Castellví et al., 2017).

As social affiliation shifts from adults to peers in adolescence (Spear, 2000), young people are particularly sensitive to peer influence (Bonell et al., 2019). Social norms theory suggests that a person’s behaviors are influenced by beliefs about what behaviors are typical (“descriptive norms”) and appropriate (“injunctive norms”) among a reference group of others whose views are important to them (Alexander-Scott et al., 2016; Cislaghi & Heise, 2018). According to this conceptualization of social norms, which has been particularly influential in gender-based violence (Alexander-Scott et al., 2016) and adolescent sexual and reproductive health (Costenbader et al., 2019) research, these norms are sustained by anticipation of social rewards (for complying with them) and social punishment (for violating them) enacted by the reference group (Alexander-Scott et al., 2016; Cislaghi & Heise, 2018).

Empirical studies demonstrate the role of peer influence in DRV victimization and perpetration, finding that inequitable gender norms (Barter et al., 2009; Shakya et al., 2022; Wesche & Dickson-Gomez, 2019) and social norms supportive of DRV (Foshee et al., 2001; Gage, 2016; Salazar et al., 2018; Vagi et al., 2013) contribute to DRV risk, even when controlling for personal attitudes toward DRV (Foshee et al., 2001; Gage, 2016; Shakya et al., 2022). Interventions to reduce DRV often incorporate strategies to influence the peer social norms that contribute to sustaining this type of abuse (Stanley et al., 2015) but have not assessed social norms as a mediator of intervention effects (Meiksin et al., 2023).

Social Norms Measurement in DRV Research

A recent global systematic review reported on the use and quality of measures assessing social norms about DRV and gender, where measures had been tested for their association with DRV outcomes (Meiksin et al., 2023). None of the 40 descriptive and injunctive measures identified by the review were used in more than one study, and the review identified no evaluations of DRV interventions that explored whether changes in social norms mediated intervention impact (Meiksin et al., 2023). Fewer than one-quarter of included measures had been developed using input from young people (Meiksin et al., 2023) despite evidence suggesting that young people might struggle to distinguish between their own and their friends’ views when responding to survey items (Moreau, 2018; Moreau et al., 2021).

In preparation for the evaluation of a new intervention in England that aimed, in part, to change social norms to reduce DRV (Meiksin 2020), we sought to test candidate social norms measures via cognitive interviews. Cognitive interviewing is a qualitative method for pretesting survey measures by exploring whether survey items function as intended and the cognitive processes participants use to answer these items (Willis & Artino, 2013). The approach allows researchers to identify any problems (Streiner & Norman, 2008) and refine items before administering surveys. The recommended approach for adolescents uses a combination of the think-aloud method and verbal probing (de Leeuw et al., 2002). The former is more open ended, asking participants to narrate their thoughts as they answer survey items (Collins, 2003; Willis, 1999). The latter involves asking specific questions about participants’ experience responding to tested items, allowing the interviewer to explore aspects they suspect might be a source of response error (Collins, 2003; Willis, 1999).

In the present study, we conducted cognitive interviews with adolescents in England to assess the understandability and answerability of candidate measures of social norms relating to DRV and gender and to refine these survey measures based on our findings.

Methods

Study Overview

We conducted cognitive interviews to refine measures used in student surveys administered for Project Respect, a pilot cluster randomized controlled trial of a school-based intervention to reduce DRV in England (Meiksin 2020). Cognitive interviews tested selected survey items from measures of descriptive and injunctive DRV norms, and injunctive gender norms. They also tested survey instructions explaining safeguarding procedures, items on attitudes toward gender roles and stereotypes, items on relationship history, and two DRV measures. Findings from the testing of social norms and attitudes items are the focus of this paper. Ethical approval for this research was granted by the London School of Hygiene and Tropical Medicine Ethics Committee (11986).

Recruitment and Informed Consent

Drawing on our existing networks, we recruited one London state secondary school to take part. We asked school staff to select students of diverse academic ability across years 8, 9, and 10 (aged 13–15 years), based on their overall knowledge of the students, including at least two girls and two boys from each year-group. Students deemed by school staff to be unable to give informed consent due to severe cognitive limitations were not eligible to take part. Due to the sensitive nature of tested items, we recommended that students with known experience of DRV not be selected.

Participants’ parents/carers received information describing the study and could opt their child out of taking part. Before beginning the interview, the researcher reviewed the written informed consent form with participants and explained that responses would be kept confidential except in the case of safeguarding concerns, which would include the following: reports of sexual activity before age 13, ongoing risk of serious harm, or disclosures for which the participant asked the researcher to breach confidentiality. A safeguarding concern arose for one participant, which was reported to the school’s safeguarding officer per our policy. Participants had the opportunity to ask questions before providing consent.

Interviews

Cognitive interviews took place at the participating school during the school day. Lasting around 40 min, interviews tested instructions, survey items, and response options for measures of descriptive and injunctive DRV norms, injunctive gender norms, and (for comparison) attitudes toward gender roles and stereotypes (Table 1). We did not test descriptive gender norms items because we found no appropriate measure in the DRV literature and developing a new measure would require formative research outside the scope of this study.

Table 1.

Original Measures, Tested Items, and Final Measures.

Original measure (tested items in bold, retained items underlined)	Items tested in cognitive interviews	Final measure (tested items in bold)
Attitudes toward DRV ^a	Injunctive DRV norms	Injunctive DRV norms
1. It is OK for a boy to hit his girlfriend if she did something to make him mad. 2. It is OK for a boy to hit his girlfriend if she insulted him in front of friends. 3. Girls sometimes deserve to be hit by the boys they date. 4. A girl who makes her boyfriend jealous on purpose, deserves to be hit 5. Boys sometimes deserve to be hit by the girls they date. 6. Sometimes boys have to hit their girlfriends to get them back under control. 7. It is OK for a boy to hit a girl if she hits him first. 8. It is OK for a girl to hit a boy if he hits her first. 9. If someone hits their boyfriend or girlfriend, their boyfriend or girlfriend should break up with them ROs: Strongly agree, Agree, Disagree, Strongly Disagree	Please tick one box to show how most other students in your school would feel if a student in your school did each of the following: a. A boy hit his girlfriend to get her back under control ROs: Approve, Disapprove, Neither	Please tick a box to show whether your friends would agree or disagree with each statement: a. It is NOT okay for a boy to hit his girlfriend if she did something to make him mad. b. Girls sometimes deserve to be hit by their boyfriends. c. Boys sometimes deserve to be hit by their girlfriends. d. It is okay for a boy to hit a girl if she hit him first. e. It is NOT okay for a girl to hit a boy if he hit her first. f. If someone hits their boyfriend or girlfriend, the boyfriend or girlfriend should break up with them ROs: My friends would agree, My friends would disagree, My friends would neither agree nor disagree
		Tested item removed.
Descriptive DRV norms ^b	Descriptive DRV norms	Descriptive DRV norms
1. How many of your friends have forced someone to have sexual activity with them that caused their partner to cry, scream, plead, hit, or fight back? 2. How many of your friends have used physical force, such as hitting to solve fights with their girlfriends or boyfriends? 3. How many of your friends insult their girlfriend or boyfriend, swear at them, or try to control everything their boyfriend or girlfriend does? ROs: 0 friends, 1–2, 3–5, 6+	Please tick one box on each line to show how many students in your school you think have done each of the following: a. How many boys in your school insult their girlfriend, swear at her, or try to control everything she does? b. How many girls in your school insult their boyfriend, swear at him, or try to control everything he does? ROs: None, Some, Many, Most	Do you have friends who have girlfriends or boyfriends? ROs: Yes, No (if Yes) Please tick a box to show your best guess of how many of your friends have done the following: a. How many of your friends have used physical force, such as hitting, to solve fights with their girlfriend or boyfriend? b. How many of your friends insult or swear at their girlfriend or boyfriend? c. How many of your friends try to control everything their girlfriend or boyfriend does? ROs: None, Some, Many, Most
Attitudes toward gender roles & stereotypes ^c	Attitudes toward gender roles & stereotypes	Attitudes toward gender roles & stereotypes
1. Swearing is worse for a girl than for a boy. 2. On a date, the boy should be expected to pay all expenses. 3. On the average, girls are as smart as boys. 4. More encouragement in a family should be given to sons than daughters to go to college. 5. It is all right for a girl to want to play rough sports like football. 6. In general, the father should have greater authority than the mother in making family decisions. 7. It is all right for a girl to ask a boy out on a date. 8. It is more important for boys than girls to do well in school. 9. If both husband and wife have jobs, the husband should do a share of the housework such as washing dishes and doing the laundry. 10. Boys are better leaders than girls. 11. Girls should be more concerned with becoming good wives and mothers rather than desiring a professional or business career. 12. Girls should have the same freedom as boys. 13. Most girls like to show off their bodies. 14. Most boys like to go out with girls just for sex. 15. Most girls cannot be trusted. 16. It is more accepted for a boy to have many sexual partners than for a girl. ROs: Strongly agree, Agree, Disagree, Strongly disagree	Please tick one box on each line to show how much you personally agree or disagree with each statement. a. On a date, the boy should pay all the expenses. ROs: Strongly agree, Agree, Disagree, Strongly disagree	Please tick a box to show how much you personally agree or disagree with each statement. a. Swearing is worse for a girl than for a boy. b. It is more acceptable for a boy to have a lot of sexual partners than for a girl. c. Most girls can’t be trusted. d. On average, girls are as smart as boys. e. Girls should have the same freedom as boys. ROs: I strongly agree, I agree, I disagree, I strongly disagree
		Tested item removed.
	Injunctive gender norms	Injunctive gender norms
	Please tick one box on each line to show how most other students in your school would feel about each of the following scenarios: a. A girl and a boy go on a date, and the boy pays all the expenses ROs: Approve, Disapprove, Neither Please tick one box on each line to show how most other students in your school would feel about a girl or boy in your school who does each of the following: a. A girl in your school who has a lot of sex partners. ROs: Approve, Disapprove, Neither	Please tick a box to show whether your friends would agree or disagree with each statement. a. Swearing is worse for a girl than for a boy. b. It is more acceptable for a boy to have a lot of sexual partners than for a girl. c. Most girls can’t be trusted. d. On average, girls are as smart as boys. e. Girls should have the same freedom as boys. ROs: My friends would agree, My friends would disagree, My friends would neither agree nor disagree
		Second tested item removed.

Note: DRV = dating and relationship violence; ROs = response options.

Items adapted from Foshee et al. (2001).

Items adapted from Cook-Craig et al. (2014).

Items adapted from Sotiriou et al. (2011).

After each participant self-completed a brief demographic questionnaire, the interviewer explained that the participant would be asked to “think aloud,” describing their thought process as they responded to each tested item. To practice carrying out this process, participants completed an exercise adapted from Willis (1999) which instructed them to “try to imagine your home, and think about how many windows there are in it. As you count up the windows, tell me what you are seeing and thinking about” (Willis, 1999, p. 4). Interviews then proceeded using a combination of the think-aloud and verbal probing approaches (de Leeuw et al., 2002). Participants were asked to think aloud as they answered tested items, which were displayed on show cards as they would appear on a survey. Verbal probes explored: alternative reference groups for norms items (i.e., the participant’s friends; or their friends in the school); how easy/difficult items were to answer; understanding of terminology; alternative terminology (i.e., how the participant would phrase the question to their friends); and experiences of answering attitudes versus social norms items. The interviewer (RM) used a laptop to type detailed notes on participants’ responses during interviews (Willis, 1999).

Measures

The demographic questionnaire asked for participants’ age, year-group, ethnic group (White British; any other White background; Asian or Asian British; Black, African, Caribbean, or Black British; Mixed/multiple ethnic background; or any other ethnic group), sex assigned at birth, gender (male; female; transgender male; transgender female; or do not identify as male, female, or transgender), and religious group (none; Christian; Jewish; Muslim/Islam; Hindu; Buddhist; Sikh; I don’t know/not sure; other religious group).

Injunctive DRV Norms

We developed an item measuring injunctive DRV norms (see Table 1) based on a scale assessing attitudes toward DRV which was used with adequate reliability (Lewis et al., 2015) of α = .69 in a trial of the Safe Dates DRV intervention (Foshee et al., 2001). The new measure instructed participants to indicate the views of “most other students in your school” on a series of behaviors attributed to students at the school, assessing norms at the site of intervention. We adapted the item “Sometimes boys have to hit their girlfriends to get them back under control” to ask about injunctive norms governing this behavior. We simplified response options from four levels of agreement (“strongly agree,” “agree,” “disagree,” and “strongly disagree”) to “approve,” “disapprove,” and “neither” (Cislaghi, 2016).

Descriptive DRV Norms

We adapted the descriptive DRV norms item “How many of your friends insult their girlfriend or boyfriend, swear at them, or try to control everything their boyfriend or girlfriend does?” to create two items concerning psychological DRV, complementing the injunctive norms item concerning physical DRV (see Table 1). The original item was drawn from a descriptive norms measure used with good reliability (Lewis et al., 2015) of α = .70 in a trial of Green Dot, a DRV and sexual violence intervention (Cook-Craig et al., 2014). We simplified response options from asking for the number of people to four options: “none,” “some,” “many,” or “most” (Cislaghi, 2016). We changed the reference group from “your friends” to “girls in your school” and (in a separate item) “boys in your school,” assessing norms at the site of intervention and separately by gender given that reported rates of DRV can differ between girls and boys (Barter et al., 2014; Leen et al., 2013). While evidence suggests that DRV rates might be higher among gender minorities as compared to cisgender young people (Dietz, 2019), we restricted this item to the two gender reference groups used in existing valid and reliable DRV descriptive norms measures (Meiksin et al., 2023).

Attitudes Toward Gender Roles and Stereotypes

We adapted an item from the 16-item Attitudes Toward Women Scale (ATWS), a measure of attitudes toward gender roles and stereotypes that combines items from previous measures (Sotiriou et al., 2011) and was used with excellent reliability (Lewis et al., 2015) of α = .82 in a 2011 study in Greece (see Table 1) (Sotiriou et al., 2011). We identified this measure via an ad hoc search for relevant measures used with good reliability in gender-based violence research among adolescents within the previous decade. We selected an item to test that concerned gender roles in dating, and instructed participants to indicate “how much you personally agree or disagree.” We simplified language from “On a date, the boy should be expected to pay all expenses” to “On a date, the boy should pay all the expenses.”

Injunctive Gender Norms

We adapted two items from the ATWS (Sotiriou et al., 2011) to develop injunctive gender norms items asking participants to indicate the views of “most other students in your school” on a series of behaviors and scenarios (see Table 1). The first item assessed norms governing sexual behavior. To simplify language and focus on norms at the site of the intervention, we adapted the original item (“It is more accepted for a boy to have many sexual partners than for a girl”) to ask about “a girl in your school who has a lot of sex partners.” The second item was paired with the tested item on attitudes toward gender roles and stereotypes, allowing for comparison between responses about participants’ own and others’ views on the same behavior. We simplified response options for both items from four levels of agreement to “approve,” “disapprove,” and “neither” (Cislaghi, 2016).

Analysis

The interviewer took detailed notes on each participant’s response to each interview question and probe during the interview (Willis, 1999) and, after reading and re-reading these notes after data collection, produced written summaries of the results for each question and probe by participant year-group and then overall sample (Willis & Artino, 2013). Summaries detailed both “dominant trends” and “discoveries” (i.e., problems might be significant despite arising rarely) (Willis, 1999, p. 28) and differences and similarities by gender and year-group. The detailed notes and written summaries were then subjected to thematic analysis (Green & Thorogood, 2018). Informed by the notion of constant comparison (Green & Thorogood, 2018), data and codes were compared throughout the analysis process, and newly emerging codes were applied to the full dataset.

When responding to a survey item, a survey participant must comprehend the question, retrieve information from long-term memory, make a judgment about how to answer, and then select from among the response options provided (Collins, 2003). Drawing on Young et al.’s work developing survey measures for young people (Young et al., 2016), we conceptualized these processes as falling within two distinct analytic categories: understandability (encompassing comprehension) and answerability (encompassing retrieval, judgment, and response). Individual codes were developed inductively under the headings of “understandability” and “answerability,” with two sets of such codes: one applied to data on social norms items and the other (for comparison) applied to data on attitude items. Interview data were coded for evidence of good or poor understandability and answerability and for aspects of the tested items that enhanced or detracted from understandability and answerability. Axial coding drew together initial codes relating to the same themes—for example, the role of the framing of the reference group or observed manifestations of social norms—facilitating analysis within these themes.

Analysis of written summaries provided an overview of our findings and facilitated comparison by gender and year-group. Further analysis of notes on individual interviews identified the evidence supporting overall findings.

Results

In all, 11 students took part in cognitive testing of social norms and attitudes items (Table 2). All were cisgender comprising seven girls and four boys. Participants were spread across year-groups with three in year 8 (age 13), five in year 9 (ages 13–14), and three in year 10 (ages 14–15). All but one identified as White British and all but two selected “none” for a religious group. Injunctive gender norms items were skipped with one participant, who did not reach these items before having to return to class; all other items were tested with all 11 participants.

Table 2.

Background Characteristics of Cognitive Interview Participants.

Characteristics	Year 8 N	Year 9 N	Year 10 N	Total N (%)
Gender
Girls	2	3	2	7 (63.6)
Boys	1	2	1	4 (33.4)
Age in years
13	3	1	0	4 (33.4)
14	0	4	2	6 (54.5)
15	0	0	1	1 (9.1)
Ethnicity
Black African, Caribbean, or Black British	1	0	0	1 (9.1)
White British	2	5	3	10 (90.9)
Religion
Christian	1	0	1	2 (18.2)
None	2	5	2	9 (81.8)
Total N (%)	3 (27.3)	5 (45.5)	3 (27.3)	11 (100.0)

Summary results relating to understandability and answerability of each tested measure, and refinements made based on these findings, are available in Supplemental Appendix A.

Understandability

Item Clarity

There was some initial difficulty with understanding the intended meaning of all three injunctive norms measures for some participants. Rather than difficulty with specific terms or phrases, some confusion appeared to stem from the framing of the items which, when the instructions and item were read aloud together, were somewhat lengthy (see Table 1). When presented with injunctive norms measures, participants from all year-groups often asked whether the item was asking for their own or others’ views, or answered initially in terms of their own views. In an example of the former, one boy asked the interviewer to clarify whether the injunctive DRV norms measure was asking for his views, those of other boys in the school or those of girls. In an example of the latter, in response to the item assessing injunctive gender norms relating to a girl with many sex partners, a girl said, “If it was my friend I’d disapprove, but if it was someone I didn’t know, I wouldn’t care.” Similarly, another girl described the measure of injunctive DRV norms as easy to answer because “I just think boys shouldn’t hit girls,” suggesting that she had interpreted the item as assessing her own views on DRV.

These findings suggest that injunctive norms items tended not to work well in their tested form, as their meanings were often not initially clear to participants. Where this was the case, the interviewer explained the intended meaning of the item, including (where needed) explaining that the question was asking about the participant’s perception of others’ views. These explanations were effective in clarifying item meaning, suggesting that for participants who had difficulty with understandability, this reflected a lack of clarity of wording rather than a more fundamental inability to distinguish injunctive norms from personal views. Students’ ability to make this distinction was especially apparent in responses to the parallel items (see Table 1) that explored both personal attitudes and injunctive social norms regarding the gendered behavior of a boy paying the expenses on a date with a girl. In their responses, two girls highlighted where they personally disagreed with others’ views (as they perceived them).

Answerability

Level of Certainty About Others’ Behavior and Views

Participants tended to have difficulty responding to the measure of descriptive DRV norms because they were uncertain about the prevalence of psychological DRV perpetration among their peers. Some qualified their answers; for example, participants added “that I know of” and one of these participants also specified that they were responding with estimates among people whom they knew. Furthermore, asking about multiple behaviors within the same item detracted from answerability. For example, one girl commented that some boys might swear at their partner but would not necessarily insult or try to control her. Contributing to this uncertainty was that psychological DRV perpetration might be unobservable. As one boy explained, some might try to control their partner due to jealousy but he did not think they would “broadcast” this behavior because people would disapprove and the person would feel embarrassed by others’ disapproval.

Overall, participants tended to report that they could respond more easily to measures of attitudes than to measures of injunctive norms. They explained that they knew their own mind better while imagining what others thought was more difficult. The level of difficulty in answering injunctive norms items varied based on the specified reference group and on the observability of social sanctions for, and on the strength of, the assessed norm, as described below.

Reference group

The reference group for injunctive norms measures, “most other students in your school,” brought to mind a range of different groups for participants. A few said they thought of their friends when responding to these items; others reported thinking of older students or their own year-group. Some reported thinking of other students of the same gender, including older or popular boys. Our data suggest that responses to injunctive norms items would differ depending on the gender of the reference group students had in mind. For example, regarding an injunctive DRV norms item, one girl responded, “I know a lot of the girls would disapprove. I think it depends on who the boy’s friends are. . ..” A gendered distinction arose also for the descriptive norms items, where levels of perceived DRV differed for items asking about perpetration by girls and by boys.

When asked about changing the reference group to “your friends,” some participants said this could make some norms items easier to answer. This change tended to improve the answerability of the injunctive DRV norms measure, with one boy suggesting that this was because he would be more likely to know the views of his friends than views among the broader reference group. However, it made less of a difference to the answerability of the measure of injunctive gender norms relating to paying on a date because the absence of a strong norm governing this behavior also detracted from that item’s answerability.

Observability of Behaviors and Social Sanctions

Answerability was improved where participants could draw concretely on past observations and conversations to respond to social norms items. For example, in discussing how she arrived at her response to the descriptive DRV norms item, a girl recalled seeing a boy screaming at another girl because she had thrown away a ring he had given to her. Discussing how they became aware of injunctive gender norms governing sexual behavior, participants described the public visibility of social sanctions. For example, one girl reported that a boy with many sexual partners would be high-fived while a girl would be called a “slag.” Similarly, a boy explained that “all the students in the school” would talk judgmentally with their friends about a student with many sexual partners or when a nude image of a student was circulated.

Presence of a Strong Norm

Once the intention to assess others’ views had been clarified where needed, participants answered the measure assessing injunctive gender norms relating to a girl with many sexual partners more easily and confidently than they did other social norms measures. Participants were able to describe social repercussions for violating this norm, suggesting that the item taps a norm that is strong in the reference group and they could thus easily draw on examples of observed behavior related to this norm. By contrast, participants tended to have more difficulty responding to the injunctive gender norms item assessing expectations of who should pay on a date between a girl and a boy. While some answered this item with little apparent difficulty, participants often expressed some uncertainty about their response, for example using words like “maybe” or “probably” or describing variable views among the reference group.

Responses to the parallel measure of attitudes provided further insight. For several participants, the response depended on context, for example, who had paid last time, the cost of the bill, or whether this was a first date. Responses to these paired attitude and injunctive norms items suggest that the injunctive norms measure did not tap a strong social norm among this population.

Measure Refinements

Based on our cognitive interview findings, we made a number of refinements to social norms and attitude measures in preparation for piloting. To address variability in who the reference group “most other students in your school” brought to mind, and difficulty reporting perceived views of this reference group, all social norms measures were adapted to ask about a reference group of “your friends.” To improve clarity and readability, we simplified the instructions for all measures and made minor changes to wording to reflect common parlance in England. To clarify that injunctive norms items ask about the views of others, we simplified the measures’ instructions and adapted items and response options to mirror corresponding attitudes measures (i.e., “Please tick a box to show whether your friends would agree or disagree with each statement”). Refined survey measures thus asked about one’s own and others’ views on the same behaviors and presented similar Likert scale response options that reinforced the perspective in question (e.g., “I agree” for attitudes items and “My friends would agree” for injunctive norms items). While attitudes items had four response options (two levels of agreement and two levels of disagreement), refined injunctive norms measures had three (agreement, disagreement, or neither) to improve answerability and to accommodate the possibility of items representing weak or absent norms. We also removed three items from the injunctive DRV norms measure (and corresponding attitudes measure) that specified a rationale for DRV (e.g., “A girl who makes her boyfriend jealous on purpose deserves to be hit”), which we judged to be less readable than other items and more difficult to respond to from the perspective of a reference group; and reframed two items from pro- to anti-DRV statements to ensure a mixture of statements supporting and opposing DRV.

For the descriptive DRV norms measure, we added instructions to “show your best guess” and we added a filter question so that only participants reporting more than one friend with a partner would be routed to these items. Based on findings that items assessing behaviors that were more likely to be observed were easier to answer and that assessing multiple behaviors within one item reduced answerability, we removed the item about sexual DRV and separated items on controlling behavior and insulting/swearing at a partner. We also adapted descriptive DRV norms items to be gender-neutral, more closely reflecting the original measure (see Table 1) and enabling us to ask about a more meaningful reference group (friends with partners, as opposed to smaller groups specifying female friends with boyfriends and male friends with girlfriends) while reducing the number of items in the measure. Attitude and injunctive gender norms items relating to who should pay on a date were dropped. To reduce the length of the injunctive gender norms measure and the corresponding attitudes measure, we removed items about gender roles among adults (father/mother, husband/wife) and items we judged to be less likely to represent strong norms among young people in England.

Discussion

Our findings suggest that participants were able to understand both descriptive and injunctive norms items and distinguish between the latter and their own views. Some participants showed initial confusion about whether injunctive norms items were asking for their own views or the views of others, suggesting that the wording of tested measures should be refined to improve clarity. To this end, injunctive norms measures were adapted to mirror attitude measures, that is, to ask about the same behaviors using similar Likert scale response options, a format used in other research with young people (Shamu et al., 2016).

We found that inconsistency in who the reference group “most other students in your school” brought to mind, lack of certainty about DRV behaviors, and perceived heterogeneity in views among this population, detracted from the answerability of norms items. Our data suggest that narrowing the reference group to “your friends” might improve answerability where norms are salient among this reference group and where the behaviors in question are likely to have been discussed or observed. The use of this narrower reference group is supported by evidence from a recent systematic review of social norms measures in DRV research showing that for nearly all included measures, DRV supportive norms among friends were associated with young people’s own experience of DRV (Meiksin et al., 2023).

Research with young people finds that levels of support for DRV (Pöllänen et al., 2018) and DRV prevalence (Barter et al., 2014; Leen et al., 2013) can both vary by gender and that girls tend to report less support for inequitable gender norms (Kågesten et al., 2016) than do boys. Findings from our study suggest that young people are sensitive to these differences among their peers: participants consider the gender of reference group members in their responses to norms items. Where they thought that the views of girls and boys differed, this detracted from the answerability of tested injunctive norms items. However, repeating all norms measures for reference groups of girls and boys separately could result in lengthy scales that would be unfeasible to include in surveys. Narrowing the reference group to “your friends” allows participants to bring to mind a smaller social group. Peers with whom young people identify or feel connected (such as friends) can be particularly influential (Wolfe & Temple, 2018). While the majority of adolescents’ friends are those of the same sex (Deutsch et al., 2014), this approach also allows for individual variation in the gender composition of the reference group while minimizing the number of measure items. Piloting these measures among a representative sample of young people in England will provide important information about their acceptability, reliability, and validity.

During the development of gender norms measures for the Global Early Adolescent Study, participants aged 10 to 14 years were surveyed about their own attitudes and those of their friends toward the same gender norms (Moreau, 2018). The study found that many reported not knowing what their friends thought and that, overall, participants tended to report their friends’ views as very similar to their own, raising concerns about whether data collected via surveys can distinguish between these two concepts in this age group (Moreau, 2018; Moreau et al., 2021). The present study builds on these findings by identifying features of norms measures that improve answerability. We found that participants were most easily and confidently able to respond where norms appeared relatively strong and where they could draw on concrete experiences of seeing norms on public display; that is, where they had discussed or seen the specified behaviors or where they had observed social rewards/repercussions for complying with/violating injunctive norms.

As with any research on social norms important to a particular health outcome, norms measures in DRV research should focus on social norms that are linked theoretically or empirically to DRV outcomes. Based on our findings, we recommend that decisions about which social norms items to include in DRV research should be based on local formative research identifying norms (a) held among a cohesive and influential reference group; (b) strong enough among the reference group for respondents to discern; and (c) for which the relevant behavior (for descriptive norms) or social sanctioning (injunctive norms) is discussed or directly observable. Measures should be worded as clearly and concisely as possible and, where surveys include corresponding attitudinal measures, researchers should consider using parallel formatting, items, and response options for both types of measures to improve the understandability of injunctive norms measures.

Limitations

As the interviewer took notes during cognitive interviews, it is possible that some nonverbal signs of participants’ confusion or tentativeness could have been missed.

Our data come from a sample of 11 participants aged 13 to 15 years recruited from one school in England. While our sample included cisgender girls and boys, it was not diverse by ethnicity, religion, or other gender identities and no data were available on sexual orientation. In addition, given time constraints on the length of the interviews as well as the early stage of our work to refine social norms measures, only two descriptive norms items and three injunctive norms items were tested and this did not include testing of refined items. We were, therefore, unable to test directly whether refinements improved understandability and answerability, including assessing whether the gender-neutral framing of the reference group “your friends” detracts from answerability due to the gendered nature of the tested items.

Conclusions

Our findings suggest that it is possible to develop social norms measures about gender and DRV that are understandable and answerable for young people aged 13 to 15 years in England. Future research should cognitively test a broader range of items, including those assessing norms suspected to be more and less publicly manifest, and should do so among a sample of young people that is diverse in terms of backgrounds and sexual and gender identities. The acceptability, reliability, and validity of the social norms measures refined through cognitive testing in the present study should be assessed among a representative sample of young people in England. Where new social norms measures are developed or existing measures adapted for DRV research, these should assess norms which are salient and publicly manifest among a cohesive, influential reference group. Careful consideration is needed to establish the value of including measures of social norms where this is not known to be the case.

Supplemental Material

sj-docx-1-jiv-10.1177_08862605231204561 – Supplemental material for Assessing Survey Items on Social Norms Relating to Dating and Relationship Violence and to Gender: Cognitive Interviews with Young People in England

Supplemental material, sj-docx-1-jiv-10.1177_08862605231204561 for Assessing Survey Items on Social Norms Relating to Dating and Relationship Violence and to Gender: Cognitive Interviews with Young People in England by Rebecca Meiksin, Ruth Ponsford, Nambusi Kyegombe, Anjalee Kohli and Chris Bonell in Journal of Interpersonal Violence

Footnotes

Acknowledgements

The authors gratefully acknowledge the young people who generously gave their time and shared their views for this study, the staff who warmly welcomed us into their school and facilitated data collection, and Ben Pelhan for his invaluable support.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interests with respect to the authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This study was funded by the NIHR Public Health Research Board (project 15/03/09). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. The funder was not involved in study design; collection, analysis, or interpretation of data; the writing of this article; or the decision to submit this article for publication.

ORCID iD

Rebecca Meiksin

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Rebecca Meiksin, MPH, is a Research Fellow in Social Science at the London School of Hygiene & Tropical Medicine. Her research focuses on sexual and reproductive health and on the role of social and gender norms in the prevention of violence in young people’s dating and relationships.

Ruth Ponsford, PhD, is an Assistant Professor in School Health Intervention Research at the London School of Hygiene & Tropical Medicine. Her research focuses on the health and well-being of young people and the role of school in promoting health, including evaluations of sexual health interventions for young people in England.

Nambusi Kyegombe, PhD, is the Head of Social Sciences at the MRC/UVRI & LSHTM Uganda Research Unit. Her work on adolescent health focuses on low- and middle-income country settings. This includes formative research on the context in which health is located, methodological research to improve measurement, and research on the conduct of high quality, ethical research, particularly among marginalized populations.

Anjalee Kohli, PhD, is an Assistant Professor-adjunct at Georgetown University and an independent consultant. Through participatory and mixed methods research and learning partnerships, Dr. Kohli advances the evidence and practice to prevent violence against women and children, to improve gender equity and sexual and reproductive health.

Professor Chris Bonell, PhD, is a Professor of Public Health Sociology at the London School of Hygiene & Tropical Medicine. His research on adolescent health focuses on how schools and school-based interventions can benefit or harm young people’s health. His research on sexual health examines the broader social determinants of adolescent sexual health, and interventions to address these.

References

Alexander-Scott

Bell

Holden

(2016). Shifting social norms to tackle violence against women and girls (VAWG). VAWG Helpdesk.

Barter

McCarry

Berridge

Evans

(2009). Partner exploitation and violence in teenage intimate relationships. NSPCC.

Barter

Stanley

(2016). Inter-personal violence and abuse in adolescent intimate relationships: Mental health impact and implications for practice. International Review of Psychiatry, 28(5), 485–503. https://doi.org/10.1080/09540261.2016.1215295

Bonell

Blakemore

S.-J.

Fletcher

Patton

(2019). Role theory of schools and adolescent health. The Lancet Child & Adolescent Health, 3(10), 742–748. https://doi.org/10.1016/S2352-4642(19)30183-X

Barter

Aghtaie

Wood

Stanley

Pavlou

Apostolov

, et al. (2014). Safeguarding teenage intimate relationships (STIR): Connecting online and offline contexts and risks Briefing Paper 2: Incidence Rates and Impact of Experiencing Interpersonal Violence and Abuse in Young People’s Relationships. (2015). University of Bristol.

Castellví

Miranda-Mendizábal

Parés-Badell

Almenara

Alonso

Blasco

M. J.

Cebrià

Gabilondo

Gili

Lagares

Piqueras

J. A.

Roca

Rodríguez-Marín

Rodríguez-Jimenez

Soto-Sanz

Alonso

(2017). Exposure to violence, a risk for suicide in youths and young adults. A meta-analysis of longitudinal studies. Acta Psychiatrica Scandinavica, 135(3), 195–211. https://doi.org/10.1111/acps.12679

Cislaghi

(2016). LSHTM group on social norms and GBV: Strategies to diagnose and measure social norms related to gender-based violence: Key lessons from the Baltimore working meeting. STRIVE.

Cislaghi

Heise

(2018). Four avenues of normative influence: A research agenda for health promotion in low and mid-income countries. Health Psychology, 37(6), 562–573. https://doi.org/10.1037/hea0000618

Collins

(2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 12(3), 229–238. https://doi.org/10.1023/a:1023254226592

10.

Cook-Craig

P. G.

Coker

A. L.

Clear

E. R.

Garcia

L. S.

Bush

H. M.

Brancato

C. J.

Williams

C. M.

Fisher

B. S.

(2014). Challenge and opportunity in evaluating a diffusion-based active bystanding prevention program: green dot in high schools. Violence Against Women, 20(10), 1179–1202. https://doi.org/10.1177/1077801214551288

11.

Costenbader

Cislaghi

Clark

C. J.

Hinson

Lenzi

McCarraher

D. R.

McLarnon-Silk

Pulerwitz

Shaw

Stefanik

(2019). Social norms measurement: Catching up with programs and moving the field forward. The Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, 64(4S), S4–S6. https://doi.org/10.1016/j.jadohealth.2019.01.001

12.

de Leeuw

Borgers

Strijbos-Smits

. (2002, November). Children as respondents: Developing, evaluating, and testing questionnaires for children. In: International conference in questionnaire development evaluation and testing methods. Charleton, SC.

13.

Deutsch

A. R.

Steinley

Slutske

W. S.

(2014). The role of gender and friends’ gender on peer socialization of adolescent drinking: A prospective multilevel social network analysis. Journal of Youth and Adolescence, 43(9), 1421–1435. https://doi.org/10.1007/s10964-013-0048-9

14.

Dietz

(2019). Dating and relationship violence in sexual and gender minority youth: A systematic review of the literature and cost-effectiveness analysis of a targeted school-based preventative intervention. London School of Hygiene & Tropical Medicine.

15.

Exner-Cortens

Eckenrode

Rothman

(2013). Longitudinal associations between teen dating violence victimization and adverse health outcomes. Pediatrics, 131(1), 71–78. https://doi.org/10.1542/peds.2012-1029

16.

Foshee

V. A.

Linder

MacDougall

J. E.

Bangdiwala

(2001). Gender differences in the longitudinal predictors of adolescent dating violence. Preventive Medicine, 32(2), 128–141. https://doi.org/10.1006/pmed.2000.0793

17.

Foshee

V. A.

Linder

G. F.

Bauman

K. E.

Langwick

S. A.

Arriaga

X. B.

Heath

J. L.

McMahon

P. M.

Bangdiwala

(1996). The safe dates project: Theoretical basis, evaluation design, and selected baseline findings. American Journal of Preventive Medicine, 12(5 Suppl), 39–47.

18.

Gage

A. J.

(2016). Exposure to spousal violence in the family, attitudes and dating violence perpetration among high school students in Port-au-Prince. Journal of Interpersonal Violence, 31(14), 2445–2474. https://doi.org/10.1177/0886260515576971

19.

Green

Thorogood

(2018). Qualitative methods for health research. SAGE.

20.

Kågesten

Gibbs

Blum

R. W.

Moreau

Chandra-Mouli

Herbert

Amin

(2016). Understanding factors that shape gender attitudes in early adolescence globally: A mixed-methods systematic review. PLoS One, 11(6), e0157805. https://doi.org/10.1371/journal.pone.0157805

21.

Leen

Sorbring

Mawer

Holdsworth

Helsing

Bowen

(2013). Prevalence, dynamic risk factors and the efficacy of primary interventions for adolescent dating violence: An international review. Aggression and Violent Behavior, 18(1), 159–174. https://doi.org/10.1016/j.avb.2012.11.015

22.

Lewis

C. C.

Stanick

C. F.

Martinez

R. G.

Weiner

B. J.

Kim

Barwick

Comtois

K. A.

(2015). The society for implementation research collaboration instrument review project: A methodology to promote rigorous evaluation. Implementation Science, 10(1), 1–18. https://doi.org/10.1186/s13012-014-0193-x

23.

Meiksin

Bonell

Bhatia

Melendez-Torres

G. J.

Kyegombe

Kohli

(2023). Social norms about dating and relationship violence and gender among adolescents: Systematic review of measures used in dating and relationship violence research. Trauma, Violence, & Abuse, 152483802311555. https://doi.org/10.1177/15248380231155526

24.

Meiksin

Crichton

Dodd

Morgan

G. S.

Williams

Willmott

Allen

Tilouche

Sturgess

Morris

Barter

Young

Melendez-Torres

Taylor

Reyes

H. L. M.

Elbourne

Sweeting

Hunt

Ponsford

Bonell

(2020). A school intervention for 13- to 15-year-olds to prevent dating and relationship violence: The Project Respect pilot cluster RCT. Public Health Research, 8(5), 1–338. https://doi.org/10.3310/phr08050

25.

Moreau

(2018, February 1). Development and validation of cross-cultural gender norms scales for early adolescents [Webinar]. http://www.geastudy.org/webinars/

26.

Moreau

Ahmed

Zuo

Cislaghi

(2021). Assessing the spectrum of gender norms perceptions in early adolescence: A cross-cultural analysis of the global early adolescent study. Journal of Adolescent Health, 69(1), S16–S22. https://doi.org/10.1016/j.jadohealth.2021.03.010

27.

Pöllänen

de Vries

Mathews

Schneider

de Vries

P. J.

(2018). Beliefs about sexual intimate partner violence perpetration among adolescents in South Africa. Journal of Interpersonal Violence, 36(3–4), NP2056-2078NP. https://doi.org/10.1177/0886260518756114

28.

Roberts

T. A.

Klein

J. D.

Fisher

(2003). Longitudinal effect of intimate partner abuse on high-risk behavior among adolescents. Archives of Pediatrics & Adolescent Medicine, 157(9), 875. https://doi.org/10.1001/archpedi.157.9.875

29.

Salazar

L. F.

Swartout

K. M.

Swahn

M. H.

Bellis

A. L.

Carney

Vagi

K. J.

Lokey

(2018). Precollege sexual violence perpetration and associated risk and protective factors among male college freshmen in Georgia. Journal of Adolescent Health, 62(3), S51–S57. https://doi.org/10.1016/j.jadohealth.2017.09.028

30.

Shakya

H. B.

Cislaghi

Fleming

Levtov

R. G.

Boyce

S. C.

Raj

Silverman

J. G.

(2022). Associations of attitudes and social norms with experiences of intimate partner violence among married adolescents and their husbands in rural Niger: A dyadic cross-sectional study. BMC Women’s Health, 22(1), 180. https://doi.org/10.1186/s12905-022-01724-y

31.

Shamu

Gevers

Mahlangu

B. P.

Jama Shai

P. N.

Chirwa

E. D.

Jewkes

R. K.

(2016). Prevalence and risk factors for intimate partner violence among Grade 8 learners in urban South Africa: Baseline analysis from the Skhokho Supporting Success cluster randomised controlled trial. International Health, 8(1), 18–26. https://doi.org/10.1093/inthealth/ihv068

32.

Sotiriou

Ntinapogias

Petroulaki

(2011). Attitudes on gender stereotypes and gender-based violence among youth. Country report: Greece (Daphne III Programme: Youth4Youth: Empowering Young People in Preventing Gender-Based Violence through Peer Education). European Anti-Violence Network.

33.

Spear

L. P.

(2000). The adolescent brain and age-related behavioral manifestations. Neuroscience & Biobehavioral Reviews, 24(4), 417–463. https://doi.org/10.1016/S0149-7634(00)00014-2

34.

Stanley

Ellis

Farrelly

Hollinghurst

Downe

(2015). Preventing domestic abuse for children and young people: A review of school-based interventions. Children and Youth Services Review, 59, 120–131. https://doi.org/10.1016/j.childyouth.2015.10.018

35.

Streiner

D. L.

Norman

G. R.

(2008). Health measurement scales: A practical guide to their development and use (4th ed.). Oxford University Press.

36.

Vagi

K. J.

Rothman

E. F.

Latzman

N. E.

Tharp

A. T.

Hall

D. M.

Breiding

M. J.

(2013). Beyond correlates: A review of risk and protective factors for adolescent dating violence perpetration. Journal of Youth and Adolescence, 42(4), 633–649. https://doi.org/10.1007/s10964-013-9907-7

37.

Wesche

Dickson-Gomez

(2019). Gender attitudes, sexual risk, intimate partner violence, and coercive sex among adolescent gang members. Journal of Adolescent Health, 64(5), 648–656. https://doi.org/10.1016/j.jadohealth.2018.10.292

38.

Willis

G. B.

(1999). Cognitive interviewing, A “How To” guide. Research Triangle Institute.

39.

Willis

G. B.

Artino

A. R.

(2013). What do our respondents think we’re asking? Using cognitive interviewing to improve medical education surveys. Journal of Graduate Medical Education, 5(3), 353–356. https://doi.org/10.4300/JGME-D-13-00154.1

40.

Wolfe

D. A.

Temple

J. R.

(Eds.). (2018). Adolescent dating violence: Theory, research, and prevention. Academic Press.

41.

Young

Költő

Reis

Saewyc

E. M.

Moreau

Burke

Cosma

Windlin

Gabhainn

S. N.

Godeau

(2016). Sexual Health questions included in the Health Behaviour in School-aged Children (HBSC) Study: An international methodological pilot investigation. BMC Medical Research Methodology, 16(1), 169. https://doi.org/10.1186/s12874-016-0270-8

42.

Young

Turney

White

Bonell

Lewis

Fletcher

(2017). Dating and relationship violence among 16–19 year olds in England and Wales: A cross-sectional study of victimization. Journal of Public Health (Oxford, England), 40(4), 738–746. https://doi.org/10.1093/pubmed/fdx139

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB