Abstract
Violent ideations (VIs) have potential significance across clinical, forensic, and research contexts. They feature in dominant theories of violence, are a candidate risk factor in violence prediction, and are a potential target for therapeutic intervention. Given this, there is a need for multi-item psychometrically supported measures of VIs. We report on the development and validation of the “Violent Ideations Scale” (VIS): a brief measure of VIs. In a normative sample of N = 1,276 older adolescents, we evaluated the dimensionality, sex invariance, concurrent validity, and discriminative power of the VIS. The VIS showed unidimensionality, minor measurement differences across males and females, correlated well with a preexisting measure of VIs and showed a strong relation to criminal violence. These features support the use of the VIS as a research tool and as a possible source of information regarding violence risk in clinical and forensic settings.
Violent ideations (VIs) can be defined as thoughts, daydreams, or fantasies of inflicting harm on another individual. VIs play a central role in several theories of violence and aggression, prospectively predict violent behavior, and could represent a target of intervention for violence reduction. VI research thus has implications of potential theoretical and practical import; however, its robustness depends on valid and reliable VI measurement. Despite the potential relevance of VIs as a core element of the neurocognitive and emotional processes associated with violence aggression, there is currently no extensively validated instrument to measure the prevalence and intensity of VIs in normative, clinical, or forensic populations. In this study, we, therefore, report on the development and validation of the “Violent Ideations Scale (VIS)”: a multi-item instrument designed to provide a brief but psychometrically supported assessment of VIs.
VIs feature prominently in a number of psychological theories concerned with ultimate- and proximate-level explanations of violence. They play an important role in evolutionary theories of violence (Duntley & Buss, 2011; Eisner, 2009); the “cathartic” model (J. Murray & Feshbach, 1978); social learning theories (e.g., Huesmann, 1998), more recently incorporated into the general aggression model (e.g., DeWall, Anderson, & Bushman, 2011); judgment and decision-making models (e.g., Van Gelder, 2013); theories focused on the role of self-control in aggression (Denson, Pedersen, Friese, Hahm, & Roberts, 2011); and are recognized as relevant to violence theory even by critics of these models (e.g., Durrant, 2009; Ferguson & Dyck, 2012). Reliable and valid measurement of VIs is, therefore, critical to violence theory development and testing. This is especially so, given the number of outstanding questions regarding, for example, their triggers, stability, relation to psychopathology, and the features of the person and situation with which they may interact to influence violent behavior.
VIs are important as candidate indicators for future violence risk, which is particularly significant in forensic and clinical settings. Commensurate with the potential consequences of inaccurate judgments, risk assessments carry important legal and clinical implications (Gellerman & Suddath, 2005). There is, thus, considerable interest in developing tools that can accurately predict whether an individual is likely to commit a violent act. In one of only the handful of studies to consider VIs as a prospective risk factor, their prospective correlation with the occurrence of a violent act in the 20 weeks following discharge from a mental health hospital setting was on a par with recent violence (r = .13 for VIs and r = .14 for recent violence; Monahan et al., 2000). As such, a VI measure was incorporated into the multiple iterative classification tree model (Monahan et al., 2005) for violence risk assessment. Optimization of VI measurement through the development and evaluation of psychometrically sophisticated instruments has the potential to capitalize on this predictive relation and improve the limited predictive accuracy of currently available risk assessments (Yang, Wong, & Coid, 2010).
VIs—should they prove to be causally linked to violence—could also represent a target for intervention for violence reduction or, in patients for whom VIs are a source of distress, their management could represent an important outcome in its own right. For example, VIs can be considered a special case of the kinds of cognitions already targeted by psychological therapies such as cognitive behavioral therapy. VI interventions could be incorporated into this and related therapeutic frameworks (e.g., as noted by Nagtegaal, Rassin, & Muris, 2006) analogous to the existing cognitive behavioral therapy interventions to treat sex offending that include a component addressing deviant sexual ideations (Beech, Beckett, & Fisher, 1998). Proof of principle for VI-based intervention was provided by a small pilot study within a therapeutic community which provided promising preliminary results (Akerman, 2008), however, much work remains to be done. The emerging area of evaluating therapeutic interventions involving VIs is another in which psychometrically robust measures of VIs will be critical to facilitate the detection or lack thereof of therapeutic change.
Previous studies involving VIs have tended to rely on single item measures, which have limited breadth and reliability. Few multi-item inventories of VI are currently available and the most commonly utilized—the Schedule of Imagined Violence (SIV; Grisso, Davis, Vesselinov, Appelbaum, & Monahan, 2000)—is not used in a truly multi-item manner because each item is designed to be used in isolation. The SIV was developed as part of the MacArthur Violence Risk Assessment Study (Steadman et al., 1998). It has thus far been used in both clinical and research contexts, for example, in risk assessment (e.g., Monahan et al., 2000) and in evaluating theories of violence (e.g., Nagtegaal et al., 2006). The SIV contains eight items, the first of which asks whether the respondent has experienced a recent VI. Contingent on an affirmative answer, respondents complete a further seven items which probe other features of their VIs such as frequency and consistency of content. Although the SIV provides a richness of information on VIs, any potential reliability advantages of multiple indicators of a VI construct are not realized because no composite or multi-indicator latent variable can be formed.
In sum, there is growing interest in, and potential importance of VIs as a research topic. Given the associated need for an appropriate psychometric measure, our aim in the current study was to develop and validate a reliable and valid assessment of VIs for use across research, forensic, and clinical contexts.
Method
VIS Development
An initial pool of items was generated by two subject matter experts drawing on theories of the structure of violence and its common precursors. Traditionally, violence is defined with respect to the use or threat of physical force; however, definitions of VI in previous studies have been variable (Gellerman & Suddath, 2005). Some have—consistent with traditional notions of violence—focused exclusively on thoughts of physically harming another (e.g., Grisso et al., 2000), while others have expanded the definition to include nonphysical harm such as verbal aggression or destruction of property (e.g., Fehr & Russ, 2013; Smith, Fischer, & Watson, 2009); self-harm and/or suicide (e.g., Uusitalo-Malmivaara, 2013), or sexual violence (Gellerman & Suddath, 2005; Hutton, Parker, Bowe, & Ford, 2012). Common to all definitions, however, VIs can be distinguished from intention to commit a violent act and from delusional states in which the imagined violence is felt as reality.
Considering these definitions, we began by allowing for a broad definition of VIs that included ideations of physical and nonphysical harm, sexual violence, and violence toward the self. This approach allowed us to test whether and how the various imagined acts in an expanded concept of violence fit within or are related to a narrower and more traditional definition of VIs. That is, we allowed the data to dictate whether this more inclusive definition was supported in regard to the current measure. From our initially generated pool of items, a subset was chosen to be administered to participants. These items were selected based on judgments of their clarity, content, and face validity and with the goal of ensuring that all key content areas of interest were included in the scale. The wording of items was also considered with respect to minimizing socially desirable responding given the potentially sensitive nature of the topic. In a previous study of sexual ideations, for example, respondents tended to report having fantasies of “using force for sex” at a considerably higher rate than of “raping” (55.9% vs. 35.7%) even though both terms refer to essentially the same behavior (Greendlinger & Byrne, 1987). Thus, we refrained from using labels such as “murder,” “rape,” or “homicide” and instead describe the behavior we are interested in without drawing attention to the serious legal and moral implications of such acts. In addition, to encourage honest responding, the test instructions included a statement that “people often think about things that they never actually do,” to emphasize that VIs are quite normal.
Items were initially developed in German—the language spoken by the participants in the current study. The items administered and their English translations are provided in the appendix. Participants were asked to report the frequency of specific VIs in the past month using a 5-point Likert-type scale response format ranging from never to very often. The time scale of the past month was selected to provide an optimal balance between minimizing recall bias and allowing sufficient time for at least one VI to have occurred among those liable to experience VIs.
VIS Validation and Translation
Participants
Testing of items was conducted in a normative sample of youths in Switzerland. Participants were part of the Zurich study on the Social Development of Children and Youths (z-proso): a longitudinal cohort study concerned with the development of pro- and antisocial behavior across the course of late childhood into early adulthood. The study has been comprehensively described in terms of recruitment, assessment, retention, and sample characteristics in previous publications (e.g., Eisner & Ribeaud, 2007; Ribeaud & Eisner, 2010b) and further details can be found on the study website (http://www.z-proso.ethz.ch/). In brief, the sample represents the outcome of a stratified random sampling of 56 schools in Zurich taking into account school size and location. All children (aged 7 years) who attended one of the selected schools were invited to participate. Those who participated at baseline can be considered largely representative of the target sample, except that children of parents who did not speak German as a first language were slightly underrepresented. In being drawn from the Zurich population, the sample is ethnically diverse. At baseline, approximately 70 different nationalities were represented among the primary caregivers, with the biggest proportions of parents from Switzerland (38.4%), Italy (8.8%), Serbia-Montenegro (8.7%), Germany (6%), and Portugal (5%). The sample also shows considerable variation in socioeconomic status. For example, at baseline (based on the parents from whom data were available) the highest educational levels of male primary caregivers were 21% mandatory school or less, 35.2% apprenticeship, 7.8% A-levels, 15.5% higher vocational education, and 16% University.
The VIS was administered at the seventh-measurement wave when the majority of participants were aged 17 years. At this point, after providing informed consent, 1,276 participants provided data on VIS (629 female, 647 male), which represents 81% of participants who provided data at any wave and 76% of the original target sample at intake.
Questionnaire Administration
All measures: The VIS items and the measures used to evaluate concurrent validity, discriminative ability, and to construct a nomological net were administered in pencil-and-paper format in a single questionnaire. Measures were embedded in a larger Wave 7 z-proso questionnaire that included questions on family situation and interactions with parents, friends and peers, attitudes to criminal and aggressive behavior, mental health, interactions with and attitude to the legal system, life events, and leisure activities. Participants were informed that their responses were completely anonymous and that the research assistants administering the questionnaires would not have access to their responses.
Factor Structure and Invariance
Participants were randomly assigned to either a calibration or validation sample. In the calibration sample, scale dimensionality was investigated using exploratory factor analysis (EFA). In the validation sample, a confirmatory factor analysis (CFA) was used to assess the generalizability of the solution developed in the calibration sample. If confirmed, the two samples were recombined and further analyses of the validity and functioning of the items conducted.
With a 5-point response format and response distributions that were skewed, our items fell near the borderline of whether ordered-categorical or continuous response format estimation methods were more appropriate. Considering the relative advantages of maximum likelihood (ML) estimation, we (unless otherwise stated) opted to treat the items as continuous and use this estimation method, allowing us to benefit from the superior treatment of missingness and to use information theoretic criteria for model comparisons. The main implications of this choice are that the factor loadings and model fits will be reduced relative to those obtained from using an ordered-categorical estimation method such as weighted least squares means and variances (WLSMV). As a check on the sensitivity of results to estimation method, we also estimated a subset of models with WLSMV to ensure that our conclusions would not be affected by choice of estimation method in any substantive way. These are reported only if there was a substantive discrepancy between this and the corresponding ML-estimated model.
EFA was conducted on a randomly assigned calibration sample of N = 638 participants. The number of factors to retain was investigated using parallel analysis with principal components (PA-PCA; Horn, 1965), the minimum average partial (MAP; Velicer, Eaton, & Fava, 2000) test and visual inspection of a scree plot. Depending on the level of agreement between these methods, a number of factor solutions were estimated and the patterns of loadings were examined. Considering the interpretability of these factor solutions together with the evidence from the statistical dimensionality assessments, we chose an optimal factor solution as the basis for further study. All EFA analyses were conducted using the “psych” package in R statistical software (R Core Team, 2014; Revelle, 2014).
A CFA was conducted in the validation sample to assess whether the model developed using EFA generalized to an independent sample. As the CFA model depended on the results of the EFA, the details of the model specification are provided in the Results section. The model was considered to fit well if Tucker–Lewis index (TLI) and comparative fit index (CFI) were >.95, root mean square error of approximation (RMSEA) was <.08, and standardized root mean square residual (SRMR) was <.08 (Beauducel & Wittmann, 2005; Hu & Bentler, 1999).
If the model developed in the calibration sample could be generalized to the validation sample, we recombined these samples for further analysis. Given the past evidence that VIs have a different prevalence and pattern in males versus females (e.g., Auvinen-Lintunen, Häkkänen-Nyholm, Ilonen, & Tikkanen, 2015) and the importance of sex differences in VIs in theoretical perspectives on their evolution (e.g., Duntley & Buss, 2011), we evaluated whether measurement invariance by sex held for the scale. We did this using a CFA approach in which increasingly constrained factor models were tested against one another.
We began with a configural invariance model in which the pattern of loadings is fixed equal across males and females. To achieve scaling and identification in this model, we fixed the means and variances of the latent variable(s) in females to 0 and 1, respectively, and fixed the loading and intercept of the first item equal across males and females. To test metric invariance, we then added invariance constraints on the remaining factor loadings. To test scalar invariance, we added invariance constraints on the remaining intercepts. At each stage, we compared the fit of the model with the previous, less constrained model. We considered metric invariance to hold if the decrease in fit with the addition of loading constraints amounted to a less than a 0.010 decrease in CFI, a less than 0.015 increase in RMSEA, and a less than 0.030 increase in SRMR (Chen, 2007). We considered scalar invariance to hold if the decrease in fit with the addition of intercept constraints amounted to less than 0.010 for CFI, accompanied by an increase of no more than 0.015 in RMSEA and an increase of no more than 0.010 in SRMR (Chen, 2007). We also examined information criteria, focusing on Akaike’s information criteria (AIC), Bayesian information criterion (BIC), and sample size adjusted BIC (saBIC). In all cases, smaller (or more negative) values of AIC, BIC, and saBIC indicate a better fitting model. In addition, a BIC differences between models greater than 10 can be taken to be indicative of a difference in fit that is practically significant (Raftery, 1995).
If invariance did not hold at any stage, we investigated whether any intercepts or loading constraints could be released to give a partially invariant model. This was guided by an examination of the modification indices in the more constrained model. Finally, once we had achieved an invariant or partially invariant measurement model, we constrained the means in males and females to equality to evaluate the significance of sex differences in VIs.
Concurrent Validity
There is currently no gold standard measure of VIs against which to compare the VIS. To assess the concurrent validity of the scale, we, therefore, used the measurement model developed for the VIS to assess the correlation of the latent VI variable with a second, previously developed measure tapping imagined violence (Averdijk, Van Gelder, Eisner, & Ribeaud, 2016). Strong correlations between the VIS and the previously developed measure of imagined violence would support the idea that—as intended—they are measuring very similar constructs. This measure was designed within a judgment and decision-making paradigm but includes a component measuring VIs. We focus on this component in the current study. The VI component yields three scores, derived from three separate sections. In each section, a vignette adapted from Huizinga and Esbensen (1990) and Wetzels, Enzmann, Mecklenburg, and Pfeiffer (2001) was presented. The vignette described a hypothetical scenario in which the respondent behaves aggressively. The respondent is asked to imagine behaving in this way and to report on the extent to which they had thought of behaving similarly in the past month. Scenarios were the respondent is subject to verbal aggression by a peer and responds aggressively (reactive aggression); the respondent threatens a peer with violence in order to rob them (proactive aggression); and the respondent intimidates a peer (verbal/relational aggression). In each, the hypothetical victim was matched in gender to the respondent. Frequency of thoughts of behaving in a similar manner to that presented in each scenario was measured on 4-point scale from never to (almost) daily. We expected that the VIS should be significantly positively correlated with each of these measures.
Nomological Net
Nomological nets concern the theoretical relations between a construct and associated constructs (see Cronbach & Meehl, 1955). If empirical relations between test scores match those implied by a theoretical nomological net, this provides some evidence for the validity of their interpretation and/or helps refine this interpretation. We evaluated the association between the VIS sum score and a range of relevant constructs that represent candidate correlates, causes, and outcomes of VIs. The following constructs were examined: reactive aggression, proactive aggression, violent victimization, bullying victimization, self-control, and moral neutralization.
Reactive aggression refers to emotion-driven aggressive responses to perceived threats and provocations. It was measured using the sum of four items referring, respectively, to being the respondent being aggressive when teased, insulted, having something from them, and when not getting something. Proactive aggression refers to emotionally cool, instrumental aggressive behaviors. It was measured using the sum of four items referring, respectively, to scaring others to force them to do something, bossing others around, humiliating others, and threatening others to get something. All eight items were derived from the Social Behavior Questionnaire (Tremblay et al., 1991) and used a 5-point Likert-type response format ranging from never to very often.
Violent victimization refers to being the victim of a violent act. It was measured with four items referring to violent theft, being injured by a weapon or object, being hit hard enough to cause injury, and being sexually assaulted. The respondent is asked to report whether each has happened to them in the past 12 months. We used a sum over the four items as a measure of violent victimization. Bullying victimization refers to being the victim of bullying. It was measured with four items referring to being insulted or taunted, being physically attacked, having belongings taken or destroyed, and sexual harassment. The respondent is asked to report how frequently each has occurred in on a 6-point Likert-type scale from never to almost daily. We used a sum over the four items to obtain a measure of overall bullying victimization. Both victimization measures and their rationales are more comprehensively described in Averdijk et al. (2016).
Self-control was measured using an adapted form of Grasmick, Tittle, Bursik, and Arneklev’s (1993) Low Self-Control Scale, later modified by Longshore, Turner, and Stein (1996). In the form administered in z-proso, Wave 7, it comprises 10 items, covering referring to behaviors indicative of risk-seeking, disregard for the consequences of actions, impulsiveness, volatile temper, and lack of deliberation. Responses are provided in on a 4-point scale from fully untrue to fully true and (re)coded such that high scores represent a lack of self-control. We used the sum over the 10 items as a measure of overall low self-control.
Moral neutralization unifies the concepts of neutralization techniques, moral disengagement, and self-serving cognitive distortions. It was measured using the sum of 16 items tapping cognitive restructuring, distortion of or disregard for the consequences, victim blaming, and assuming the worst in regard to aggressive acts. The aggressive acts referred to include verbal aggression, physical aggression, bullying, and general aggression. Responses are provided on a 4-point scale from true to false. This measure is comprehensively described in Ribeaud and Eisner (2010a).
Discriminative Ability
We used a Receiver Operator Characteristic (ROC) analysis to evaluate whether the VIS has utility in identifying criminally violent individuals, defined as having engaged in any of the following behaviors: carrying a weapon, extortion, robbery, or assault according to self-report over the past 12 months. ROC analysis can be used to evaluate the overall discriminative power of a scale and to identify an optimal cut-point for classifying individuals on the basis of scale scores. An ROC curve is a plot of the sensitivity and 1-specificity at each cut-point on the scale. Here, sensitivity reflects the proportion of individuals who are correctly assigned to the criminally violent category (true positives), while specificity reflects the proportion of individuals who are correctly assigned to the noncriminally violent class (true negatives). As an overall measure of the discriminative power of a scale, the area under the ROC curve (AUC) can be computed. AUC can vary between .5 (reflecting chance classification) and 1 (reflecting perfect classification).
To select an appropriate cut-point for classifying individuals as at risk of criminal violence, one strategy is to select a cut-point based on the Youden index which identifies the cut-point c where:
This strategy provides an optimal cut-point where sensitivity and specificity are given equal weight; however, as the relative seriousness of false positives versus false negatives may vary by context, we also present sensitivity and specificity values for a range of plausible cut-points.
Translation
The instrument was translated into English to facilitate its use in English-speaking contexts. We used a procedure based on recommended best practices (van Widenfelt, Treffers, de Beurs, Siebelink, & Koudijs, 2005). First, two independent initial German-to-English translations were provided. The first was provided by a bilingual test author. The second was provided by an independent bilingual translator with no specialist subject matter expertise. Both translators had familiarity with English speaking culture, as well as language. Based on these translations, a third monolingual test author with psychometrics and subject matter expertise made amendments to the proposed items to improve readability and cultural appropriateness. A final translation was selected based on a consensus reached by the translation team. This was then reviewed by two additional monolingual English speakers: one with practical and academic subject matter and test development expertise (a clinical psychologist with extensive experience working in forensic services) and one with no specialist subject matter expertise. Feedback provided here was then incorporated in to the provisional translation to be tested in English-speaking samples.
Results
Descriptive Statistics
Item category response frequencies are provided in Table 1. For all items, the most frequently selected response option was “never,” with only a small proportion of participants selecting “very often.” The proportion of individuals who experienced a specific VI at least once ranged from .05 (homicidal, rape, and sexual assault ideations) to .37 (taking violent revenge).
Category Response Distributions for the Violent Ideations Scale (Proportions).
Factor Structure and Invariance
PA-PCA suggested three and MAP suggested one dimension to retain and a scree plot is suggested one dimension to retain. Given the disagreement between factor retention methods, we examined a one-, two-, and three-factor solution. These are provided in Table 2, with an oblimin rotation applied to the multifactor solutions. Together, the factor solutions suggested that both sexual aggression ideation and suicidal ideation could be distinguished from ideations involving aggression. From here on, we, therefore, focus on the items pertaining to a narrower definition of VIs, excluding the suicidal and sexual ideation analyses from further analyses.
Factor Solutions in Calibration and Validation Samples.
Note. CFA = confirmatory factor analysis. Not showing loadings <|.3|.
In contrast to the sexual aggression and suicidal ideation items, there was no indication that ideations of nonphysical aggression should be treated as distinct from ideations of physical aggression. For example, an item referring to humiliating a target had a loading on the VI factor of .54 which was both >.3 and not markedly smaller than the items referring to physical violence.
The three-factor solution suggested an additional distinction could be made between “serious” VIs and “less serious” VIs; however, examining the distributions of the items loading on this factor, we judged the distinction to be more a function of a statistical than substantive difference. This was based on the observation that the items tending load on the “serious” VIs factor had response distributions very skewed toward the low end of the scale. Thus, it was likely that this factor represented a “difficulty” factor arising because more similarly distributed items will tend to correlate more strongly with one another than less similarly distributed items (e.g., Gorsuch, 1983).
Repeating our dimensionality assessments and factor analyses excluding the sexual assault and suicidal ideation items, we found evidence for only one dimension to retain according to both PA-PCA and MAP. A scree plot also suggested only one factor to retain. Given the agreement between the dimensionality assessments, we estimated a one-factor solution only, provided in Table 2. All loadings were >|.30| in this solution. The highest loading items referred to inflicting pain and injury, while the lowest loading item referred to bullying. The items referring to homicide had loadings intermediate in strength.
A CFA was conducted in the validation sample using the items retained from the previous stage of analysis. Based on the EFA results from the calibration sample, a one-factor model was fit, fixing the factor variance to one for scaling and identification purposes. The model was fit using ML estimation in Mplus 7.0 (Muthén & Muthén, 2010). The model showed reasonable fit (TLI = .92, CFI = .93, RMSEA = .08). Factor loadings are provided in the last column of Table 2.
The one-factor VI model described in the preceding section was used as the basis for assessing sex invariance. Model fit for each invariance model is provided in Table 3. The fit at each stage fell below consensus standards for good fit. This was not judged to be problematic given our choice to treat the indicators as continuous (e.g., fitting the male and female models using WLSMV estimation gave RMSEA = .06 and RMSEA = .04, respectively).
Model Fits for Sex Invariance Models.
Note. TLI = Tucker–Lewis index; CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; AIC = Akaike’s information criteria; BIC = Bayesian information criterion; saBIC = sample size adjusted BIC.
According to the fit difference criteria recommended by Chen (2007), both metric invariance and scalar invariance held; however, AIC and saBIC (but not BIC) suggested that a model without metric invariance constraints fit better. Examining the modification indices in the metric invariance model (M2) suggested that releasing the invariance constraints on the loadings of Items 3, 5, and 7 could improve the fit of the model. The fit of the partial metric invariance model (M4) with these constraints released are provided in Table 3 labelled “partial metric.” This model fit better than the full metric model (M2) according to all criteria except RMSEA. It also fit better than the configural model (M1). We, therefore, also tested a model “M5” in which scalar invariance constraints were added to this more relaxed model. In M5, scalar invariance constraints were not imposed on Items 3, 5, and 7 for which metric invariance was not supported. Fits for this model are also provided in Table 3. According to Chen’s (2007) criteria, these invariance constraints were supported. We, therefore, on balance judged M5 to be the model that provided the best representation of our data.
The parameter estimates for M5 are provided in Table 4. For Item 3 (violent payback), males had a higher intercept than for females and for Items 5 and 7 females had a higher intercept although the latter two differences were very small. Thus, the primary difference between the model in males and females was that for a male and female of the same latent trait level, a male would be expected to score higher on Item 3. The factor loading for Items 3 and 7 were higher in females; however, the factor loading for Item 5 was larger in males. Therefore, Items 3 and 7 are more informative about VI levels for females than males, while the opposite is true for Item 5.
Unstandardized Parameter Estimates for M5.
The results from M5 also suggested that males had higher levels of VIs than females. Comparing M5 with M6 in which the male and female factor means were constrained to equality suggested that this difference was both statistically and practically significant (Δχ2(1) = 82.91, p < .001; ΔBIC > 10).
Concurrent Validity
We assessed the correlation between the VIS latent variable and the previously developed VI measures (reactive, proactive, and relational/verbal aggression) using M0, that is, a one-factor CFA model fit to the whole sample. The response distributions for these VI measures are provided in Table 5. Responses were skewed toward the low end of the response scale, with the highest response option almost entirely unused. We used WLSMV estimation for this analysis, correlating each VI measure with the VIS latent variable and with one another. The correlation between the VIS latent variable and the reactive, proactive, and verbal/relational aggression ideations were r = .65 (p < .001), r = .48 (p < .001), and r = .54 (p < .001), respectively. These results thus support the concurrent validity of the VIS.
Category Response Distributions for Preestablished Violent Ideation Measures.
ROC Analysis
The distribution of the sum score generated from the VIS is shown in Figure 1. The sum score is skewed toward low levels of VI. The mean score was 16 (SD = 6.35), the median score was 13, and the percentage of people who scored 12 (the minimum score) was 40%. There were 155 individuals who met our criteria for criminal violence (12% of the sample). Using this as our outcome in an ROC analysis, we aimed to identify a cutoff point on the VIS sum scores that would indicate risk for physical aggression. The AUC was .78. The optimal cut-point according to the Youden index was 15.5. At this cut-point, specificity was .71 and sensitivity was .75. Sensitivity and specificity values at a range of other cut-points are provided in Table 6. These values support the discriminative ability of the scale.
Histogram of Violent Ideations Scale sum score.
Sensitivity and Specificity of the Violent Ideations Scale at a Range of Cut-Points.
Nomological Net
The correlations between the VIS score and a range of other relevant constructs is shown in Table 7 and graphically displayed in Figure 2. In Figure 2, the thicknesses of the vertices are proportional to the strength of the association it represents. The VIS was significantly (p < .01) associated with all included constructs. The strongest associations were with other cognitions related to aggression and with aggressive behavior itself. It was less strongly related to victimization. In terms of aggressive behavior, it was approximately equally related to proactive aggression and reactive aggression.
Correlations of VIS With Other Relevant Constructs.
Note. VIS = Violent Ideations Scale; SC = self-control;
RA = reactive aggression; PA = proactive aggression;
BV = bullying victimization; VV = violent victimization;
MN = moral neutralization. Pearson correlations below the diagonal, p values above the diagonal.
Nomological net of VIS.
Translation
The English-language translation is provided in the appendix. In general, we aimed for a translation that was as linguistically and conceptually similar to the original; however, we prioritized conceptual similarity when any conflicts arose. Early iterations of the translations were linguistically similar to the original German but the wording was iteratively edited for readability and cultural appropriateness. A consensus was reached among members of the translation team on optimal wording by the end of the translation, editing, and review process. Work is currently underway to validate the resulting translation in English-speaking samples.
Discussion
In the current study, we aimed to develop a brief reliable and valid measure of VIs. VIs feature in contemporary theoretical perspectives concerned with the ultimate and proximate causes of violence as well as having significance for violence prediction, especially in clinical and forensic settings. The availability of an appropriate measure of VIs is, therefore, important to facilitate research into VIs, their role in violence, and their ability to help predict violent acts. Selected through a series of psychometric evaluations, we present a set of 12 items forming the VIS with the aim of meeting this need.
The selected set of items refer to inflicting harm on an individual and include serious acts of physical violence such as homicide as well as acts such as bullying and humiliation which do not necessarily have a physical harm component. Among the set of selected items, those with the highest factor loadings referred to physical but nonlethal violence such as inflicting pain or serious injury.
In contrast, ideations of sexual violence (raping or undressing someone) appeared to represent a distinct phenomenon from nonsexual VIs although it should be noted that our results were here somewhat ambiguous. First, ideations of sexual violence were very rarely reported so there is a possibility that their separation into a distinct dimension was merely a “difficulty factor” phenomenon, that is, the result of differing response distributions between these and the remaining items. Second, statistical tests of dimensionality were not unanimous in suggesting that the set of initial items including sexual violence and suicidal ideations should be described by more than one dimension. With regard to sexual violence ideations, the situation most likely mirrors actual sex offending, namely, that sex offender tends to engage in a whole range of criminal behaviors (i.e., showing a lack of specialization), while also evidencing specific risk factors for sex offending (e.g., sexual entitlement attitudes; Jewkes, Fuli, Roselli, & Garcia-Moreno, 2012). Thus, in psychometric terms, the structure of sexual offending can be considered hierarchical, with both general and specific influences, leading to ambiguity as to whether the behaviors should be treated as belonging to the same domain as other violent behaviors. As only two items measuring ideations of sexual violence were included in the pilot version of the VIS, we could not fully explore the hierarchical structure of ideations; however, future studies could do so by utilizing a more comprehensive set of ideations of sexual and nonsexual violence indicators and candidate general and specific predictors.
Some previous studies have included suicidal ideations in their definitions of VIs (e.g., Uusitalo-Malmivaara, 2012); however, similar to the results on ideations of sexual violence, our factor analyses suggested that it may be more appropriate to treat them as belonging to a separate domain. Indeed, in a factor analysis of the VIS with an omnibus psychopathology inventory, suicidal ideations loaded not on the VIs factor but on an internalizing factor (A. L. Murray, Obsuth, Eisner, & Ribeaud, 2016). Suicidal ideations were, nonetheless—and consistent with previous research—positively correlated with VIs (Harter, Low, & Whitesell, 2003). Indeed, at a general level, both appear to be associated with psychological illness and could be considered markers of psychological distress (e.g., Grisso et al., 2000; Miles, 1977).
A sex invariance analysis suggested that there were some minor measurement differences between males and females in Items 3, 5, and 7. These differences were small in practical terms and would be unlikely to introduce much bias; however, researchers may consider omitting Items 3, 5, and 7 or fitting a partially invariant measurement model when testing male–female differences in means, variances, and correlations. Arguably, the most important difference identified was in the intercept of Item 3: an item referring to violent revenge. For the same level of VI tendency, males would be expected to have a higher score on this item than females. This may reflect a greater willingness among males to report this kind of VI, possibly because it is consistent with masculinity norms (e.g., Krumpal, 2013; Reidy, Sloan, & Zeichner, 2009), although it does not explain why the bias was restricted to this one item.
Confirming (effective) sex invariance in the measure was an important step in its validation because sex differences in VIs represent a potentially important substantive issues: Males vastly outnumber females as perpetrators of violence; however, our and previous studies would suggest that the homicidal/VI differences are a long way from proportional to the perpetration differences (e.g., Auvinen-Lintunen et al., 2015; Kenrick & Sheets, 1993). For example, while males are responsible for approximately 80% of all homicides, they barely differed from females on VIS scores in the current study. Understanding these sex differences may help illuminate causal mechanisms in violence and inform prevention.
Although we would recommend using an explicit measurement model for the VIS when used in empirical research, a sum score can be also be computed to provide a reasonable proxy, particularly in applied contexts. The distribution of the sum score derived from the VIS was skewed toward low levels of ideation with 40% of the respondents reporting none at all. One interpretation of this is that there exists a subpopulation of individuals who tend to experience VIs infrequently or not at all. However, another possibility is that the scale did not include enough sufficiently “mild” VI items to capture variation at low levels. This could be tested by supplementing the current 12 VIS items with items referring to ideations of arguably less serious acts of violence and evaluating whether: (a) a single dimension of severe to mild VI provides a good model of responses to these items and (b) whether the addition of mild items improves the precision of measurement at low levels of VI (e.g., see A. L. Murray, Booth, McKenzie, Kuenssberg, 2015; Thomas, 2011). Examples of “mild” items could include ideations of harm passively befalling an individual; of a target being distressed, embarrassed, angered, or shamed; or of verbally or psychologically abusing a target. In previous research, for example, ideations of verbal aggression were substantially more common than ideations of causing physical harm, suggesting that the former could represent the milder end of the VI spectrum (Nagtegaal et al., 2006). That 40% of individuals reported no VIs could also reflect the relatively short recollection period of only 1 month: Longer recollection periods would be expected to yield further incidences.
We tested the utility of the VIS sum score in to discriminate between those who do and not engage in criminal violence. An ROC analysis suggested that the cut-point that provided the best balance of specificity and sensitivity was 15.5; a cut-point at which both sensitivity and specificity were reasonable. However, we also presented the sensitivity and specificities at a range of alternative cut-points because the appropriate cut-point will depend on the particular context in which the VIS is used; specifically, on the extent to which failing to predict criminal violence is considered more or less costly than falsely labelling an individual as likely to be criminally violent. It is also important to consider the base rates of criminal violence in the particular population in which the VIS is administered: Unacceptably high false positive rate are likely in populations in which criminal violence is rare (e.g., Elwood, 1993). This issue can be mitigated by choosing a strict cut-off point to maximize specificity.
In spite of its concurrent ability to discriminate between those who do and not commit acts of criminal violence, we would caution against using the VIS in isolation in making high stakes decisions and without further validation as a prospective predictor of violence. Even with further validation, we recommend that it be used merely as one source of information contributing to risk estimation within a more comprehensive assessment that considers a multitude of factors relevant to criminal violence. In addition, the relation between VIS scores and criminal violence will need to be verified using official records that do not rely on self-report. Although participants were assured of the confidentiality of their data, it is possible that participants under- or overreported their engagement in criminal violence due to concerns about admitting to illegal activities or other response biases.
We anticipate that with additional research and refinement, the discriminative power, interpretation, and overall clinical and forensic utility of the VIS can be further improved. For example, it likely that the predictive power of the VIS depends on a range of other situational and person factors. Given that VIs appear to be extremely common (Auvinen-Lintunen et al., 2015; Bruns & Disorbio, 2000; Crabb, 2000; Grisso et al., 2000; Kenrick & Sheets, 1993; Nagtegaal et al., 2006; Watt, Kohphet, Oberin, & Keating, 2013), identifying the factors that differentiate between those with VIs who do and no not act on them is, therefore, a key future direction in VI research. If such moderators of the VI–violence relation can be identified, the interpretation of VIS scores can be adapted according to the presence of these features. Some preliminary evidence points to problematic alcohol use and violence exposure as moderators of the relation between violent thoughts and violent behavior (e.g., Watt et al., 2013) and in individuals who have been exposed to violence (Smith et al., 2009). Other candidate moderators, yet to be tested, include characteristics such as impulse control deficits, moral attitudes, perceived norms, callous-unemotional traits and the level and quality of emotional arousal associated with VIs. Similarly, the use of the VIS can be targeted to contexts in which its predictive power is high and its administration in contexts in which it is likely to be of little informational value avoided.
Finally, we found that VIs were significantly and moderately–strongly associated with a range of other relevant constructs: bullying victimization, violent victimization, moral neutralization, and proactive and reactive aggression. Establishing the causal processes, underpinning these associations and associations between VIs and other constructs to which it has been theoretically linked (e.g., violent media consumption; Anderson et al., 2010) represents a newly emerging line of research with considerable potential for improving understanding violence causation, prevention, and treatment.
Limitations and Future Directions
There remain some outstanding questions regarding the VIS. First, while we tested factorial validity, sex invariance, criterion validity, and discriminative ability of the VIS, we did not assess test–retest stability, invariance across language, culture or age, or examine social desirability effects. These will be the subject of future research. For example, it will be of interest to test whether including a statement that notes that VIs are a common and normal phenomenon affects the reporting of VIs; whether scores are stable over short time periods; and whether they predict future, informant-reported violent behavior. Second, it will be necessary to evaluate the psychometric properties of the VIS in other populations for which it may have clinical or research utility, including general population adults and clinical and forensic populations. It should not be assumed that the VIS will measure the same construct in the same way across these populations. For example, there are considerable differences in aggression and violence and associated risk factors across the life course with perpetration levels following an average trajectory that sees a peak in late adolescence followed by declines in young adulthood (e.g., see Sampson & Laub, 1992). These age-related changes in aggression and violence underline the importance of evaluating age-related changes in the measurement properties of aggression- and violence-related constructs such as VIs. Such changes can be particularly informative when examined in the same individuals measured repeatedly over time (e.g., Widaman, Ferrer, & Conger, 2010). Finally, there were also some limitations that likely cannot be easily addressed with future research. There is no preexisting gold standard measure against which to evaluate the VIS, although longer, more comprehensive measures than the VIS could be developed. In addition, it will not be possible to evaluate the VIS using multirater data because VIs are internal phenomena which informant raters do not have access to.
Conclusions
Initial evidence suggests that the VIS provides a brief but reliable and valid measure of VIs. Using the VIS, we confirmed that VIs are common; that they show sex differences; and they discriminate between those who have and have not engaged in criminal violence. We also found evidence that a distinction should be made between suicidal, sexual, and violent ideations but not between ideations of lethal and nonlethal violence.
Footnotes
Appendix
Items Administered.
Menschen denken manchmal auch über Dinge nach, die sie eigentlich nie tun würden. Wie ist das bei dir?
Gib unten bitte an, wie häufig du im letzten Monat an diese Dinge gedacht hast.
English Translation.
People sometimes think about doing things that they would never actually do. How about you?
Please indicate below how often you thought about these things in the past month.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.