Multi-method cross-linguistic validation of the Activism and Radicalism Intention Scale for adolescents

Abstract

The world is rapidly changing, and many societies face radicalization involving activism and sometimes violence. Adolescents are at increased risk of radicalization, yet validated scales to assess violent and non-violent radical intentions among youth are lacking. This study aimed to validate the Activism-Radicalism Intention Scale (ARIS) for use among French- and English-speaking adolescents (14–18 years) using data from 1,911 Canadian high school students (M_age = 15.67; SD_age = 0.98). We employed confirmatory factor analysis (CFA) and Rasch analysis to evaluate the scale’s dimensionality, reliability, validity, and invariance across sociodemographic factors. The CFA supported distinct Activism Intention Scale (AIS) and Radicalism Intention Scale (RIS) for both language versions of the scale. The Rasch analysis provided item-level diagnostics for each subscale, with satisfactory model fit indices observed for both the AIS and RIS. Both the CFA and Rasch supported the scales’ reliability and convergent and structural validity, including its measurement invariance across languages, age, and gender. An algorithm for converting ordinal data into interval-level scores using Rasch estimations was developed to enhance the precision of the scales. Both the validated ordinal and interval-level scores of the ARIS provide evidence of its robustness for assessing violent and non-violent radical intentions in adolescents across the two languages.

Keywords

Radicalism activism confirmatory factor analysis Rasch analysis psychometrics adolescents

Introduction

Radicalization can pose a significant societal challenge, particularly when it fosters division or conflict via violence, undermining societal harmony and functionality (Hong et al., 2021; Kriesi, 2016). Radicalization involves legal, non-violent measures (activism) or illegal, violent actions (violent radicalism; McCauley & Moskalenko, 2008; Schmid, 2013). Although non-violent activism and violent radicalism can be positively associated, empirical evidence suggests that they are two separate constructs that do not necessarily coexist in all individuals (Miconi et al., 2024; Moskalenko & McCauley, 2009). While activism can drive social transformation through non-violent means, violent radicalism may escalate into various forms of violence, including hate crimes, shootings, and terrorism (Thelma et al., 2024).

Despite growing interest in addressing violent radicalization, research lacks valid and reliable measures, especially for adolescents who are at increased risk of radicalization and extremism (Rousseau et al., 2019, 2020). Existing instruments are also limited in scope, often focusing solely on activism or violent radicalization without distinguishing or examining both simultaneously as separate dimensions (Gopal & Verma, 2017; Opozda-Suder et al., 2024). Although violent and non-violent radical intentions can be measured within a single scale framework, aggregating their scores can be theoretically problematic by not allowing for a distinction between democratic and undemocratic forms of political mobilization (Moskalenko & McCauley, 2009). For instance, non-violent protests can legitimately channel the expression of grievances. Failing to distinguish between violent and non-violent intentions may lead to non-violent radical intentions as automatic precursors of violent actions, thus limiting and biasing intervention avenues.

The Activism-Radicalism Intention Scale (ARIS) is a widely recognized and relatively robust scale for measuring violent and non-violent radicalization intentions (Moskalenko & McCauley, 2009). Initially developed with adult samples, the ARIS has shown mixed findings on its dimensionality across contexts (Moskalenko & McCauley, 2009; Opozda-Suder et al., 2024). Typically, Moskalenko and McCauley (2009) found that the Activism Intention Scale (AIS) and the Radicalism Intention Scale (RIS) did not form a unidimensional measure in Ukrainian and American adult samples. Similar findings were reported by Pavlović et al. (2022) in Spanish and Croatian adult samples. In contrast, Opozda-Suder et al. (2024) showed that the scale could be treated as unidimensional in a sample of Polish university students. Moreover, psychometric evidence across languages, sociodemographics, and adolescent samples remains limited. The available ARIS validation studies were also predominately conducted using the traditional confirmatory factor analysis (CFA) methods, based on the Classical Test Theory (CTT). Though useful for validation, this method is sample-dependent, limiting generalizability and often violates the interval-level assumption for parametric analysis (Boone, 2016). These limitations underscore the need to complement CFA with robust models like those offered by Rasch analysis. Rasch analysis, rooted in Item Response Theory (IRT), transforms ordinal data into interval scales and addresses item bias by accounting for item difficulty and person ability (Rasch, 1960). Unlike CFA, it offers a robust evaluation of measures, estimating item parameters independently of sample size and enhancing generalizability (Boone, 2016). To date, the ARIS has not been validated using Rasch analysis.

This study examined the psychometric statistics of the ARIS using CFA and Rasch analysis among a French and English-speaking adolescent sample from Quebec, Canada. Combining both methods allowed for triangulation and more robust validation, improving ARIS’s accuracy, applicability, and cross-linguistic relevance. We assessed reliability, structural, and convergent validity, dimensionality, and invariance across language, age, and gender. Finally, we aimed to develop an algorithm to convert ordinal scores into interval-level scores, contingent on Rasch model fit, to enhance precision (Medvedev & Krägeloh, 2022). We hypothesized a significant positive correlation between the RIS scores and measures of discrimination, victimization, and support for violent radicalization, supporting convergent validity (Bhui et al., 2014; Rousseau et al., 2019). Similarly, the AIS was expected to correlate significantly with support for non-violent radicalization (Bhui et al., 2014) for convergent validity. Given rising global conflict and extremism (Ozer et al., 2024), this study offers timely validation of tools assessing radical intentions in adolescents.

Method

Participants and Procedure

Our study included 1,911 students from six public high schools in Quebec, Canada. Among all the participants, 899 (47.0%) identified as boys, 883 (46.2%) as girls, 43 (2.3%) as non-binary, and 9 (0.5%) as transgender boys and girls. Participants’ ages ranged from 14 to 18 years (M_age = 15.67; SD_age = 0.98). Among these, 56.72% (n = 1,084) completed the French version of the questionnaire, while 43.28% (n = 827) completed the English version. Regarding gender, among the French-speaking respondents, 462 (42.6%) identified as boys, 545 (50.3%) as girls, 25 (2.3%) as non-binary, and 1 (0.1%) as transgender. In the English-speaking group, 338 (40.9%) identified as girls, 437 (52.8%) as boys, 18 (2.2%) as non-binary, and 8 (1.0%) transgender boys and girls. Participants’ ages ranged from 14 to 18 years for both English-speaking (M = 15.5, SD = 1.00) and French-speaking (M = 15.8, SD = 0.96) respondents.

Power Analysis

Recommended software for a priori sample size calculation in structural equation modeling indicated a minimum required sample size of 100 cases for the current model structure (Adu et al., 2023; Soper, 2021). However, for robust findings, 947 cases were required to detect an effect size of 0.01 for a scale with eight observed variables and two latent factors, with 80% power and a significance level of 0.05. Our sample exceeds these thresholds, ensuring the reliability of our findings and confirming that the sample size is sufficient for conducting the CFA of the ARIS. For the Rasch analysis, we randomly selected 1,000 participants (500 from each group) using Microsoft Excel. This selection ensured the required sample size for Rasch analysis using the Rasch Unidimensional Measurement Model (RUMM2030) was met (Hagell & Westergren, 2016). This also addressed the chi-square test’s tendency to overemphasize statistical significance with larger samples, potentially amplifying data misfit with minimal practical implications (Adu et al., 2024a; Pelton, 2002). The randomly selected sample accurately mirrored the characteristics and diversity of the entire sample.

The study received approval from University of Montreal Institutional Committee on Ethics in Educational and Psychological Research (#CEREP-22-123-D). Ethics approval was obtained from each of the six participating public schools in Quebec, Canada, before data collection. Data were collected between late 2022 and early 2023 through partnerships with these schools. A purposive sampling via convenience sampling technique was employed to collect data online via Limesurvey, which took approximately 30min to complete. The current research team had previously adapted the ARIS scale into French using the forward-backward translation method (Adu et al., 2023; Brislin, 1970). However, a separate psychometric study on this version of the scale has not yet been published in the literature on adolescent samples. Supplemental Text S1 provides all the items for both French and English versions of the ARIS. This study is part of a larger project on social polarization in Quebec schools, with sections already published (e.g., Miconi et al., 2024).

Measures

Violent and Non-Violent Radical Intentions

Violent and non-violent radical intentions were assessed using the ARIS (Moskalenko & McCauley, 2009). This 8-item intention scale comprises two subscales: the 4-item AIS and the 4-item RIS. A sample item for the AIS is, “I would join an organization that fights for my group’s political and legal rights,” while a sample item from the RIS is “I would continue to support an organization that fights for my group’s political and legal rights even if the organization sometimes resorts to violence.” Responses are rated on a 7-point Likert-type scale ranging from 1 =“Disagree completely” to 7 =“Agree completely.”

Support for Violent and Non-Violent Radicalization

We used the Sympathy for Radicalization Scale (SYFOR; Bhui et al., 2014) to measure support for violent and non-violent radicalization. This nine-item scale is rated on a 7-point Likert-type scale, ranging from 1 =“Disagree completely” to 7 =“Agree completely.” A single item measures support for non-violent radicalization (“Take part in non-violent political protests”), while eight items (e.g., “The use of violence to fight against injustice by the police”) assess support for violent radicalization. The internal consistency of the scale was excellent, with Cronbach’s alpha (α) = 0.90 and McDonald’s omega (ω) = 0.90 (M = 2.84; SD = 1.36).

Victimization

Victimization was assessed with the traditional victimization scale (Pozzoli et al., 2016). This self-reported 8-item scale is rated on a 5-point Likert-type scale, ranging from 1 =“Never” to 5 = “Almost Always.” An example item on the scale is as follows: “Some classmates spread rumors about me or say mean things when I can’t hear.” The scale demonstrated very good reliability in this study (α = 0.84; ω = 0.84; M = 1.56; SD = 0.67).

Discrimination

We examined discrimination using the 10-item Perception of Racism in Children and Youth scale (PRaCY; Pachter et al., 2010). Participants rated their personal experiences of discrimination on a 5-point Likert-type response format, ranging from 1 = “Never” to 5 = “Every week.” An example item from the scale is as follows: “Have you ever been treated badly or unfairly by a teacher?.” The scale showed very good reliability in this study (α = 0.86; ω = 0.86; M = 2.30, SD = 0.90).

Statistical Analyses

Data preparation included computing reliability and descriptive statistics for all scales. Little’s (1988) test indicated that the data were not missing completely at random (MCAR; p > .05). The negligible percentage of missing data in the overall sample (3.41%; Lee & Huber, 2021) was handled using the robust expectation maximization (EM). EM reduces sensitivity to non-normality and outliers, thereby improving the stability of parameter estimates without altering the ordinal nature of the indicators (Malan et al., 2020). CFA analyses used the diagonally weighted least squares (DWLS) estimation to assess model fit, dimensionality, and measurement invariance (MI) across gender, age, and language. Notably, EM was used solely to address missing data and does not alter the measurement level of the indicators. Although EM may yield fractional expected values internally, these values do not represent observed responses and do not change the ordinal nature of the items (Little et al., 2019). Accordingly, the choice of the DWLS estimator was guided by the ordinal nature of the indicators rather than the imputation method, strategy commonly used in the literature (Adu et al., 2024b; Bahamón et al., 2025). DWLS is well suited for ordinal data modeled using polychoric correlations and threshold parameters, and it is more robust than methods like maximum likelihood when normality assumptions may be violated (Li, 2016; Mindrila, 2010). Importantly, DWLS does not rely on raw observed scores but preserves the ordinal structure of the data (Little et al., 2019).

In addition, model fit was evaluated using indices like the Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR), with recommended thresholds of CFI, TLI > 0.95 and RMSEA and SRMR < 0.08 (Hu & Bentler, 1999). A bifactor model further examined dimensionality of the ARIS. Rasch analysis evaluated item fit, dimensionality, threshold ordering, local dependence, reliability using Person Separation Index (PSI), and sample targeting, with differential item functioning (DIF) testing for invariance. Where model fit was acceptable, ordinal scores were converted to interval-level scores. Supplemental “Text S2” provides detailed description of statistical analyses.

Results

CFA Findings

Our CFA results indicated the best model fit for the two-factor structure for both the French and English versions of the ARIS, as well as in the total sample, as evidenced by CFI and TLI values exceeding 0.95, and RMSEA and SRMR values all below 0.08 (Table 1). In contrast, the one-factor structure for both language versions and the total sample showed poor model fit indices, with CFI and TLI values below the 0.95 cutoff, and RMSEA and SRMR values exceeding 0.08 (Table 1). The chi-square to degrees of freedom ratio for the two-factor structure was below 5 for the English version, but slightly higher for the French version, suggesting a closer match between the observed and predicted data patterns for this model. However, for the one-factor structure, the chi-square to degrees of freedom ratio was significantly higher for both versions. Factor loadings for the two-factor structure of both the French and English versions as well as with the overall sample were good, ranging from 0.54 to 0.90 (Supplementary Table S1).

Table 1.

Model Fit Indices and Measurement Invariance for the CFA Factor Solution of the Validated English and French Versions of the Activism and Radicalism Intention Scale for Adolescents (N = 1,911).

Models/variables		CFI	ΔCFI	TLI	ΔTLI	RMSEA	ΔRMSEA	SRMR	ΔSRMR	df/x² (p)
English version
Factor structures	Two-factor model	0.994		0.990		0.049		0.040		2.96 (<.001)
	One-factor model	0.944		0.918		0.139		0.101		16.89 (<.001)
	Bifactor model	0.996		0.993		0.039		0.035		2.27 (<.001)
French version
	Two-factor model	0.983		0.975		0.072		0.055		6.58 (<.001)
	One-factor model	0.912		0.876		0.161		0.120		29.00 (<.001
	Bifactor model	1.000		1.000		0.004		0.019		1.01(<.425)
General sample
	Two-factor model	0.988		0.982		0.063		0.048		8.63 (<.000)
	One-factor model	0.925		0.895		0.152		0.112		45.00 (<.000)
	Bifactor model	0.999		0.996		0.028		0.018		2.50 (.007)
Invariance testing
Language	Configural	0.988	–	0.983	–	0.062	–	0.049	–	4.61 (<.001)
	Metric	0.988	0.000	0.985	0.002	0.058	−0.004	0.50	0.001	4.25 (<.001)
	Scalar	0.987	−0.001	0.986	0.003	0.056	−0.002	0.051	0.001	3.96 (<.001)
	Strict	0.987	0.000	0.988	0.003	0.052	−0.004	0.052	0.001	3.55 (<.001)
Gender	Configural	0.992	–	0.988	–	0.050	–	0.046	–	2.58 (<.001)
	Metric	0.992	0.000	0.990	0.002	0.044	0.006	0.047	−0.001	2.22 (<.001)
	Scalar	0.991	0.001	0.990	0.000	0.045	0.001	0.051	0.004	2.25 (<.001)
	Strict	0.991	0.000	0.992	0.002	0.040	−0.005	0.052	0.001	2.01 (<.001)
Age
	Configural	0.989	–	0.983	–	0.061	–	0.049	–	4.55 (<.001)
	Metric	0.989	0.000	0.986	0.003	0.056	−0.005	0.049	0.000	4.00 (<.001)
	Scalar	0.989	0.000	0.988	0.002	0.052	−0.004	0.049	0.000	3.56 (<.001)
	Strict	0.989	0.000	0.990	0.002	0.048	−0.004	0.050	0.001	3.17(<.001)

In contrast, some items in the one-factor structure exhibited low factor loadings (e.g., Item 8 with a loading of 0.45). The same trend was observed across the English-speaking, French-speaking, and total samples (Table 1, Supplementary Tables S1 and S2). Refer to Supplementary Figures S1 through S8 for the path diagrams of all models. Since these initial analyses supported the two-factor model of ARIS across all considered samples, MI analyses were conducted for this structure. MI was achieved across language, gender, and age. The configural invariance model demonstrated acceptable fit indices for all sociodemographic variables, with CFI and TLI values above the 0.95 cutoff, and RMSEA and SRMR values below 0.08 (Table 1). Changes in CFI and TLI for metric, scalar, and strict invariances were below 0.010, and RMSEA and SRMR changes did not exceed 0.015, and all chi-square values in the IM test were less than 5 (Table 1).

We tested the dimensionality of the ARIS using the bifactor model separately for each language and the total sample. Supplementary Tables S1 and S2 provide comprehensive statistics of all models across all samples. The bifactor model showed excellent fit, with most items loading higher on the subscales compared with the general factor (e.g., Item 6 loaded 0.75 on RIS and 0.23 on the general factor; Supplemental Table S2; Supplementary Figures S3, S6, and S9), showing the significance of the subscales. The low (e.g., r = .49) intercorrelations between the general factor and RIS indicated independence of factors (see Supplementary Figures S3, S6, and S9 for all other intercorrelations among factors). The AIS and RIS showed a negative and small correlation (−0.07), demonstrating distinct factors. The Hierarchical Omega (ω_H) was below 0.80, while the Subscale Omega (ω_s) for RIS was above the 0.50 cutoff, suggesting the general factor was not dominant and subscales reliably captured distinct aspects. The ω values for both subscales were above 0.80, showing the subscales reliably measured the underlying construct, further evidence of the multidimensional nature of the ARIS. The explained common variance (ECV) values for the general factor (43%), RIS (42%), and AIS (15%) emphasized the relative importance of each factor, confirming ARIS’s multidimensional structure. The scales demonstrated very good reliability for both versions: French (AIS: ω = 0.87, α = 0.87; RIS: ω = 0.86, α = 0.86) and English (AIS: ω = 0.89, α = 0.89; RIS: ω = 0.86, α = 0.85).

Rasch Model Findings

We applied the Rasch model to the multifactor structure of the ARIS, informed by evidence from the CFA analyses. Since the ARIS demonstrated MI across the two languages, for Rasch analyses we assessed their validity using the combined data from both languages. We did not examine language differences in validity but rather focused on the validity of the latent variables in the overall sample. Our initial analysis (Table 3) showed that the AIS fitted the overall Rasch model, χ²(24) = 31.10, p = .15, indicated by a non-significant interaction between the items and the latent variable, with all item thresholds falling within the acceptable range of ±2.5 (Table 2). In contrast, the RIS, χ²(24) = 150.38, p < .001, did not meet Rasch model expectations due to a significant interaction between the items and the latent variable (Table 2). Specifically, Items 6, 7, and 8 on the RIS exhibited significant misfit, as reflected by elevated fit residuals (denoted by asterisks in Table 2). Removing these misfitting items could compromise the construct validity of the RIS. Since the Rasch model misfit may also result from local item dependency, we addressed this by examining the residual correlation matrix to identify locally dependent items contributing to the misfit. We addressed local dependency using an advanced method by combining locally dependent items into testlets (Table 2). As a result, the RIS, χ²(12) = 16.01, p = .19, demonstrated good fit to the Rasch model (Table 3). Both scales showed strong evidence of strict unidimensionality, with the lower bound of significant t-tests within the 5% threshold (Table 2). The scales also exhibited very good reliability, with PSI values of 0.81 for AIS and 0.82 for RIS. Table 2 summarizes the item fit statistics, including item locations, fit residuals, and chi-square parameters. In addition, the final items, including the testlets, performed well in the final models, as indicated by the monotonous patterns observed in the Item Charateristic Curves (ICCs) (Figure 1).

Table 2.

Items Fit Statistics Including the Initial and Final Analyses for all the subscales of the 8-Item Activism and Radicalism Intention Scale (N = 1,911).

Scale	Analysis	No.	Items/Details	Location	Fit residual	Chi-square
AIS	Initial analysis: Four items	1	I would join an organization that fights for my group’s political and legal rights	−0.01	0.63	4.83
		2	I would give money to an organization that fights for my group’s political and legal rights	−0.18	0.77	3.33
		3	I would volunteer my time (writing petitions, distributing flyers, recruiting people, and so on) for an organization that fights for my group’s political and legal rights	−0.07	−1.27	8.95
		4	I would travel for an hour to attend a public rally or demonstration for my group’s support	0.26	0.62	13.99
RIS	Initial analysis: Four items	5	I would continue to support an organization that fights for my group’s political and legal rights even if the organization sometimes resorts to violence	−0.32	0.92	10.24
		6	I would continue to support an organization that fights for my group’s political and legal rights, even if the organization sometimes breaks the law	0.16	−3.17	43.70*
		7	I would take part in a public demonstration against my group’s oppression, even if I think the demonstration could turn violent.	0.09	−2.74	42.90*
		8	I would attack the police or security forces if I saw them beating up my group’s members	0.08	5.95	53.54*
	Three testlets (T)	T1	Items: 5 + 8	−0.12	−1.59	6.48
			6	0.10	0.39	16.20
			7	0.02	0.93	13.74
	Two testlets (T)	T1	Items: 7 + T1	−0.11	−6.61	10.12
			6	0.11	3.30	5.89

*significant misfit reflected by elevated fit residuals.

Table 3.

Rasch Model Fit Statistics for the Initial and Final Analysis for all the subscales of the 8-Item Activism and Radicalism Intention Scale (N = 1,911).

Scale	Analyses (A)	Item fit residual (M, SD)	Person fit residual (M, SD)	Goodness of fit (χ², df, p)	PSI	Unidimensionality t-test (% lower bound)
AIS	A1 initial	0.19, 0.97	−0.85, 1.59	31.10 (24), .15	0.81	0.7	−0.7 (YES)
RIS	A1 initial	0.24, 4.23	−0.78, 1.56	150.38 (24), <.001	0.74	1.1	−0.3 (YES)
	A2 sublet 1	−0.09, 1.33	−0.86, 1.55	36.41 (18), <.006	0.80	0.2	−1.2 (YES)
	A3 final	−1.65, 7.01	−0.63, 0.91	16.01 (12), .19	0.82	2.5	1.1 (YES)

Note: PSI = Person Separation Index without extremes.

Figure1.

Examples of Ordered ICCs for the Final Items on the Scales.

Items were found to be invariant across sociodemographic variables, as no significant DIF was detected for gender, age, and language (Figures 2 and 3). There was no evidence of ceiling or floor effects, and the person-item thresholds aligned well with the sample distribution (Figure 4). Once the Rasch model fit indices were confirmed for all the subscales, algorithms were created to transform ordinal scores into interval-level data based on Rasch person estimates (Table 4). Paired samples t-tests were conducted to compare ordinal scores with interval scores on the same scale range. The paired samples t-tests using the combined sample revealed significant differences between ordinal and interval-level scores, with standard error favoring the interval-level scores.

Figure 2.

DIF Curves of the Activism Intention Scale for Age (First Column), Gender (Second Column), and Language (Third Column).

Figure 3.

DIF Curves of the Radicalism Intention Scale for Age (First Column), Gender (Second Column), and Language (Third Column).

Figure 4.

Person-Item Thresholds Distributions for the Activism Intention Scale (Top) and Radicalism Intention Scale (Bottom).

Table 4.

Ordinal-to-Interval Transformation for the 8-Item Activism and Radicalism Intention Scale Ordinal Total Scores in Logits and the Scale Metric.

	Activism Intention Scale			Radicalism Intention Scale
Ordinal	Interval		Ordinal	Interval
Scores	logits	Scale	Scores	logits	Scale
4	−2.47	4	4	−3.62	4
5	−1.98	6.05	5	−2.89	6.63
6	−1.64	7.44	6	−2.34	8.59
7	−1.42	8.39	7	−1.94	10.05
8	−1.24	9.13	8	−1.60	11.24
9	−1.09	9.77	9	−1.32	12.25
10	−0.95	10.34	10	−1.08	13.13
11	−0.83	10.86	11	−0.86	13.91
12	−0.70	11.38	12	−0.66	14.63
13	−0.58	11.89	13	−0.47	15.3
14	−0.46	12.39	14	−0.29	15.94
15	−0.33	12.92	15	−0.12	16.54
16	−0.20	13.48	16	0.04	17.12
17	−0.06	14.07	17	0.20	17.69
18	0.10	14.72	18	0.36	18.26
19	0.27	15.43	19	0.52	18.83
20	0.45	16.20	20	0.68	19.42
21	0.65	17.03	21	0.86	20.04
22	0.86	17.92	22	1.04	20.69
23	1.09	18.88	23	1.23	21.4
24	1.34	19.94	24	1.45	22.18
25	1.64	21.17	25	1.70	23.08
26	2.00	22.69	26	2.01	24.19
27	2.52	24.87	27	2.45	25.77
28	3.27	28.00	28	3.38	28.00

Note: This conversion table can only be used for complete responses to each of scale’s items. To use this table, ordinal raw scores (ordinal scores) should be obtained by adding the observed scores for items on each of the scales. Next, match the ordinal total score (4–28) to the corresponding interval score in the right column of each scale (4–28). A final converted score between 4 and 28 for each subscale will be obtained, with higher scores corresponding to higher levels of either activism, or violent radicalism. Alternatively, the current authors can be contacted for guidance on how to use this conversion algorithm.

Convergent Validity

Positive associations were found between the ordinal and interval-level scores of the RIS and support for violent radicalization as measured by the violent SYFOR (ordinal: RIS r = .25, p < .001; interval: RIS r = .19, p < .001), discrimination (ordinal: RIS r = .19, p < .001; interval: RIS r = .09, p < .001), and victimization (ordinal: RIS r = .16, p < .001; interval: RIS r = .14, p < .001), supporting convergent validity. Positive associations were also found between the ordinal and interval-level scores of the AIS and support for non-violent radicalization as measured with the non-violent one-item of the SYFOR scale (ordinal: AIS r = .52, p < .001; interval: AIS r = .50, p < .001), indicating convergent validity. We also observed statistically non-significant link between the AIS scores and discrimination (ordinal r = −.05, p = .125; interval: AIS r = −.04, p = .125) and victimization (ordinal: AIS r = .03, p = .240; interval: AIS r = .03, p = .147). These demonstrate divergent validity of the AIS.

Discussion

This study validated the ARIS among English- and French-speaking adolescents in Quebec. The CFA assessed the scale’s structure, while Rasch analysis offered item-level insights beyond CFA (Adu et al., 2024a). Using both methods allowed for robust triangulation of the scale’s psychometric statistics (Turner-Stokes et al., 2019). The two-factor CFA model showed the best fit, supporting the distinction between AIS and RIS in both English and French versions of the ARIS. The bifactor model confirmed the relevance of the separate subscales, indicating the presence of an overarching construct of political mobilization that is articulated mainly through two distinct subscales: RIS and AIS. The negative intercorrelation between AIS and RIS further supported their conceptual independence. These findings suggest that AIS and RIS should be treated as distinct constructs rather than combined into a single political mobilization score, aligning with Pavlović et al. (2022) and Moskalenko and McCauley (2009), and contrasting with Opozda-Suder et al. (2024).

Our findings warn against such an approach and contribute to mounting literature emphasizing that violent and non-violent radicalization intentions are distinct constructs that present complex associations (McCauley & Moskalenko, 2008; Miconi et al., 2024). Given that activism may serve as a protective factor against violent radicalization (Bramsen, 2019), merging AIS and RIS scores could obscure key differences and misguide interventions. Notably, the low ECV of the AIS indicates that it captures a distinct and more specific construct with a limited contribution to the common variance. Together with additional supporting evidence, including strong factor loadings and theoretical considerations, this finding supports treating the AIS and RIS as separate subscales rather than combining them into a single aggregated score. These distinct scales showed strong reliability as well. While CFA reliability captures average item consistency at the group level, Rasch offers individual-level precision, enhancing generalizability (Boone, 2016). This dual approach adds value by combining complementary strengths. Overall, both the ordinal and interval scores of the English and French RIS and AIS subscales were reliable for individual, within-group, and group-level assessments (Tennant & Conaghan, 2007). The evidence of MI across age, gender, and language supports the ARIS’s generalizability, enabling valid comparisons across diverse adolescent groups, especially in French and English contexts (Welzel et al., 2023). The Rasch model supported each subscale’s unidimensionality and supported item functioning, with all showing satisfactory model fit after applying the testlet creation modification to the RIS. This method groups items with shared variance unrelated to the main latent trait, reducing measurement error while preserving construct validity, and benefits only Rasch-transformed interval scores (Lundgren-Nilsson et al., 2013; Tennant & Küçükdeveci, 2023).

The Rasch analysis also showed that the items captured the full trait range, with strong sample targeting, monotonic item patterns, and no floor or ceiling effects, confirming good discriminatory power of the scales. The scales met Rasch model assumptions (Rasch, 1960), and all items were invariant across sociodemographic groups. The developed algorithm to convert ordinal scores to interval-level scores based on Rasch estimates presents novelty for this study as no such scores exist for the ARIS. However, this method has been successfully applied to other tools (Adu et al., 2024a; Medvedev et al., 2020). Both methods provided evidence of convergent validity, showing the scales align with related constructs. This supports their use in measuring activist and violent radicalization intentions among adolescents and strengthens confidence in their predictive power for future research (Adu et al., 2024a; Rönkkö & Cho, 2022).

In essence, we have provided both CFA-based ordinal and Rasch-based interval scores to assess violent and non-violent radical intentions among adolescents in two languages, improving scale precision and enabling robust analysis. This dual scoring approach contributes meaningfully to the psychometric validation of the ARIS (Turner-Stokes et al., 2019). Interval scores are better suited for parametric analyses due to their increased level of precision (Adu et al., 2024b). For instance, these scores showed reduced lower standard error in the t-tests analysis. Ordinal scores, however, remain useful for ranking adolescents’ extremist intentions (Boone, 2016). A practical example of using the ordinal and interval-level scores can be demonstrated through two scenarios. In the first scenario, the adolescent violent radical intention score decreases from 15 to 10. In the second, another adolescent score drops from 20 to 15. While both reductions seem equal on the ordinal scale, the interval scale reveals a different story. The first adolescent reduction is 5.97 units, while the second’s is 3.33 units. This illustrates that the Rasch interval-level scores provide more precise measurements, offering clearer insights into changes in radical intentions at both group and individual levels.

This study is the first to validate the ARIS among adolescents in both English and French, offering key insights into violent and non-violent radical intentions in this population. Validating both versions allows for initial cross-linguistic comparisons, although further research on transnational groups is needed to confirm transcultural validity. It is also the first to generate ordinal and interval-level measurements within a single study, enhancing understanding of the scale’s reliability and precision. Despite an 85% response rate, the convenience sample may limit generalizability to the wider adolescent population in Quebec or Canada. While Rasch analysis is less sensitive to sample characteristics, cultural and language factors may still influence responses. Future studies using both CFA and Rasch across diverse populations and translations are recommended to strengthen the scale’s global applicability.

Conclusion

This study used CFA and Rasch methods to assess the psychometric properties of the ARIS among English- and French-speaking adolescents in Quebec. Findings confirmed that both language versions are reliable and valid using either ordinal or interval scores. Results support treating the ARIS as a multidimensional tool that distinctly measures activist and violent radical intentions. All items showed strong discriminatory power and reliability at individual levels. The scales also demonstrated sound convergent validity and MI across sociodemographic groups. Algorithms for converting ordinal to interval-level scores were developed and are included to aid future research. This strengthens ARIS’s applicability for parametric analyses in adolescent samples. Further validation in other languages and contexts is recommended to improve generalizability and assess its utility in public health prevention efforts.

Supplemental Material

sj-docx-1-jbd-10.1177_01650254261423823 – Supplemental material for Multi-method cross-linguistic validation of the Activism and Radicalism Intention Scale for adolescents

Supplemental material, sj-docx-1-jbd-10.1177_01650254261423823 for Multi-method cross-linguistic validation of the Activism and Radicalism Intention Scale for adolescents by Peter Adu, Diana Miconi and Cécile Rousseau in International Journal of Behavioral Development

Footnotes

ORCID iDs

Peter Adu

Diana Miconi

Cécile Rousseau

Ethical Considerations

All procedures performed in this study involving human participants were in accordance with the ethical standards of the 1964 Helsinki Declaration and its later amendments or comparable ethical standards (World Medical Association, 2013). The study was approved by the involved school boards and by the Committee on Ethics in Educational and Psychological Research at the University of Montréal (#CEREP-22-123-D).

Consent to Participate

All participants were informed that their involvement was voluntary and confidential and provided an electronic informed consent. All participants also agreed that the aggregate data (i.e., combined with data from other participants and not individually identifiable) could be used for academic purposes, including reports, presentations, and public documentation.

Author contributions

P.A.: Writing—original draft preparation, Data analyses. D.M.: Conceptualization, Writing—reviewing and editing, Supervision. C.R.: Conceptualization, Writing—reviewing and Editing, Supervision. All authors contributed to the study design, drafting the paper, revising it for important intellectual content, and gave final approval for publication.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Our work was funded by an Insight Development grant awarded to D.M. by the Social Sciences and Humanities Research Council of Canada (SSHRC).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The study participants did not consent to having their data shared publicly. The deidentified participant data set from this study can be made available to researchers upon a reasonable request to the corresponding author.

Supplemental Material

Supplemental material for this article is available online.

References

Adu

Popoola

Bartholomew

Iqbal

Roemer

Jurcik

Collings

Aspin

Medvedev

O. N.

Simpson

C. R.

(2024a). Enhancing the Precision of the Self-Compassion Scale Short Form (SCS-SF) with Rasch Methodology. Mindfulness. Advance online publication. https://doi.org/10.1007/s12671-024-02462-y

Adu

Popoola

Collings

Aspin

Medvedev

O. N.

Simpson

C. R.

(2024b). Psychometric properties of the motors of COVID-19 vaccination acceptance scale in New Zealand: Insights from confirmatory factor analysis. Current Psychology. Advance online publication. https://doi.org/10.1007/s12144-024-05877-x

Adu

Popoola

Roemer

Collings

Aspin

Medvedev

O. N.

Simpson

C. R.

(2023). Validation and cultural adaptation of the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S) in German. Psychological Test Adaptation and Development, 4(1), 319–329. https://doi.org/10.1027/2698-1866/a000064

Bahamón

M. J.

Cudris-Torres

Javela

J. J.

Ortega-Bechara

Cabezas-Corcione

Barrios-Nuñez

Á.

Bermúdez

(2025). Life satisfaction in children: An analysis of the psychometric properties of the SWLS-C scale by Diener. BMC Psychology, 13(1), Article 383. https://doi.org/10.1186/s40359-025-02690-4

Bhui

Warfa

Jones

(2014). Is violent radicalisation associated with poverty, migration, poor self-reported health and common mental disorders? PLOS ONE, 9(3), Article e90718. https://doi.org/10.1371/journal.pone.0090718

Boone

W. J.

(2016). Rasch analysis for instrument development: Why, when, and how? CBE—Life Sciences Education, 15(4), Article rm4. https://doi.org/10.1187/cbe.16-04-0148

Bramsen

(2019). Avoiding violence: Eleven ways activists can confine violence in civil resistance campaigns. Conflict Resolution Quarterly, 36(4), 329–344. https://doi.org/10.1002/crq.21254

Brislin

R. W.

(1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1(3), 185–216. https://doi.org/10.1177/135910457000100301

Gopal

Verma

(2017). Political Participation: Scale development and validation. International Journal of Applied Business and Economic Research, 21(2), 391–403.

10.

Hagell

Westergren

(2016). Sample size and statistical conclusions from tests of fit to the Rasch model according to the Rasch unidimensional measurement model (Rumm) program in health outcome measurement. Journal of Applied Measurement, 17(4), 416–431.

11.

Hong

Rutherford

Cebrian

(2021). Social mobilization and polarization can create volatility in COVID-19 pandemic control. Applied Network Science, 6, 1–13. https://doi.org/10.1007/s41109-021-00356-9

12.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

13.

Kriesi

(2016). Political mobilization in times of crises: The relationship between economic and political crises. In Giugni

Grasso

(Eds.), Austerity and protest (pp. 19–33). Routledge.

14.

C. H.

(2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369–387. https://doi.org/10.1037/met0000093

15.

Little

R. J.

(1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.1080/01621459.1988.10478722

16.

Little

R. J.

Rubin

D. B.

(2019). Statistical analysis with missing data. John Wiley & Sons.

17.

Lundgren-Nilsson

Å.

Jonsdottir

I. H.

Ahlborg

Tennant

. (2013). Construct validity of the psychological general well-being index (PGWBI) in a sample of patients undergoing treatment for stress-related exhaustion: A rasch analysis. Health and Quality of Life Outcomes, 11, 1–9. https://doi.org/10.1186/1477-7525-11-2

18.

Lee

J. H.

Huber Jr

J. C

. (2021). Evaluation of multiple imputation with large proportions of missing data: How much is too much?. Iranian journal of public health, 50(7), 1372. https://doi.org/10.18502/ijph.v50i7.6626

19.

Malan

Smuts

C. M.

Baumgartner

Ricci

(2020). Missing data imputation via the expectation-maximization algorithm can improve principal component analysis aimed at deriving biomarker profiles and dietary patterns. Nutrition Research, 75, 67–76. https://doi.org/10.1016/j.nutres.2020.01.001

20.

McCauley

Moskalenko

(2008). Mechanisms of political radicalization: Pathways toward terrorism. Terrorism and Political Violence, 20(3), 415–433. https://doi.org/10.1080/09546550802073367

21.

Medvedev

O. N.

Krägeloh

C. U.

Titkova

E. A.

Siegert

R. J.

(2020). Rasch analysis and ordinal-to-interval conversion tables for the Depression, Anxiety and Stress Scale. Journal of Health Psychology, 25(10–11), 1374–1383. https://doi.org/10.1177/1359105318755261

22.

Medvedev

O. N.

Krägeloh

C. U.

(2022). Rasch measurement model. In Handbook of assessment in mindfulness research (pp. 1–18). Cham: Springer International Publishing.

23.

Miconi

Njingouo Mounchingam

Zambelli

Rousseau

(2024). Disentangling support for violent and non-violent radicalization among adolescents: A latent profile analysis. Journal of youth and adolescence, 53(9), 1953–1970. https://doi.org/10.1007/s10964-024-01988-7

24.

Mindrila

(2010). Maximum likelihood (ML) and diagonally weighted least squares (DWLS) estimation procedures: A comparison of estimation bias with ordinal and multivariate non-normal data. International Journal of Digital Society, 1(1), 60–66.

25.

Moskalenko

McCauley

(2009). Measuring political mobilization: The distinction between activism and radicalism. Terrorism and Political Violence, 21(2), 239–260. https://doi.org/10.1080/09546550902765508

26.

Opozda-Suder

Łopatkiewicz

Michel

(2024). The Activism and Radicalism Intention Scales (ARIS) as a Measure of Political Mobilization: Verification of Factor Structure and Invariance across Gender in a Sample of Polish Academic Youth. Terrorism and Political Violence. Advance online publication. https://doi.org/10.1080/09546553.2024.2308795

27.

Ozer

Obaidi

Anjum

(2024). Extreme reactions to globalization: Investigating indirect, longitudinal, and experimental effects of the globalization–radicalization nexus. Personality and Social Psychology Bulletin, 50(12), 1635–1660. https://doi.org/10.1080/09546553.2024.2308795

28.

Pachter

L. M.

Szalacha

L. A.

Bernstein

B. A.

García Coll

(2010). Perceptions of Racism in Children and Youth (PRaCY): Properties of a self-report instrument for research on children’s health and development. Ethnicity & Health, 15(1), 33–46. https://doi.org/10.1080/13557850903383196

29.

Pavlović

Moskalenko

McCauley

(2022). Bifactor analyses provide uncorrelated measures of activism intentions and radicalism intentions. Dynamics of Asymmetric Conflict, 15(2), 123–140. https://doi.org/10.1080/17467586.2021.1980220

30.

Pelton

(2002). Where are the limits to the Rasch advantage? Paper presented at the International Objective Measurement Workshop.

31.

Pozzoli

Gini

Thornberg

(2016). Bullying and defending behavior: The role of explicit and implicit moral cognition. Journal of School Psychology, 59, 67–81. https://doi.org/10.1016/j.jsp.2016.09.005

32.

Rasch

(1960). Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests. Nielsen & Lydiche.

33.

Rönkkö

Cho

(2022). An updated guideline for assessing discriminant validity. Organizational Research Methods, 25(1), 6–14. https://doi.org/10.1177/1094428120968614

34.

Rousseau

Hassan

Miconi

Lecompte

Mekki-Berrada

El Hage

Oulhote

(2019). From social adversity to sympathy for violent radicalization: The role of depression, religiosity and social support. Archives of Public Health, 77, 1–12. https://doi.org/10.1186/s13690-019-0372-y

35.

Rousseau

Miconi

Frounfelker

R. L.

Hassan

Oulhote

(2020). A repeated cross-sectional study of sympathy for violent radicalization in Canadian college students. American Journal of Orthopsychiatry, 90(4), 406–418. https://doi.org/10.1037/ort0000444

36.

Schmid

A. P.

(2013). Radicalisation, de-radicalisation, counter-radicalisation: A conceptual discussion and literature review. ICCT research paper, 97(1), 22.

37.

Soper

D. S.

(2021). A-priori sample size calculator for structural equationmodels [Software] https://www.danielsoper.com/statcalc/calculator.aspx?id=89

38.

Tennant

Conaghan

P. G.

(2007). The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Care & Research, 57(8), 1358–1362.

39.

Tennant

Küçükdeveci

A. A.

(2023). Application of the Rasch measurement model in rehabilitation research and practice: Early developments, current practice, and future challenges. Frontiers in Rehabilitation Sciences, 4, Article 1208670. https://doi.org/10.3389/fresc.2023.1208670

40.

Thelma

C. C.

Sain

Z. H.

Phiri

E. V.

Sylvester

Patrick

(2024). The role of ethnic identity in political mobilization: An analytical review. https://doi.org/10.5281/zenodo.13986835

41.

Turner-Stokes

Medvedev

O. N.

Siegert

R. J.

(2019). Rasch analysis of the UK Functional Assessment Measure in a sample of patients with traumatic brain injury from the UK national clinical database. Journal of Rehabilitation Medicine (Stiftelsen Rehabiliteringsinformation), 51(8), 566–574. https://doi.org/10.2340/16501977-2580

42.

Welzel

Brunkert

Kruse

Inglehart

R. F.

(2023). Non-invariance? An overstated problem with misconceived causes. Sociological Methods & Research, 52(3), 1368–1400. https://doi.org/10.1177/00491241219955

43.

World Medical Association. (2013). WMA Declaration of Helsinki—Ethical principles for medical research involving human subjects (10th ed.). https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

3.40 MB