Abstract
Objectives
Adverse event (AE) reporting is crucial to ensure patient safety and prevent future incidents, yet AEs are often underreported. Exploring healthcare professionals’ attitudes toward AE reporting is essential for addressing this gap. This study aimed to translate and validate the Italian version of the reporting of clinical adverse events scale (RoCAES), originally developed in 2008, and assess its reliability and applicability in the Italian healthcare system.
Methods
We developed the Italian RoCAES (I-RoCAES) through translation and back-translation. Healthcare professionals were given an electronic scale version created using Google Forms for anonymous data collection between October 2021 and October 2023. Confirmatory factor analysis (CFA) was conducted to evaluate the original scale's factor structure, followed by exploratory factor analysis (EFA) to refine the model. Internal consistency (Cronbach's alpha and Guttmann coefficient) and CFA results for the I-RoCAES were also analyzed.
Results
The study sample consisted of 778 healthcare professionals, including physicians, nurses, and nursing students, aged 44.2 years (SD = 15.4) on average. In total, 51.9% had been involved in AEs, but only 33.7% reported it. The data we retrieved from Italian healthcare professionals did not fit well during CFA, which led to an EFA identifying a four-factor solution that accounts for 43.9% of the variance. The I-RoCAES showed good internal consistency (Cronbach's alpha = 0.79; Guttmann coefficient = 0.77) and improved CFA fit indices.
Conclusions
Due to its good psychometric properties, the I-RoCAES can be utilized to study and promote AE reporting among Italian healthcare professionals.
Background
Adverse events (AEs), which are common and harmful outcomes in any medical system, can be caused by medical care, including procedures, surgery, and medication, 1 and affect at least 10% of patients. 2 Healthcare professionals can experience AEs in all areas of care, including clinical, surgical, pediatric, territorial, outpatient, primary care, and intensive care settings.3–5 Whether AEs are preventable or not, they are often connected to human events or general structural or systemic problems. 6 Medical errors can lead to AEs, which may be caused by poor patient communication, improper orders, or case documentation. 7
About 8% of AEs lead to death, with an estimated 50% of them being preventable. 8 Surgical specialties, medications, and fluid-related and healthcare-associated infections are the most frequently reported AEs. 9
Reporting AEs is a crucial aspect of clinical governance and helps improve the safety and quality of services by allowing staff to learn from past errors. 10 Several healthcare systems have developed reporting and learning systems to enable healthcare staff to record events and identify patterns needing correction.11–13 Despite a no-blame culture's constant diffusion, scientific evidence suggests that many events are not reported, posing a threat to the credibility of reporting and learning systems.14–16
The importance of incident reporting systems for clinical risk management and patient safety has been highlighted from a cultural and juridical perspective, with mandatory implementation in several countries, including Italy, since 2017. 17 Incident reporting objectives require targeted reporting, robust analysis, healthcare professional engagement, and visible, sustainable actions.18,19 In spite of the use and operation of incident reporting systems in healthcare systems, the voluntary reporting of AEs could be a potential source of confounding because of selection bias and underreporting. 20 Consequently, to promote a safety culture it is necessary to understand employees’ attitudes toward AE reporting. 21 This information could assist in better understanding the factors that contribute to a poor culture of reporting AEs by identifying areas for possible intervention and implementation of prevention strategies.
Assessing the attitudes of health professionals toward reporting AEs requires tools that are not yet available in Italian. The reporting of clinical adverse events scale (RoCAES) was developed and validated in the UK in 2008 and assessed five domains of staff attitudes to AE: perceived blame (six items); perceived criteria for identifying events that should be reported (six items); perceptions of colleagues’ expectations (six items); perceived benefits of reporting (five items); and perceived clarity of reporting procedures (two items). 22 The RoCAES has already been translated and validated for use by health professionals in China, Germany, and Poland.23–25 The original RoCAES was tested in the UK on 201 health professionals (61% nurses/nurse-midwives, 37% doctors). 22 The German validation study included 120 health professionals and students who worked in the inpatient care sector. 24 A sample of 1557 nurses was used to complete the RoCAES Chinese validation study, 23 while the more recent Polish version involved 101 participants (41% nurses, 59% nursing students). 25
The present study aims to translate and validate the Italian version of the RoCAES (I-RoCAES) questionnaire. 22 To do so we aimed at testing the I-RoCAES on a large mixed sample of healthcare personnel including doctors, nurses, and nursing students.
Methods
The original RoCAES 22 was translated from English to Italian following international translation recommendations.26–28 Two bilingual translators performed a back-to-back translation, and another consensus meeting was held to discuss differences and unify an Italian consensus version. Independent experts conducted pilot testing on an Italian version draft to evaluate the clarity of instructions, items, and response format. To collect data, the questionnaire was converted into an electronic Google Forms version. Between October 2021 and October 2023, the link to the online survey was emailed to physicians, nurses, and medical and nursing students who were asked to complete it. Participation was voluntary and anonymous, and all the recipients were invited to share the link with their teamwork (snowball sampling). Completing the I-RoCAES items section was mandatory, ensuring there were no missing values. The online form provided a preliminary question that inhibited repeated compilation by the same subject and comprised of sections to acquire information about demographic characteristics and professional information, previous involvement in or reported AEs, and future reporting intention.
According to the original scale, the I-RoCAES consisted of 25 items rated on a 4-point Likert scale corresponding to four possible judgments: strongly agree (score = 1), agree (score = 2), disagree (score = 3), or strongly disagree (score = 4), of which 15 required reversed scoring. Total scores were calculated by summing up all the scores for each item. Following the German validation study, 24 we assumed that the scoring guidelines for item 4 (“reporting adverse events protects the patient”), which results in a positive attitude, were switched in the original study with item 3 (“It is not my responsibility to report adverse events involving colleagues”) that is formulated negatively, so we reversed the score for these two items for our analysis. Factor scores were calculated from statements within each dimension by summing all items within a dimension, having reverse-scored negative-phrased statements.
A confirmatory factor analysis (CFA) of the original factor structure of the RoCAES 22 was performed to assess the factor structure of the measurement model. Model fit was evaluated through various indices, including chi-square (χ2), root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index, and Tucker–Lewis index. Subsequently, an exploratory factor analysis (EFA) was performed to improve and modify the model through the eigenvalue method >1 to extract the factors, also observing the scree plot. The principal component analysis with varimax orthogonal rotation was conducted to find a satisfactory factor structure. Residuals and covariances were calculated within the EFA to modify the existing factor model. Items with factor loading scores >0.40 were reported and retained for factor contribution. Model modifications were applied based on EFA, and then the modified model's fit was re-evaluated.
Cronbach's alpha coefficient was calculated to verify internal consistency and Guttmann split-half reliability coefficient. Differences between groups in continuous variables were analyzed using a Kruskal–Wallis test. A chi-square or fisher's exact test was used to compare categorical variables. The P-value was set to 0.5; all tests were two-tailed. Data were analyzed using SPSS version 25.0 and Jamovi version 2.3.13 to conduct CFA.
An exploratory post-hoc subgroup analysis was conducted to examine potential differences in I-RoCAES scores between physicians and nurses. To do so, a random sample of physicians was selected to match the number of nurses using standard randomization procedures available in SPSS. Group comparisons were performed using a Mann–Whitney U test to evaluate the consistency of the scale's constructs between the two professional groups.
Results
Out of the 816 respondents who answered the survey, only 778 healthcare professionals and students provided valid responses; therefore, the responses of the other 38 individuals were excluded. The study sample, aged 44.2 years (SD = 15.4; range 19–79) on average, was nearly evenly divided between sexes, with 49% female participants. The sample comprised of participants from various professions and levels of work experience, including 70.1% physicians, 10.4% nurses, and 18.5% nursing students (Table 1). The participants declared an average work experience of 17.8 years (SD = 14.1; range 0‒50), with more than half (51.9%) reporting a previous involvement in an AE during their professional career. Among participants who reported having previously experienced AEs, only 33.7% indicated that they had reported the event, while 42.9% did not (23.4% unanswered). In terms of the likelihood of reporting an AE in the future, 70% of participants responded positively.
Sociodemographic data of the study sample (N = 778).
Confirmatory factor analysis
Before conducting CFA, the normal distribution of each I-RoCAES item was assessed, and none exhibited a skewness value greater than the cut-off of |3|. CFA was conducted following the factor structure of the English version of the RoCAES. 22 Initial CFA results indicated a poor model fit for the original structure within the Italian context (see Table 2). However, the modified I-RoCAES model demonstrated an improved fit, as evidenced by a reduction in the RMSEA from 0.0664 to 0.0622, indicating a closer alignment between the observed data and the model's predictions. The reduction in the chi-square value from 1072 to 898 also revealed a significant enhancement in the model's overall fit, suggesting a better match between the observed data and the model, resulting in a more accurate representation of the data structure. The I-RoCAES also exhibited a lower SRMR, reflecting a more accurate correspondence between the modified model's implied covariance structure and the observed data and a lower Akaike information criterion value, suggesting a more optimal balance between model fit (see Table 2). Therefore, the I-RoCAES demonstrates a better fit in the Italian context, attributable to its enhanced contextual relevance and improved statistical fit indices.
Fit indices for confirmatory factor analysis models in an Italian sample (N = 778): comparison of the original five-factor RoCAES and the revised four-factor I-RoCAES.
Note: RoCAES: The reporting of clinical adverse events scale; I-RoCAES: Italian version of reporting of clinical adverse events scale; P values by chi-square test; df: degrees of freedom; RMSEA: root mean square error of approximation; SRMR: standardized root mean square residual; CFI: Bentler's comparative fit index; TLI: Tucker–Lewis index; AIC: Akaike's information criterion.
Exploratory factor analysis
The Kaiser–Meyer–Olkin (KMO) index (KMO = 0.87), and Bartlett's test of sphericity (P < 0.001) revealed that the data were suitable for EFA. 29 The analysis showed that seven factors outweighed the eigenvalue (>1.0) obtained by chance. Consequently, we forced a seven-factor solution, which explained 56.9% of the variance. Three of the seven factors had all or most of their items with very low loads, so a solution of four factors, which explained 43.9% of the variance, was forced with a loading factor >0.4, and if the item loading was less than ±0.4 were excluded for factor structure. Specifically, items 11 and 15 had factor loadings lower than the abovementioned cut-off.
The analysis of the items included in the four factors has resulted in a consistent solution for content and interpretability, and the factors have been identified as follows: (1) “consequences of event reporting” (seven items); (2) “characteristics of the event” (eight items); (3) “benefits of event reporting” (five items); and (4) “reporting procedures” (three items). These four factors explained 43.9% of the total variance (Table 3). Differences in I-RoCAES factors and total scores among physicians, nurses, and nursing students are shown in Table 4.
I-RoCAES principal component analysis.
Note: The percentage of variance accounted for by each factor is indicated in parentheses. Loading refers to the numerical values that indicate the relationships between each item and each factor in the exploratory factor analysis.
Items requiring reverse-scoring.
I-RoCAES: Italian version of reporting of clinical adverse events scale.
Differences in I-RoCAES factors among physicians, nurses, and nursing students.
Note: I-RoCAES: Italian version of reporting of clinical adverse events scale, comprising four factors.
n = 8 are not specified.
P values by Kruskal–Wallis test; ns: non-significant.
Cronbach's alpha revealed a good I-RoCAES internal consistency (alpha = 0.79), while factors revealed acceptable to low internal consistency (factor 1: 0.59; factor 2: 0.76; factor 3: 0.61; factor 4: 0.59). Guttmann split-half reliability coefficient was 0.77. The 25-item I-RoCAES is attached in the Appendix.
Due to the significant disparity in the frequency of responses between physicians (n = 545) and nurses (n = 81), we randomly selected a subsample of n = 81 physicians to compare to the total sample of nurses. Mann–Whitney U test disclosed no significant differences between physicians and nurses’ I-RoCAES factor scores (factor 1: P = 0.190; factor 2: P = 0.775; factor 3: P = 0.432; factor 4: P = 0.106) whereas nurses reported significant slightly higher I-RoCAES total scores (M = 58.46 ± 4.85 vs M = 57.30 ± 4.31, P < 0.05) (Table 5).
Comparative analysis of I-RoCAES factor scores between nurses and a randomized subsample of physicians.
Note: The nurse group (n = 81) includes the entire subset of nurses from the total sample (N = 778). The physician group (n = 81) was selected randomly from the entire group of physicians (n = 545) using SPSS randomization tools.
I-RoCAES: Italian version of reporting of clinical adverse events scale, comprising four factors.
P values by Mann–Whitney U test; ns: non-significant.
Discussion
The I-RoCAES questionnaire was formulated and implemented in this study on a large mixed sample of health personnel, an outcome that has been desired since the original scale was created.
The original research and subsequent validation studies have highlighted the importance of expanding the investigation of the RoCAES to different populations. The present study involved a variety of healthcare personnel, such as physicians, nurses, and students. Previous RoCAES translation and validation studies relied on a smaller number of participants (English: 201; German: 120; Polish: 101), while the Chinese study was conducted on 1557 nurses.22–25
The I-RoCAES (N = 778) proved good internal consistency (Cronbach's alpha = 0.79) and adequate split-half reliability. Cronbach's alpha value was comparable to the English RoCAES (0.83), Chinese RoCAES (0.85), Polish RoCAES (0.82), and German RoCAES (0.79).22–25 Nonetheless, the initial CFA demonstrated a poor match between our data and the original five-factor structure. This result aligns with the findings of the Chinese validation study, which suggest that this outcome may be due to specific elements in the RoCAES that do not sufficiently capture the reporting attitudes among different cultural contexts. 23
The subsequent EFA yielded a four-factor solution, which comprised of factors that were interpreted and identified four main domains different from the original scale: consequences of event reporting, characteristics of the event, benefits of event reporting, and reporting procedures.
Comparing the individual factors of the I-RoCAES with those of the original scale, factor 1 was almost identical in both scales, except for the addition of item 24, “Reporting adverse events makes people accountable for their actions,” in the I-RoCAES. The inclusion of item 24 in factor 1 could be attributed to a different interpretation of the term's original meaning, “accountable,” which might have been understood differently in the two questionnaires.
In factor 2 of the I-RoCAES, all original items were retained except for item 15, which was removed due to low factor loading. Additionally, items 6, 22, and 19 were included, resulting in consistency and appropriateness for factor 2, which focused on the perceived criteria for identifying events that should be reported.
Factor 3 of I-RoCAES closely resembled factor 4 of the original scale, and both focused on the perceived benefits of reporting. Both factors consisted of five items, four identical (Items 1, 4, 16, and 17). Specifically, item 24 was also included in factor 3 of the original scale, while in I-RoCAES, item 9 (“I am not doing my job properly unless I report adverse events”) was included. Both were coherent with the factor's label of the respective scale. One potential reason for these variations could be the potential misinterpretation of the term “accountable” in item 24 by Italian healthcare personnel, as discussed in factor 1. Furthermore, item 9, “I am not doing my job properly unless I report adverse events,” was categorized under factor 3 in the original scale, relating to colleagues’ expectations. In contrast, the I-RoCAES presents this idea more comprehensively as a key part of the advantages of reporting.
Factor 4 of the I-RoCAES aligns with factor 5 in the original scale and encompasses the two items from factor 5 of the original scale (items 8 and 13) along with item 25 (“Colleagues seem unconcerned when adverse events occur”).
In addition, two items were excluded from the I-RoCAES four-factor solution due to low factor loadings observed in the EFA (item 11: “My colleagues expect me to report adverse events”; item 15: “Writing in a patient's notes that an adverse event has happened is just as good as filling in a separate reporting form”). These items were included in factors 2 and 3 of the original scale and subsequent scales and were related to criteria for identifying events that should be reported and the colleagues’ perceptions about the reporting. The results suggest that the lower factor load on factors 11 and 15 examined in I-RoCAES may be due to cultural differences identified during reallocation and the inclusion of participants from different professional backgrounds. 30 For example, item 11, “My colleagues expect me to report adverse events,” might be influenced by the blame culture identified in several studies among healthcare workers, which is closely associated with the underreporting phenomenon. 31 Studies have found that reporting systems detect 7–15% of AEs, 32 which may reflect a healthcare culture that leads to blame rather than identifying areas for learning and improvement. 31
Similarly, item 15, “Writing in a patient's notes that an adverse event has happened is just as good as filling in a separate reporting form,” might be driven by a lack of knowledge about proper reporting procedures, highlighting the need for clinical risk management training programs for all healthcare workers. Furthermore, numerous studies have explored strategies to encourage healthcare providers to submit incident reports, uncovering several barriers, including the lack of knowledge about reporting procedures, limited time, and the absence of feedback from the incident reporting system, all of which reduce the motivation to report incidents.30,33 Successful improvement in patient safety through the analysis of incident reports is only possible by achieving a no-blame culture. 31
Compared to previous studies, the sample in this study shows some differences, including a relatively greater presence of physicians, which could influence the interpretation of interprofessional dynamics. 34 The diversity of roles within our sample provided an opportunity to examine whether professional background might influence attitudes toward AE reporting. While previous international validation studies of the RoCAES have predominantly focused on nurses and nursing students, we conducted an internal comparison between nurses and a matched, randomly selected group of physicians from the larger dataset. The analysis did not reveal significant differences across the four factors of the I-RoCAES, while the slight difference in the total score—higher among nurses—invites reflection on the enlightening influence of professional culture and hierarchy on attitudes toward AE reporting.
Previous studies have highlighted how physicians may be less likely to report incidents, such as medication errors, patient falls, or adverse drug reactions, due to concerns related to accountability, reputational risk, or legal consequences.15,31 In contrast, nurses are frequently more engaged in frontline risk management processes and may perceive the reporting of incidents as a more integral aspect of their professional responsibilities. 33 However, it is essential to acknowledge that shared perceptions measured by the I-RoCAES do not necessarily reflect equality in reporting behaviors, which are still shaped by institutional norms, workplace dynamics, and medico-legal frameworks.
In this light, the robustness of the I-RoCAES across roles supports its broader applicability while also underscoring the need for culturally and professionally sensitive strategies when interpreting or applying its results at the operational level.
In light of the results from our validation study, the I-RoCAES may serve as a practical screening tool for identifying potential areas for improvement and further investigation within Italian healthcare settings, offering insights that are sensitive to the cultural and organizational specificities of the national context. As recommended by Litke et al., the scale's brevity and specificity make it well-suited for inclusion in broader institutional surveys, such as those addressing safety culture or related subjects. 24 This can facilitate the exploration of cultural nuances and enable tailored interventions based on survey findings to improve patient safety and healthcare quality. 32
This study has several limitations. First, although the sample included a variety of healthcare professionals, it was predominantly composed of physicians, which may limit the generalizability of the findings to the broader healthcare personnel. Second, the single-center design may constrain the applicability of the results to other institutional or regional contexts. Third, the use of self-reported data introduces potential biases, such as social desirability and recall bias, which may affect the accuracy of the responses. Finally, the findings should be interpreted in light of the specific organizational culture, policies, and resource availability of the participating institution.
Conclusion
The validation of the I-RoCAES scale represents an important step toward enhancing healthcare professionals’ understanding and reporting of AEs in Italy. Implementing the I-RoCAES in Italian healthcare facilities as a practical screening tool can help identify areas requiring further investigation. The I-RoCAES represents a key instrument for fostering a culture of safety and supporting continuous improvement in clinical risk management within Italian healthcare institutions, ultimately enhancing patient safety through more effective AE reporting.
Supplemental Material
sj-pdf-1-cri-10.1177_25160435251365981 - Supplemental material for The reporting of clinical adverse events scale (I-RoCAES): Validation and psychometric characteristics of the Italian version
Supplemental material, sj-pdf-1-cri-10.1177_25160435251365981 for The reporting of clinical adverse events scale (I-RoCAES): Validation and psychometric characteristics of the Italian version by Davide Ferorelli, Luigi Buongiorno, Federica Mele, Biagio Solarino, Giampiero Bottari, Lorenzo Spagnolo, Alessandro Dell’Erba, Roberto Catanesi and Gabriele Mandarelli in Journal of Patient Safety and Risk Management
Footnotes
Acknowledgment
The authors sincerely acknowledge the Bari Medical Council (OMCEO Bari) for their invaluable support and collaboration during this study.
Ethical considerations
The module tutors approved contacting the student. For the participation of clinical staff, permission of the respective staff council was asked. The study was conducted in accordance with the Declaration of Helsinki, and approval from the local ethics committee was not deemed necessary.
Consent to participate
As the survey was conducted using an anonymous online questionnaire, participants gave their consent by actively sending their completed questionnaires to the study center. All participants were informed about this procedure in writing and were given the opportunity to ask questions.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data availability
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
