Bullying experiences of dental interns working at four dental institutions of a developing country: A cross-sectional study

Abstract

BACKGROUND:

Bullying is an aggressive and violent behavior marked by repetitive harassment of a weaker victim, which may also occur in the workplace including healthcare settings. Although extensively studied in the west, bullying of workers in the healthcare setting is largely underexplored in the South Asian context.

OBJECTIVE:

The aim of our study was to explore the phenomena of workplace bullying among dental interns in selected dental institutes of Karachi, Pakistan.

METHODS:

A cross-sectional study was conducted among dental interns working at four dental institutions in Karachi, Pakistan. The Negative acts questionnaire (NAQ-R), a standardized, validated tool was administered to identify bullying experiences among dental interns over the past six months; in addition, demographic information of participants as well as details about their exposure to bullying was collected. A multivariable binary logistic regression was used to identify the correlates of bullying in this population. The study was performed and reported according to the STROBE guidelines. Data was analyzed using STATA 12.0 and SPSS 19.0.

RESULTS:

A total of 125 participants were included in our analysis. Bullying prevalence among dental interns based on the operational definition by Mikkelsen stood at 36.8%, while self-labelled bullying was observed in 55 %. Males and participants from private institutions were more likely to self-label themselves as victims. 67% of respondents reported having witnessed bullying. Clinical faculty was identified as the most common perpetrator (23%) followed by colleagues 20% and the dental support staff 17%. Report of bullying among victims was low (14.5%) the most common reason being that “complaining is of no use” (28.8%) and “being afraid of the consequences” (22%).

CONCLUSIONS:

The results of the study indicate a high prevalence of bullying in the participating dental institutions. Our results indicate a clear need to implement antiviolence regulations, anti-bullying educational programs and advocate further research on interventions to minimize bullying, enhance learning and professional engagement of interns in dental institutions.

Keywords

Bullying workplace violence mobbing work harassment emotional abuse occupational health dental staff hospital Pakistan

1 Introduction

Bullying is a complex aggressive violent, antisocial behavior marked by repetitive harassment of a weaker victim without any provocation [1]. It is a subset of repetitive aggressive and hostile behaviors with an imbalance of power making it difficult for the victims to defend themselves [2]. “Workplace bullying” means harassing, offending or socially excluding someone or negatively affecting work tasks of victims. The term mobbing, referring to emotional abuse, work harassment and psychological harassment has also been used to describe the same experience [3, 4]. The phenomenon of bullying is broad and complex and is characterized by a) Presence of a power differential between the perpetrator and victim b) persistent and frequent negative behavior that is goal directed c) negative acts that may range from subtle to the overt abuse d) intentional nature of the harm [3 , 6].

Bullying exposures may be direct, indirect or a combination of the two, based on the observed behavior. Direct forms include physical aggression such as hitting, harassment, and belittling comments in front of others, whereas indirect forms includes covert behaviors like spreading rumors, assigning unrealistic work targets, mean gestures, posing barriers, discrimination and social isolation [5 , 7–9]. Globally, workplace bullying has been recognized as an occupational health issue with devastating consequences (Fig. 1) for all the people involved [7 –13]. The questionnaire method is the most common means to investigate the exposure to workplace bullying, which may be based on either self-labelling by respondents or standardized, operational definitions [13].

Fig.1

Consequences of bullying.

Considerable research has been carried out in developed countries exploring the prevalence and experience of bullying in workplaces. A study conducted at a Finish University indicated 5% bullying prevalence in which teaching faculty identified as the most common perpetrator [14]. A multinational study involving dental schools indicated 35% prevalence of bullying in which common perpetrators are fellow students or a member of teaching staff [15]. In a Czech University 7.9% of the employees reported exposure to bullying during the past twelve months and mostly by their colleagues [16]. In contrast to the developed countries, the phenomenon is largely underexplored in the South Asian sub-continent including Pakistan; to date, only a few studies have assessed bullying exposures among medical undergraduate students, postgraduate trainees and doctors in Pakistan, reporting bullying to be as high as 63–66% [17 –19].

To date, no research has explored workplace bullying among young dental graduates in the healthcare sector. The rapid growth of dental teaching institutions and affiliated hospitals in Pakistan recently has led to considerable growth in this workforce presenting a crucial research and knowledge gap when it comes to the phenomenon of bullying. Through this study, we aim to explore bullying among dental graduates in a workplace setting from the largest city of Pakistan through addressing the following research questions: 1) What is the frequency of workplace bullying experienced by dental interns? 2) What are the correlates of bullying exposures in this population? 3) Who are the common perpetrators of bullying at the dental workplace? 4) What are the commonly perceived barriers in lodging the complaint against the perpetrator?

2 Materials and methods

An analytical, cross-sectional study was conducted between February and May, 2016 in four dental institutes of Karachi, Pakistan following ethical approval by the Institutional review board, Jinnah Sindh Medical University (JSMU/IRB-FT/2015/003). Selection of dental institutes was based on administrative and ethical approvals at each study site. Our study population was defined as graduate dentists working as interns in dental teaching hospitals in Karachi, Pakistan.

2.1 Eligibility criteria

Eligible respondents included dental interns of either sex, having a valid practicing license and at least 6 months’ work experience of clinical rotations in their respective institution. The minimum work experience requirement was put in place because the time reference for which the Negative Acts questionnaire measures bullying exposures covers the previous 6 months. We excluded undergraduate dental students and post graduate trainees, as well as fresh dental graduates who were doing clinical observer-ship. To ensure maximum participation, the participating institutes were informed of the survey date ahead of the actual visit. We included all eligible, consenting participants who were present on the day of the survey using a purposive sampling approach.

2.2 Data collection

A structured, self-administered interview questionnaire was provided to study participants following a written, informed consent. The questionnaires and consent forms were administered in English since all our participants were able to understand the content. To clarify the terms and study objectives, the purpose of the study, definition of bullying and the time reference of six months were mentioned in the consent form as given by Mikkelsen and Einarsen bullying is defined as a situation in which one or several individuals persistently over a period of time perceive themselves to be on the receiving end of the negative actions from one or several individuals, in a situation in which victim has difficulty in defending him or herself [20]. The interview questionnaire comprised two sections. Section 1 included questions relating to participants’ demographic information, including age, gender, educational background and family structure. Section 2 contained the Negative Acts Questionnaire-Revised (NAQ-R) which was used with permission from the Bergen Bullying research group. The NAQ-R is a reliable, validated questionnaire containing 22 Likert type questions that assess both indirect and direct forms of workplace bullying and negative acts experienced by the respondent [21]. The responses for each item reflect an increasing frequency of exposure, ranging from “never” coded as “1” to “daily” coded as “5”. We used the operational definition of bullying by Mikkelsen and Einarsen to dichotomize respondents into those who were bullied and those who were not. According to the operational definition, respondents were labelled as bullying victims if they reported exposure to at least two or more negative acts per week [6]. We also used additional questions from the NAQ-R in order to measure i) self-labelled bullying by the respondents, in which participants themselves responded to a direct question asking whether they had been bullied or not, based on a provided definition of the term “bullying” ii) information on type and number of perpetrators, iii) whether complaints against the perpetrator were lodged, and iv) common reasons for non-complaint. A separate question assessing whether respondents had witnessing bullying of their co-workers was also asked.

2.3 Sample size

We calculated a minimum sample size requirement of 125 respondents using the formula for single sample proportions, $[n = (Z_{α / 2} \sqrt{p . q}) / b^{2}]$ in order to estimate the prevalence of bullying (p) between 63–66% as previously reported among healthcare workers [17 –19], assuming a 95% level of confidence (α) and 8% margin of error (b). The sample size calculation was performed using OpenEpi software.

2.4 Statistical analysis

Descriptive statistics were calculated as mean±SD for quantitative variables, and absolute & relative frequencies for categorical variables. Prevalence of bullying among the dental interns was estimated using the operational definition given by Mikkelsen and Einarsen, described above. In addition, bullying prevalence based on respondents’ self-labelling method was also calculated. We used the Cohen’s kappa coefficient to assess the degree of agreement between these two methods.

Information on type and gender of perpetrators, registration of complaints following bullying and reasons for non-complaint were analyzed. A separate question was also asked regarding witnessing bullying of others. We calculated the frequency of each negative act on the NAQ-R for all study participants. In order to identify the independent predictors of bullying among the dental interns, we performed a multi-variable, binary logistic regression analysis using a stepwise approach. Crude and adjusted odds ratios along with 95% confidence intervals were reported. We used p < 0.05 to denote statistical significance. The Crohnbach’s alpha was computed for the overall scale and by item wise deletion to assess the reliability of the data. The study results were reported according to STROBE guidelines [22] for cross-sectional studies. Microsoft Access was used for data entry, while all statistical analyses were performed on STATA v12.0 and SPSS v19.

3 Results

Our study was an analytical, cross-sectional survey conducted from February to May, 2016 in dental hospitals of four teaching institutes of Karachi, Pakistan. Out of a total of 152 subjects present on the day of the survey, 135 provided written, informed consent. Our statistical analyses and write up is based on data from a total of 125 participants who returned completed questionnaires (Fig. 2).

Fig.2

Flowchart of participant recruitment.

We achieved a fairly equal representation of both private and public institutes in our sample, with a mean age of 24 years and a predominance of females. In public sector institutions, the gender imbalance was particularly pronounced with male representation being less than 10%. The most of our respondents lived with their parents, and received schooling locally. The general characteristics of respondents stratified by type of institution are presented as Table 1.

Table 1

General characteristics of study participants stratified by type of institution (n = 125)

	Public (02)	Private (02)	Total (04)
No. of participants (n)	58 (46.4)	67 (53.6)	125
Mean age (SD)	23.8 (1.2)	24.2 (1.3)	24.0 (1.3)
Gender of respondent (%)
Female	50 (86.2)	37 (55.2)	87 (69.6)
Male	8 (13.8)	30 (44.8)	38 (30.4)
Place of schooling (%)
Local	56 (96.6)	60 (89.5)	116 (92.8)
Overseas	2 (3.4)	7 (10.5)	9 (7.2)
System of education (%)
Intermediate/F.Sc	49 (86.0)	56 (83.6)	105 (84.7)
O/A levels	7 (12.3)	11 (16.4)	18 (14.5)
AKU-EB	1 (1.7)	0 (0)	1 (0.8)
Live with parents (%)
Yes	52 (89.7)	56 (83.6)	108 (86.4)
No	6 (10.3)	11 (16.4)	17 (13.6)
If no, then reason* (%)
Separated/divorced	2 (33.3)	2 (18.2)	4 (23.5)
One or both parents not alive	0 (0)	2 (18.2)	2 (11.7)
One/both parents residing overseas	2 (33.3)	3 (27.3)	5 (29.4)
Others	2 (33.3)	4 (36.3)	6 (35.2)

The revised version of the Negative Acts Questionnaire (NAQ-R) was used, which is a validated instrument for the measurement of workplace bullying. The internal consistency of the 23-item scale was high (Crohnbach alpha = 0.928) which did not change with item deletion. In our study, the overall prevalence of workplace bullying over the previous 6 month period, based on the operational definition by Mikkelsen and Einarsen stood at 36.8%, and did not differ by gender, site, age, place of schooling and whether respondents lived with their family or not. On the other hand, more than 55% of respondents labelled themselves as bullied based on the self-labelling method. There was only a moderate degree of agreement between these two methods (65.6% agreement, kappa: 0.33), suggesting that respondents tend to over-report bullying when they were asked directly. Male respondents (68.4%) and those working in private sector hospitals (67.2%) were significantly more likely to self-label themselves as victims of bullying (p < 0.05) (Table 2).

Table 2

Prevalence of bullying among dental interns in four dental institutions of Karachi, Pakistan

	Operational definition		Self-labelled
	n (%)	p-value	n (%)	p-value
Overall	46 (36.8)		69 (55.2)	–
Site				0.04
Public	25 (43.1)	0.17	24 (41.4)
Private	21 (31.3)		45 (67.2)
Age
22–24 years	34 (40.0)	0.28	45 (52.9)	0.45
25 years & above	12 (30.0)		24 (60.0)
Gender				0.003
Female	30 (34.4)	0.41	43 (49.4)
Male	16 (42.1)		26 (68.4)
Place of schooling		0.82		0.50
Local	43 (37.0)		65 (56.0)
Overseas	3 (33.1)		4 (44.4)
Living with parents				0.39
Yes	40 (37.0)	0.89	58 (53.7)
No	6 (35.3)		11 (64.7)

The most commonly reported negative acts to be experienced daily included “being exposed to an unmanageable workload” (11.2%), “Excessive monitoring of work”, (8.8%), “spreading of gossip and rumors” (8%) and “being ignored or excluded” (8%). The negative acts most frequently encountered on a weekly basis included “being ordered work below level of competence” (8.8%), “practical jokes” (8.0%) and “repeated reminders of your errors or mistakes” (7.2%), The frequency of responses to each item on the NAQ-R is provided separately (Supplemental Table 1).

A high proportion of respondents (67.2%) in our sample reported witnessing bullying of their co-workers. Those who reported witnessing bullying acts were not likely to have been victims themselves (p = 0.45). Of respondents who were bullied, only 14.5% ever reported against the bullying acts while the majority chose to remain silent. The most common reasons for not reporting complaints included “complaining is no use” (28.8%), being afraid of the consequences (22.0%) and having dealt with the problem on their own (20.3%) (Fig. 3). The most common perpetrators of bullying identified by respondents was faculty, co-workers and auxiliary staff (Fig. 4). The results of multivariable logistic regression analysis are presented in Table 3. Following adjustment, we did not find any factors to be significantly associated with bullying. However, the highest odds (aOR: 1.92, 95% CI:0.87–4.24) were found among interns in private institutions, followed by male gender (aOR:1.64, 95% CI: 0.69–3.89), while only slightly higher odds were reported for interns living away from their parents or those younger than 24 years.

Fig.3

Reasons for not complaining among victims of bullying.

Fig.4

Common perpetrators identified by victims of bullying.

Table 3

Crude and adjusted logistic regression estimates for factors associated with bullying among dental interns (n:125)

	Crude OR (95% CI)	Adj. OR (95% CI)	p-value 1*
Age (22–24 years)	1.05 (0.48–2.29)	1.06 (0.46–2.41)	0.88
Male gender	1.25 (0.57–2.72)	1.64 (0.69–3.89)	0.26
Private institution	1.66 (0.80–3.43)	1.92 (0.87–4.24)	0.10
Not living with family	1.16 (0.40–3.39)	1.12 (0.37–3.38)	0.83

^*<0.05 considered statistically significant. Values presented are for the adjusted model.

4 Discussion

Bullying is one of the most commonly reported negative acts in the workplace that carries several immediate and long term consequences. Our study adds to the existing literature on bullying in the local context, particularly with respect to bullying behaviors in the academic and healthcare sector. We analyzed data from 125 participants in four public and private dental institutions of Karachi, using the NAQ-R which has been validated and used previously in different settings, including health care professionals such as nurses [23], surgical technologists, perioperative personnel and Trainee doctors [24] and consultant surgeons [21 , 26].

The major finding of this study was a high prevalence of bullying exposures among dental interns across public and private dental teaching hospitals, affecting more than one third of our study participants. Since dental interns represent the junior most workforce at dental hospitals, it may be assumed that such exposures are likely to cumulate and be at even higher levels among dentists with more work experience. We also observed a high proportion of interns reporting that they witnessed bullying of their colleagues, suggesting that these events may not necessarily occur in isolation. Self-perceived victimization was higher than actual bullying based on the operational definition; because the respondents may have different threshold together with this other potential biases such as emotional, personality factors. Even in the presence of precise definition of bullying, the respondents may apply their own definition of bullying when responding to self-labelling question. Thus justifies our reason for using an objective definition for bullying prevalence [27].

A comprehensive review by Ciby and Raya highlighted wide variations in the prevalence of workplace bullying across continents and in different settings. While lowest rates were observed in Scandinavian (2% – 14%) and European countries (8% – 42%), comparatively higher estimates were observed in North America, Africa and Asia. The highest rates (52–55%) were observed in Turkish and Pakistani employees respectively [3]. In Pakistan, Ahmer et al. [19] in their survey of 60 psychiatry residents reported bullying at 80%, while Hussain and Rahim [28] reported a prevalence of 89% among postgraduate trainees at three tertiary care hospitals. Imran and colleagues surveyed junior doctors and highlighted up to 63% prevalence of bullying over the previous 12 months [17]. Globally, a wide variation is observed in prevalence rates due to differences in methods used for instance self-labelling, behavioral experience method or a combination of both methods and use of different inventories most common among them are the Negative Acts Questionnaire, Workplace Aggression Research Questionnaire and Leymann Inventory of Psychological Terror [27]. The variation observed in the published research can partly be explained by cultural differences for instance the cultures with masculine values or higher power distance reported to have high bullying prevalence as compared to the countries with feminine values and lower power distance [3], similarly the investigators also reported that nationality and organizational culture are interrelated [29]. In the local literature, the alarmingly high prevalence could possibly be due to non-standardized measurement techniques, use of convenience sampling methods or a small sample size [20, 30]. Although our study also reports a high prevalence of bullying, our estimates are comparatively more conservative and correlate with international data.

In our study sample, teaching faculty including both junior and senior faculty members were the most commonly identified as perpetrators of bullying, jointly accounting for up to 45% of all responses. It was observed that victims were mostly subject to acts, which impeded their professional growth. This finding is consistent with previous research in Pakistan [17 , 28], as well among healthcare professionals in the UK [31] and Australia [32], which reported the highest bullying exposures in junior most cadres with consultants, professors and senior registrars implicated as perpetrators in the majority of cases. We observed an alarmingly high frequency of unreported occurrences (85.5%) and an overall apprehensive attitude of respondents in making complaints, which suggests either a lack of, or loose implementation of anti-bullying policies within the institutions resulting in poor accountability of perpetrators.

It is recognized that bullying behavior may lead to adverse outcomes such as depression, post-traumatic stress, low job satisfaction, higher absenteeism and an increased tendency to quit [31 –33]. In the health care sector, the consequences are borne by the organization and the patient as well, due to the compromise in quality of care being provided [33, 34]. At present, the mechanisms explaining bullying are complex and not fully understood. Some theories suggested that traits highlighting vulnerability among victims (anxiety, low self-esteem) may predispose them to bullying behaviors, marking them as potential targets whereas the “work environment” hypothesis implies poor organizational culture, poor job design and social climate as triggers [35]. In the context of South Asian population, high rates of bullying may also be partially explained by the commonly observed practice of intimidation and submission in the teaching and professional environment, which along with lack of mentorship and counseling fosters fear and submissiveness [17].

This is the first study from Pakistan that highlights bullying experience of interns in dental institutes through a standardized measurement tool (NAQ-R), having a minimum recall limit of six months. Bullying is still an under-researched yet relevant problem in developing countries like Pakistan. Our data provides fresh evidence of bullying in dental workplaces among the local setting, laying the groundwork for further research in this area and encourages administrative efforts to prevent the consequences of such exposures. Despite the strengths, the authors acknowledge the limitations of this study. We only focused on dental interns as a vulnerable group and thus were unable to draw comparative estimates of bullying across other groups that may have included students, trainees and staff. Furthermore, we did not assess the impact of bullying on victims, or differences with respect to the socioeconomic class or ethnicity of the victim. A direct comparison of our survey findings with studies done in other settings may not be very accurate, due to the complexities and variations in the use of measurement tools, operational criteria used to define bullying as well as differences in study methodology. Further, our current objective was to quantify the bullying experience in local context, a qualitative inquiry could provide a deeper understanding of bullying, however was not possible at this time. With a limited number of study sites, our study may have limited generalizability; however, it lays the groundwork for more comprehensive work in this area.

5 Conclusions

The consequences of bullying may have serious implications for the victim, the perpetrator, bystander and the organization. In such instances, victims may require prompt intervention, advocacy and mentoring by administrative and academic heads. Future research should also incorporate the perspective of clinical faculty, which was identified as a key perpetrator of bullying by our respondents. Our findings advocate the need of action plans developed by the institutions to minimize and address bullying. Our analysis suggests high prevalence of bullying in dental interns, and although organizations cannot be entirely held responsible for individual acts. An organizational implementation of the following reforms is advocated to curb harmful exposures among the dental workforce.

Secure mechanisms of complaint including regular meetings with the faculty, students and staff, central database for reporting because non reporting make the monitoring and resolution of bullying difficult for the stakeholders [13, 36].

Anti-violence programs and education about developing positive attitudes and behaviors through mentorship and counselling facilities. In the case of appointed faculty, the perpetrator may consider it as part of training process and not be aware of the resultant distress inflicted on the victim [36].

Clarification of ambiguities in work expectations from dental interns, with clear demarcation of unacceptable behaviors and consequences to promote a conducive working environment [37].

Bystanders should be motivated to stand against or report negative behaviors for effective intervention rather than being silent or the part of this act [38].

Conflict of interest

Authors declared no potential conflicts of interest.

Footnotes

Acknowledgments

We wish to thank all the participants who took part in this study. We also like to extend our special thanks to the heads of the dental institutions for granting permission and assistance during data collection.

The supplementary material is available in the electronic version of this article: .

References

Ramachandran

. Encyclopedia of human behavior. London, UK: Elsevier. 2012.

Vie

, Glasø

, Einarsen

. Health outcomes and self-labeling as a victim of workplace bullying. J Psychosom Res. 2011;70(1):37–43.

Ciby

, Raya

. Workplace bullying: A review of the defining features, measurement methods and prevalence across continents. IIM Kozhikode Society & Management Review. 2015;4(1):38–47.

Chirilă

, Constantin

. Understanding workplace bullying phenomenon through its concepts: A literature Review. Procedia-Social and Behavioral Sciences. 2013;84:1175–9.

Volk

, Dane

, Marini

. What is bullying? A theoretical redefinition. Developmental Review. 2014;34(4):327–43.

Nielsen

, Matthiesen

, Einarsen

. The impact of methodological moderators on prevalence rates of workplace bullying. A meta-analysis. J Occup Organ Psychol. 2010;83(4):955–79.

Vanderbilt

, Augustyn

. The effects of bullying. Paediatrics and Child Health. 2010;20(7):315–20.

Hakojärvi

, Salminen

, Suhonen

. Health care students’ personal experiences and coping with bullying in clinical training. Nurse Educ Today. 2014;34(1):138–44.

O’Moore

, Seigne

, McGuire

, Smith

. Victims of workplace bullying in Ireland. The Irish Journal of Psychology. 1998;19(2-3):345–57.

10.

Huntingdon

, Petherick

. Serial Bullying and Harassment. In: Profiling and serial crime. Oxford, UK: Elsevier. 2012, pp. 263–93.

11.

Samnani

, Singh

. 20 years of workplace bullying research: A review of the antecedents and consequences of bullying in the workplace. Aggression and Violent Behavior. 2012;17(6):581–9.

12.

Kivimaki

, Elovainio

, Vahtera

. Workplace bullying and sickness absence in hospital staff. Occup Environ Med. 2000;57(10):656–60.

13.

Arnetz

, Hamblin

, Ager

, Aranyos

, Essenmacher

, Upfal

, Luborsky

. Using database reports to reduce workplace violence: Perceptions of hospital stakeholders. Work. 2015;51(1):51–9.

14.

Sinkkonen

, Puhakka

, Meriläinen

. Bullying at a university: Students’ experiences of bullying. Studies in Higher Education. 2014;39(1):153–65.

15.

Rowland

, Naidoo

, AbdulKadir

, Moraru

, Huang

, Pau

. Perceptions of intimidation and bullying in dental schools: A multi-national study. Int Dent J. 2010;60(2):106–12.

16.

Zabrodska

, Kveton

. Prevalence and forms of workplace bullying among university employees. Employee Responsibilities and Rights Journal. 2013;25(2):89–108.

17.

Imran

, Jawaid

, Haider

, Masood

. Bullying of junior doctors in Pakistan: A cross-sectional survey. Singapore Med J. 2010;51(7):592–5.

18.

Mukhtar

, Daud

, Manzoor

, Amjad

, Saeed

, Naeem

, Javed

. Bullying of medical students. J Coll Physicians Surg Pak. 2010;20(12):814–8.

19.

Ahmer

, Yousafzai

, Bhutto

, Alam

, Sarangzai

, Iqbal

. Bullying of medical students in Pakistan: A cross-sectional questionnaire survey. PLoS One. 2008;3(12):e3889.

20.

Nielsen

. Methodological issues in research on workplace bullying. Unpublished PhD thesis, University of Bergen, Bergen, Norway, 2009.

21.

Einarsen

, Hoel

, Notelaers

. Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the Negative Acts Questionnaire-Revised. Work & Stress. 2009;23(1):24–44.

22.

Vandenbroucke

, Von Elm

, Altman

, Gøtzsche

, Mulrow

, Pocock

, Poole

, Schlesselman

, Egger

, Strobe Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. PLoS Med. 2007;4(10):e297.

23.

Nam

, Kim

, Koo

, Park

. The reliability and validity of the negative acts questionnaire-revised (NAQ-R) for nurses for the assessment of workplace bullying. Korean Journal of Occupational and Environmental Medicine. 2010;22(2):129–39.

24.

Chipps

, Stelmaschuk

, Albert

, Bernhard

, Holloman

. Workplace bullying in the OR: Results of a descriptive study. AORN J. 2013;98(5):479–93.

25.

Jiménez

, Muñoz

, Gamarra

, Herrer

. Assessing workplace bullying: Spanish validation of a reduced version of the Negative Acts Questionnaire. The Spanish Journal of Psychology. 2007;10(02):449–457.

26.

Ling

, Young

, Shepherd

, Mak

, Saw

. Workplace bullying in surgery. World J Surg. 2016;40(11):2560–6.

27.

Nielsen

, Notelaers

, Einarsen

. Measuring exposure to workplace bullying. Bullying and harassment in the workplace: Developments in theory, research, and practice. 2011;2:149–74.

28.

Hussain

, Rahim

. Bullying of postgraduate medical trainees in tertiary care hospitals. Journal of Postgraduate Medical Institute (Peshawar-Pakistan). 2014;28(3).

29.

Jacobson

, Hood

, Van Buren III , Harry

. Workplace bullying across cultures: A research agenda. International Journal of Cross Cultural Management. 2014;14(1):47–65.

30.

Báguena

, Beleña

MÁ

, Toldos

, Martínez

. Psychological harassment in the workplace: Methods of evaluation and prevalence. The Open Criminology Journal. 2011;4(Suppl 2-M7):102–8.

31.

Paice

, Aitken

, Houghton

, Firth-Cozens

. Bullying among doctors in training: Cross sectional questionnaire survey. BMJ. 2004;329(7467):658–9.

32.

Askew

, Schluter

, Dick

, Régo

, Turner

, Wilkinson

. Bullying in the Australian medical workforce: Cross-sectional data from an Australian e-Cohort study. Australian Health Review. 2012;36(2):197–204.

33.

Martin

. Is your hospital safe? Disruptive behavior and workplace bullying. Hosp To. 2008;86(3):21–8.

34.

Bentley

, Catley

, Cooper-Thomas

, Gardner

, O’Driscoll

, Trenberth

Understanding stress and bullying in New Zealand workplaces: Final report to OH&S steering committee. Wellington, New Zealand. 2009.

35.

Nordin

Supporting Sleep: The Importance of Social Relations at Work. Hampshire, UK: Springer. 2015.

36.

Stagg

, Sheridan

. Effectiveness of bullying and violence prevention programs: A systematic review. AAOHN J. 2010;58(10):419–24.

37.

Gillespie

, Gates

, Fisher

. Individual, relationship, workplace, and societal recommendations for addressing healthcare workplace violence. Work. 2015;51(1):67–71.

38.

Midgett

, Doumas

, Trull

, Johnson

. Training students who occasionally bully to be peer advocates: Is a bystander intervention effective in reducing bullying behavior? Journal of Child and Adolescent Counseling. 2017;3(1):1–13.