Digital training for psychosocial risk assessment as an approach to foster primary prevention for SMEs: An evaluation study

Abstract

BACKGROUND:

Rapidly changing stressful working conditions put new challenges on mental health in future work, especially for small- and medium-sized enterprises (SMEs) which need to be addressed on an organisational level. To promote, secure and sustain a healthy workforce in the long run, primary prevention of psychosocial risks is needed. Still, 70% of EU companies and over 85% of German SMEs lack the legally required implementation of psychosocial risk assessment (PRA) in their occupational safety and health (OSH) management.

OBJECTIVE:

The aim of the study was to evaluate the digital training PsyHealth worXs! as a suitable approach to teach OSH stakeholders how to conduct PRA.

METHODS:

We conducted a longitudinal evaluation study with two measurement times in the first and last week of the digital training based on N = 312 questionnaires.

RESULTS:

After the training, participants’ knowledge of the PRA process was significantly higher, and they felt significantly more competent to derive OSH interventions. Overall, the process of PRA and the involvement of stakeholders were perceived as significantly easier.

CONCLUSION:

Results suggest that the digital training provides an easily accessible opportunity for SMEs to successfully enable their OSH management to implement PRA strategies. Future research will have to evaluate the overall long-term implementation increase of PRA in German SME companies.

Keywords

Psychosocial risk assessment training training evaluation organisational prevention

1 Introduction

In the context of globalisation, demographic trends and digitalisation, todays’ work environment, work conditions and work organisation are rapidly changing and posing new and complex challenges on companies and their employees [1]. More and new stressful psychosocial working conditions occur on a daily level [2] and the subjective perception of work intensity characteristics like working quickly or working at the limit of capacity has been rising continuously in recent years [3]. The digital transformation of work and the associated performance requirements are also changing in terms of emerging digital stressors that add to existing psychosocial stressors [4]. In future work, especially for small and medium-sized enterprises (SMEs), preventive approaches for promoting mental health on an organisational level will become more and more important to secure a healthy workforce in the long term [5, 6]. While legal regulations and occupational safety and health (OSH) management are already relevant topics of present work organisation, employee health is also central for increasing company attractiveness and securing skilled and competitive labour [5 , 8].

Assessing psychosocial characteristics at the workplace in a risk assessment is the first preventive step when it comes to health promotion [5, 9]. Still, the implementation of psychosocial working conditions in a risk assessment is far from advanced in Europe, and specifically in Germany [10]. Especially, SMEs or smaller companies show large deficiencies in the implementation of psychosocial risk assessment (PRA) and even a tendency to avoid it [11]. Therefore, the goal of the present study was to evaluate a newly developed educational program on psychosocial health prevention and work standards ‘PsyHealth worXs!’. It is a digital training on how to conduct a PRA empowering OSH stakeholders to make sustainable changes in working conditions and job characteristics.

2 Background

Psychosocial factors are neutral working conditions and job characteristics that influence mental health and well-being. The factors can be categorised into the characteristic areas of work content, work organisation, social relationships and work environment [12, 13]. Depending on how these factors or characteristics are pronounced they can impede employee recovery [14] and thus pose adverse outcomes on employee health [15, 16], for example if the work tasks have to be completed under time pressure or with many interruptions. To prevent psychosocial factors from turning into risks, mental health promotion at the workplace necessitates addressing these factors by conducting a PRA [12], which requires positive changes at both the individual and organisational levels [5].

In the EU, more than 70% of companies lack the legally required PRA implementation in their OSH management [17, 18]. This is even more astonishing considering the sick leave rates in the EU due to work-related problems ranges between 42.7% and 49.8% [19]. Direct and indirect costs of job strain in Germany were estimated and resulted in a total cost of €19 29.2 billion per year. Direct costs for prevention, rehabilitation, care treatments and administration were about €19 9.9 billion. Even more indirect costs (€19.3 billion) accumulated through lost working years [20]. In summary, there is strong evidence that promoting mental health by investing in workplace interventions focusing on stress prevention and improving psychosocial work environment is cost-effective [21, 22]. Promoting employee well-being leads to positive outcomes like higher job satisfaction and reducing work-related stress will increase employee productivity and therefore profitability for companies [9]. The WHO emphasises the importance of safe and supportive working conditions and attention shifting to work organisational improvements [23].

Evidently, there is a wide policy-practice gap between research findings, legal requirements and actual mental health prevention. The company size is one of the strongest determinants of the scope of psychosocial risk management [24]. Large organisations have more psychosocial risk management policies and procedures in place than SMEs [13]. Especially for SMEs in Germany, the percentage of risk assessment strategies that include psychosocial risks is limited to 15% [11]. It has been shown that this policy-practice gap is mainly due to a self-reported lack of knowledge about PRA, as OSH stakeholders are often unaware on how to efficiently handle psychosocial risks in companies, especially in SMEs [24, 25]. In addition, the major underlying factors for the gap between policy and practice in SMEs are comparatively limited personnel and financial resources and a general lack of a systematic approach of OSH management [11 , 26]. The gap is aggravated by the lack of support from trained experts, as in SMEs often no specialised and experienced OSH management is installed. Rather a single person is assigned OSH duties in addition to their usual tasks [11, 27]. Moreover, smaller companies tend to neglect psychosocial risk assessments more frequently, because they are either convinced all risks are already known or that there are no risks at all [2]. Psychological stress is less addressed in a systematic manner, or even ignored or dismissed by the management level [28]. In fact, problems are often handled in very personal informal manners, rather than implemented with structured procedures [11, 29]. Awareness of psychosocial risks is still not high enough among employees and their employers at the workplace [27, 30], and it has to be raised especially in company managements [25]. Dealing with psychosocial risks is generally perceived more difficult than managing other OSH risks [10, 27]. Consequently, there is a strong need for practical tools identified to deal with work-related stress and increase awareness for psychosocial risks [24, 28]. SMEs might have the biggest need of action regarding trainings in PRA since they have limited resources, implemented systematic strategies less and have not implemented OSH management yet [11 , 27]. For this reason, a low threshold approach to gain access to training and practical implementation ideas might be especially important for SMEs.

Based on the gap between research findings, legislation and the practical application of PRA, in 2020, we co-created a digital training tool that helps those responsible in OSH management to gain knowledge and skills on how to implement a PRA. The training was designed as first overarching interdisciplinary educational training with a multi-method learning approach that is based on recent research evidence from different fields (e.g. occupational health, occupational health psychology, and occupational safety), EU regulations as well as country-specific information and regulations. More importantly, the training was co-created with OSH stakeholders who are the target group of the training. Co-creation occurred in two workshops that were led by occupational health psychologists from Germany and the Netherlands. Invited participants of the workshop were 45 OSH stakeholders in total. Participants were occupational physicians, occupational safety experts, human resources (HR) managers, OSH managers and psychologists. After evaluating the co-creation workshops, the following needs of the OSH stakeholders for a digital training concept were identified as the most pressing: practice-oriented and user-friendly, easily accessible, comprehensive and a step-by-step approach.

The digital nature of the tool ensures a low threshold approach due to its easy accessibility, time flexibility and location independence. The digital training is taught in the format of a massive open online course (MOOC) that does not require more than a free registration and an internet connection to follow it. A MOOC is a suitable educative approach for people in our target group as it requires very low time effort in their normal daily work routines [31]. Learners can additionally benefit from interdisciplinary exchange from other areas of expertise, experience and branches throughout Europe. By adding variability and interactivity as the basis for high learner engagement, the MOOC strives for increased knowledge and developing competencies for the implementation of the PRA process.

The main goal of PsyHealth worXs! is sensitising OSH stakeholders to employee health promotion through systematic prevention and early changes. Training participants learn how to analyse, evaluate and adapt working conditions to reduce psychosocial risks. The training enables companies to tackle the demands employees encounter through the changing world of work. It aims at overcoming and minimising known obstacles for starting PRA and promotes behavioural changes for implementing psychosocial risk management. By shifting the attention to mental health and sustaining a healthy workforce, jobs will be secured in the long term. The long-term goal of PsyHealth worXs! is an increased implementation rate of PRA strategies in the EU.

The purpose of PsyHealth worXs! includes shifting the attention to the working conditions and their adaption to the workers’ mental health. By now, research and practice has focused on mental-health promotion like stress-management trainings to make individuals resilient to work stressors [32]. The goal of current interventions is to adjust the worker to the stressful work environment by focusing on individual change [33]. Individual-focused measures are effective [32, 34], however, the dramatic increase in absence rates due to mental health problems indicates that overall this is not the solution to keep a sustainable workforce [35]. The majority of work-related stressors in office work are the result of inadequately designed work organisation and management [21], such as interruptions and multi-tasking demands. Traditional psychosocial working conditions even play a role in modern open space offices (OSO) and influence the well-being at the workplace [36]. Therefore, systematically improving these working conditions on both an individual and organisational level with often minimal interventions promotes well-being, productivity and job satisfaction [5, 6].

To ensure that the training achieved its desired learning goals and is effective, we designed an evaluation questionnaire which is based on theoretical concepts. Therefore, we applied the evaluation model from Kirkpatrick as well as the theory of planned behaviour (TPB) to check whether the training is a valuable tool to change behaviour predicting factors [37]. These concepts are based on years of research and are still relevant in today’s applied research [38, 39]. Training evaluation according to Kirkpatrick (2006) is built on four distinct, but consecutive levels. The four levels represent a sequence of precise measurements of the training effectiveness. The levels are labelled Reaction, Learning, Behaviour and Results, with Reaction at the base and Results at the top of the evaluation pyramid [37, 40]. Due to the design and setting of the present study, we were able to incorporate the first two levels of reaction and learning. Kirkpatrick’s level of reaction means the participants’ individual reaction to the training like satisfaction [37]. With learning, the Kirkpatrick model wants to explicitly assess the extent of knowledge gain, increased skills and changed attitudes as consequence of completing the training [37]. The other two higher levels require more than a pre-post-test design and can only be measured over a longer period of post-training time. Since behaviour cannot be measured in this questionnaire, we resorted to the TPB and assessed changes for relevant indicators for behaviour change. In addition, the Theory of Planned Behaviour (TPB; [41, 42]) is a psychological model about behavioural changes and is commonly used as theoretical basis in psychological research studies that predict human behaviour. Specifically, in our training evaluation, we wanted to assess behavioural intentions and actual behavioural changes in favour of conducting a psychosocial risk assessment. The predictors for the intention to change are the participants’ attitudes, their subjective norms and their perceived behavioural control towards the process of conducting PRA. By evaluating the attitudes of the participants, we wanted to explore their behavioural beliefs and the emotional valence they put into their outcome evaluations [43]. The attitudes suggest that a specific behaviour is more likely when the individual considers it important or positive. By measuring the subjective norms, the participants’ normative beliefs and their motivation to comply is assessed [43]. The subjective norms imply that a specific behaviour is more likely if the individual perceives relevant interaction partners to be in favour of the behaviour in question. For the present study the subjective norm experience should be exercised through the perspective adoption of the management responsible for work design, employees as those primarily affected by working conditions, politicians as those who form the legal basis of OSH and OSH stakeholders as the individuals who have to bring in their consultation efforts within the legal regulations and organisational circumstances. Perceived behavioural control considers that a behaviour gets more likely if the individual has a certain amount of influence on how to conduct the behaviour properly. Intended behaviour is predicted through one’s attitudes, subjective norms and perceived behavioural control. Finally, the TPB proposes that the actual behaviour is predicted by the individual intention to conduct a specific behaviour. An actual behavioural change requires the respective behavioural intentions to shift. With those constructs of the TPB, past research predicted a variety of different behaviours related to prevention through health and compliance behaviour [44 –46]. Considering the evaluation criteria of the four-level training evaluation model [37], we primarily expected the training to be perceived highly satisfactory and to improve knowledge regarding PRA in the second questionnaire. The main hypothesis concerning the TPB [41] is a significant improvement of the attitudes, subjective norms, perceived behavioural control and behavioural intention for conducting a PRA after the training.

3 Methods

3.1 Procedure

We conducted a six-week digital training twice in between November 2020 and June 2021 on the platform edX. Marketing for learner recruitment was done in professional and business networks in Germany, such as placing the training invitation in newsletters of statutory accident insurances and professional associations, but also in EU-wide organisations. We conducted a longitudinal evaluation study with two measurement times in the first and in the last week of training. The target audience comprised all stakeholders involved in OSH management, such as occupational physicians, occupational safety experts, HR managers, OSH managers, psychologists and managers. The voluntary evaluation questionnaire was presented in the training itself via GoogleForms.

3.2 Participants

Overall, 985 OSH professional and executives participated in the training and N = 312 questionnaires were filled out. In total, N = 225 participants filled out the questionnaire before the start of the training and N = 87 replied after completion of the training. The participation and response to the questionnaire were voluntary for all items. No participants were excluded from the sample due to incomplete responses. 51.3% (N = 160) of the participants identified as female, 36.5% (N = 114) identified as male and 12.2% (N = 38) preferred not to answer the question. The mean age of the participants was 44.26 years (SD = 10.86) with a range between 21 to 67 years. The participants were asked to choose one or several answers to describe their occupational roles during the course which resulted in N = 225 total cases. The most frequent statements were occupational safety specialist (N = 54), occupational health management (N = 44), occupational health physician (N = 36) and manager or supervisor (N = 30). The complete list is shown in Table 1. The motivation to join the training is due to own interest or motivation, further training or education and deepening of knowledge on the subject (see Table 2).

Table 1
Frequencies of participants’ professional roles (multiple answers possible)

Self-identified role(s) n %

Occupational safety specialist 54 24.0

Occupational health management 44 19.6

Occupational health physician 36 16.0

Manager / supervisor 30 13.3

Employee 21 9.3

Psychologist 19 8.4

Medical specialist / consultant 14 6.2

Human resources 13 5.8

Workers’ or staff council member 6 2.7

Student 6 2.7

Ergonomist 5 2.2

PhD candidate 2 0.9

Research assistant 1 0.4

Graduate 1 0.4

Professor 1 0.4

Total cases N = 225

Self-identified role(s)	n	%
Occupational safety specialist	54	24.0
Occupational health management	44	19.6
Occupational health physician	36	16.0
Manager / supervisor	30	13.3
Employee	21	9.3
Psychologist	19	8.4
Medical specialist / consultant	14	6.2
Human resources	13	5.8
Workers’ or staff council member	6	2.7
Student	6	2.7
Ergonomist	5	2.2
PhD candidate	2	0.9
Research assistant	1	0.4
Graduate	1	0.4
Professor	1	0.4
Total cases	N = 225

Note. Participants were asked to choose one or several roles to describe their status during the course.

Table 2

Frequencies of participants’ motivation to participate in the training (multiple answers possible)

	2020		2021
Motivation for training participation	n	%	n	%
Own interest or motivation	82	36.4	28	29.5
Further training / education	72	32.0	32	33.7
Deepen knowledge on subject	70	31.1	26	27.4
Receive course certificate	18	8.0	2	2.1
Proposed by employer (voluntary)	6	2.7	3	3.2
Requested by employer (mandatory)	1	0.4	2	2.1
Other option	0	0.0	2	2.1
Total cases	N = 130	N = 95

Note. Participants in 2020 were asked to choose one or several reasons for which they joined the course. Participants in 2021 were asked to choose one single reason.

3.3 Training outcome measures

Regarding the variables operationalising the Kirkpatrick Model, we developed pre- and post-survey questions tailored to the training. The constructs of the TPB were developed according to a manual for health and psychological researchers to create effective and precise questionnaires that assess constructs based on the TPB relevant for predicting individual health-related behaviour [43].

Reaction (Kirkpatrick’s level 1): Participants were asked to rate eight statements in the post-questionnaire regarding their training experience on a five-point scale ranging from strongly disagree (1) to strongly agree (5). Training experiences included enjoyment, perceived training success and satisfaction with the content and structure. A sample item would be “The course was successful in teaching me PRA.”. The internal consistency was α= 0.909.

Learning (Kirkpatrick’s level 2): To measure the level of learning success, the participants answered two statements in the pre- and post-version and three items only in the post-questionnaire, all on the five-point scale ranging from strongly disagree (1) to strongly agree (5) which were analysed on item-level. Learning included items regarding knowledge gain about the process and interventions of PRA and regarding perceived competence and awareness. A sample item is: “I know how to plan and implement interventions.”.

In our questionnaire for the present evaluation, we included three variables of the TPB model that predict the intention to perform a specific behaviour which was also included as scale in the questionnaire.

Attitudes (TPB): The attitudes towards the behaviour of conducting a PRA were directly measured by using four bipolar evaluative adjectives (e.g. good–bad). On the scale 1 to 7, they could choose between pairs of opposites. The internal consistency of the scale varied between the measurement times from α= 0.780 at T1 to α= 0.875 at T2.

Subjective norm (TPB): Participants were asked to rate four statements regarding their subjective norms on the importance of PRA on a five-point scale from strongly disagree (1) to strongly agree (5). The internal consistency for the scale of subjective norms was α= 0.604 at t1 and α= 0.747 at t2. A sample item is: “Politicians strengthen the need for PRA.”.

Perceived Behavioural Control (TPB): The perceived behavioural control on the PRA process was described on a seven-point scale with twelve pairs of adjectives. The pairs of adjectives represent two poles of a continuum (e.g. very easy = 1; very difficult = 7) between the perceived difficulty of each step of the PRA process and some related tasks in that process. The internal consistency of the scale varied between the measurement times from α= 0.886 to α= 0.908.

Intention (TPB): The behavioural intentions were explored through seven statements on a five-point scale ranging from strongly disagree (1) to strongly agree (5). Of those items about behavioural intentions, two scales on the intention to conduct a PRA (α= 0.911 to α= 0.891) and the intention to conduct a cultural change (α= 0.938 to α= 0.818) were built with three items each. A sample item for conducting a PRA was “I intend to conduct a PRA after the course.”. A sample item for conducting a cultural change in the organisation is “I want to establish a culture in my company / consultancy in which PRA is respected and psychosocial risks are regularly and continuously assessed.”.

3.4 Data analysis

All data were processed and recoded using Excel and analysed with SPSS. The items regarding reaction and learning according to Kirkpatrick’s model were analysed on a descriptive level. In order to test whether the attitudes, subjective norms, perceived behavioural control, behavioural intentions and the knowledge improved through participation in the training, mean comparisons were calculated using t-tests for independent samples (p < 0.05). The effect sizes of the mean differences are given as Cohen’s d, in order to interpret the significance of the results. Here, Cohen’s d of 0.20 corresponds to a small effect,0.50 to a medium and.80 a large effect [47]. All hypotheses were tested one-sided.

4 Results

4.1 Reaction

The participants’ overall reaction to the training was highly satisfactory (M = 4.40, SD = 0.51). The distribution of the single items is displayed in Fig. 1 and Table 3. The participants gave very positive feedback regarding training enjoyment and found the training content interesting. According to them, the structure and content were easy to understand. The participants further stated that the training was successful in teaching them PRA.

Fig.1

Participants’ reaction to the course in the post questionnaire.

Table 3

Responses to the scale Reactions in the post training questionnaire

	M	SD	n
I enjoyed the course overall.	4.3	0.634	86
The course was worth my time.	4.3	0.771	86
The course was successful in teaching me PRA.	4.3	0.758	87
I was interested in the content of the course.	4.6	0.515	86
The diverse components of the course have made it interesting.	4.3	0.74	86
The structure of the course made sense in terms of content.	4.5	0.696	87
The structure of the course made sense in terms of form (lectures, interviews, quizzes and assignments).	4.4	0.734	86
The course contents were easy to understand for me.	4.4	0.794	86

4.2 Learning

85.5% (N = 71) of all respondents in the post-questionnaire (N = 83) felt more competent in conducting PRA than before the training (M = 4.19, SD = 0.77). 81.9% (N = 68) of the participants thought they learned a lot in the training (M = 4.08, SD = 0.74). 75.6% of the participants (N = 62) saw the relevance of the PRA more clearly after finishing the training (M = 3.94, SD = 0.95). After the training, participants’ knowledge of the risk assessment process was significantly greater (t(258.965)=–11.701, p = 0.000, d = 1.112), and the proportion of those who felt competent to derive OSH measures increased from 33% to 88% (t(240.298)=–10.891, p = 0.000, d = 1.043). Their perceived competence and knowledge increased overall.

4.3 Attitudes

With regard to the attitudes of the participants towards conducting PRA and being aware of the relevance of PRA, a significant training effect (t(288)=2.397, p = 0.017, d = 0.84) was found. In both the pre- and post-questionnaires, PRA was assessed as beneficial (= 1; oppositely harmful = 7) (pre M = 1.78, SD = 1.203; post M = 1.55, SD = 1.124). For both measurements, PRA was clearly described as good (= 1; instead of bad = 7) (pre M = 1.82, SD = 0.986; post M = 1.54, SD = 0.967) and as pleasant for oneself (= 1; instead of unpleasant = 7) (pre M = 2.38, SD = 1.23; post M = 2.04, SD = 1.13). All three items showed significant training effects on item-level evaluation. In both measurements, the participants thought PRA was useful (= 1; instead of worthless = 7; pre M = 1.63, SD = 0.876; post M = 1.45, SD = 0.848), but no significant difference was observed.

4.4 Subjective norms

With regard to items on subjective norms and the participants’ perspective on other stakeholder groups, no significant training effects were detected when tested on scale level. However, when testing for single statements, one significant training effect was found. After the training, the participants thought that politicians do strengthen the need of PRA (t(293)=–2.031, p = 0.043, d = 1.021). The other mean values, both at scale and item level, showed a clear error of central tendency and hardly any differences (all between M = 3.25 to M = 3.48).

4.5 Perceived behavioural control

The participants’ expectations regarding perceived behavioural control to conduct PRA were significantly easier after completion of the training (t(290)=6.365, p = 0.000, d = 0.843). Participants indicated significant simplification of the PRA process and tasks with very large effects between d = 1.126 and d = 1.345 in their perception in all statements except for motivating the management. Here, no significant training effect could be found. All training effects are displayed in Fig. 2 and Table 4.

Fig.2

Participants’ perception on the difficulty of steps of PRA in pre- and post- questionnaire.

Table 4

Responses to the scale Perceived Behavioural Control and effect sizes of t-tests with pre and post questionnaires

	Pre			Post
	M	SD	n	M	SD	n	Effect size d
I think conducting a PRA is...	4.59	1.131	221	4.11	1.207	82	1.152**
I think building proper assessment units is...	4.59	1.135	221	3.77	1.103	82	1.126**
I think assessing psychosocial risks is...	4.53	1.186	219	3.84	1.073	80	1.157**
I think interpreting the results is...	4.57	1.216	220	3.79	1.201	81	1.212**
I think implementing interventions is...	4.90	1.166	221	4.14	1.292	81	1.201**
I think evaluating interventions is...	4.59	1.256	220	3.58	1.171	81	1.234**
I think documenting the process of PRA is...	3.81	1.324	220	3.01	1.401	81	1.345**
I think continuing PRA is...	4.63	1.203	217	3.75	1.392	81	1.257**
I think building trust for employees is...	4.88	1.296	220	4.37	1.318	81	1.302**
I think motivating employees to participate is...	4.70	1.340	219	4.17	1.321	81	1.335**
I think motivating the management to support PRA is...	4.94	1.411	219	4.65	1.416	81
I think managing the needs of different stakeholders involved in the process is...	5.00	1.153	219	4.35	1.233	80	1.175**
Scale average	4.66	0.820	213	3.95	0.900	79	0.843**

**p < 0.01.

4.6 Behavioural intentions

59 of 79 participants (74.6%) reported to recommend the training to others. The intention to conduct PRA remained stable on a relatively high level from the very beginning of training participation (pre M = 3.79, SD = 0.88; post M = 3.83, SD = 0.96). A comparable picture was shown in intention for a cultural change (pre M = 3.76, SD = 0.94; post M = 3.90, SD = 0.75). With regard to the intention to conduct a risk assessment after the training no significant training effects were detected. Similarly, no significant training effects were found with respect to the intention to make a culture change toward respectful treatment of PRA.

5 Discussion

The present study aimed at evaluating the newly developed online training PsyHealth worXs! that teaches OSH stakeholders how to conduct and implement a PRA as a primary approach for health prevention at work. Results indicate that the training is well received within the acquired target group of OSH stakeholders. The evaluation results give hints that the training provides knowledge and skills in conducting PRA and is affecting relevant behavioural antecedents like attitudes and perceived behavioural control known to be relevant for behaviour changes.

5.1 Reaction

The reaction of the participants to PsyHealth worXs! was very positive and satisfactory. The digital training seemed to be highly accepted and appreciated among the participants. The training was originally co-created with OSH stakeholders and evidently it was successful in implementing the needs expressed by the target group. The content of the training was rated interesting and educative, and the overall MOOC was rated joyful and suitable which makes it an effective tool especially for SMEs as it allows to deal with the topic of PRA implementation in an easy, low-threshold approach, considering that SMEs reported to have only limited resources in terms of personnel and time matters [11].

5.2 Learning

Significant improvements in all areas are visible in the results of all learning levels after the training. Participants showed a significant increase in terms of perceived competence, knowledge and awareness and relevance of the topic. Those improvements exactly tackle the known big obstacles for not conducting and implementing PRA procedures [11]. The participants not only showed a knowledge gain, but also felt more competent in adapting skills on how to take action.

5.3 Attitudes

By evaluating the attitudes of the participants, we wanted to explore their behavioural beliefs and the emotional valence they put into their outcome evaluations, which according to the TPB is relevant for initiating behaviour change. The attribution to the conduction of PRA as being good and beneficial shows awareness and preventive thinking of the participants. Those high levels of attribution may be biased by the target audience of OSH stakeholders that are already highly interested in health prevention strategies and therefore joined this training. In fact, learners who were asked about their motivation to participate in the training reported “own interest or motivation”, “further training or education”, and “deepen knowledge on subject” as being their main drivers for joining the training. Attributing the PRA process as more pleasant than before the training may be due to a now more familiar access in handling psychosocial risk management. It also confirms learners’ willingness to start changing their behaviour. Specifically, a significant training effect was detected in the participants’ attitudes towards conducting PRA which corresponds to their initially already high perception of the relevance of PRA. A high level of awareness and attributed relevance of the topic was already given at the start of the training and even increased in the course of the training. It is likely that this will encourage PRA implementation.

5.4 Subjective norms

By measuring the subjective norms, the participants’ normative beliefs and their motivation to comply with the behaviour to conduct a PRA was assessed. Since the training was designed as an interdisciplinary training, considering the perspective of all stakeholder groups involved in PRA, our expectation that participants shift their perspective to other stakeholder groups’ perceptions on the relevance and importance of PRA after the training was not met. In additional analyses, only the perception that politicians are aware of the importance of PRA changed which might be due to their deepened knowledge from training material about the legal background of PRA in Europe as well as their respective countries Since on average, the participants responded that they neither agreed nor disagreed whether the management or employees think that PRA is important, a rather indecisive or insecure response behaviour was shown. Also, it could be a matter of timing the question. While participating in the six-week course, learners could not bring in their newly acquired knowledge and skills into practice and thus could not yet influence their managements or employees, immediately. The impact of subjective norms may thus evoke only over the period of several months or even years. Similarly, the participants were unsure whether they think that OSH stakeholders push the PRA process forward. This could be due to the fact that they think they have no influence on other stakeholder groups or that the interdisciplinary exchange remains difficult [48]. The participants may be unsure how to get sceptical stakeholders to commit to psychosocial risk management.

5.5 Perceived behavioural control

The results showed a successful training effect and strong improvement in feeling able to handle PRA. The training simplified all steps of the PRA process and the requirements related to it, like involving the management. These findings corresponded with the successful learning outcomes. By knowing more about PRA and feeling more competent in handling it, the participants’ self-efficacy expectation for conducting PRA was dramatically increased. They perceived their own competency and improvement. These findings predict that their intention or willingness to act is given, but may fail for other reasons (see perceived influence on others which is considered to be low), but their own readiness is there.

5.6 Behavioural intentions

After completing a six-week online training, it is not possible to immediately evaluate individuals’ behaviour regarding the conduction of a psychosocial risk assessment, therefore we evaluated their behavioural intentions. With regard to the intention to conduct a risk assessment after the training or to create a respectful company culture towards psychosocial risk management, no significant training effects were detected. The predictors for behavioural intentions have all changed positively in the course of the training or are perceived as not being able to change by their own efforts. One reason for the lack of effects might be that intention was already high from the very beginning in this sample. After completion of the training, a possibly increased intention to conduct PRA could be hidden by missing or still unclarified responsibilities, which still have to be discussed at the management level. An obstacle for changing the intentions may be due to the belief of not being able to convince other stakeholder groups of getting involved and committing themselves to psychosocial risk management. Thus with regard to the long term training effects, future research should follow up on the learners to assess their actual behaviour in changing and adopting or implementing the PRA after six months or one year.

To sum up, the results of the training evaluation harmonised in terms of content and show a coherent picture. A general acceptance of the training was shown through the satisfactory reaction evaluation. A great learning success in terms of knowledge and skills was achieved by the training. The positive attitudes and evaluation of topic went along with the increased feeling of competency and self-efficacy. A subjective change happened for the participants, but not for their perception of stakeholders they collaborate with in daily work life. Considering the challenges to PRA scholars discussed [11], the most important obstacles were tackled by improving knowledge, competence and awareness. A remaining obstacle is the perceived influence on other stakeholder groups which leads to three important implications. First, an important stakeholder group may not be reached yet, namely the stakeholders in control of decision-making and change. Management commitment is one of the most effective drivers for an organisational psychosocial risk management [25] and would increase the implementation rate of PRA procedures. Second, to reach the managerial group, a different tool may be needed to specifically address leadership and management positions, and involve and commit sceptic stakeholders in charge of psychosocial risk management [49]. Third, since a significant proportion of participants doubted their ability to influence decision-makers in the PRA, future research should focus on ways to increase learners’ self-efficacy and provide tools to address this specific topic.

The MOOC was offered as EU-wide training and designed for all stakeholder groups involved in OSH management. For most participants (72.4%) no data was available about their country of origin, only 50% of the participants disclosed that they were German (N = 43). The company size was only included in the updated 2021 version of the pre-questionnaire and 44.2% participants (N = 42 of 95 participants) stated that they work in micro-, small- or medium-sized companies. In summary, half of the sample size was German and working in or for SMEs, so the results are still a generalisable indicator for the training impact in German SMEs.

6 Limitations

The results of this evaluation study need to be interpreted with the knowledge of potential limitations. First, regarding the survey of training effectiveness, all statements and motivations on intended behavioural changes after the training were self-reported here. Thus, results may be biased by social desirability issues [50]. However, since the repeated measurement of the survey items occurred in a time frame of six weeks, it is unlikely that respondents were able to recall their initial responses to the same questions. In addition, no objective knowledge gain was assessed in the evaluation questionnaire. The objective results could be measured by quizzes and assignments that are available on edX, but until now data was kept separate from the evaluation results.

Second, the results lack a comparison with a control group, since the training was freely available on edX. Thus, it cannot be ruled out that the effects are specifically due to the training and not because of an unmeasured third variable [51]. Future research should conduct a randomised controlled experiment for measuring training effectiveness. In addition, future research should follow up on learners over several months to survey the actual and not only intended behaviour change.

A third limitation concerns the drop-out rate of the training. More than 60% of participants, who initially started the voluntary evaluation survey dropped out by the end of the six-week training. These high drop-out rates could be due to generally low completion rates of MOOCs [52] and limit the generalisability of results. That is, only a rather small proportion of participants were motivated to invest extra time to complete the evaluation survey. When comparing the sample of the evaluation survey with internal learner statistics of edX, an equivalent composition in terms of gender and age was found. A general problem in MOOC retention rates is that for various factors including academic and personal reasons, participants enroll in courses without the intention to complete them [53]. Reviews which compared MOOCs on popular platforms showed average retention rates of circa 7–20% [52]; the overall retention rate of 21.6% for PsyHealth worXs! was satisfactory. In addition, the proportion of participants who completed the evaluation survey among those who were active by the end of the course was rather high with 39.7%.

7 Conclusion

The overall positive feedback on the MOOC indicates that learning with the digital training PsyHealth worXs! is pleasant, educational and effective for OSH stakeholders. Instead of new challenges and more stressful workload, the OSH stakeholders have a user-friendly tool which is successful in teaching PRA through a low-threshold approach, which is especially important for SMEs. The evaluation of the digital training PsyHealth worXs! shows significant improvements in the perceived competence to take action, a significant knowledge gain and greater awareness of topics of mental health and psychosocial risk prevention. That means that the training tackles the main obstacles in SMEs for not yet implementing PRA procedures in their OSH prevention. According to the TPB, the prediction of whether an individual intends to act or not requires knowledge of the three predictors attitudes, subjective norms and perceived behavioural control [43]. Two of these factors changed significantly in the present study. One factor is connected to insecurities, but there are indicators that it still improved. Predictors for behavioural intentions (and therefore behaviour change) are all on the right track to reach the long-term goal of increased implementation of PRA in Germany.

Ethical approval

Not applicable. This study is exempt from Institutional Review Board approval. Evaluation data is completely anonymous with no personal information being collected (apart from their age and gender). None of the data is considered to be sensitive or confidential in nature.

Informed consent

Not applicable. The participation in the survey was based on a voluntary participation in a freely available online training. All participants were informed that the data is anonymously assessed, stored and used before answering the questionnaire. No conclusions can be drawn about individuals.

Conflict of interest

The authors declare that there is no conflict of interest.

Footnotes

Acknowledgments

We would like to thank everyone involved in the process of creating the MOOC and its evaluation. In particular, we would like to thank the participants of the co-creation workshops and the interview partners who shared their experiences, knowledge and lessons learned with us. Further thanks go to our student assistants, the teams of AVMZ and MfL for their constant and professional support, and David Spencer for proofreading.

Funding

PsyHealth worXs! Psychosocial Health Prevention and work standards: Training for OSH Professionals and Executives was funded by EIT Health (Grant ID 20458).

References

Latos

, Harlacher

, Przybysz

, Mutze-Niewohner

. Transformation of working environments through digi- talization: Exploration and systematization of complexity drivers; 2017.

European Agency for Safety and Health at Work. Third European Survey of Enterprises on New and Emerging Risks (ESENER 3): First findings. Bilbao: Publications Office; 2019.

BAuA. Stressreport Deutschland 2019: Psychische Anforderungen, Ressourcen und Befinden. Dortmund: Bundesanstalt fur Arbeitsschutz und Arbeitsmedizin (BAuA); 2020.

Hünefeld

. öffentlicher Dienst: hohe Arbeitsintensitat, starke Belastung. Dortmund: Bundesanstalt für Arbeitss- chutz und Arbeitsmedizin; 2020.

Barry

. Promoting Mentally Healthy Workplaces. In: Barry MM, Clarke AM, Petersen I, Jenkins R, editors. Implementing mental health promotion. Cham: Springer International Publishing; 2019. pp. 389–427.

Burton

WHO Healthy workplace framework and model: Background and supporting literature and practices. Geneva: World Health Organization; 2010.

Koppel

, Plünnecke

Fachkräftemangel in Deutschland: Bildungsökonomische Analyse, politische Handlungsempfehlungen, Wachstums- und Fiskaleffekte. IW- Analysen, Nr. 46. Köln: Deutscher Instituts-Verlag; 2009.

Institut für angewandte Arbeitswissenschaft e. V. Im Fokus der Pandemie: der gesetzliche Arbeits- und Gesundheitsschutz. Düsseldorf; 2020.

Leka

, Jain

Mental health in the workplace in Europe: Consensus paper; 2017.

10.

European Agency for Safety and Health at Work. Second European Survey of Enterprises on New and Emerging Risks (ESENER-2). Overview Report: Managing Safety and Health at Work 2016.: Publications Office; 2016.

11.

Beck

, Lenhardt

. Consideration of psychosocial factors in workplace risk assessments: findings from a company survey in Germany. Int Arch Occup Environ Health. 2019;92:435–51.

12.

Beck

, Berger

, Breutmann

, Fergen

, Gregersen

, Morschhäuser

, et al. Empfehlungen zur Umsetzung der Gefährdungsbeurteilung psychischer Belastung. Berlin: GDA-Arbeitsprogramms Psyche; 2017.

13.

Council Directive of 12 June 1989 on the introduction of measures to encourage improvements in the safety and health of workers at work: 89/391/EEC; 1989.

14.

Pauli

, Lang

. Collective resources for individual recovery: The moderating role of social climate on the relationship between job stressors and work-related rumination –A multilevel approach. German Journal of Human Resource Management. 2021;35:152–75.

15.

Garst

, Frese

, Molenaar

. The temporal factor of change instressor-strain relationships: a growth curve model on alongitudinal study in east Germany. The Journal of AppliedPsychology. 2000;85:417–38.

16.

Demerouti

, Bakker

, Bulters

. The loss spiral of workpressure, work–home interference andexhaustion: Reciprocal relations in a three-wave study. Journal ofVocational Behavior. 2004;64:131–49.

17.

Ales

, Bell

, Deinert

, Robin-Olivier

, editors. International and European labour law: Article-by-article commentary. 1st ed. International and European business law. Baden-Baden, Mönchen, Oxford: Nomos; Beck; Hart; 2018.

18.

Eurofound and EU-OSHA. Psychosocial risks in Europe: Prevalence and strategies for prevention. Luxembourg: Publications Office of the European Union; 2014.

19.

Spasova

, Bouget

, Vanhercke

. Sick pay and sickness benefit schemes in the European Union: background report for the Social Protection Committee’s: in depth review on sickness benefits. Brussels: Publications Office; 2016.

20.

Bödeker

, Friedrichs

. zu den Kosten der psychischen Erkrankungen und Belastungen in Deutsch land. Kamp L, Pickshaus K, editors. Regelungslücke psychische Belastungen schließen. Düsseldorf; 2011. pp. 69–102.

21.

Hassard

, Teoh

, Cox

, Dewe

, Cosmar

, Gründler

, et alCalculating the costs of work-related stress and psychosocial risks: literature review. Bilbao: Publications Office; 2014.

22.

Živković

, Milenović

, Krstić

, Veljković

. Correlation between psychosocial work factors and the degree ofstress. Work (Reading, Mass.). 2021;69:235–45.

23.

World Health Organization. Mental health action plan 2013–2020. Geneva: World Health Organization; 2013.

24.

European Agency for Safety and Health at Work. Annual report 2012: summary. Bilbao: Publications Office; 2013.

25.

van den Heuvel

, Bakhuys Roozeboom

, Eekhout

, Venema

. Management of psychosocial risks in European workplaces: Evidence from the second European survey of enterprises on new emerging risks (ESENER-2): European Agency for Safety and Health at Work; 2018.

26.

Champoux

, Brun

J-P

. Occupational health and safety management in small size enterprises: an overview of the situation and avenues for intervention and research. Safety Science. 2003;41:301–18.

27.

European Agency for Safety and Health atWork. ESENER 2019: what does it tell us about safety and health in Europe’s workplaces? Bilbao: Publications Office; 2020.

28.

Lenhardt

. Psychische Belastung in der betrieblichen Praxis. Z.Arb. Wiss.6-13. Available from. 2017;71;6–13. https://link.springer.com/article/10.1007/s41449-017-0045-z.

29.

Beck

, Schuller

, Schulz-Dadaczynski

. Psychosocial risk management in company practice. Opportunities and limitations. Präv Gesundheitsf. 2017302–10.

30.

Ahlers

. Leistungsdruck, Arbeitsverdichtung und die (ungenutzte) Rolle von Gefährdungsbeurteilungen. WSI. 2015;68:194–201.

31.

Pomerol

J-C

, Epelboin

, Thoury

. A MOOC for Whom and for What Purposes? In: Pomerol J-C, editor. MOOCs: Design, use and business models. Focus series. London, UK, Hoboken, NJ: ISTE Ltd; 2015. pp. 49–69.

32.

Richardson

, Rothstein

. Effects of occupational stressmanagement intervention programs: a meta-analysis. Journal ofOccupational Health Psychology.. 2008;13:69–93.

33.

Probst TM . Conducting effective stress intervention research: strategies for achieving an elusive goal. Stress and health: journal of the International Society for the Investigation of Stress. 2013;29:1–4.

34.

Restrepo

, Lemos

. Addressing psychosocial work-related stress interventions: A systematic review. Work (Reading, Mass.). 2021;70:53–62.

35.

Grobe

, Bessel

, Meyer

, Zill

, Schuhmann

. Ein Jahr Coronapandemie: Wie geht es Deutschlands Beschäftigten?: Dossier 2020–Corona 2020: Gesundheit, Belastungen, Möglichkeiten. Hamburg; 2021.

36.

Lütke Lanfer

, Becker

, Göritz

. Well-being in open space offices: The role of office features and psychosocial working conditions. Work (Reading, Mass.). 2021;68:317–32.

37.

Kirkpatrick

, Kirkpatrick

. Evaluating training programs: The four levels. 3rd ed. San Francisco: Berrett- Koehler; 2006.

38.

Opoku

, Cuskelly

, Pedersen

, Rayner

. Applying the theory of planned behaviour in assessments of teachers’ intentions towards practicing inclusive education: a scoping review. European Journal of Special Needs Education. 2021;36:577–92.

39.

Goh

, Wong

, Ayub

. The Effectiveness of MOOC Among Learners Based on Kirkpatrick’s Model. In: Tang SF, Cheah SE, editors. Redesigning Learning for Greater Social Impact: Taylor’s 9th Teaching and Learning Con- ference 2016 Proceedings. Singapore: Springer Singapore; 2017. pp. 313–23.

40.

Kirkpatrick

Great Ideas Revisited: Revisiting Kirk- patrick’s Four-Level Model. Training & Development. 1996:54-7.

41.

Ajzen

. The theory of planned behavior. Organizational Behavior and Human Decision Processes. 1991;50:179–211.

42.

Ajzen

Attitudes, personality and behavior. Mapping social psychologyBuckingham: Open Univ. Press; 2022.

43.

Francis

., Eccles

M. P

., Johnston,

, Walker

A.E

, Grimshaw

, Foy

., Kaner

EFS

Smith

, Bonetti

., Constructing questionnaires based on the theory of planned behaviour: A manual for health services researchers. New- castle upon Tyne, UK: Centre for Health Services Research; 2004.

44.

Hardeman

, Johnston

, Bonetti

, Wareham

, Kinmonth

. Application of the Theory of Planned Behaviour in Behaviour Change Interventions: A Systematic Review. Psychology & Health. 2010;17:123–58.

45.

Poulter

, McKenna

. Evaluating the effectiveness of a road safety education intervention for pre-drivers: An application of the theory of planned behaviour. British Journal of Educational Psychology. 2010;80:163–81.

46.

Quine

, Rutter

, Arnold

. Persuading school-age cyclists to use safety helmets: Effectiveness of an intervention based on the Theory of Planned Behaviour. British Journal of Health Psychology. 2001;6:327–45.

47.

Cohen

. A power primer. Psychological Bulletin. 1992;112:155–59.

48.

Wulf

, Süß

, Diebig

. Akteure derGefährdungsbeurteilung psychischer Belastung –Perspektiven und Konflikte im betrieblichen Arbeits- undGesundheitsschutz. Z. Arb. Wiss.296-304. Available from2017;71:296–304. https://link.springer.com/content/pdf/10.1007/s41449-017-0085-4.pdf

49.

Lang

, Mattyssek

Psychological health promotion as a leadership tasks. psyGA eLearning Tool for Leaders.: Bundesanstalt für Arbeitsschutz und Arbeitsmedizin; 2013.

50.

Tourangeau

, Rips

, Rasinski

. The psychology of survey response. 10th ed. Cambridge: Cambridge Univ. Press; 2099.

51.

Stolberg

, Norman

, Trop

. Randomized controlled trials. American Journal of Roentgenology. 2004;183:1539–44.

52.

Khalil

, Ebner

. MOOCs Completion Rates and Possible Methods toImprove Retention - a Literature Review. Proceedings of WorldConference on Educational Multimedia, Hypermedia andTelecommunications. 20141236–44.

53.

Gütl

, Rizzardini

, Chang

, Morales

. Attrition in MOOC: Lessons Learned from Drop-Out Students. In: pp. 37-48.