Putting context into organizational intervention design: Using tailored questionnaires to measure initiatives for worker well-being

Abstract

Realistic evaluation emphasizes the importance of exploring the mechanisms through which organizational interventions are effected. A well-known mechanism in organizational interventions is the screening process. Standardized questionnaires, in popular use, neither consider individuals’ appraisals of working conditions nor the specific context of the workplace. Screening with items tailored to intervention contexts may overcome the limitations of standardized questionnaires. In the present study, we evaluate an approach to develop a tailored questionnaire to measure employees’ appraisals of their specific working conditions. First, we interviewed 56 employees and 17 managers and, later, developed tailored items focused on the working conditions in a postal service. In follow-up interviews, we explore participants’ experiences with the tailored questionnaire, including the development of initiatives, compared with their previous experiences with the company’s annual attitude survey that used standardized scales. Results indicated that participants felt the tailored questionnaire highlighted issues that had previously been ignored, that initiatives were easier to develop owing to its specificity and that the feedback strategy was useful in prioritizing questionnaires. Overall, it can be concluded that tailored questionnaires may be appropriate for use in organizational intervention research and more broadly that evaluations of organizational interventions need to be contextually grounded.

Keywords

cognitive appraisal theory cognitive mapping context critical realism organizational intervention questionnaire development realistic evaluation validation

Studies consistently show associations between indicators of the work environment and the health and well-being of employees (Crawford et al., 2010; De Lange et al., 2004). On the basis of such findings, organizational interventions to improve employee health and well-being are generally recommended (Cousins et al., 2004; European Trade Union Confederation, 2004; European Agency for Safety and Health at Work, 2010; ILO, 2001; Mackay et al., 2004). Organizational interventions can be defined as planned, behavioral, theory-based actions that aim to improve employee health and well-being through changing the way work is designed, organized and managed (e.g. Nielsen, 2013).

Positivism has been the dominant paradigm for evaluating organizational interventions and holds the randomized control trial as the methodological gold standard. The objective has been to evaluate whether the intervention fulfilled its stated goals (i.e. improve employee health and well-being; McEvoy and Richards, 2003). Based on this paradigm, researchers have argued that compared with individual interventions, organizational interventions are ineffective (Richardson and Rothstein, 2008). Others have argued that organizational interventions embody intricate processes embedded in social contexts (Nielsen and Abildgaard, 2013) and should focus on evaluating why interventions work the way they do (McEvoy and Richards, 2003; Nielsen and Abildgaard, 2013; Pawson and Tilley, 1997).

In realistic evaluation, the focus shifts from the research question of ‘what works?’ to ‘what works for whom in which circumstances and why?’ (Pawson and Manzano-Santaella, 2012). Realistic evaluation springs from critical realism and suggests that the main purpose of evaluation research is to obtain knowledge about the underlying causal mechanisms by which change is brought about in order to identify which processes may improve the successful implementation of the intervention (Greenhalgh, 2014). According to critical realism (Bhaskar, 1986), there exists a reality independent of our thoughts that can be differentiated at three levels: the causal level concerns the ‘mechanisms’ that generate events, the actual level concerns whether events actually take place, and the empirical level concerns the observations of experienced events. The first step in evaluating complex interventions becomes identifying the ‘working mechanism’ or ‘programme theory’ (i.e. the theoretical basis for why the intervention is expected to have the expected effect) (Pawson and Tilley, 1997). We propose that one important mechanism of the organizational interventions is screening. In the context of interventions to improve employee health and well-being, screening entails the identification of work environment factors that influence employee health and well-being and therefore need to be managed (Nielsen et al., 2010). In translating the levels of critical realism to organizational intervention research, researchers need to consider: 1) the mechanisms that effect change in the social situation, for example, the screening tool (the causal level), 2) how the screening tool is used by participants (the actual level) and 3) how the screening tool and its results lead to the development and implementation of action plans (the empirical level).

Analyses of the studies included in recent reviews of intervention processes reveal that most of the studies referenced use standardized questionnaires as their screening tool (Nielsen, 2013; Nielsen and Randall, 2013; Nielsen et al., 2010), and national policy approaches encourage the use of standardized measures (Daniels, 2011). Studies have pointed to difficulties in developing activities based on screening using standardized questionnaires in terms of translating abstract concepts such as job autonomy or social support into concrete initiatives (Daniels et al., 2004, 2011; Nielsen et al., 2010; Rick et al., 2001). Furthermore, influential stress theories suggest that individual appraisal of the work environment plays a role in determining health and well-being (Lazarus and Folkman, 1992). Therefore, screening that captures the specific organizational context and participants’ appraisals rather than trying to assume an ‘objective’ work environment (Daniels, 2011) may function as a working mechanism facilitating the development and implementation of action plans.

A central element of critical realism is the exploration of how the actions of human agents (i.e. participants in the intervention) are influenced by innate psychological mechanisms (cognitive appraisal), and the wider social context (collective sensemaking of the results of the tailored questionnaire) (Bhaskar, 1986; Edwards, 2005). There is a gap in the intervention literature on how to design screening tools that match the theoretical processes of appraisal and the context that employees inhabit. One problem is that standardized measures assume a simple relationship between underlying/causal processes and the manifestation of working conditions in any given context (Daniels et al., 2006). Critical realism offers a way forward in that it highlights the context as a key influence on how things at the causal level become manifest at the empirical level. In the present study, we describe the development and evaluation of a questionnaire tailored to a target group of postal service mail carriers taking into account the appraisals of the target group and the local context and how it was used to make sense of the social context and develop action plans. The tailored approach offers a sensitivity of day-to-day realities as advocated by critical realism (Edwards, 2005).

We explore how a tailored questionnaire and the subsequent translation process of its results act as a mechanism enabling employees and managers to make sense of their work environment (Greenhalgh, 2014; Pawson and Tilley, 1997). We assess the impact on the development of initiatives that are sufficiently detailed and contextualized to be perceived as useful. Our intention is not to present a new questionnaire, but to show how a rigorous process of tailoring and structured translation can provide a better basis for intervention activities to be developed. To the best of our knowledge, no studies have focused on whether the use of tailored questionnaires can help organizations develop better initiatives compared with when initiatives are developed based on standardized questionnaires. Therefore, we have little systematic knowledge on the methods that may incorporate cognitive appraisal of local contexts into organizational intervention design. Guided by realistic evaluation (Pawson and Tilley, 1997), our specific contribution is to examine an important mechanism in organizational intervention processes (i.e. how screening facilitates sensemaking and the opportunity to develop detailed and contextualized action plans to improve the psychosocial work environment and employee health and well-being).

Theoretical challenges in using standardized questionnaires in intervention research

Many current tools for screening are based on the assumption that an ‘objective’ work environment exists that has an effect on employees (Nielsen et al., 2010). The underlying assumption of such measures is that there is an increased probability of an aspect of the work environment being harmful if it exceeds a certain threshold (Cousins et al., 2004; Mackay et al., 2004). This view has been challenged (e.g. Rick and Briner, 2000). In relation to developing screening tools, a corollary of the assumption that aspects of the work environment are objective is that two people in the same job would experience work in the same way (Daniels, 2011; Daniels et al., 2012), yet research has found that employees in similar jobs do not rate their work environment the same (Persson et al., 2012). The relationships between working conditions and outcomes also differ depending on the organizational context (John, 2001).

The way individuals appraise and give meaning to their experiences at work is a trigger to their well-being (Lazarus and Folkman, 1992). According to cognitive appraisal theory (CAT), individuals categorize features of their environment based on the extent to which they perceive an aspect of the environment to be harmful, challenging or irrelevant (Lazarus and Folkman, 1992). Thus, it is important to measure whether a situation or a condition is appraised to be good or bad for the individual’s well-being in order to determine the severity of the problem (Lazarus and Folkman, 1992). Numerous studies have found support for the importance of cognitive appraisal in employee health and well-being (Daniels et al., 2006; Dewe, 1989; Harris and Daniels, 2005; Webster et al., 2011) and it thus becomes essential to consider appraisals when aiming to improve employee well-being (Daniels, 2011).

In organizational intervention research, CAT may be attractive in that the variation in perceptions is not attributed to individual differences but rather to interpretations of the work environment (Dewe and Trenberth, 2012; Harris and Daniels, 2005), thus making it possible to influence these appraisals through changing the way work is organized, designed and managed (Dewe and Trenberth, 2012). To capture which appraisals are shared by individuals in the workplace, it is important to examine the extent to which an issue is experienced as problematic in a group. If, for example, 50 percent of employees experience a certain aspect of the work environment to be problematic, this would make the argument that it is a shared problem that needs to be managed collectively.

We suggest that when developing initiatives to improve employee health and well-being, we need to a) measure positive and negative appraisals of the work environment, b) examine the prevalence of appraisals in the target population to assess the degree to which these appraisals approximate shared mental models, and c) examine the severity of these appraisals in relation to well-being outcomes.

Methodological challenges in using standardized questionnaires in intervention research

Difficulties in interpreting the results of standardized questionnaires have been identified (Daniels et al., 2012). Although some aspects of the work environment may be shared by many occupations, others may be specific to certain occupations, and even specific workplaces (Trenberth and Dewe, 2006), and standardized questionnaires fail to identify issues deriving from actual work duties (Evans and Coman, 1993). For example, influence over the length and composition of the postal route may be a critical feature for postal workers, yet would not be picked up by standardized measures of working conditions. In a qualitative study, Dewe (1989) concluded that globalized measures do not lead to a better understanding of the nature and structure of, for example, actual problems in the workplace. We argue that the choice of instruments to assess working conditions depends on the purpose of the assessment. Studies that aim to assess specific aspects of the work environment and target these for intervention, as is the case in organizational interventions, may be better served using a tailored questionnaire (Hurrell et al., 1998).

When conducting interventions to improve the psychosocial work environment, the recommendation is to conduct a thorough screening to identify factors relevant to the group in question, and to target activities to the specific problems of the workplace as these will be more effective (Murphy and Sauter, 2004). Standardized measures make it difficult to get a nuanced understanding of the complex situation in a specific target group, and it is therefore difficult to develop targeted initiatives to change the way work is designed, organized and managed in that group (Dewe, 1989). As the goal of intervention implementation research is to develop knowledge concerning initiatives that produce a sustainable, positive impact on well-being, it would appear that tailored measures may be preferable because they allow for a contextualized screening of the issues relevant to the target group.

Daniels et al. (2004) presented a cognitive model for how employees appraise their situations and suggested that assessment of the work environment should be made on the basis of categories of experience and the language used to describe those categories by the target population rather than forcing external classification through standardized measures. Daniels et al. (2004) argued that this tailored approach provides a better basis for intervention because it reflects how employees construe their reality and make predictions about what actions will be effective.

We argue that in contextually dense research such as intervention studies, the focus should be on local explanations based on tailoring of frameworks, thus producing local, accurate and simple explanations (Weick, 1995). The need for local measures is based on the fact that cognitions are not just individual, but also based on social interaction (Edwards, 2005; Weick, 1995). According to sensemaking theory, individuals’ appraisals can become shared by a group of employees over time, through social interaction and sharing contextual surroundings (Weick, 1990). An example is Harkness et al.’s (2005) study of a group of clerical workers who developed a shared understanding of management as the cause of poor well-being. We argue that a contextual cognitive approach focusing on appraisal (Lazarus and Folkman, 1992), shared mental models (Weick, 1995) and the organizational context (Johns, 2001) should be used when screening employees’ working conditions in organization intervention research. According to Weick (1990), representations of reality not only inform us about the world but also facilitate action. This aspect of assessment is accentuated when local language and perceptions are considered.

The present study

We present and evaluate an approach to designing and using a tailored questionnaire that 1) asks respondents to appraise whether aspects of their work environment are ‘problematic’ or ‘good’ and 2) is tailored to the population that is the target of interventions (mail delivery service workers) based on assessments of the work environment. In line with critical realism and realistic evaluation, we propose that a tailored questionnaire functions as a resource or mechanism (Bhaskar, 1986; Pawson and Tilley, 1997) that may help employees and managers make sense of their working environment, offering them tools to prioritize and develop activities to improve the psychosocial work environment.

First, we describe a method to develop tailored items. Second, using qualitative methods, we report on employees’ and managers’ perceptions of the tailored questionnaire and its usefulness in the intervention process compared with the company’s standardized annual attitude survey. To the best of our knowledge, this is the first study to present and evaluate such an approach.

More specifically, we investigate five research questions:

Which, if any, problems did the organization experience with the existing standardized questionnaire?

What types of questions were developed based on interviews?

How did participants evaluate the ability of the tailored approach to detect issues relevant to the target group compared with the standardized questionnaire?

How did participants evaluate the usefulness of the feedback method that reported on prevalence and severity of appraisals of harmful and benign aspects of the work environment compared with their experiences with the standardized questionnaire?

How suitable was the tailored approach to develop initiatives compared with the standardized questionnaire?

Methods

Context of intervention

The intervention took place in the Danish national postal service. An internal occupational health consultant in the postal service approached the first author, inviting her to conduct an intervention study to improve employee health and well-being in the postal service using a tailored, systematic approach that the consultant knew the first author had experience with. Postal areas in Jutland were invited to participate by the internal occupational health consultant. Four geographically distinct postal areas, two in the North of Jutland and two in the Mid of Jutland volunteered after meetings with the area and the Human Resources managers in the invited areas. In the postal service, employees are organized into teams who are responsible for delivering mail to a smaller geographical area within the overall postal area. In total, 24 teams with 363 employees and 17 managers participated in the intervention. The two geographical areas (North and Mid) were randomly assigned either to receive the intervention or be on the waiting list. We conducted a baseline survey with two follow-ups with 12 months in between them. Furthermore, we conducted interviews and observed meetings and workshops related to the project between surveys.

The intervention employed a participatory problem solving cycle design (Kompier et al., 1998; Nielsen et al., 2010, 2013), with the phases of preparation, screening, action planning, implementation and evaluation. Throughout the project, a researcher would observe meetings and workshops relevant to the project and take extensive notes.

Preparation phase

In the first phase, steering groups were established that assumed overall responsibility for implementing the project. The steering groups consisted of employee and manager representatives. From each team in the participating areas, there was at least one employee representative. An internal consultant functioned as a facilitator and the researchers functioned as observers and would provide information on the method. Organizational data were collected including the content and results of the standardized annual attitude survey.

Screening phase

The second phase was the screening phase. We followed the guidelines of Hinkin (1998) in developing items for a tailored questionnaire. For the development of the tailored questionnaire, a series of interviews were conducted in both the intervention and waiting list groups. The interviews were based on a cognitive mapping approach (Harris et al., 2002) and began with open questions about the positive and negative aspects of the working environment (Nielsen et al., 2013). Employees were then asked about which measures had been taken at different levels (individual, group, managerial, or organizational) to maximize positive demands and minimize adverse demands. All responses were written down on coloured sticky notes and placed on a large piece of paper. The responses were linked with the problem/resource they addressed, thus producing a map of the perceived work environment. All managers in the participating areas were interviewed; from smaller teams (N < 25) two employees were interviewed individually; and from larger teams (N > 25) three to four employees were interviewed in focus groups and one individual interview was held. Using a stratified approach, researchers randomly selected all interviewees from alphabetical personnel lists provided by the organization. If a person was absent for the duration of the interview period, the next person on the personnel list was selected. None of the employees refused to take part in the study; however, two participants asked to be interviewed together. Seventeen managers and 56 employees were interviewed (including six group interviews). Interviews lasted between 29 minutes and two hours and eight minutes. The majority of interviews lasted between one hour and one hour and 15 minutes. Interviews were tape-recorded, and transcribed into NVivo and matched with photos of the cognitive maps.

All job features reported in the interviews were analyzed by two researchers to develop items. The reported aspects of the work environment were translated into neutrally worded items (for some examples of items included see Appendix 1). As problems with the follow-up of the annual attitude survey were identified during the cognitive mapping interviews, we also included items about the annual survey. Tailored items were discussed in steering group meetings with employee and manager representatives, HR and occupational health consultants to ensure face validity. Tailored items were presented to participants as statements (e.g. Degree of involvement in connection with changes; see Appendix 1) on ‘Very problematic’ = 1 to ‘Very good’ = 5 Likert-type scales. We used standardized outcome measures of job insecurity (Hellgren et al., 1999), work engagement (Schaufeli et al., 2006), and burnout (Kristensen et al., 2005) because the content of these outcome measures are not context-dependent.

The tailored questionnaire was distributed to all employees and managers in the intervention and waiting list areas (with the exemption of temporary staff) and confidentiality was assured. The questionnaire was distributed to 380 employees and managers and 340 returned the questionnaire (response rate 89%, response rates varied across teams between 75% and 100%). Questionnaires were returned directly to the research team in pre-stamped envelopes.

After completion of the survey, data were analyzed and results shared with the organization. For feedback purposes, we first identified which demands and resources were linked to well-being outcomes by calculating the odds ratios of risks of being burnt out or engaged when a given aspect of the working environment was rated positively or negatively. The responses were dichotomized into 1= ‘problematic, very problematic’ and 0 = ‘good, very good’ and neither/nor was recorded as missing. Outcome variables were dichotomized around the mean and into ‘high’ and ‘low’ values. Odds ratios provided respondents with an indication of whether a given statement was related to work engagement or burnout. For example, the odds ratio for the statement ‘amount of changes’ and burnout was 4.28. This was explained to participants as ‘the risk of being burned out is more than four times higher if the respondent has reported the amount of changes as a problem’. The odds ratios gave participants an indication of which areas to prioritize (e.g. if a statement had high odds ratios for both work engagement and burnout). Chi-square was used to calculate the significance levels.

Second, frequencies were calculated to investigate which aspects of work were perceived to be either positive or negative by a majority of staff. We believed this feedback approach would aid participants’ understanding of the issues that needed to be addressed insofar that aspects of the working environment would be prioritized for change if they were more severe (gauged by odds ratios on job insecurity, burnout, lack of engagement) and reported to be a problem by a large number of staff.

An overall report of the results was produced to the steering groups in both the intervention and the waiting list areas. Short reports to each team outlining the frequencies in their team were produced to help target activities at the team level (see Appendix 2). To interpret the results, reports were discussed in steering group meetings and short reports at team meetings. In the first instance, results were fed back to the steering groups in facilitated discussions. Employee representatives and line managers would discuss the results for each team and group the items associated with job insecurity, burnout and work engagement into themes (for examples of themes see Appendix 2) and then prioritize which themes to focus on based on the severity and frequency.

Contextualized action planning phase

In the third phase, action plans were developed at the team level in the intervention group. At team meetings, employees would discuss the themes identified by representatives of their teams. In team meetings, employees developed action plans that addressed these themes and identified what should be done, by whom, when and how to ensure implementation of the action plans. If appropriate, action plans were also developed at the area level.

Implementation phase

In the fourth phase, organizational members in the intervention area would implement action plans. Regular meetings were held in the steering groups to monitor and discuss progress.

Evaluation of the intervention and its process

In the fifth, and last, phase, the research team conducted semi-structured interviews 12 and 24 months after cognitive mapping interviews. In the waiting list group, employees were asked general questions about what had happened in the last year. In the intervention group, employees were asked about their experiences with the intervention and its process. Of particular relevance to the present study, employees and managers were asked about their reactions to the tailored questionnaire itself and about its usefulness in developing initiatives to improve the work environment and employee health and well-being. Results from these later phases were shared with the organization through an overall report, short reports for each team, at steering group meetings, and at team meetings. The feedback included information on which areas had improved or deteriorated significantly and information about the process.

For the purpose of this study, we included data from the first intervention group from the first follow-up and from both the intervention and the waiting list group from the second follow-up as the first intervention group had continued to work with the intervention.

In the follow-up, the following strategy was used. If a manager had left during the past year his or her successor was interviewed and if an employee from the initial cognitive mapping interviews was not available (no longer employed or on holiday), the next person on the personnel list was interviewed. In total, 20 employees were interviewed, six of these were interviewed in two focus groups. Eight new employees were included. Eleven managers were interviewed, six were newly appointed managers. Interviews lasted between 28 minutes and one hour and 38 minutes. Most interviews lasted about one hour. At the second interview follow-up, the same sampling procedure was employed as in the first follow-up. Fifty employees were interviewed, two were interviewed for the first time, and 15 of these were interviewed in five focus groups. Thirteen managers were interviewed, of which seven were newly appointed. Interviews lasted between 17 minutes and one hour and 37 minutes. Most interviews lasted about one hour.

Data analysis for the purpose of this study

For the purpose of the study presented here, we analyzed the follow-up interviews, meeting observations and compared action plans developed on the basis of the tailored questionnaire with action plans for initiatives developed on the basis of the company’s standardized questionnaire that was distributed every year in October. These post-intervention data and analysis of action plans formed the bulk of data analyzed to address the research questions; however, we also consulted data from the original cognitive mapping, questionnaire data and observations of meetings to further inform our interpretations.

We categorized the qualitative data using content analysis (Krippendorff, 2004; Tesch, 1990). The coding unit was one statement in the interviews. Content categories were: 1) Perceptions of the annual company survey using a standardized approach, 2) Perceptions/reception of tailored questionnaire, 3) Perceptions/reception of questionnaire feedback and 4) Perceptions of the usefulness of the tailored questionnaire in developing initiatives. Although not prompted to compare the tailored questionnaire with the standardized annual attitude survey, respondents often did so.

Results

Research question 1: Which problems, if any, did the organization experience with the existing standardized questionnaire?

Before entering the project, the organization had, for a number of years, conducted an annual attitude survey. However, this survey was perceived not to lead to the desired improvements in the psychosocial work environment. Employees and managers shared, together with the internal consultant, a frustration about the lack of results. At steering group meetings, it was agreed that the intervention project with its tailored questionnaire approach was an opportunity to review the standardized annual attitude survey to explore whether the most appropriate items were included, the extent to which it was possible to work with minimizing the negative aspects of work and enhance the positive aspects, rather than just focusing on firefighting and ‘soft’ wellness initiatives such as arranging barbeques after mail delivery service workers had ended their shift.

In the annual attitude survey, two items were included concerning the ‘Outcome of the annual attitude survey on my team’ and ‘The dialogue in my team about the annual attitude survey’ and these items were among the five items (out of a total of 66 items) that employees were most dissatisfied with in the majority of teams. As the cognitive mapping interviews also revealed problems with the annual attitude survey, an item was included in the tailored questionnaire: ‘The management of follow-up on the annual attitude survey’ item was reported to be problematic by 21 percent of employees, while 56 percent responded neither/nor and 24 percent reported the management of follow-up to be good. When fed back to the steering group in the intervention group, the results created a debate as to why so many had responded neither/nor. It was suggested that employees were disheartened by the annual attitude survey because it included many items that teams could not do anything about; meetings were frequently cancelled; it was perceived that management was only interested in a high response rate rather than actually doing something about the problems raised in the annual attitude survey; and no resources were allocated to solve problems. As a result, employees became cynical about the whole process and gave up trying to tackle issues raised in the survey. It was discussed that employees would only complete the annual attitude survey because their line manager put pressure on them; they did not expect any results or real engagement in improving the psychosocial work environment (Data from fourth steering group meeting, district 1). In the interviews, employees and managers raised issues with regards to the relevance of the annual attitude survey and the difficulties in developing and following up on action plans because they were too broad (see Table 1).

Table 1.

Illustrative quotes from interviews.

Theme	Illustrative quotes
Issues with the standardized questionnaire	‘I think the annual attitude survey was a little tired because we have asked the same questions for the past 10 years – and when you don’t feel it helps . . .’ (Manager 1101001)
	‘The actions just aren’t detailed enough . . . the annual attitude survey action plans are very broad.’ (Manager 1102002)
	‘It may be that the response rate is high. Basically, I don’t really think people believe in it [the annual attitude survey]. When you go to meetings about the annual attitude survey you don’t get a reaction.’ (Employee 1131008)
	‘Working with the action plans of the annual attitude survey is difficult because they do not feel relevant.’ (Comment at steering group meeting)
	‘We do sit and discuss them [the results of the annual attitude survey] and we agree what we want to focus on but there is no follow-up.’ (Employee 122 1005)
Ability of the tailored approach to detect relevant issues compared with the standardized questionnaire	‘. . . We have talked about well-being and that it was important that some issues emerged that didn’t emerge in the annual attitude survey. The [company attitude survey] has been the same for many years. So we hoped that some new issues would emerge that were more appropriate for the group. And I actually think that has happened.’ (Manager no. 2401005)
	‘Personally, I think it was good that there were new questions in this questionnaire compared with the old questions we have in the [company attitude survey]. The items covering the psychosocial work environment were better measures. And because there were more items it was also more detailed.’ (Employee no. 2411010)
	‘But the [tailored questionnaire] is much more detailed and that opens new opportunities. For example, it became evident that the days are very different. Today they work seven hours and on Friday it is seven to eight. It is very different how many are at work each day and how many routes the mail is divided into and that may confuse some – and that came out in the results.’ (Manager no. 2401005)
	‘I think it is a good way [to assess work environment issues] – for once it is the employees with their frustrations and problems and so on.’ (Employee no. 1131018)
Usefulness of the feedback method compared with the standardized questionnaire	‘. . . I think it is the degree of specification, that resulted in the problem being detected in the [tailored questionnaire study], because if only a few people are dissatisfied in a team then it may happen that it disappears and you don’t know it is a real great cause of stress. That you cannot see in the [company annual attitude survey].’ (Manager no. 1101001)
	‘What I think is interesting about this questionnaire is that it says that 50% and then the weighting part of it . . .The weighting part [reporting on prevalence and severity and adding these up to get an overall understanding of a problem area] of it is good in terms of helping us prioritize what issues do we need to work with to make initiatives.’ (Manager no. 2303004)
	‘It is helpful that you get the angle where you can see whether it is job insecurity or burnout that an aspect is related to – or work engagement. Then you can see what we are really good at, so let’s do more of that. This has clearly resulted in a focus on something we normally wouldn’t have done – what works instead of what doesn’t work.’ (Employee no. 2401004)
	‘I think the questionnaire the way you ask is fantastic. I like the odds ratios you make and I like the probability calculation, that is something that has been really powerful . . . I think there is something good in there, because you can feel they [employees and managers] have caught onto something. Perhaps it doesn’t have the level of systematics and in-depth understanding but you can feel that there is something . . . that they say, wow, we really are very different, and there are these and these issues, so they have caught something, but I don’t think the numbers come into their right, you could perhaps get even more out of it. I think you can use the complexity if you have more room to immerse yourself in the results.’ (Internal consultant)
Suitability of the tailored approach to develop initiatives compared with the standardized questionnaire	‘I think some of the questions are more specific, so it is perhaps easier to find concrete action, whereas the [company attitude survey] is broader. E.g. “Managers should be better at informing”, there you can get a more specific answer in the [tailored questionnaire] about what the problem really is.’ (Manager no. 1101001)
	‘The fact that there are more questions means that you go deeper into the topic and perhaps it also puts things in a different light, because we have very few questions in the [company attitude survey], but that is also because they try to make it easy but it also means it becomes a little superficial. So it is easy when you sit at a team meeting to say “I didn’t answer like this”. Whereas when you have 20 questions on the topic then it becomes difficult to avoid the issue and say “none of us said this”.’ (Manager no. 1201001)
	‘I think that when the [tailored questionnaire] sheds light on new issues then you have to deal with them and you become aware of them. You may think about the same issues during your daily routine, but you don’t think aloud and you don’t do anything about it.’ (Employee no. 1191008)

Research question 2: What types of questions were developed based on interviews?

In total, 167 statements were developed on the basis of the cognitive mapping interviews. In the questionnaire, these were grouped around nine themes: social relations and colleagues, working hours, self-managing work teams, the work tasks, working in the postal service, physical work environment, changes, senior management and line management. The groupings were developed by the research teams and agreed with the steering group. The statements did not easily lend themselves to the traditional categories such as social support, job demands, job control, role clarity and role conflict.

Some statements were seemingly unrelated to the postal service while others explicitly referred to the mail carrier job or working in the postal service; however, the underlying theme of most statements was that the postal service was an organization undergoing major changes. For example, the statements ‘Demands that I take on additional tasks’ and ‘Willingness among colleagues to take on extra tasks’ could on the surface have been asked in any organizational survey. However, these statements were embedded in a social context where changes were happening rapidly and roles changed. The context behind the first statement was that employees were asked to take additional tasks such as bringing food to the elderly and to keep an eye on holiday homes in remote areas. The context behind the second statement was that although previously the amount of mail had been relatively stable from day to day, changes in how people were using postal services (e.g. using email instead of postal letters) meant that there was greater day-to-day and route-to-route variability in the amount of mail to be delivered. In some of the postal areas, a solution to this problem was that mail delivery service workers could call each other to help each other out during the day if they realized they would have problems delivering all the mail on their route. This created issues as in some teams colleagues were more willing to lend a helping hand than in others.

Other items were explicitly related to the postal service as an organization (e.g. ‘The use of the after-hours guarantee’). This was an agreement between the union and the postal service. If a mail delivery service worker at the beginning of the working day recognized that he or she would not be able to deliver the mail on his route on time he could use the guarantee to say ‘I need to leave on time today’ and the line manager would have to call in extra staff. This relieved the individual mail delivery service worker of the pressure of relying on the goodwill of his or her colleagues. A downside of this system was that temping staff were expensive and had to be taken out of the team’s budget, which meant that in teams where the guarantee was often used they would struggle to keep within budget. This issue was related to a statement of ‘Having to keep within budget’. Another way of solving the problem with extensive use of the guarantee was to call in staff who were on leave.

In other areas, the problem with the uneven amounts of mail was addressed by changing the route design. Traditionally, the postal routes had been organized such that each mail delivery service worker had his or her own area they would deliver mail to – all starting from the postal service center and returning to the postal service center at the end of the shift. This pattern was changing to what was known as the ‘snake pattern’. The postal area would be planned not as separate routes but as one long ‘snake’ where the start and end point were at the postal center and mail delivery service workers cover varying lengths of the ‘snake’. This meant that mail delivery service workers would have (slightly) varying routes every day. The issues with the ‘snake pattern’ was translated into statements in the questionnaire such as ‘The level of familiarity with the routes’, ‘The length of the routes and the pattern’ and ‘That routes on the day are unpredictable’.

Research question 3: How did participants evaluate the ability of the tailored approach to detect issues relevant to the target group compared with the standardized questionnaire?

Both employees and managers reported that they felt topics were identified that were not included in the company’s annual attitude survey (see Table 1). They felt the tailored items made the questionnaire easier to relate to with its focus on the working day of the mail delivery service worker. For example, through inclusion of items relating to the challenges in receiving mail that had been sorted incorrectly, distributing the mail among carriers in the morning, delivering the mail throughout the day, having unequal amounts of mail from day to day and the declining amounts of mail in general. Managers reported it had helped raise awareness about specific problems that they had not been aware of before and employees reported they felt heard. An added benefit was that participants felt ownership over the project. At steering group meetings, both employee representatives and managers reported that the combined interview and questionnaire approach had increased ownership: employees who had been interviewed actively encouraged colleagues to complete the questionnaire.

Research question 4: How did participants evaluate the usefulness of the feedback method reported on prevalence and severity of appraisals of harmful and benign aspects of the work environment compared with the standardized questionnaire?

Reports were distributed to steering group members five days before the meeting. At the meeting, a member of the research team presented the results of the survey at the departmental level and explained how results should be interpreted. Steering group members then could ask questions. From each team, a team manager and an employee representative were present and provided with strips of paper each with a statement that was either related to burnout or work engagement or both, and the prevalence of statement at the team level. These smaller groups were asked to categorize the statements into themes (e.g. changes, well-being or colleagues). This approach helped develop participants’ understanding of their work environment and how different aspects of work related to each other. Managers felt that the approach helped them identify issues otherwise overlooked and that it had helped them prioritize which initiatives to develop. Participants in the team meetings reported that it was very important to discuss the results at the team level to enable sensemaking at this level.

The reporting of frequencies at both the overall area and the team level allowed the steering group and teams to determine at which level interventions should be developed. For problems prevalent in the whole group the steering group would develop action plans, whereas for problems at the team level the team would develop its own action plan.

Research question 5: How suitable was the tailored approach to develop initiatives compared with the standardized questionnaire?

Both employees and managers reported that they felt the tailored questionnaire had helped them develop initiatives. The degree of specificity and the length of the questionnaire made the answers more credible and helped ensure ownership in that employees felt obliged to take action. The specificity made it possible to develop initiatives targeting more specific aspects of the work environment. A review of the action plans based on the company attitude survey revealed they had been limited to either obvious and collectively agreed upon simple problems such as ‘Problems with faulty equipment should be reported immediately’ or ‘More tidiness in the workplace’, or non-specific plans addressing macro aspects of the workplace without explicit instructions for action (e.g. ‘Accept that everybody does not work at the same pace. Fast employees shouldn’t do all the work’). In contrast, the initiatives based on the tailored measure were both specific and addressed issues of perceived importance.

For example, in the team where the action plan on social climate had previously been ‘Accept that everybody does not work at the same pace. Fast-working employees shouldn’t do all the work’, the short report indicated that the particular problem was that employees did not communicate in a respectful manner during meetings and had verbal conflicts. An item in the tailored questionnaire related specifically to the fora used by postal workers to communicate with each other. This item was ‘Colleagues’ reactions to attitudes and comments made in plenary sessions’ (36% of employees reported this to be a problem) and was found to be problematic in this team. Based on the tailored questionnaire, the team developed concrete action that included use of a ‘red card’, used like a referee in a football match:

‘Sometimes some colleagues start screaming at each other, but then we have the red card . . . We have a red card that we use, where we say timeout’ (Employee 1181005).

Employees agreed that all had the opportunity to use the red card when the conduct was out of line and thus stop abusive verbal behavior. The experience with the use of the solution was positive:

It’s not that they shouldn’t square their differences but maybe they need to go home and reflect on the issues. Then they may discuss how to get on. The rest of us don’t need to listen to two people arguing and getting upset, this is why we chose to implement the red card as a way to improve dialogue. (Manager 1101001)

In another team, the team report revealed problems with employee involvement in, and subsequently commitment to, re-planning of delivery routes. The routes were laid out by a computer system based on mail amounts and other statistics; this did not consider local factors, such as where it was possible to cross the road easily. An example of an item included in the tailored questionnaire is: ‘Frequency of route planning’, and responses to this item were found to indicate problems within the team (57% of staff reported this to be a problem). The plan of action was to have a large board in the sorting room where the new routes were shown. Postal service mail carriers could then suggest improvements to the preplanned route layout. This plan both improved the flow of mail delivery, evened out the routes, and created a sense of involvement. One of the employees involved in the re-planning explained that:

. . . everybody has been a lot more involved. We did something that was really, really clever this time. We had a board . . . and sticky notes. It’s so clever, we put the time schedule and the routes up and see what belongs to what route. We spent two days on this. Every time someone would come in we said, ‘if you have five minutes please look at this because we really need your help, what do you think of this route layout’. So everybody was participating . . . that time was a damn good investment. (Employee 1211014)

This plan made the unpopular task of replanning routes a more positive experience, both for the ones responsible for creating the new route layout but also for those employees whose daily routes would be changed.

In summary, interviews indicated that both employees and managers perceived the tailored questionnaire as a better alternative to manage well-being in the workplace than the existing standardized survey. They felt the tailored questionnaire helped identify issues specific to the group, issues that were not captured by standardized scales, and that it gave employees the opportunity to voice their problems. The feedback was perceived as helpful in getting a better understanding of the severity and prevalence of the problems, in a way that made initiatives possible. Finally, the initiatives developed on the basis of the tailored measurement were perceived to be more relevant and were more comprehensive than previous action plans.

Discussion

Currently, most organizational intervention studies use standardized questionnaires to measure potential harm to employee health and well-being (Nielsen et al., 2010; Nielsen and Abildgaard, 2013; Nielsen and Randall, 2013). However, this use is problematic (Daniels, 2011; Daniels et al., 2004; Nielsen et al., 2010). In the present study, we have described the design of a tailored questionnaire that considers employee appraisals and is tailored to a specific intervention group of postal service mail carriers. Our study builds on the critical realism paradigm to explore how a mechanism (the screening tool) influenced employees’ sensemaking processes and their subsequent development of action plans (Bhaskar, 1986). To the best of our knowledge, this is the first study that explores and assesses how tailored questionnaires, that capture both the participants’ cognitive appraisals and the local context, may be used in organizational intervention research.

At a practical level, the present study offers valuable insights into how this approach can be used to develop initiatives that are more easily understood by participants and therefore can be used to develop detailed action plans on how to improve employee health and well-being. With respect to evaluation, our results support the importance of extending the randomized control trial design to examine more than whether an intervention worked. At a theoretical level, the present study suggests that we should see organizational interventions and their screening process not as a technical and structural issue but rather as a process of sensemaking (Weick, 1995) where representations of reality are collectively interpreted to facilitate the development of action plans. Critical realism stipulates that social structures provide resources that enable individuals to act but at the same time they also place limits on human agency (Bhaskar, 1986). It would appear that the tailored questionnaire, together with a structured action planning process, enabled the participants in this intervention to develop action plans they perceived to be detailed and sustainable, whereas the standardized annual attitude survey and its social context of senior management not investing in the process did not enable such agency. In translating the results of the tailored questionnaire into action plans that were perceived to be useful, participants were able to transform their work environment and how they interacted in daily work life (e.g. when experiencing conflicts and managing changes in the postal routes).

To answer our research questions, we found that problems were reported with the annual attitude survey using a standardized questionnaire. It was felt that items did not capture issues that could be dealt with at the team level, that there was an overarching focus on negative aspects rather than a balanced focus of problems and resources, and employees were cynical about the commitment of management to follow up on issues raised. Employees and managers felt that the tailored questionnaire revealed issues that they had not been aware of previously and that it provided a greater level of detail than the company’s standardized questionnaire. The statements included in the tailored questionnaire did not easily lend themselves to the traditional categories of job demands, job control, social support and role clarity and conflict and so forth (e.g. Cousins et al., 2004; Karasek and Theorell, 1990; Warr, 1987). Instead statements seemed for the most part to be related to an underlying theme of change. While some statements were related to the work of the mail delivery service workers explicitly (e.g. the route planning), other statements could have been included in any organizational survey but the meaning attributed to the statements were specific to the postal service.

These findings indicate that standardized measures may not map onto the processes through which workers make sense of their working environments (Daniels et al., 2004) and consequently how working environments may come to be enacted (Daniels et al., 2006). Beyond the confines of intervention research, such findings may suggest standardized measures might have limited utility in understanding the inter- and intra-personal processes through which work environments influence the experience of work, and that more contextually-grounded methods may be more suitable in some instances (see also Daniels, 2011; Dewe, 1989; Trenberth and Dewe, 2012). Further support for taking a contextually grounded approach to assessment comes from comparisons between action plans developed on the basis of the standardized questionnaire and the action plans developed on the basis of the tailored questionnaire. Representations of reality, such as results from questionnaires, not only promote understanding and sensemaking but also facilitate action (Weick, 1995). It would appear that the avoidance of pre-imposed classifications made it easier for participants to relate to the content of the questionnaire and use this to develop detailed action plans that were perceived to be meaningful and that were detailed and contextualized.

Our study shows using tailored questionnaires may be one way of encouraging participation and human agency in organizational interventions (Nielsen, 2013; Nielsen and Randall, 2012). Through interviews, issues relevant to the target population were identified and participants reported it led to increased ownership of the process. Through the detailed information gained from the tailored questionnaire, it was easier for employees and managers to develop initiatives, thus facilitating the translation from survey results to detailed action plans. Participation at this phase of the intervention project ensures that the screening tool is easily understood by participants. It would appear that the tailored approach also succeeded in creating ownership and commitment among a group of employees who perceived their management not to take an interest in improving the work environment but rather chase high response rates in the annual attitude survey.

The tailored questionnaire was designed to assess workers’ appraisals of their working conditions. Appraisal-based theories have been criticized for ‘blaming the victim’ (i.e. placing responsibility for poor well-being on the employees themselves). However, in the tailored questionnaire approach, variation in perceptions is not attributed to individual differences but rather interpretations of the work environment (Dewe and Trenberth, 2012; Harris and Daniels, 2005), thus making it possible to influence these appraisals through changing the way work is organized, designed and managed (Dewe and Trenberth, 2012). Work on shared mental models suggests that over time groups of employees develop a shared understanding of their environment (Levesque et al., 2001; Weick, 1995). In this study, we incorporated the level of sharedness by calculating the prevalence of specific perceptions of work factors to determine the extent to which problems or positive aspects of work were experienced by large groups of employees, either at the departmental or the team level.

Strengths and limitations

The main strength of this study is the qualitative, multi-source design used to evaluate the use of tailored questionnaires in organizational intervention research. However, the study also has a number of limitations that must be considered.

First, as the scale development was part of an intervention program, we did not have the opportunity to test the items elsewhere in the postal service. Second, this approach is time-consuming and requires skills in questionnaire development. Further research should investigate whether the gains of this approach in terms of ability to develop targeted initiatives to improve employee health and well-being outweigh the costs of developing items or practically, whether a simple toolkit or web-based system can be developed for practitioners to develop their own tailored items. Certainly, employees and managers in the present study reported the tailored approach to be worthwhile. Third, it could be argued that the standardized questionnaire used in the organization was a poorly designed questionnaire, and therefore any other measure would be perceived more positively. However, the standardized questionnaire both covered established constructs (e.g. role clarity) and items either resembled or were adopted from established questionnaires (Kristensen et al., 2005; Morgeson and Humphrey, 2006).

Fourth, it could be argued that respondents were apparently positive about the tailored approach as they were interviewed by the researchers who had developed the questionnaire. This is unlikely, as the intervention process was managed by an internal consultant, who served as the ‘face of the project’ and interviews in the second round were primarily conducted by a newly appointed researcher whom interviewees had not met before, and interviews were conducted 8–9 months after tailored questionnaire administration.

Finally, it could be argued that the novelty of the questionnaire could explain the increased activity in developing action plans and that the company survey in its earlier years had been perceived more positive. However, a review of the company survey revealed that at no point during its history had employees felt the company survey fulfilled its purpose (in the survey there was a question on the satisfaction with the follow-up of the previous survey). Also, during the project, the postal service relaunched the standardized survey without any increases in the subsequent action planning and implementation.

Conclusion

In line with realistic evaluation (Pawson and Tilley, 1997) we examined the questions of ‘What works for whom in which circumstances and why?’ (Pawson and Manzano-Santaella, 2012) of a tailored questionnaire approach in an organizational intervention. What works seems thus to be a tailored questionnaire that incorporates cognitive appraisal and considers the local context of the intervention group. For whom the intervention works appears to be employees in an organization where a standardized questionnaire did not capture the local context of the intervention group and in particular the rapid pace of changes at the group and individual levels within the organization. The circumstances in which the approach works seem to be when the screening is followed by a structured sensemaking process where employees and managers collectively discuss and interpret the screening results in order to make sense of these and to develop intervention action plans. Why the screening tool was perceived to work better than a standardized screening tool was because it captured participants’ cognitive appraisal and the local context and enabled participants to make sense of their work environment. The present study indicates that intervention researchers need to critically review their screening tools and consider both the sensemaking processes embedded in using screening data to identify appropriate initiatives and whether the underlying constructs of job design are better represented by tailored items that reflect how the constructs become manifest in any given target population rather than standardized items that may trade off universal application for contextualized understanding.

Footnotes

Appendix

Appendix 2.

Examples of themes emerged at the team level

Team	Themes and items in theme	Prevalence
Team 1	Changes	Problematic	Good
	Opportunities to influence my future job situation	60%	10%
	Time set aside for getting used to new tasks	30%	40%
	Support for changes from colleagues	10%	60%
	Degree of influence in connection with changes	30%	40%
	Degree of involvement in connection with changes	10%	60%
	Demands on the mail of the future	30%	50%
	Understanding of the necessity for changes among colleagues	10%	60%
	Amount of changes	20%	40%
	Collaboration & social interaction in the team	10%	60%
	Use of rules for social interaction in the team	10%	60%
	Demands that I take on extra tasks	10%	60%
	Acceptance that we are all different	30%	50%
	Degree of rumours in my team	20%	40%
Team 2	The future
	Amount of changes	38%	13%
	Time set aside for getting used to new tasks	38%	38%
	Opportunities to influence my future job situation	38%	19%
	Degree of involvement in connection with changes	31%	13%
	Degree of influence in connection with changes	38%	19%
	Demands on the mail man of the future	25%	50%
	Support for changes from colleagues	19%	13%
	Well-being
	Degree of rumours in my team	44%	13%
	Accept in the team that we are all different	25%	50%
	Use of rules for social interaction in the team	25%	31%
	Willingness among colleagues to take on extra tasks	13%	56%
Team 3	Change/Future
	Amount of changes	41%	14%
	Time set aside for getting used to new tasks	46%	14%
	Opportunities to influence my future job situation	41%	9%
	Degree of involvement in connection with changes	27%	23%
	Degree of influence in connection with changes	41%	14%
	Demands on the mailman [sic] of the future	18%	36%
	Colleagues
	Support for changes from colleagues	27%	23%
	Willingness among colleagues to take on extra tasks	27%	36%
	Use of rules for social interaction in the team	18%	36%
	Understanding of the necessity for changes among colleagues	18%	50%
	Acceptance in the team that we are all different	18%	64%

Funding

This research was funded by the National Work Environment Research Fund (Grant no. 14- 2009-09).

Karina Nielsen is Professor of Work and Organisational Psychology at the Norwich Business School, the University of East Anglia, UK, research affiliate at the CPH-NEW, USA and Karolinska Institutet, Sweden, and Honorary Professor at the University of Leicester, UK. Her research interests lie within the area of new ways of working and job redesign. She is particularly interested in the evaluation of organizational interventions and ways to develop methods to understand how and why such interventions succeed or fail. She is currently on the editorial boards of Human Relations, The Leadership Quarterly and Journal of Business and Psychology and is an associate editor of Work & Stress. She has published her work in journals such as Human Relations, Work & Stress, The Leadership Quarterly, Journal of Occupational Health Psychology and Journal of Organizational Behavior. [Email: k.nielsen@uea.ac.uk]

Johan Simonsen Abildgaard is a researcher at the National Research Centre for the Working Environment in Denmark. His research interests relate to materiality and process in organization theory, conceptualization and measurement of working conditions and mixed methods evaluation of organizational level work environment interventions. He has published his work in Work & Stress and Nordic Journal of Working Life Studies. [Email: jss@nrcwe.dk]

Kevin Daniels is Professor of Organizational Behaviour at the University of East Anglia, UK and a Fellow of the British Psychological Society. His main research interests relate to job design, especially in relation to how workers use job characteristics for self-regulation. He is a member of the editorial boards of Human Relations, Journal of Management and Journal of Occupational and Organizational Psychology. He has published in journals such as Human Relations, Journal of Applied Psychology, Journal of Management, Journal of Occupational and Organizational Psychology and Journal of Organizational Behavior. [Email: kevin.daniels@uea.ac.uk].

References

Bhaskar

(1986) Scientific Realism and Human Emancipation. London: Verso.

Cousins

Mackay

Simon

Kelly

McCaig

(2004) Management standards and work-related stress in the UK. Work & Stress 18(2): 113–136.

Crawford

LePine

Rich

(2010) Linking job demands and resources to employee engagement and burnout: A theoretical extension and meta-analytic test. Journal of Applied Psychology 95(5): 834–848.

Daniels

(2011) Stress and well-being are still issues and something still needs to be done: Or why agency and interpretation are important for policy and practice. In: Hodgkinson

Ford

(eds) Review of Industrial and Organizational Psychology. Chichester: Wiley, 1–45.

Daniels

Harris

Briner

(2004) Linking work conditions to unpleasant affect: Cognition, categorization and goals. Journal of Occupational and Organizational Psychology 77(3): 343–363.

Daniels

Hartley

Travers

(2006) Beliefs about stressors alter stressors’ impact: Evidence from two experience-sampling methods. Human Relations 59(9): 1261–1285.

Daniels

Karanika-Murray

Mellor

Van Veldhoven

(2012) Moving policy and practice forward: Beyond descriptions of job characteristics. In: Biron

Karanika-Murray

Cooper

(eds) Improving Organizational Interventions on Stress and Well-being: Addressing Process and Context. London: Psychology Press, 313–332.

De Lange

Taris

Kompier

MAJ

Houtman

ILD

Bongers

(2004) The relationships between work characteristics and mental health: Examining normal, reversed and reciprocal relationships in a 4-wave study. Work & Stress 18(2): 149–166.

Dewe

(1989) Examining the nature of work stress: Individual evaluations of stressful experiences and coping. Human Relations 42(11): 993–1013.

10.

Dewe

Trenberth

(2012) Exploring the relationships between appraisals of stressful encounters and the associated emotions in a work setting. Work & Stress 26(2): 161–174.

11.

Edwards

(2005) The challenging but promising future of industrial relations: Developing theory and method in context-sensitive research. Industrial Relations Journal 36(4): 264–282.

12.

European Trade Union Confederation (2004) Framework Agreement on Work-related Stress. Brussels: European Trade Union Confederation.

13.

European Agency for Safety and Health at Work (EU-OSHA) (2010) Enterprise survey of enterprises on new and emerging risks. European Agency for Safety and Health at Work. Available at: http://www.esener.eu.

14.

Evans

Coman

(1993) General versus specific measures of occupational stress: An Australian police survey. Stress Medicine 9(1): 11–20.

15.

Greenhalgh

(2014) Realist synthesis. In: Edwards

O’Mahoney

Vincent

(eds) Studying Organizations Using Critical Realism: A Practical Guide. Oxford: Oxford University Press.

16.

Harkness

AMB

Long

Bermbach

Patterson

Jordan

Kahn

(2005) Talking about work stress: Discourse analysis and implications for stress interventions. Work & Stress 19(2): 121–136.

17.

Harris

Daniels

(2005) Daily affect and daily beliefs. Journal of Occupational Health Psychology 10(4): 415–428.

18.

Harris

Daniels

Briner

(2002) Using cognitive mapping for psychosocial risk assessment. Risk Management: An International Journal 4(3): 7–21.

19.

Hellgren

Sverke

Isaksson

(1999) A two-dimensional approach to job insecurity: Consequences for employee attitudes and well-being. European Journal of Work and Organizational Psychology 8(2): 179–195.

20.

Hinkin

(1998) A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods 1(1): 104–121.

21.

Hurrell

Nelson

Simmons

(1998) Measuring job stressors and strains: Where we have been, where we are, and where we need to go. Journal of Occupational Health Psychology 3(4): 368–389.

22.

ILO (2001) Guidelines on Occupational Safety and Health Management Systems. Geneva: International Labor Office.

23.

Johns

(2001) In praise of context. Journal of Organizational Behavior 22(1): 31–42.

24.

Karasek

Theorell

(1990) Healthy Work: Stress, Productivity and the Reconstruction of Working Life. New York: Basic Books.

25.

Kompier

MAJ

Cooper

Geurts

(2000) A multiple case study approach to work stress prevention in Europe. European Journal of Work and Organizational Psychology 9(3): 371–400.

26.

Kompier

MAJ

Geurts

Grundemann

Vink

Smulders

(1998) Cases in stress prevention: The success of a participative and stepwise approach. Stress Medicine 14(3): 155–168.

27.

Krippendorff

(2004) Content Analysis: An Introduction to Its Methodology, 2nd edn. Thousand Oaks, CA: SAGE.

28.

Kristensen

Borritz

Villadsen

Christensen

(2005) The Copenhagen Burnout Inventory: A new tool for the assessment of burnout. Work & Stress 19(3): 192–207.

29.

Kristensen

Hannerz

Hogh

Borg

(2005) The Copenhagen Psychosial questionnaire (COPSOQ): A tool for the assessment and improvement of the psychosocial work environment. Scandinavian Journal of Work and Environmental Health 31(6): 438–449.

30.

Lazarus

Folkman

(1992) Stress, Appraisal and Coping. New York: Springer Publications.

31.

Levesque

Wilson

Wholey

(2001) Cognitive divergence and shared mental models in software development project teams. Journal of Organizational Behavior 22(2): 135–144.

32.

McEvoy

Richards

(2003) Critical realism: A way forward for evaluation research in nursing? Journal of Advanced Nursing 43(4): 411–418.

33.

Mackay

Cousins

Kelly

Lee

McCaig

(2004) ‘Management standards’ and work-related stress in the UK: Policy background and science. Work & Stress 18(2): 91–112.

34.

Morgeson

Humphrey

(2006) The Work Design Questionnaire (WDQ): Developing and validating a comprehensive measure for assessing job design and the nature of work. Journal of Applied Psychology 91(6): 1321–1339.

35.

Murphy

Sauter

(2004) Work organization interventions: State of knowledge and future directions. Sozial Praventivmedizin 49(2): 79–86.

36.

Nielsen

(2013) How can we make organizational interventions work? Employees and line managers as actively crafting interventions. Human Relations 66(8): 1029–1050.

37.

Nielsen

Abildgaard

(2013) Evaluating organizational interventions: A research-based framework of process and effect evaluation. Work & Stress 27(3): 278–297.

38.

Nielsen

Randall

(2013) Opening the black box: A framework for evaluating organizational-level occupational health interventions. European Journal of Work & Organizational Psychology 22(5): 601–616.

39.

Nielsen

Randall

(2012) The importance of employee participation and perception of changes in procedures in a teamworking intervention. Work & Stress 26(2): 91–111.

40.

Nielsen

Randall

Holten

Rial González

(2010) Conducting organizational-level occupational health interventions: What works? Work & Stress 24(3): 234–259.

41.

Nielsen

Stage

Abildgaard

Brauer

(2013) Participatory intervention from an organizational perspective: Employees as active agents in creating a healthy work environment. In: Bauer

Jenny

(eds) Salutogenic Organizations and Change: The Concepts between Organizational Health Intervention Research. New York: Springer Publications, 327–349.

42.

Pawson

Manzano-Santaella

(2012) A realist diagnostic workshop. Evaluation 18(2): 176–191.

43.

Pawson

Tilley

(1997) Realistic Evaluation. London: SAGE.

44.

Persson

Hansen

ÅM

Kristiansen

Nordander

Balogh

Ohlsson

. (2012) Can the job content questionnaire be used to assess structural and organizational properties of the work environment? International Archives of Occupational and Environmental Health 85(1): 45–55.

45.

Richardson

Rothstein

(2008) Effects of occupational stress management intervention programs: A meta-analysis. Journal of Occupational Health Psychology 13(1): 69–93.

46.

Rick

Briner

(2000) Psychosocial risk assessment: Problems and prospects. Occupational Medicine 50(5): 310–314.

47.

Rick

Briner

Daniels

Perryman

Guppy

(2001) A Critical Review of Psychosocial Hazard Measures. Sudbury: HSE Books.

48.

Schaufeli

Bakker

Salanova

(2006) The measurement of work engagement with a short questionnaire. Educational and Psychological Measurement 66(4): 701–716.

49.

Tesch

(1990) Qualitative Research: Analysis Types and Software Tools. Bristol, PA: Falmer.

50.

Trenberth

Dewe

(2006) Understanding the experience of stressors: The use of sequential analysis for exploring the patterns between various work stressors and strain. Work & Stress 20(3): 191–209.

51.

Warr

(1987) Work, Unemployment, and Mental Health. Oxford: Clarendon Press.

52.

Webster

Beehr

Love

(2011) Extending the challenge–hindrance model of occupational stress: The role of appraisal. Journal of Vocational Behavior 79(2): 505–516.

53.

Weick

(1990) Cartographic myths in organizations. In: Huff

(ed.) Mapping Strategic Thought. Chichester: John Wiley & Sons, 1–10.

54.

Weick

(1995) Sensemaking in Organizations. Thousand Oaks, CA: SAGE.

55.

Weick

(1999) Theory construction as disciplined reflexivity: Tradeoffs in the 90s. Academy of Management Review 24(4): 797–806.

56.

Weick

Sufcliffe

Obstfeld

(2005) Organizing and the process of sensemaking. Organization Science 16(4): 409–421.