Structured Multidisciplinary work Evaluation Tool: Development and validation of a multidisciplinary work questionnaire

Abstract

BACKGROUND: Important success factors for the Occupational Health Service (OHS) include services being based on active participation and risk identification from a multidisciplinary/multifactorial perspective. Despite an extensive search, no questionnaire with this approach was found so a new questionnaire was developed at the OHS. The aim of this study was to develop and validate the new questionnaire named Structured Multidisciplinary work Evaluation Tool (SMET) through action research.

METHOD: Communicative and pragmatic validity were tested through the development of the questionnaire using action theory and presented in a descriptive portrayal. The Content Validity Index (CVI) was used to test content validity for each item as well as for the questionnaire as a whole.

RESULT: Communicative and pragmatic validity were developed and tested over time in four different periods between 2008 and 2014, in 24 clinics (with a total of approximately 1000 employees) in Region Jönköping County.

The content validity of the SMET questionnaire as a whole was close to excellent and the validity of the questions regarding physically and psychosocially demanding work factors were found to be excellent. The questions regarding environmentally demanding work factors were found to have a lower, but still good, validity.

CONCLUSION: The SMET questionnaire has very good content validity. The pervasive work with the SMET questionnaire also shows good pragmatic and communicative validity.

Keywords

Occupational health care work questionnaire validity participatory ergonomics

Abbreviations

SMET

Structured Multidisciplinary Work Evaluation Tool

OHS

Occupational health service

OSHA

European Agency for Safety and Health at Work

NIOSH

National Institute for Occupational Safety and Health at work

SWEA

Swedish Work Environment Authority

DOEM

Departments of Occupational and Environmental Medicine

CVI

Content Validity Index

I-CVI

Item – Content Validity Index

S-CVI/ave

Scale – Content Validity Index/average

S-CVI/UA

Scale – Content Validity index/Universal agreement

OSHC

Occupational safety and health care, Region Jönköping County

1 Introduction

In spite of the Occupational Health Service (OHS) existing in Sweden for many years, diseases in the musculoskeletal system, environmental problems, and psychosocial ill health are common in working life [1].

In 2013, 35% of the Swedish working population (women 44%, men 27%) reported pain in the neck or upper back at least one day per week and 32% reported pain in the lower back at least one day per week (men 27%, woman 35%) [1]. In addition to the discomfort and decreased quality of life suffered by these individuals, these disorders lead to a heavy economic burden on society due to costs associated with sick leave, poorer work performance, and reduced productivity [2, 3]. In addition, 23% of the Swedish working population is exposed to high noise levels at least 25% of the time they spend at work (men 30%, woman 16%) [1]. Work demands are experienced as high and 36% of the workforce has a workload that forces them to reduce their regular lunchtime. Support or encouragement from management staff during periods when work is considered challenging is lacking in 34% of the working population (men 35%, woman 31%) [1]. The OHS is a multidisciplinary knowledge area whose purpose is to promote safe, healthy, progressive, and productive work conditions [4]. An important success factor for the development of OHS activities is that the services are based on a participative methodology with active participation from both employers and employees and risk identification from a multidisciplinary perspective. A participative attitude and multidisciplinary approach (which includes organizational, technical, and individually based measures) has shown positive effects in the management of the work environment [5].

Swedish work environment regulations and legislation have pointed out the importance of evaluating working environments in a systematic way [6]. Assistance from the OHS can be an important help factor during the implementation of the systematic management of the work environment in working life [6].

To strengthen the participative approach of the OHS during this process, a valid questionnaire that can be used for the assessment of both physical and psychosocial exposures is needed.

A search for a questionnaire that can be used for the evaluation of the work environment from multidisciplinary/multifactorial perspectives was carried out. This included the Swedish Work Environment Agency (SWEA) [7], Prevent (a Swedish work environment management company) [8] fhvmetodik.se (a Swedish work environment management project between Gothenburg and Lund universities) [7, 9], OSHA [10], NIOSH [11], and PubMed (keywords: occupational, health, questionnaire, survey). A large amount of checklists were found, but only a limited number of questionnaires. Questionnaires used for the evaluation of psychosocial factors [12], work-related physical pain [13], and health and lifestyle factors [14] were found, but no single questionnaire that included all relevant factors was identified. Therefore, it was decided that a multidisciplinary/multifactorial work questionnaire would be developed – through the employees’ knowledge of their work – to be used as an aid in identifying problems experienced in the work environment. The work model was called Structured Multidisciplinary work Evaluation Tool (SMET).

The purpose of this study was to develop and validate the SMET questionnaire regarding pragmatic and communicative validity as well as content validity, through action research.

2 Method

2.1 Description of the SMET questionnaire

The questionnaire evaluates three areas of work: Physically experienced demands (9 items), environmentally experienced demands (8 items), and psychosocially experienced demands (13 items). It has a total of 30 questions and takes approximately 20 minutes to answer. Questions regarding physically and environmentally experienced work demands were developed partly from an existing questionnaire [15] and partly from SWEA’s code of statutes [16 –20]. The psychosocial questions have been tested previously and originates from the demand, control, and support model [21] that has been used, in different formats, in work and employee investigations [22].

A numeric scale of 1–10 was used for the questionnaire, with the extreme statements concerning the actual condition on either end of the scale. Low values indicated desirable (good) statements and high values indicated undesirable (bad) statements. These questions were followed up by one question that ranked which of the previous questions constituted the worst demands. Finally, each area was completed with an open question asking about the most strenuous work situations. There was a general question at the end of the questionnaire about how respondents perceive their work environment and their level of job satisfaction. Here, low values indicated a high degree of job dissatisfaction and high values indicated a high degree of job satisfaction.

2.2 Pragmatic and communicative validity

The general view in the literature is that traditional criteria for scientific validity do not in itself guarantee usefulness to the practitioners [23]. Pragmatic validity is the work that identifies the characteristics that lead to usefulness [23].

Communicative validity emerges from the interaction between readers and the reported research. To achieve communicative validity researchers must ensure that readers can judge the argument to be coherent, logical and substantiated [24].

Pragmatic and communicative validity were developed from the autumn of 2008 to the spring of 2014 using action research (AR). AR works through a cyclical four step process: planning, taking action and evaluating the action, leading to further planning and so on. AR is participative, the members of the system which is being studied participate actively in the cyclical process above. AR is concurrent with action, the goal is to make that action more effective while simultaneously building up a body of scientific knowledge [25, 26]. A descriptive portrayal of the SMET questionnaire development was compiled.

To establish pragmatic and communicative validity of the SMET questionnaire it has been tested over time, with continuous development taking place in 24 clinics in Region Jönköping County. The work was described in the actions, observations, and reflections of every step of the process. The working team was broadened and more professions were included successively (Fig. 1). Time needed to fill in the SMET questionnaire was approximately 10–20 minutes depending on the different stages of development.

2.3 Development of the SMET questionnaire 2008–2014

2.3.1 Action 2008–2009

An evaluation of physically demanding work factors was conducted in two nursing departments (with a total of 76 employees). A paper-based questionnaire was used. Evaluation of musculoskeletal problems was investigated using questions from the Standardised Nordic questionnaire for the analyses of musculoskeletal symptoms [24] and one open question where the employees described the most physically demanding work tasks they carry out.

2.3.2 Action 2009–2012

An evaluation of physically and psychosocially demanding work items in 11 nursing departments (435 employees) was conducted. The web-based questionnaire system was introduced (esMakerNX). Different evaluations of individual questions (for example: age, sex, height, weight, number of years in the profession) and different questions about musculoskeletal problems were tested in the questionnaire. In some of the nursing departments the experience of physical workloads was evaluated. An open question where the employees described psychosocial factors in the work environment and a question about individual experiences regarding work satisfaction and work environment were evaluated.

2.3.3 Action 2013

An evaluation of physically, environmentally, and psychosocially demanding work factors in six nursing departments (231 employees) was executed. A revised questionnaire was constructed and the questions in the questionnaire were delimited into the three following areas.

Physically demanding work factors: The questionnaire contains nine questions regarding physically demanding work. Those questions were answered using a numeric scale of 1–10. Questions 1–7 evaluate the presence of unfavourable physical exposure. Question eight evaluates which of the previous seven questions constitutes the highest physical demands. The ninth question asks employees to describe physically demanding factors in the work environment.

Environmentally demanding work factors: The questionnaire contains six questions regarding environmentally demanding work factors. Four of these questions are answered using a numeric scale of 1–10. These questions evaluate the presence of unfavourable environmental exposure. Question five evaluates which of the previous questions constitutes the worst environmental demands and question six asks the employees to describe environmentally demanding factors in the work environment.

Psychosocially demanding work factors: The questionnaire contains 13 questions regarding psychosocially demanding work factors. Six of these questions are answered using a numeric scale of 1–10 and six are answered using a numeric scale of 1–5. These questions evaluate the presence of unfavourable psychosocial exposure and question 13 asks the employees to describe psychosocially demanding factors in the work environment.

2.3.4 Action 2014

Evaluation of physically, environmentally, and psychosocially demanding work factors in 6 nursing departments (192 employees). The questions were adapted to focus on problem identification (Do you experience any problems associated with...). A question in the psychosocial area where the employee answered if they had sufficient resources in the department was excluded.

2.3.5 Action (autumn 2014) final version

The question regarding which factor is the most demanding to the psychosocial section was added. The involved team-members started to use a structured method for a qualitative analysis of the open, individually written questions to ensure that no qualitative information is lost [27].

2.4 Content validity

Content validity has been defined as the extent to which an instrument adequately samples the research domain of interest when attempting to measure phenomena [28].

An evaluation of content validity was carried out in the final version of the SMET questionnaire. The evaluation was carried out in two steps: Face validity and item-level Content Validity Index (I -CVI) [29 –31].

Step 1: (Face validity): A multidisciplinary team (consisting of occupational health nurses, ergonomists, occupational health psychologists, and industrial hygiene engineers) from OSCH in Region Jönköping County, met four times and by using both field- and professional experience they designed a questionnaire consisting of relevant questions from a work environment perspective.

Step 2: Item level content validity index (I – CVI)

When the questionnaire design was finalized in the autumn of 2014, a SMET web-based version (esMakerNX) of the questionnaire was distributed to an external group of work environment experts. Responses were required within two weeks. The expert group consisted of 15 safety delegates in the three hospital areas of Region Jönköping County, and two ergonomists, two industrial hygiene engineers, and two occupational health psychologists from the Department of Occupational and Environmental Medicine (DOEM) in three regions in Sweden. This group of experts was considered expert from two different perspectives. The safety delegates were considered to be experts in the clinical/practical work environment management and the representatives from DOEM were considered to be experts from an academic perspective.

In order to evaluate item content validity, the I-CVI was calculated for each item in the three areas of work (physical, environmental, psychosocial). The experts were asked to rate their relevance on a four-point scale (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant) [32] For each item, the I-CVI was computed as the proportion of experts giving a rating of either 3 or 4, i.e. that the item was quite relevant or highly relevant [30].

When the I-CVI proportion value was <0.50, an item was excluded. Values between 0.50–0.70 were considered fair validity, while values between 0.70–0.78 were considered good validity. Values >0.78 were considered excellent validity [31].

In order to evaluate the content validity of the questionnaire as a whole, Scale CVI/average (S-CVI/ave) was used. The S-CVI/ave was calculated by dividing the total sum of each single I-CVI with the total number of questions in the questionnaire [30, 31]. S-CVI/ave values have to be >0.80 to be approved and for validity to be considered as excellent, S-CVI/ave values have to be >0.90 [22].

3 Results

3.1 Pragmatic and communicative validity during the development of the SMET questionnaire 2008–2014

3.1.1 Observation and reflection 2008–2009

Questions regarding the presence of musculoskeletal problems gave us information about musculoskeletal problems but did not provide information about the work environment. The open question where the employees described their most strenuous work tasks gave us detailed information about the physical work environment, which was subjectively described from each of the employees perspective. The paper-based questionnaire was perceived as very time consuming for analysis and synthesis of the result.

3.1.2 Observation and reflection 2009–2012

Questions regarding the prevalence of physical complaints provided information about the level of physical complaints, but did not provide any information about the work environment. Furthermore, this information is hard to interpret. However, the employee descriptions of physically demanding work items provided good information about the work environment. The open question where the employees described psychosocial factors in the work environment provided valuable information about problems in practical working life. The question regarding experienced work satisfaction and work environment was used to compare different nursing departments.

3.1.3 Observation and reflection 2013

The problem with the questions about the estimated exposure was that almost every employee in the nursing departments described heavy lifting, uncomfortable working positions, and high demands as a part of their workload, however, when we visited the department it became evident that the described exposure was not always a problem. The question about which item was the most demanding (physically and environmentally) gave us good information and helped us to sort all the demanding work factors. The open question that the employees answered is a complement to the result. The usefulness about the estimation of sufficient resources at the department is doubted because no one feels that the resources meet the needs and therefore it gives us no information to work with.

3.1.4 Observation and reflection 2014

The adaptation towards problem identification focus gave us a good picture of problems and risks in the work environment compared to the earlier questions about exposure. Our synthesis of the open question gave us inadequate information. We became aware that we only extracted quantitative information from the three open qualitative questions, which meant that we were missing a lot of qualitative information. We were missing the question about which work factor is the most demanding within the psychosocial area. These questions are good questions that help us to extract information and find the most important work environmental risks.

3.1.5 Final version of the SMET questionnaire (autumn 2014)

The final version of the SMET questionnaire was tested for content validity.

3.2 Content validity

3.2.1 Face validity

The multidisciplinary team met and discussed the questionnaire. Both from the scientific area and field experience the team in OSHC considered face validity to be high in the SMET questionnaire and that the continuous development has improved face validity over time.

3.2.2 Content validity index

Eleven of a total of twenty-one respondents answered the questionnaire resulting in a response rate of 52.4% (response rate 66.6% academics DOEM, 46.6% safety delegates).

The content validity of the SMET questionnaire was close to excellent. The S-CVI/ave of the total SMET questionnaire was 0.89 (I-CVI 0.64–1.0). For the separate sections, the validity of the physical items was excellent. The S-CVI of the physically demanding factors was 0.90 (I-CVI 0.8–1.0) (Table 1). The validity of the environmental items was good, S-CVI was 0.83 (I-CVI 0.64–1.0) (Table 2). Two items in the SMET questionnaire are not considered to have excellent content validity. The question about chemical risks is considered to have good validity (I-CVI = 0.73) and the question about narrow spaces is considered to have fair validity (I-CVI = 0.64). Neither of these questions has such poor content validity that they have to be excluded. The validity of the psychosocial items was excellent, S-CVI of the psychosocially demanding factors was 0.92 (I-CVI 0.82–1.0) (Table 3).

4 Discussion

Initially the SMET questionnaire was evaluating work exposure at the current workplace. When the employees were asked if the exposure constituted a problem, it became obvious that exposure is not always equivalent with problems. So, for that reason, the SMET questionnaire was gradually developed and adapted from evaluating exposure to evaluating problems at work – a questionnaire for the identification of work-related problems. The SMET questionnaire identifies work-related problems both by confirming known problems and by identifying, for the workplace, unknown work-related problems. This enables the work environment management efforts to be more correctly aimed, which can produce greater benefit and diminished costs.

The clear and well-described structure in SMET is valuable in OHS. An evaluation of five leading management consulting companies in Sweden found that a clear structured working method was important for the internal work in the consulting company, the internal work in the client company, and the cooperation between them [33]. Besides, a clear work structure increased the competence in the consulting company and acted as a competence bank and contained more knowledge and experience than an individual consultant could manage [33]. The success factors for user-friendliness in these studies have been important principles during the development of SMET. The pragmatic and communicative validity, showed in the development process of the SMET work, strengthens the usefulness of the SMET questionnaire regarding employee participation through multidisciplinary and multifactorial evaluation. During the practical work with the SMET analysis at each clinic, the employees have shown both a great interest and participation in describing their work circumstances, as well as giving suggestions for improvements.

The SMET questionnaire may simplify the collaboration between the different occupational categories within OHS and clarify which competence is needed in the continued work environment management at the specific workplace. The practical work with SMET shows that the clear, simple and user-friendly work structure may contribute to high quality and benefits both for the clients and for OHS.

The response rate of the participants in the content validity test can be considered a limitation to the study’s results. However, an evaluation of the content validity is still possible when comparisons between CVI and a modified kappa statistic show that I-CVI >0.78 and S-CVI/ave >0.90 means excellent content validity regardless of the number of experts [31]. The result of the content validity with face validity and CVI shows very good validity in the whole SMET questionnaire (S-CVI = 0.89). Regarding the single items in the questionnaire, 28 of 30 questions are considered to have excellent content validity (I-CVI <0.78). Content validity was evaluated according to Polit and Beck because it can be regarded as the gold standard for CVI (21, 22). CVI was chosen because the method has several attractive qualities, it is easy to carry out, the results are easy to understand, and it evaluates both the individual items and the questionnaire as a whole [31]. As with Polit and Beck [30, 31] S-CVI/Universal Agreement (S-CVI/UA) was not used, since this method was considered to be too strict in its evaluation.

The SMET questionnaire multidisciplinary/ multifactorial structure strengthens the validity through its three perspectives on the workplace: physically, environmentally, and psychosocially. Practical instruments, such as the Test Instrument for Profile of Physical Ability (TIPPA) [34] and PLIBEL (a method assigned for the identification of ergonomic hazards) (PLIBEL) [35] are often used in OHS, but these instruments do not have a multidisciplinary/multifactorial perspective. Evaluation regarding content validity in TIPPA and PLIBEL has been used as a complement during the planning of the SMET validation. Content validity in TIPPA has been conducted with an expert panel approach similar to our validation of the SMET questionnaire [34]. Validation of PLIBEL was conducted through an extensive search in the scientific literature, which strengthens the content validity [28]. A search in the scientific literature has been going on continuously during the work with the SMET questionnaire, which strengthens the content validity.

The general picture in the literature is that traditional criteria for scientific validity do not guarantee practical benefits and user friendliness [23]. During the work with the SMET questionnaire the authors have found only one scientific article with a clear description of pragmatic and communicative validity for an instrument within OHS [36]. The Quick Exposure Check (QEC) is an instrument for the evaluation of risk factors for work-related musculoskeletal disorders, where pragmatic and communicative validity is carefully described [36]. QEC was developed in two phases (1996-98 and 2000-03) and begun with a focus on important factors for user-friendliness. Evaluation of QEC showed that the instrument must be simple, easy, and quick to conduct, useful in many different work situations, based on science, etc.

Question 14, that asks about chemical risks, does not have the same problematic focus as the other questions in the questionnaire. The reason for this is that the handling of chemical substances is tightly regulated in SWEA’s codes of statues. According to AFS 2005 : 5, all work involving chemicals shall be planned, executed, and followed up in a way that does not affect the health of the employees [19, 20]. For this reason we are only interested in finding out if chemical substances are handled at the workplace and if so, which chemical substances. If the handling of chemical substances occurs in the workplace the next step is to evaluate if the handling of these substances follows the current regulations. Therefore it is not relevant if an employee experiences problems with chemicals or not. The content validity in the environmentally demanding work factors is not as strong as the physically and psychosocially demanding work factors in the questionnaire, but they can be complemented with objective measurements of real conditions (temperature, airflow etc.), which is more difficult to measure in physically and psychosocially demanding work factors. The content validity is strengthened by the fact that the qualitative result from the questionnaire agree with the quantitative result that has emerged by the use of SMET in clinical practice.

In spite of an extensive search, no multidisciplinary/multifactorial questionnaire for evaluation of work conditions has been found. For that reason it is hard to make comparisons regarding content validity in the SMET questionnaire with comparable questionnaires.

The instruments for the assessments of physical and psychosocial items which are included in the SMET questionnaire has been shown to be valid as described in the method section [15, 21]. This might explain the good content validity for these items. The environmental items were based on Swedish work regulations and legislations, which might be a reason why these items had a lower content validity. The lack of construct validity testing of the environmental items, as well as of the SMET questionnaire as a whole, might be considered a limitation. Further studies of sensitivity and reliability of the SMET questionnaire are recommended. However, considering the good validity results for content and communicative/pragmatic validity, we as authors consider the SMET questionnaire to be valid as a practical tool.

5 Conclusion

The SMET questionnaire has very good content validity. The pervasive work with the SMET questionnaire also shows good pragmatic and communicative validity. The SMET questionnaire is a valid method for the evaluation of the work environment from a multidisciplinary/multifactorial perspective.

Conflict of interest

None to report.

Author contributions

Haraldsson: Planned the study, contributed to the study design, conducted a specific literature review, wrote the introduction, method, result, and discussion, approved the final manuscript, and send in the final manuscript.

Jonker: Planned the study, contributed to the study design, conducted continuous reviews and discussions regarding the manuscript, approved the final manuscript.

Strengbom: Planned the study, contributed to the study design, conducted continuous reviews and discussions regarding the manuscript, approved the final manuscript.

Areskoug: Planned the study, contributed to the study design, conducted continuous reviews and discussions regarding the manuscript, approved the final manuscript, scientific supervisor.

Footnotes

Acknowledgments

Thanks to Ann Lund for all your work with SMET since the start 2008. Thanks to the SMET team (Ann Lund, Inga-Lill Andersson Hjelm, Marianne Rosgren, Erik Strengbom) for all your work that has made SMET what it is today. Thanks to our colleagues at OSHC, for all your practical work and your thoughts about how to develop SMET further. Thanks to Futurum, Academy for healthcare, Region Jönköping County, who, through financial support, made it possible to carry out this study.

References

Arbetsmiljoverket [the Swedish Work Environment Authority]. Arbetsmiljöstatistisk rapport 2014:3 [Work Environment statistics Report 2014:3]. Stockholm; 2014.

Hagberg

, Vilhemsson

, Tornqvist

, Toomingas

. Incidence of self-reported reduced productivity owing to musculoskeletal symptoms: Association with workplace and individual factors among computer users. Ergonomics 2007;50(11):1820–34.

Bostrom

, Dellve

, Thomee

, Hagberg

. Risk factors for generally reduced productivity–a prospective cohort study of young adults with neck or upper-extremity musculoskeletal symptoms. Scandinavian Journal of Work, Environment & Health 2008;34(2):120–32.

Rantanen

. Future perspectives in developing high-quality occupational health services for all.. in International Symposium on Good Occupational Health Practice and Evaluation of Occupational Health Services. People and work; Helsinki: Finnish Institute of Occupational Health 1991.

European Agency for Safety and Health at Work. Work-related musculoskeletal disorders: Prevention report 2008.

Arbetsmiljöverket [The Swedish Work Environment Authority]. Arbetsmiljöverkets författningssamling AFS 2001:1, Systematiskt arbetsmiljöarbete [Provision AFS 2001:1 Systematic Work Environment Management]. Stockholm.

Arbetsmiljöverket [The Swedish Work Environment Authority]. [homepage on the Internet]. Available from: www.av.se

Svensk näringsliv LO PTK [Swedish business world]. Prevent – Arbetsmiljö i samverkan [Prevent – Work Environment in Collaboration] [homepage on the Internet]. [2014-10-07]. Available from: www.prevent.se

Arbets- och miljömedicin [Departments of Occupational and Environmental Medicine]. FHV Metodik [OHS Method], http://fhvmetodik.se [2014-09-12].

10.

European Agency for Safety and Health at Work. www.osha.europa.eu [2014-10-07].

11.

CDC – The National Institute for Occupational Safety and Health (NIOSH). www.cdc.gov [2014-10-07].

12.

Österberg

, Karlsson

. LUQSUS. Manual packet version 1.0. Lund. Division of Occupational and Environmental Health. Lund University (SWE: LUQSUS. Manualpaket version 1.0.) 2010.

13.

Kuorinka

, Jonsson

, Kilbom

, Vinterberg

, Biering-Sorensen

, Andersson

, Jorgensen

. Standardised Nordic questionnaires for the analysis of musculoskeletal symptoms. Applied Ergonomics 1987;18(3):233–7.

14.

Bergstrom

, Bjorklund

, Fried

, Lisspers

, Nathell

, Hermansson

, Helander

, Bodin

, Jensen

. A comprehensive workplace intervention and its outcome with regard to lifestyle, health and sick leave: The AHA study. Work (Reading, Mass) 2008;31(2):167–80.

15.

Ekberg

, Bjorkqvist

, Malm

, Bjerre-Kiely

, Karlsson

, Axelson

. Case-control study of risk factors for disease in the neck and shoulder area. Occupational and Environmental Medicine 1994;51(4):262–6.

16.

Arbetsmiljöverket [The Swedish Work Environment Authority]. Arbetsmiljöverkets författningssamling AFS 2012:2 Belastningsergonomi [Provision AFS 2012:2 Workload Ergonomics]. Stockholm2012.

17.

Arbetsmiljöverket [The Swedish Work Environment Authority]. Arbetsmiljöverkets författningssamling AFS 2005:16 Buller [Provision AFS 2005:16 Noise]. Stockholm2005.

18.

Arbetsmiljöverket [The Swedish Work Environment Authority]. Arbetsmiljöverkets författningssamling AFS 2009:2 Arbetsplatsens utformning [Provision AFS 2009:2 Workplace design]. Stockholm2009.

19.

Arbetsmiljöverket [The Swedish Work Environment Authority]. Arbetsmiljöverkets författningssamling AFS 2005:5 Cytostatika och andra läkemedel med bestående toxisk effekt [Provision AFS 2005:5 Chemotherapy and other pharmaceuticals with lasting toxic effects]. Stockholm2005.

20.

Arbetsmiljöverket [The Swedish Work Environmen Authority]. Arbetsmiljöverkets författningssamling AFS 2011:19 Kemiska arbetsmiljörisker [Provision AFS 2011:19 Chemical Hazards in the Working Environment]. Stockholm2011.

21.

Karasek

, Theorell

. Healthy work: Stress, productivity, and the reconstruction of working life. New York: Basic Books; 1990.

22.

Hagman

, Wigaeus Tornqvist

, Hagberg

, Hansson Risberg

, Isaksson

, Karlkvist

, Toomingas

. Arbetsförhållanden och hälsa bland datoranvändare. Deskriptiva data från basenkäten i en longitudinell studie av manliga och kvinnliga datoranvändare [Working conditions and health among computer users]. Arbetslivsinstitutet, 2001.

23.

Worren

, Moore

, Elliott

. When theories become tools: Towards a framework for pragmatic validity. Human Relations 2002;55:1227–50.

24.

Lankshear

, Knobel

. A handbook for Teacher Research: From design to implementation.: Open University Press; 2004.

25.

Coughlan

, Coghlan

. ACTION RESEARCH. Action research for oprations management. International Journal of Operations & Production Management 2002;22(2):220–40.

26.

Kemmis

, Mc Taggart

. The action research planner. Geelong, Victoria, BC: Deaken University Press; 1982.

27.

Langemar

. Kvalitativ forskningsmetod i psykologi. Att låta en värld öppna sig [Qualitative science in psychology]. Stockholm: Liber AB; 2008.

28.

Wynd

, Schmidt

, Schaefer

. Two quantitative approaches for estimating content validity. Western Journal of Nursing Research 2003;25:508–18.

29.

Lynn

. Determination and quantification of content validity. Nursing Research 1986;35(6):382–5.

30.

Polit

, Beck

. The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health 2006;29(5):489–97.

31.

Polit

, Beck

, Owen

. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health 2007;30(4):459–67.

32.

Davis

. Instrument review: Getting the most from your panel of experts. Applied Nursing Research 1992;5:194–7.

33.

Werr

, Stjernberg

, Docherty

. The functions of methods of change in management consulting. Journal of Organizational Change Management 1997;10:288–307.

34.

Missaghi-Wedefalk

, Lindh

, Nordholm

, Schön-Ohlsson

. Test Instrument for Profile of Physical Ability=TIPPA. Nordisk Fysioterapi 2004;8:74–83.

35.

Kemmlert

. A method assigned for the identification of ergonomic hazards – PLIBEL. Applied Ergonomics 1995;26(3):199–211.

36.

David

, Woods

, Li

, Buckle

. The development of the Quick Exposure Check (QEC) for assessing exposure to risk factors for work-related musculoskeletal disorders. Applied Ergonomics 2008;39(1):57–69.