Implementation of a Calorie Menu Labeling Policy in Public Hospitals: Study Protocol for a Multiple Case Study

Abstract

Background:

Promotion of good nutrition is essential for reducing the risk of chronic disease and premature death. Evidence shows menu labeling interventions should be implemented in workplaces as part of a comprehensive approach to improve employees’ dietary habits; however, implementation challenges have arisen. This article describes a protocol for a multiple case study to explore the factors that impact on implementation of a calorie menu labeling policy in Irish public hospitals.

Methods:

Using a multiple case study design, comprising four Irish acute public hospitals, this study will draw on multiple perspectives and sources of evidence (observations followed by interviews, focus groups, and documentary analysis) to allow for a comprehensive depth and breadth of inquiry. Data collection and analysis will be guided by the Consolidated Framework for Implementation Research, bringing together constructs from implementation theories to understand the complexity of implementing policies. Hospitals will be categorized into high and low implementers of the policy based on quantitative data obtained from structured observations. Using framework analysis, within- and cross-case analyses will be performed to identify factors influencing policy implementation and to identify distinguishing patterns across high and low implementers and across hospital direct and indirect stakeholders. Strategies will be employed to ensure rigorous case study research, for example, triangulation, audit trail, reflexivity, and thick descriptions. An integrated knowledge translation approach, where researchers work with stakeholders throughout the research process, will be adopted to facilitate the translation of research into policy and practice.

Discussion:

This protocol highlights methodological insights in utilizing case study research to gain a greater understanding of the menu labeling implementation process. Study findings will be relevant to policy makers and other stakeholders involved in the rollout of such interventions and will provide a foundation to select and tailor implementation strategies to assist with scale-up of calorie menu labeling across the health service.

Keywords

nutrition food services workplace interventions menu labeling policy implementation fidelity Consolidated Framework for Implementation Research multiple case study knowledge translation

Background

Poor diet is a leading modifiable risk factor for obesity and other chronic diseases (Adams, Grandpre, Katz, & Shenson, 2019; Khandelwal, Kurpad, & Narayan, 2018). Obesity generates substantial costs for the health service, employers, and society as a whole (Tremmel, Gerdtham, Nilsson, & Saha, 2017). In the workplace setting, obesity is linked with increased sick leave, absenteeism, injuries, and discrimination (Fitzgerald, Kirby, Murphy, & Geaney, 2016; Flint & Snook, 2015; Koepp, Snedden, & Levine, 2015; Shrestha, Pedisic, Neil-Sztramko, Kukkonen-Harjula, & Hermans, 2016). Trends in obesity prevalence show an increase in all employment industries including the health-care industry (Jackson, Wee, Hurtado, & Kawachi, 2016; Luckhaupt, Cohen, Li, & Calvert, 2014). Research shows a significant proportion of health-care professionals are obese (Kyle, Neall, & Atherton, 2016; Kyle et al., 2017; Studnek, Bentley, Mac Crawford, & Fernandez, 2010; Zapka, Lemon, Magner, & Hale, 2009), with a higher prevalence of overweight and obesity among nurses and midwifes than the general population (Bogossian et al., 2012). Evidence suggests that health-care professionals’ lifestyle behaviors influence the frequency and willingness with which they engage in health promotion practice (Blake & Harrison, 2013; Dalton, 2014; Fie, Norman, & While, 2013; Lobelo & de Quevedo, 2016) and thus may have important implications for patient care (While, 2015).

As employees now spend longer periods of time in the work environment (Artazcoz et al., 2016; Cygan-Rehm & Wunder, 2018; Johnson & Lipscomb, 2006) and frequently consume meals acquired in the worksite cafeteria (Blanck et al., 2009; Roos, Sarlio-Lähteenkorva, & Lallukka, 2004), it is imperative that the health and well-being of employees is moved to the forefront of organizational agendas. In light of this, the World Health Organization (2004) has identified the workplace as a priority setting for health promotion. In particular, the health-care industry, being one of the largest employers in many countries, provides an opportunity for workplace health promotion to reach a larger portion of the population (Al-Khudairy, Uthman, Walmsley, Johnson, & Oyebode, 2019). Furthermore, with responsibilities to both employees and patients, the health-care industry increasingly recognizes their leadership position in serving as public health role models as well as health promotion advocates (Rothstein, 2014).

Modifications to workplace environments have been recommended to promote or encourage healthy behaviors (Allan, Querstret, Banas, & de Bruin, 2017; Hollands et al., 2013). One such strategy is calorie labeling in workplace cafeterias which aims to assist employees in making both informed and healthier food choices. Evidence from two recent systematic reviews suggests labeling menus with nutrition information in the restaurant setting has positive effects on consumer dietary intake (Crockett et al., 2018; Shangguan et al., 2019) and industry practices (i.e., reformulation of menu items) (Shangguan et al., 2019). Furthermore, a recent review of environmental interventions to change dietary behaviors in the health-care setting suggests labeling as part of a multiple component intervention may help to improve employees’ dietary habits (Al-Khudairy et al., 2019). Evidence also suggests the benefits of workplace dietary interventions (including labeling) may extend beyond the workplace environment to influence dietary behaviors off-duty (Fitzgerald, Buckley, Perry, & Geaney, 2019; McCurley et al., 2019). Furthermore, research shows high levels of support for labeling among consumers (Mah et al., 2013; Reynolds et al., 2019; Roberto, Schwartz, & Brownell, 2009; Vasiljevic et al., 2018).

While there is growing consensus that nutrition education strategies such as calorie labeling should be implemented in workplaces as part of a comprehensive approach to improve employees’ dietary habits (Al-Khudairy et al., 2019; Crockett et al., 2018; Vasiljevic et al., 2018), studies suggest challenges to implementation have arisen (Vanderlee, Vine, Fenton, & Hammond, 2016; Vasiljevic et al., 2018; Vyth, Van Der Meer, Seidell, & Steenhuis, 2012). To date, much research has focused on menu labeling implementation in the restaurant setting (Kerins et al., 2018), with fewer studies in the workplace (Vasiljevic et al., 2018; Vyth et al., 2012) and health-care setting (Vanderlee et al., 2016). A recent study by Vasiljevic and colleagues (2018) found worksite cafeterias varied in the proportion of cafeteria products that were calorie labeled (50–99%) and identified issues relating to accuracy of this information. In the health-care setting, Vanderlee, Vine, Fenton, & Hammond (2016) identified challenges to menu labeling implementation including financial resources, digital menu board maintenance, and availability of healthy food options from suppliers. These findings, together with the importance of providing nutrition information that is accurately calculated and communicated (Huang, Pomeranz, & Cash, 2018), highlight the need for greater understanding of the implementation process.

In Ireland, the Health Service Executive (HSE) is responsible for delivering public health services and is the largest employer in the state with over 2,500 workplaces. In 2015, the HSE published their first implementation plan for “Healthy Ireland in the Health Services” (HSE, 2015a), which prioritized the implementation of the HSE Calorie Posting Policy throughout all publicly funded health services for staff and visitors (HSE, 2015b). The purpose of this policy is to promote awareness and increase consumption of healthier food and drink choices among HSE staff and the visiting public (HSE, 2015b). Since 2015, progress reports suggest inconsistent implementation of the policy across hospitals in Ireland (HSE, personal communication, October, 2018).

This article describes the rationale and methodology for a multiple case study to explore the factors that impact on the implementation of a calorie menu labeling policy in Irish public hospitals. The study will explore the implementation process through the lens of the Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009), which brings together constructs from an array of implementation theories to understand the complexity of implementing policies and interventions. As implementation is intertwined with the context in which it takes place, this meta-theoretical framework can assist in understanding “what ‘works where and why’ across multiple contexts” (Damschroder et al., 2009; Nilsen, 2015). Unlike previous studies, this study will compare the factors influencing implementation across hospitals with high and low levels of implementation fidelity. By adopting a case study design, this allows the researchers to comprehensively study a complex issue within its context (Anthony & Jack, 2009). Findings from this study will provide a foundation to select and tailor implementation strategies to assist with scale-up of calorie posting across the health service (Powell et al., 2019).

Study Aims and Objectives

The aim of this study is to explore the factors that impact on the implementation of a calorie menu labeling policy in Irish public hospitals.

The key objectives for this study are as follows:

To assess levels of implementation fidelity to the calorie menu labeling policy in Irish public hospitals.

To describe the barriers and facilitators to implementation of a calorie menu labeling policy in Irish public hospitals.

To explore the perceptions and experiences of stakeholders in implementing a calorie menu labeling policy in Irish public hospitals.

To compare the factors influencing implementation of a calorie menu labeling policy across hospitals with high and low levels of implementation fidelity.

To compare the factors influencing implementation of a calorie menu labeling policy from the perspective of hospital direct and indirect stakeholders.

Method

Study Design

Case study, as a research design, allows for a holistic understanding of a phenomenon, in its natural real-life context and from the perspective of those involved (Anthony & Jack, 2009; Merriam, 1998; Stake, 1995; Yin, 2014). This study will employ a mixed methods explanatory multiple case study design, in which cases comprise four Irish acute public hospitals. A multiple case study allows for a more in-depth understanding of the phenomenon, through comparison of similarities and differences within and between cases (Heale & Twycross, 2018). This study will adopt an instrumental, embedded approach. The instrumental cases (i.e., four Irish acute public hospitals) will provide a general understanding of the phenomenon, thus generating a number of findings that are potentially transferable to other hospitals (Crowe et al., 2011). A minimum of four cases is recommended by Stake (2013), while Creswell (1998) suggests no more than four cases to allow individual cases to be adequately explored. In the embedded approach, subunits of analysis will reside within the main unit of analysis (Yin, 2014). There is debate in the literature about defining the unit of analysis. This study, in alignment with Grünbaum (2007), defines the unit of analysis as the reason for selecting a particular case(s) for study; it is the phenomenon or phenomena of interest within the case(s). As such, the main unit of analysis in this case study will be the implementation of the calorie menu labeling policy, and the subunits will focus on levels of implementation fidelity to the policy, the barriers and facilitators to implementation of the policy, and the perceptions and experiences of those implementing the policy. Similar to Merriam’s work on case study research, this study will adopt a pragmatic constructivist approach to case study methodology (Merriam, 1998). In this way, the study will draw on the work of both Yin (2014) and Stake (2013), resulting in a combined perspective which best serves the research purpose. Study findings will be reported using the Good Reporting of A Mixed Methods Study checklist (O’Cathain, Murphy, & Nicholl, 2008) and the Standards for Reporting Implementation Studies checklist (Pinnock et al., 2017).

Phenomenon of Interest

The phenomenon of interest in this study includes the implementation of the HSE Calorie Posting Policy in hospital staff canteens. The HSE Calorie Posting Policy was introduced in September 2015, with the aim of promoting awareness of healthier food and drink choices among HSE staff and the public using and visiting HSE health-care facilities, by highlighting the calorie content of food and drinks provided in HSE facilities (HSE, 2015b). The policy specifies the following four conditions must be adhered to in implementing calorie posting (HSE, 2015b):

Calorie posting is in place for all food and drink items on sale.

Calorie information is displayed clearly at the “point of choice” for the consumer.

Calorie information is displayed per standard portion or per meal.

Information on how many calories an average person needs in a day is prominently displayed to help consumers better understand calorie information.

Table 1 provides more detail of the policy using the Template for Intervention Description and Replication in Population Health and Policy interventions checklist (Campbell et al., 2018).

Table 1.

Intervention Description Using the Template for Intervention Description and Replication in Population Health and Policy Interventions Guidelines for Intervention Reporting (Campbell et al., 2018).

Goal and rationale	The purpose of the Health Service Executive (HSE) Calorie Posting Policy is to promote awareness of healthier food and drink choices among HSE staff and the public using and visiting HSE health-care facilities by highlighting the calorie content of food and drinks provided in HSE facilities. The policy objectives are to: create a supportive environment, including health education for patients/service users and staff to adopt healthy eating habits; encourage increased uptake of healthy food and beverage options at HSE premises; ensure that the organization reflects best practice in relation to healthy weight management; support other initiatives in relation to the broader healthy eating and active living program; serve as an exemplar of good practice and encourage other organizations to follow suit.
Materials and training	A “Calorie Posting Toolkit” has been designed to assist hospitals when implementing the policy (HSE, 2018) and consists of the following: A guideline document entitled “Guidance for Calorie Posting Implementation.” A presentation for catering staff on calorie posting. A checklist for calorie posting implementation. Presentations from a calorie posting study day. A literature review on the impact of calorie posting in the workplace including the hospital setting.
Procedures and mode of delivery	The policy applies to all food and beverage outlets on HSE premises, that is, staff canteens, staff and visitor restaurants, coffee shops, mobile shop trolleys, and vending machines. The current study will focus on implementation in one setting, that is, hospital staff canteens. The calorie information must be clearly displayed at the “point of choice” (i.e., anywhere food and drink offerings are described and the prices are displayed). The following conditions must be adhered to in implementing calorie posting in all HSE facilities: Calorie posting is in place for all food and drink items on sale. Calorie information is displayed clearly at the “point of choice” for the consumer. Calorie information is displayed per standard portion or per meal. Information on how many calories an average person needs in a day is prominently displayed to help consumers better understand calorie information. The policy does not apply to dishes produced and served less than once a month. Furthermore, the policy does not apply to inpatient menus. Monitoring implementation of the policy includes reporting on progress and a national audit to assess impact of calorie posting. The HSE Calorie Posting Policy is mandated until it is superseded by national legislation.
Intervention providers	Stakeholders with roles and responsibilities for policy implementation include: Senior Management (Director General of HSE, National Directors, Chief Officers of Community Healthcare Organizations, and Chief Executive Officers of Hospital Groups), Facilities and Catering Management, Frontline Catering Staff, Dietitians, Health Promotion and Improvement Staff, HSE Procurement/Office of Government Procurement, and External Contractors.

Guiding Conceptual Framework

The proposed study will be guided by the CFIR (Damschroder et al., 2009). CFIR is described as a “determinant framework” meaning that it assists with understanding and/or explaining the factors that influence implementation outcomes (Nilsen, 2015). The framework incorporates constructs from 19 theories on dissemination, innovation, organizational change, implementation, knowledge translation, and research uptake (Damschroder et al., 2009). This meta-theoretical framework comprises 39 constructs organized into five major domains, all of which interact to influence implementation outcomes (Damschroder et al., 2009). The five CFIR domains are the intervention, the individuals, the inner setting, the outer setting, and the implementation process (Damschroder et al., 2009).

According to Damschroder et al. (2009), the CFIR should not be applied wholesale to every problem but rather applied to the context of the study. Therefore, the selection of CFIR constructs to guide data collection will be informed by findings of a recent systematic review conducted by the authors, which synthesized the evidence on barriers and facilitators to implementation of menu labeling interventions from the food service industry perspective (under review; Kerins et al., 2018). The CFIR will also be used to guide analysis of data to develop a richer understanding of the factors that impact on implementation of a calorie menu labeling policy in Irish public hospitals.

Status of Study

The study commenced in February 2019, with the pilot hospital and three of the four study hospitals recruited. Data collection for the study is currently underway, with an anticipated completion date by end of 2019.

Hospital Selection and Recruitment

The sampling frame consisted of 35 HSE-funded acute public hospitals in the Republic of Ireland after the exclusion criteria were applied. Hospitals with external catering services in staff canteens were excluded (n = 2), as they were considered not representative of the overall sample who utilized internal catering services. Hospitals that reported no experience of calorie posting in staff canteens, as indicated by progress reports provided by individual hospitals, were excluded (n = 6). Specialist hospitals (maternity and pediatric hospitals) were also excluded (n = 7).

To achieve a diverse sample of four hospitals for study inclusion, a two-phase sampling approach was used (Yin, 2014). The first stage involved identifying hospitals with diverse levels of implementation of the calorie menu labeling policy as indicated by the most recent progress reports (October 2018) provided by individual hospitals. Progress reports are compiled by the Healthy Ireland Hospital Group Programme Leads in the HSE and provide basic information on whether calorie menu labeling has been implemented on the breakfast menu only or across the full menu in hospital staff canteens. No information on adherence to specific policy conditions was provided in these reports. Based on the information provided, hospitals were broadly categorized into low implementers (i.e., breakfast menu only) and high implementers (i.e., full menu) of the calorie menu labeling policy. The second stage involved developing a sampling matrix to achieve maximum variation across hospitals with high and low levels of implementation, based on the following criteria: hospital group (hospitals are organized into seven individual hospital groups, each with its own management structure and linked to a major academic partner), hospital model (hospitals are categorized into one of four models which determines the complexity of care delivered in each hospital), hospital size (< and >1,500 whole time equivalent staff), and hospital type (nonvoluntary hospitals—owned and funded by the HSE, and voluntary hospitals—funded but not owned by the HSE).

The four purposively selected hospitals will be invited to participate by way of e-mail to the hospital group gatekeepers (i.e., the chief executive officer and chief operating officer of the hospital group), which will include a study information leaflet. After the hospital group gatekeeper agrees to enroll in the study (via verbal consent), the general manager of the hospital will be invited to participate by e-mail, which will include a study information leaflet. The e-mail will be followed up with a phone call 2 weeks later. Verbal consent to conduct the study will be sought.

Participant Selection and Recruitment

A combination of snowball and purposive sampling techniques will be used to recruit representatives from different stakeholder groups at each participating hospital. Study participants will include those with roles and responsibilities for implementation of the HSE Policy on Calorie Posting (HSE, 2015b): hospital senior management, facilities and catering management, frontline catering staff, dietitians, and health promotion and improvement staff. Study participants may also include other stakeholders (not specifically outlined in the HSE Calorie Posting Policy) who participated in policy implementation. Stakeholders will be invited to participate by e-mail, which will include a study information leaflet. The e-mail will be sent to each stakeholder and followed up with a phone call (no earlier than 5 working days after e-mail sent). Written informed consent will be sought from participants at each hospital site.

Data Collection

The study will draw on multiple perspectives and sources of evidence to allow for a comprehensive depth and breadth of inquiry (Merriam, 1998; Stake, 2013; Yin, 2014). Data collection methods were piloted in a nonstudy hospital prior to study commencement.

Structured Observations

The outcome of interest for this study is implementation fidelity, as indicated by adherence to conditions outlined in the HSE Calorie Posting Policy in hospital staff canteens (HSE, 2015b). According to Carroll et al. (2007), adherence is the bottom-line measurement of implementation fidelity. To measure implementation fidelity, structured observations using an observer-rated implementation checklist will be used. The checklist, developed by the researchers and refined through initial testing at a pilot hospital, will assess adherence to the four specific conditions outlined in the HSE Calorie Posting Policy (HSE, 2015b). The four conditions apply to each menu separately (to breakfast, lunch, evening menus). Structured observations will be conducted by one researcher over a 12-hr period in staff canteens to assess implementation of the conditions for each menu. For each individual menu, adherence to each of the four policy conditions will be rated on a scale between 0 and 2 (0 = no, 1 = partially, 2 = yes), thus generating a total adherence score per menu ranging from 0 (no condition implemented) to 8 (all four conditions fully implemented). Written informed consent will be sought from catering management prior to conducting the structured observation. To explore the factors influencing implementation of the HSE Calorie Posting Policy, multiple sources of data will be used, as outlined below.

Unstructured Observations

Non-participant unstructured observations will be conducted in the kitchen and meal service area of staff canteens at each case (hospital) site. This will involve observing participants (i.e., catering staff and management), without actively participating, with the aim of understanding the phenomenon (i.e., implementation of the HSE Calorie Posting Policy) in its natural setting (Mays & Pope, 1995; Urquhart, 2015). Observations will be overt, in that catering staff and management will be aware, having agreed to being observed. The length of observation will depend on the number of menus being served in the staff canteen (e.g., breakfast, lunch) and the time frame for meal preparation and service; however, the maximum observation time will be 12 hrs. The focus at each meal will be on food preparation in the kitchen and meal service in the canteen area, when tasks relating to calorie posting are taking place. The researcher will document field notes during the observation and write up in full within 24 hrs after each site visit.

Written informed consent will be sought from all catering staff and management prior to conducting the unstructured observation. Catering staff and management who do not consent to participate in the study will not be included in any observation notes taken by the researcher. As observations will be conducted in a public space, staff and members of the public will be informed of the study via a study information poster displayed at the entrance to the observational area (i.e., hospital canteen). The A3-size poster will be displayed 1 week prior to the observation to allow canteen visitors time to consider if they wish to enter the canteen area on the day of observation.

A number of strategies will be used to reduce the risk of the Hawthorne effect (i.e., where study participants alter their behavior in response to their awareness of planned observations). Research shows that by revealing your identity (i.e., overt observations), giving study participants the rationale for the observation (i.e., to help increase understanding of the process involved in implementing calorie posting and the factors that may influence implementation), establishing a good rapport, and providing assurances of confidentiality, study participants may feel relaxed and unthreatened (Casey, 2006; Oswald, Sherratt, & Smith, 2014).

Semistructured Interviews and Focus Groups

A combination of snowball and purposive sampling techniques will be used to recruit representatives from different stakeholder groups at each hospital site to take part in semistructured face-to-face interviews, except for frontline catering staff who will be invited to participate in focus groups. The rationale for inviting frontline catering staff to participate in focus groups is based on this stakeholder group sharing certain experiences or backgrounds. In determining sample size sufficiency, the principle of data adequacy will be applied (Vasileiou, Barnett, Thorpe, & Young, 2018). As such, adequate amounts of evidence from different sources will be sought to address the aim and scope of this study (Fusch & Ness, 2015; Vasileiou et al., 2018).

The interview and focus group topic guides will contain a set of open-ended questions based on the CFIR and also findings from the observations (refer to Supplementary Files 1 and 2). The interview guide will be modified to suit the category of participant. The purpose of the guide will be to ensure all questions are covered; however, open narration on participants’ implementation experiences will be encouraged and probed for more details when appropriate. This approach will encourage participants to share information they deem important, minimize recall bias, and allow us to more deeply understand their context and implementation process. All interviews and focus group discussions will be audio-recorded and transcribed verbatim. Written informed consent will be obtained from all study participants prior to conducting semistructured interviews and focus groups.

Document Analysis

In conjunction with other data collection methods, gathering information from documents will be used to develop a better understanding of the phenomenon of interest (Hancock & Algozzine, 2016). Documents may be hard copy or electronic and may include canteen policies, food purchase lists, menu cards, recipes, and nutritional information. The researcher will seek permission to access these documents via the catering manager at each hospital site.

Data Analysis

Quantitative data collected via structured observations will be analyzed. Descriptive statistics will be calculated for each hospital including scores per condition for each menu and total score for each menu. An average score across the menus for each hospital will be calculated and hospitals will be compared. Finally, hospitals will be categorized into high and low levels of implementation fidelity by applying cutoff points. Hospitals with an average score over 4 will be categorized as high implementers, while hospitals with an average score less than or equal to 4 will be categorized as low implementers. Similar to previous research, these cutoff points were determined based on minimally accepted practices derived from discussion and consensus within the research group (Swindle et al., 2019).

Using NVivo 11 software, a separate database for each hospital (case) will be used for qualitative data management and analysis. Within- and cross-case analyses will be performed to identify factors influencing implementation of the HSE Calorie Posting Policy. This will involve framework analysis (Gale, Heath, Cameron, Rashid, & Redwood, 2013), using a combined deductive and inductive approach, with the CFIR as the a priori framework. Data from one hospital (case) will be independently coded by two members of the research team to check for coding consistency. Disagreements will be discussed and resolved through consensus. Triangulation of data sources (by direct and indirect stakeholders) and types will be used in each case. Case summaries from each hospital (case) will be imported into a case-ordered matrix in which cases are listed by level of implementation. The factors influencing implementation across the four hospitals will be reviewed to identify distinguishing patterns across high and low implementers and across hospital direct (e.g., catering management and staff) and indirect (e.g., hospital senior management, dietitians, health promotion and improvement staff) stakeholders.

Rigor in Case Study Research

A number of strategies will be adopted to ensure rigorous case study research is conducted (Houghton, Casey, Shaw, & Murphy, 2013; Hyett, Kenny, & Dickson-Swift, 2014; Tracy, 2010). First, reflexivity and maintaining a clear audit trail will be undertaken throughout the research process (during the design, data collection, and analysis) to enhance the study’s dependability and confirmability (Houghton et al., 2013; Tracy, 2010). Decisions made will be captured in comprehensive notes maintained by the lead researcher. Furthermore, NVivo software will provide a record of decisions during the analysis phase, capturing the original raw data to final themes, including illustrative quotes (Houghton et al., 2013). Second, multiple sources of evidence from a wide range of stakeholders will be collected and undergo triangulation to enhance the study’s credibility (Houghton, Casey, & Smyth, 2017; Tracy, 2010). In line with best practice, each step of the framework analysis will be overseen by experienced qualitative researchers (Gale et al., 2013). Furthermore, a multidisciplinary health research team from health promotion, nursing, psychology, health services research, and public health disciplines will provide different perspectives throughout the research process, thus helping to enhance the credibility and relevance of the research findings (Tracy, 2010). Lastly, thick descriptions which include accounts of the context, the research methods, and examples of raw data (e.g., direct quotes from study participants, excerpts from the observation field notes) will be provided in the final report to enhance the transferability of the study (Houghton et al., 2013; Tracy, 2010).

Ethical Approval and Consent

Ethical approval has been obtained from the National University of Ireland Galway Research Ethics Committee (ref: 18-Oct-05), Galway University Hospitals Research Ethics Committee (ref: C.A. 2083), HSE Mid-Western Regional Hospital Research Ethics Committee (ref: 006/19), and HSE North East Area Research Ethics Committee (ref: REC/19/015). The research team is currently awaiting the outcome of a submission for ethical approval to Tallaght University Hospital/St. James’ Hospital Joint Research Ethics Committee.

Ethical challenges that are specific to qualitative research (e.g., conducting unstructured observations in a public space) have been addressed in the ethics applications (Houghton, Casey, Shaw, & Murphy, 2010). Written informed consent will be sought from all study participants prior to data collection. The study information sheet will confirm that all data collected will be confidential and that data will be aggregated so that individuals will not be identifiable. Furthermore, the study information sheet will confirm that participation in the study is voluntary and that study participants may withdraw their consent at any time without giving a reason.

Stakeholder Engagement

The research project has been developed in collaboration with an advisory group established by the Healthy Eating Active Living Policy Priority Programme, as part of Health & Wellbeing, Strategic Planning and Transformation of the HSE. The advisory group consists of key stakeholders with national or local roles and responsibilities for developing/implementing the HSE Calorie Posting Policy. The advisory group advised on the initial conception of the study, the selection and recruitment of hospitals, determining the data that will be collected and providing input into study materials (e.g., interview guides, observation checklist). The advisory group will also assist with the dissemination of study findings and facilitate the translation of research into policy and practice.

Discussion

This study protocol describes the approach and methods that will be adopted to understand the factors that impact on the implementation of a calorie menu labeling policy in Irish public hospitals. It describes a systematic and iterative approach to conducting case study research in which the fundamental goal is to develop a holistic understanding of the phenomenon of interest within its real-life context (Merriam, 1998; Stake, 1995; Yin, 2014). The research seeks to generate both naturalistic generalizability (Stake, 1978, 1995), where “generalizability is reached on the basis of recognition of similarities and differences to the results with which the reader is familiar” (Smith, 2018, p. 4), and transferability (Tracy, 2010). Findings from this research will provide insight on the factors that hinder or enable implementation of calorie posting in the health service, therefore, advising policy makers and other stakeholders involved in the rollout of such interventions and informing their future development and implementation. Any deviations from this protocol will be justified and discussed upon publication of the study findings.

Study limitations include the use of self-reported retrospective data to assess factors influencing implementation. Response bias may be present as individual’s attribute failures to external (environment or other people) rather than internal (ability, effort) factors (Miller & Ross, 1975; Zuckerman, 1979). Furthermore, recall bias may arise due to retrospective accounts of menu labeling implementation. In an effort to help minimize the effects of recall and response bias, multiple perspectives from different stakeholders and different sources of data will be used to investigate the retrospective process of implementation (Atkins et al., 2017).

We anticipate that the findings from this study will be relevant to a wide array of stakeholders including policy makers within the health service and government, health professionals, the catering industry, and researchers. The integrated knowledge translation approach adopted in this study, where researchers work with stakeholders in the planning and execution of the study, is important for facilitating the translation of research into policy and practice (Gagliardi, Berta, Kothari, Boyko, & Urquhart, 2015; Kothari, McCutcheon, & Graham, 2017). Evidence shows that researchers and knowledge users coproducing research are more likely to create effective health services and strengthen the health-care system than researchers who work in isolation (Canadian Institutes of Health Research, 2013). We will engage with influential stakeholders who have authority to implement research recommendations, thereby facilitating knowledge translation (Boaz, Hanney, Borst, O’Shea, & Kok, 2018). The findings from this study will also be disseminated through publication in a peer-reviewed open-access journal, written reports and articles, policy briefs, presentations at conferences, seminars and workshops, and via social media. Study participants will also receive a “newsletter” giving an overview of the results upon completion of the study.

Conclusion

Given the growing consensus that menu labeling should form part of a comprehensive set of strategies to improve employees’ dietary habits (Al-Khudairy et al., 2019; Crockett et al., 2018; Vasiljevic et al., 2018), greater understanding of the practical issues relating to implementation is required. This protocol describes a study that seeks to explore the factors impacting on implementation of a calorie menu labeling policy in Irish public hospitals, using case study methodology and an existing conceptual implementation framework (i.e., the CFIR). Findings from this research will provide direction for improved implementation of calorie posting across all HSE facilities and provide guidance for future development and implementation of nutrition policies within health services.

Supplemental Material

Supplementary_File_1 - Implementation of a Calorie Menu Labeling Policy in Public Hospitals: Study Protocol for a Multiple Case Study

Supplementary_File_1 for Implementation of a Calorie Menu Labeling Policy in Public Hospitals: Study Protocol for a Multiple Case Study by Claire Kerins, Catherine Houghton, Sheena McHugh, Fiona Geaney, Elaine Toomey, Catherine Hayes, Ivan J. Perry and Colette Kelly in International Journal of Qualitative Methods

Supplemental Material

Supplementary_File_2 - Implementation of a Calorie Menu Labeling Policy in Public Hospitals: Study Protocol for a Multiple Case Study

Supplementary_File_2 for Implementation of a Calorie Menu Labeling Policy in Public Hospitals: Study Protocol for a Multiple Case Study by Claire Kerins, Catherine Houghton, Sheena McHugh, Fiona Geaney, Elaine Toomey, Catherine Hayes, Ivan J. Perry and Colette Kelly in International Journal of Qualitative Methods

Footnotes

Acknowledgments

The authors would like to acknowledge the contributions of the advisory group established by the Healthy Eating Active Living Policy Priority Programme, as part of Health & Wellbeing, Strategic Planning and Transformation of the HSE in Ireland, for their input to the development of this study protocol.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study is funded by the Health Research Board under Grant SPHeRE/2013/1.

ORCID iD

Claire Kerins

Catherine Houghton

Supplemental Material

Supplemental material for this article is available online.

References

Adams

M. L.

Grandpre

Katz

D. L.

Shenson

(2019). The impact of key modifiable risk factors on leading chronic conditions. Preventive Medicine, 120, 113–118. doi:10.1016/j.ypmed.2019.01.006

Al-Khudairy

Uthman

O. A.

Walmsley

Johnson

Oyebode

(2019). Choice architecture interventions to improve diet and/or dietary behaviour by healthcare staff in high-income countries: A systematic review. BMJ Open, 9, e023687. doi:10.1136/bmjopen-2018-023687

Allan

Querstret

Banas

de Bruin

(2017). Environmental interventions for altering eating behaviours of employees in the workplace: A systematic review. Obesity Reviews, 18, 214–226. doi:10.1111/obr.12470

Anthony

Jack

(2009). Qualitative case study methodology in nursing research: An integrative review. Journal of Advanced Nursing, 65, 1171–1181. doi:10.1111/j.1365-2648.2009.04998.x

Artazcoz

Cortès

Benavides

F. G.

Escribà-Agüir

Bartoll

Vargas

Borrell

(2016). Long working hours and health in Europe: Gender and welfare state differences in a context of economic crisis. Health Place, 40, 161–168. doi:10.1016/j.healthplace.2016.06.004

Atkins

Francis

Islam

O’Connor

Patey

Ivers

… Michie

. (2017). A guide to using the theoretical domains framework of behaviour change to investigate implementation problems. Implementation Science, 12, 77. doi:10.1186/s13012-017-0605-9

Blake

Harrison

(2013). Health behaviours and attitudes towards being role models. British Journal of Nursing, 22, 86–94. doi:10.12968/bjon.2013.22.2.86

Blanck

H. M.

Yaroch

A. L.

Atienza

A. A.

S. L.

Zhang

Mâsse

L. C.

(2009). Factors influencing lunchtime food choices among working Americans. Health Education & Behavior, 36, 289–301. doi:10.1177/1090198107303308

Boaz

Hanney

Borst

O’Shea

Kok

(2018). How to engage stakeholders in research: Design principles to support improvement. Health Research Policy and Systems, 16, 60. doi:10.1186/s12961-018-0337-6

10.

Bogossian

F. E.

Hepworth

Leong

G. M.

Flaws

D. F.

Gibbons

K. S.

Benefer

C. A.

Turner

C. T.

(2012). A cross-sectional analysis of patterns of obesity in a cohort of working nurses and midwives in Australia, New Zealand, and the United Kingdom. International Journal of Nursing Studies, 49, 727–738. doi:10.1016/j.ijnurstu.2012.01.003

11.

Campbell

Katikireddi

S. V.

Hoffmann

Armstrong

Waters

Craig

(2018). TIDieR-PHP: A reporting guideline for population health and policy interventions. British Medical Journal, 361, k1079. doi:10.1136/bmj.k1079

12.

Canadian Institutes of Health Research. (2013). Evaluation of CIHR’s knowledge translation funding program. Retrieved August 01, 2019, from http://www.cihr-irsc.gc.ca/e/47332.html

13.

Carroll

Patterson

Wood

Booth

Rick

Balain

(2007). A conceptual framework for implementation fidelity. Implementation Science, 2, 40. doi:10.1186/1748-5908-2-40

14.

Casey

(2006). Choosing an appropriate method of data collection. Nurse Researcher, 13, 75–92.

15.

Creswell

J. W.

(1998). Qualitative inquiry and research design: Choosing among the five traditions. Thousand Oaks, CA: Sage.

16.

Crockett

King

Marteau

Prevost

Bignardi

Roberts

… Jebb

. (2018). Nutritional labelling for healthier food or non-alcoholic drink purchasing and consumption. Cochrane Database of Systematic Reviews, 2:CD009315. doi:10.1002/14651858.CD009315.pub2

17.

Crowe

Cresswell

Robertson

Huby

Avery

Sheikh

(2011). The case study approach. BMC Medical Research Methodology, 11, 100. doi:10.1186/1471-2288-11-100

18.

Cygan-Rehm

Wunder

(2018). Do working hours affect health? Evidence from statutory workweek regulations in Germany. Labour Economics, 53, 162–171. doi:10.1016/j.labeco.2018.05.003

19.

Dalton

(2014). Implementing NICE obesity guidance for staff: An NHS trust audit. Occupational Medicine, 65, 78–85. doi:10.1093/occmed/kqu176

20.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. doi:10.1186/1748-5908-4-50

21.

Fie

Norman

I. J.

While

A. E.

(2013). The relationship between physicians’ and nurses’ personal physical activity habits and their health-promotion practice: A systematic review. Health Education Journal, 72, 102–119. doi:10.1177/0017896911430763

22.

Fitzgerald

Buckley

Perry

I. J.

Geaney

(2019). The impact of a complex workplace dietary intervention on Irish employees’ off-duty dietary intakes. Health Promotion International, 1–11. doi:10.1093/heapro/daz051

23.

Fitzgerald

Kirby

Murphy

Geaney

(2016). Obesity, diet quality and absenteeism in a working population. Public Health Nutrition, 19, 3287–3295. doi:10.1017/S1368980016001269

24.

Flint

S. W.

Snook

(2015). Disability discrimination and obesity: The big questions? Current Obesity Reports, 4, 504–509. doi:10.1007/s13679-015-0182-7

25.

Fusch

P. I.

Ness

L. R.

(2015). Are we there yet? Data saturation in qualitative research. The Qualitative Report, 20, 1408–1416.

26.

Gagliardi

A. R.

Berta

Kothari

Boyko

Urquhart

(2015). Integrated knowledge translation (IKT) in health care: A scoping review. Implementation Science, 11, 38. doi:10.1186/s13012-016-0399-1

27.

Gale

N. K.

Heath

Cameron

Rashid

Redwood

(2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13, 117. doi:10.1186/1471-2288-13-117

28.

Grünbaum

N. N.

(2007). Identification of ambiguity in the case study research typology: What is a unit of analysis? Qualitative Market Research: An International Journal, 10, 78–97. doi:10.1108/13522750710720413

29.

Hancock

D. R.

Algozzine

(2016). Doing case study research: A practical guide for beginning researchers (3rd ed.). New York, NY: Teachers College Press.

30.

Heale

Twycross

(2018). What is a case study? Evidence-Based Nursing, 21, 7–8. doi:10.1136/eb-2017-102845

31.

Health Service Executive. (2015a). Healthy Ireland in the health services. National implementation plan 2015–2017. Dublin, Ireland: Author.

32.

Health Service Executive. (2015b). HSE Policy on Calorie Posting. Dublin, Ireland: Author.

33.

Health Service Executive. (2018). Calorie posting. Retrieved August 07, 2019, from https://www.hse.ie/eng/health/hl/calorieposting/

34.

Hollands

G. J.

Shemilt

Marteau

T. M.

Jebb

S. A.

Kelly

M. P.

Nakamura

… Ogilvie

. (2013). Altering micro-environments to change population health behaviour: Towards an evidence base for choice architecture interventions. BMC Public Health, 13, 1218. doi:10.1186/1471-2458-13-1218

35.

Houghton

Casey

Shaw

Murphy

(2010). Ethical challenges in qualitative research: Examples from practice. Nurse Researcher, 18, 15–25. doi:10.7748/nr2010.10.18.1.15.c8044

36.

Houghton

Casey

Shaw

Murphy

(2013). Rigour in qualitative case-study research. Nurse Researcher, 20, 12–17. doi:10.7748/nr2013.03.20.4.12.e326

37.

Houghton

Casey

Smyth

(2017). Selection, collection and analysis as sources of evidence in case study research. Nurse Researcher, 24, 36–41. doi:10.7748/nr.2017.e1482

38.

Huang

Pomeranz

J. L.

Cash

S. B.

(2018). Effective national menu labeling requires accuracy and enforcement. Journal of the Academy of Nutrition and Dietetics, 118, 989–993. doi:10.1016/j.jand.2018.03.001

39.

Hyett

Kenny

Dickson-Swift

(2014). Methodology or method? A critical review of qualitative case study reports. International Journal of Qualitative Studies on Health and Well-being, 9, 23606. doi:10.3402/qhw.v9.23606

40.

Jackson

C. L.

Wee

C. C.

Hurtado

D. A.

Kawachi

(2016). Obesity trends by industry of employment in the United States, 2004 to 2011. BMC Obesity, 3, 20. doi:10.1186/s40608-016-0100-x

41.

Johnson

J. V.

Lipscomb

(2006). Long working hours, occupational health and the changing nature of work organization. American Journal of Industrial Medicine, 49, 921–929. doi:10.1002/ajim.20383

42.

Kerins

McSharry

Hayes

Perry

I. J.

Geaney

Kelly

(2018). Barriers and facilitators to implementation of menu labelling interventions to support healthy food choices: A mixed methods systematic review protocol. Systematic Reviews, 7, 88. doi:10.1186/s13643-018-0752-3

43.

Khandelwal

Kurpad

Narayan

(2018). Global non-communicable diseases—The nutrition conundrum. Frontiers in Public Health, 6, 9. doi:10.3389/fpubh.2018.00009

44.

Koepp

G. A.

Snedden

B. J.

Levine

J. A.

(2015). Workplace slip, trip and fall injuries and obesity. Ergonomics, 58, 674–679. doi:10.1080/00140139.2014.985260

45.

Kothari

McCutcheon

Graham

I. D.

(2017). Defining integrated knowledge translation and moving forward: A response to recent commentaries. International Journal of Health Policy and Management, 6, 299. doi:10.15171/ijhpm.2017.15

46.

Kyle

R. G.

Neall

R. A.

Atherton

I. M.

(2016). Prevalence of overweight and obesity among nurses in Scotland: A cross-sectional study using the Scottish Health Survey. International Journal of Nursing Studies, 53, 126–133. doi:10.1016/j.ijnurstu.2015.10.015

47.

Kyle

R. G.

Wills

Mahoney

Hoyle

Kelly

Atherton

I. M.

(2017). Obesity prevalence among healthcare professionals in England: A cross-sectional study using the health survey for England. BMJ Open, 7, e018498. doi:10.1136/bmjopen-2017-018498

48.

Lobelo

de Quevedo

I. G.

(2016). The evidence in support of physicians and health care providers as physical activity role models. American Journal of Lifestyle Medicine, 10, 36–52. doi:10.1177/1559827613520120

49.

Luckhaupt

S. E.

Cohen

M. A.

Calvert

G. M.

(2014). Prevalence of obesity among U.S. workers and associations with occupational factors. American Journal of Preventive Medicine, 46, 237–248. doi:10.1016/j.amepre.2013.11.002

50.

Mah

C. L.

Vanderlinden

Mamatis

Ansara

D. L.

Levy

Swimmer

(2013). Ready for policy? Stakeholder attitudes toward menu labelling in Toronto, Canada. Canadian Journal of Public Health, 104, e229–234.

51.

Mays

Pope

(1995). Qualitative research: Observational methods in health care settings. British Medical Journal, 311, 182–184. doi:10.1136/bmj.311.6998.182

52.

McCurley

J. L.

Levy

D. E.

Rimm

E. B.

Gelsomin

E. D.

Anderson

E. M.

Sanford

J. M.

Thorndike

A. N.

(2019). Association of worksite food purchases and employees’ overall dietary quality and health. American Journal of Preventive Medicine, 57, 87–94. doi:10.1016/j.amepre.2019.02.020

53.

Merriam

S. B.

(1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.

54.

Miller

D. T.

Ross

(1975). Self-serving biases in the attribution of causality: Fact or fiction? Psychological Bulletin, 82, 213–225. doi:10.1037/h0076486

55.

Nilsen

(2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10, 53. doi:10.1186/s13012-015-0242-0

56.

O’Cathain

Murphy

Nicholl

(2008). The quality of mixed methods studies in health services research. Journal of Health Services Research & Policy, 13, 92–98. doi:10.1258/jhsrp.2007.007074

57.

Oswald

Sherratt

Smith

(2014). Handling the Hawthorne effect: The challenges surrounding a participant observer. Review of Social Studies, 1, 53–73. doi:10.21586/ross0000004

58.

Pinnock

Barwick

Carpenter

C. R.

Eldridge

Grandes

Griffiths

C. J.

… Taylor

S. J

. (2017). Standards for Reporting Implementation Studies (StaRI) statement. British Medical Journal, 356, i6795. doi:10.1136/bmj.i6795

59.

Powell

B. J.

Fernandez

M. E.

Williams

N. J.

Aarons

G. A.

Beidas

R. S.

Lewis

C. C.

… Weiner

B. J

. (2019). Enhancing the impact of implementation strategies in healthcare: A research agenda. Frontiers in Public Health, 7, 3. doi:10.3389/fpubh.2019.00003

60.

Reynolds

Archer

Pilling

Kenny

Hollands

G. J.

Marteau

(2019). Public acceptability of nudging and taxing to reduce consumption of alcohol, tobacco, and food: A population-based survey experiment. Social Science & Medicine, 236, 112395. doi:10.1016/j.socscimed.2019.112395

61.

Roberto

C. A.

Schwartz

M. B.

Brownell

K. D.

(2009). Rationale and evidence for menu-labeling legislation. American Journal of Preventive Medicine, 37, 546–551. doi:10.1016/j.amepre.2009.07.015

62.

Roos

Sarlio-Lähteenkorva

Lallukka

(2004). Having lunch at a staff canteen is associated with recommended food habits. Public Health Nutrition, 7, 53–61. doi:10.1079/PHN2003511

63.

Rothstein

M. A.

(2014). Promoting public health in health care facilities. American Journal of Public Health, 104, 965–967. doi:10.2105/AJPH.2014.301885

64.

Shangguan

Afshin

Shulkin

Marsden

Smith

… Imamura

. (2019). A meta-analysis of food labeling effects on consumer diet behaviors and industry practices. American Journal of Preventive Medicine, 56, 300–314. doi:10.1016/j.amepre.2018.09.024

65.

Shrestha

Pedisic

Neil-Sztramko

Kukkonen-Harjula

K. T.

Hermans

(2016). The impact of obesity in the workplace: A review of contributing factors, consequences and potential solutions. Current Obesity Reports, 5, 344–360. doi:10.1007/s13679-016-0227-6

66.

Smith

(2018). Generalizability in qualitative research: Misunderstandings, opportunities and recommendations for the sport and exercise sciences. Qualitative Research in Sport, Exercise and Health, 10, 137–149. doi:10.1080/2159676X.2017.1393221

67.

Stake

R. E.

(1978). The case study method in social inquiry. Educational Researcher, 7, 5–8. doi:10.3102/0013189X007002005

68.

Stake

R. E.

(1995). The art of case study research. Thousand Oaks, CA: Sage.

69.

Stake

R. E.

(2013). Multiple case study analysis. New York, NY: Guilford Press.

70.

Studnek

J. R.

Bentley

Mac Crawford

Fernandez

A. R.

(2010). An assessment of key health indicators among emergency medical services professionals. Prehospital Emergency Care, 14, 14–20. doi:10.3109/10903120903144957

71.

Swindle

Johnson

S. L.

Davenport

Whiteside-Mansell

Thirunavukarasu

Sadasavin

Curran

G. M

. (2019). A mixed-methods exploration of barriers and facilitators to evidence-based practices for obesity prevention in head start. Journal of Nutrition Education and Behavior, In Press.

72.

Tracy

S. J.

(2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16, 837–851. doi:10.1177/1077800410383121

73.

Tremmel

Gerdtham

U.-G.

Nilsson

Saha

(2017). Economic burden of obesity: A systematic literature review. International Journal of Environmental Research and Public Health, 14, 435. doi:10.3390/ijerph14040435

74.

Urquhart

(2015). Observation research techniques. Journal of the European Association for Health Information and Libraries, 11, 29–31.

75.

Vanderlee

Vine

M. M.

Fenton

N. E.

Hammond

(2016). Stakeholder perspectives on implementing menu labeling in a cafeteria setting. American Journal of Health Behavior, 40, 371–380. doi:10.5993/ajhb.40.3.9

76.

Vasileiou

Barnett

Thorpe

Young

(2018). Characterising and justifying sample size sufficiency in interview-based studies: Systematic analysis of qualitative health research over a 15-year period. BMC Medical Research Methodology, 18, 148. doi:10.1186/s12874-018-0594-7

77.

Vasiljevic

Cartwright

Pilling

Lee

M.-M.

Bignardi

Pechey

… Marteau

T. M

. (2018). Impact of calorie labelling in worksite cafeterias: A stepped wedge randomised controlled pilot trial. International Journal of Behavioral Nutrition and Physical Activity, 15, 41. doi:10.1186/s12966-018-0671-7

78.

Vyth

E. L.

Van Der Meer

E. W.

Seidell

J. C.

Steenhuis

I. H.

(2012). A nutrition labeling intervention in worksite cafeterias: An implementation evaluation across two large catering companies in the Netherlands. Health Promotion International, 27, 230–237. doi:10.1093/heapro/dar034

79.

While

A. E.

(2015). Promoting healthy behaviours—Do we need to practice what we preach? London Journal of Primary Care, 7, 112–114. doi:10.1080/17571472.2015.1113716

80.

World Health Organization. (2004). Global strategy on diet, physical activity and health. 2004. Retrieved July 22, 2019, from https://www.who.int/dietphysicalactivity/strategy/eb11344/strategy_english_web.pdf

81.

Yin

R. K.

(2014). Case study research: Design and methods. Thousand Oaks, CA: Sage.

82.

Zapka

J. M.

Lemon

S. C.

Magner

R. P.

Hale

(2009). Lifestyle behaviours and weight among hospital-based nurses. Journal of Nursing Management, 17, 853–860. doi:10.1111/j.1365-2834.2008.00923.x

83.

Zuckerman

(1979). Attribution of success and failure revisited: Or the motivational bias is alive and well in attribution theory. Journal of Personality, 47, 245–287. doi:10.1111/j.1467-6494.1979.tb00202.x

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.26 MB

0.45 MB