Expert Elicitation to Estimate the Relative Risk of Food Safety Criteria Included in the Establishment-Based Risk Assessment Model for Canadian Hatcheries

Introduction

Nontyphoidal Salmonellae are the second-most burdensome, domestically acquired foodborne pathogens in Canada, after Campylobacter (Havelaar et al., 2012; Thomas et al., 2013; Butler et al., 2015). According to Canadian experts, ∼24% and 11% of foodborne illnesses in humans associated with Salmonella spp. are attributed to the consumption of contaminated poultry meat and eggs, respectively (Butler et al., 2016). In Canada, efforts are implemented at all levels of the poultry meat and egg production chains to reduce the public health impact of this zoonotic pathogen. From a regulatory perspective, risk-based oversight is also needed to optimize the allocation of inspection resources.

Within this context, the Canadian Food Inspection Agency (CFIA) developed quantitative risk assessment models aimed at prioritizing oversight based on the food safety risk associated with their regulated parties (CFIA, 2019a). One of these models—the Establishment-based Risk Assessment model for Hatcheries (ERA-H)—aims to quantify the food safety risk associated with the products derived from Canadian hatcheries based on the impact of Salmonella spp. on public health, these foods including poultry meat, eggs, and fertilized eggs (balut). In Canada, hatcheries with an incubation capacity of 1000 eggs or more fall under CFIA's jurisdiction. Our group previously selected 29 food safety-related risk factors and 96 assessment criteria that should be included in this model (Racicot et al., 2019).

The objective of the current study was to estimate the relative risk (RR) of each criterion using an expert elicitation and to assess the maximum increase or decrease in the level of risk obtained when multiple criteria belonging to the same cluster were identified in a specific hatchery. This process also aimed to assess the risk attribution of Salmonella spp. at the hatchery and bird-type levels.

Materials and Methods

A web-based questionnaire was developed using SimpleSurvey (OutSide Soft Solutions, inc., Québec, Canada). The questionnaire was pretested, in English and French, by three experts from the CFIA and the Université de Montréal. The questionnaire was administered to members of the Scientific Advisory Committee (SAC) on March 22, 2017. The SAC consisted of 15 hatchery and/or food safety experts from across Canada. Participants were gathered in one room, yet completed the questionnaire individually over a 2-h period (first round). Guidelines for completing the questionnaire and a glossary of terms were made available.

The first section of the questionnaire pertained to the expert's profile, including questions on the number of years of professional experience, the current employment sector, the highest degree held, and the field of expertise. The second section required the expert to estimate the relative contribution of each stage along the poultry meat and egg production chains and each type of bird hatched in Canadian hatcheries to the Salmonella spp. burden in poultry meat and egg products. The third section required the expert to estimate the RR associated with each criterion and to assess the maximum increase or decrease in the level of risk when multiple criteria belonging to a same cluster (inherent risk, risk mitigation, or compliance) were identified in a hatchery. To ensure all participants would provide estimates on a comparable scale, the concept of RR was explained: “If a hatchery is responsible for 100 human illnesses, a criterion with a RR of 2 would increase the contribution of this hatchery to the Salmonella burden by a factor of 2, which would translate into 200 human illnesses.”

Data from the first round were compiled using Microsoft Office Excel 2013 (Microsoft Corporation, Redmond, WA). Summarized results were shared with the participants in a printed report during the afternoon session. For each criterion and cluster, the report presented the following: (1) the number of respondents; (2) the minimum and maximum RR values; and (3) the first quartile (Q1), the median (Q2), and the third quartile (Q3) values. Using this information, participants were asked to compare their responses with those provided by the other experts. During the second round, participants raised technical questions, which were then discussed by the group to share points of view on the weighting assigned to each criterion. Participants were then invited to reconnect to their own web-based questionnaire and to modify their RR as they thought necessary. Data derived from the second round were compiled in the same manner as the first. At the end of the meeting, participants were given the opportunity to suggest, within 2 weeks, other assessment criteria that were not included in the questionnaire.

Due to the proposal of some additional criteria, a second expert elicitation was organized. A similar approach was adopted, although conducted electronically. Previous participants received an e-mail on August 28, 2017, that included a personalized link to a web-based questionnaire (SimpleSurvey), a copy of the guidelines provided in March 2017, a glossary, and the results of the first elicitation to ensure a consistent approach between the two elicitation processes. Participants were given 2 weeks to complete the questionnaire (first round). The second round took place on September 26, 2017, via WebEx (Cisco Systems, Inc., San Jose, CA).

Statistical analyses were performed using R Programming environment (3.2.2, 2015-08-14; R Foundation for Statistical Computing, Vienna, Austria) to evaluate whether there were significant differences among experts' final responses depending on their employment sector (industry vs. government vs. academia) and number of years of professional experience (≤15 vs. >15 years). To compare median values obtained from the second round of both expert elicitations, the nonparametric Kruskal–Wallis test for non-normally distributed data was used (Hollander and Wolfe, 1973). The method described by Benjamini and Hochberg (1995) was applied to control for the false discovery rate. To quantify the degree of uncertainty around each median, a distribution-free 95% confidence interval for percentiles was calculated (Hogg and Tanis, 2010). In addition, coefficients of quartile variation were investigated to evaluate experts' agreement from the first to the second round (Bonett, 2006).

Results

Thirteen members of the SAC voluntarily participated in the first expert elicitation conducted in March 2017 (one expert from Alberta, six from Ontario, and six from Québec). Of those, 12 participated in the second elicitation. The number of years of professional experience in the areas of hatcheries and/or food safety for all participating experts ranged from 7 to 33 (mean ± SD = 18 ± 8.2). At the time the elicitation was conducted, five participants (38.5%) were working for regulatory institutions and/or government, five (38.5%) for universities, colleges, and/or research institutions, and three (23%) were involved in the poultry industry. Past experience in the poultry industry was reported by three experts from government and academia. Experts were asked to provide information on their highest degree obtained: five experts (38%) reported holding a PhD degree, three (23%) held a Master's degree, and five (38%) held a Doctor of Veterinary Medicine (DVM) degree. Experts' fields of competence included veterinary medicine (46%), microbiology (23%), epidemiology (15%), biology (8%), and engineering (8%).

Tables 1 to 4 present the attribution of risk along the poultry meat and egg production chains, and across the different types of birds hatched in Canadian hatcheries, to the Salmonella spp. burden in poultry meat and egg products. Tables 5 to 8 present the number of respondents, median RR value (Q2), and 95% confidence interval of the median for each assessment criterion and cluster for the first and second rounds of the elicitation. Based on the values obtained from the second round, 63%, 32%, and 5% of the assessment criteria were given a RR of <2, between 2 and 3, and >3, respectively (Table 9). Criteria related to regulatory enforcement actions were attributed the highest RR by the experts. The median maximum increase or decrease in the RR for each cluster (with 95% confidence interval) was as follows: 5 (3–10), 6 (5–10), and 10 (5–20) for the inherent risk, risk mitigation, and compliance cluster, respectively.

Histograms showing the RR estimated by the experts during the second round for each criterion and cluster are presented in Appendix Figure A1. The majority of the 100 histograms (96 criteria and 4 clusters; the compliance cluster was assessed both during the March and September elicitations) showed a skewed distribution. Indeed, 63% of the histograms were highly skewed to the right with a skewness >1 (Bulmer, 1979). Bar charts representing the median RR and 95% confidence interval for each criterion (second round), by cluster, are presented in Appendix Figure A2.

Overall, there was better agreement among the experts in their RR estimates in the second round. Compared with the first round, the coefficients of quartile variation during the second round were lower for 51% of the criteria and unchanged for 40% of the criteria. The median RR values did not significantly depend on the experts' employment sector nor on the number of years of professional experience after adjustment for multiple comparisons.

Discussion

The current study estimated the RR of criteria for the new ERA-H model. A total of 13 Canadian hatchery and/or food safety experts, with significant experience (18 years on average) and knowledge, and who collectively represented all key employment sectors took part in an expert elicitation process. According to Delbecq et al. (1975, p. 174), 10 to 15 individuals are sufficient in a Delphi process when participants are homogeneous (e.g., specific expertise), which was the case for this study. Results showed that the range of RR values attributed by experts to the criteria narrowed between rounds 1 and 2, or remained unchanged; hence, having more than two rounds for the elicitation was deemed unnecessary.

Experts were initially asked to estimate the health burden of Salmonella spp. in humans that was attributed to each stage of the poultry meat and egg production chains, and to each poultry species. For poultry meat products, experts pointed out that the relative contribution of the rearing facilities and of the slaughter/processing plants were dominant. This result is in accordance with previous studies that identified rearing, transportation, and slaughtering stages as key influencers for the contamination of poultry meat products with Salmonella spp. (Heyndrickx et al., 2002; Namata et al., 2009; Bucher et al., 2012; Henry et al., 2013). However, results of a Dutch study (Nauta et al., 2000) involving nine experts identified the hatchery and the slaughter plant as critical steps in the transmission pathway of foodborne pathogens of poultry origin, and it was suggested that the application of effective control measures at those stages would give the best results. Experts of the current study estimated that broiler chicken meat contributes three times more to the human health burden attributed to Salmonella spp. than turkey meat. Based on a study conducted in Québec, Canada, the prevalence of Salmonella-positive carcasses was comparable between broiler chicken and turkey flocks at the time of slaughter (Arsenault et al., 2007). This difference might be explained by different marketing and consumption habits for the two products in Canada. Indeed, when attributing a level of risk to each commodity, the experts likely considered the domestic consumption aspect, which is ∼31.9 and 4.2 kg per capita for chicken and turkey meat, respectively (Agriculture and Agri-Food Canada, 2017). For egg products, layer birds and their production facilities were identified by the experts as critical elements contributing to the Salmonella spp. burden in humans, which is a well-defined picture all around the world (Gantois et al., 2009; Arnold et al., 2014; Denagamage et al., 2015).

The experts also estimated a RR value for 96 criteria that can impact the risk of Salmonella spp. in Canadian hatcheries. Within the inherent risk cluster, criteria related to incoming material and management practices, mixing eggs from different supplying breeder flocks in the same hatcher, frequently obtaining imported eggs from the spot market due to unexpected surge demand (i.e., unexpectedly importing incubation eggs from known and/or unknown sources to meet clients' needs), and hatching more than one species of poultry, were estimated to be the most important contributors to the overall inherent risk of a hatchery. In Canada, the poultry sector is subject to supply management, a system that matches the domestic production to the consumer demand across the country. When events affecting egg supply or hatchability happen, importing eggs with potentially unknown status is an alternate way for Canadian hatcheries to meet a demand that cannot be fulfilled domestically. Volkova et al. (2011) showed that a higher number of supplying parent flocks was associated with a significantly higher probability of detecting Salmonella spp. in the gastrointestinal tract of day-old broilers delivered to grow-out farms. The fact that the Salmonella spp. status of some supplying flocks might be unknown probably prompted experts to give more weight to this criterion. Operational features of a hatchery, such as having fixed incubator trolleys or using an evaporative cooling ventilation system, were considered by the experts as affecting a hatchery's food safety risk to a lesser extent. In Volkova et al. (2011), evaporative cooling increased the odds of detecting Salmonella spp. by more than four times when compared with an air conditioning system. However, this risk factor was not retained in their final models.

Within the risk mitigation cluster, requiring information on the foodborne pathogen status of the supplying breeder flocks and using this information to mitigate the risk was considered by experts of the current elicitation to contribute to the greatest reduction in the food safety risk of a hatchery (threefold reduction). It has been reported that a high Salmonella spp. prevalence in a specific breeder flock is correlated with a higher prevalence of Salmonella spp.-positive eggs laid by those birds, and more specifically, with a higher contamination rate of the eggshells (Arnold et al., 2014). This emphasizes the importance of monitoring programs for supplying breeder flocks when evaluating the risk that they may pose to a hatchery. For the current risk assessment model, an increase in the inherent risk of a hatchery would be offset by the reduction in risk for requiring and utilizing the information on the foodborne pathogen status of the supply flocks.

From the criteria that were estimated to contribute to a twofold reduction in risk, having a sampling plan to identify hazards, coupled with a trend analysis and an action plan when triggers are identified, was determined by the experts as an important mitigation measure, and is in agreement with previously published literature (Samberg and Meroz, 1995; USDA, 2014). Similarly, the implementation of a quality assurance program at the hatchery level, such as Hazard Analysis and Critical Control Point (HACCP), was recognized to mitigate the food safety risk. Such programs are key elements for ensuring the quality of the end-product (birds), although they do not specifically tackle foodborne pathogens along this production chain (Billy and Wachsmuth, 1997).

Experts also estimated a twofold reduction of risk to certain methods of egg sanitation during hatching, including the administration of formaldehyde (37%) using a constant rate infusion method and a fumigation approach. It is generally accepted that hatching is the critical point among all of the activities taking place in a hatchery for the control of Salmonella spp., and it is recognized that egg sanitation positively impacts the overall hygiene status of a hatchery (Samberg and Meroz, 1995; Bailey et al., 1996; Mueller-Doblies et al., 2013; USDA, 2014; Gottselig et al., 2016). Indeed, it is well known that bacterial contamination will increase in the hatcher as hatch nears completion due to the increasing density of hatchlings in a confined space and the increasing amount of fluff disseminated throughout the hatcher by air movement (Kim and Kim, 2010). Research has shown that hatchers and their related ventilation system and equipment are more likely to be contaminated than all other areas or devices in a hatchery (Kim and Kim, 2010; Mueller-Doblies et al., 2013). The hatching room has also been identified as a major source of Salmonella spp. contamination in a hatchery (Chao et al., 2007; Kim and Kim, 2010). However, no fumigation program should be used to replace cleanliness (Wright and Epps, 1958). Experts also identified the use of new cardboard trays or chick boxes (not reused) as an important mitigation approach.

All of the experts agreed that compliance with control measures and regulatory requirements imposed by the CFIA influenced the level of risk. Indeed, experts considered a hatchery to represent a risk for food safety that can be up to 10 times higher when all measures pertaining to compliance were not implemented as requested. Noncompliance related to process control (critical control points) and sanitation programs received the highest increase in risk among all of the Preventive Control Plan sub-elements (CFIA, 2019b). The role of sanitation in controlling foodborne pathogen contamination has been clearly demonstrated (Christensen et al., 1997). In addition, criteria related to enforcement actions by the CFIA were considered by the experts to have the greatest contribution to increasing the global risk.

Experts consulted during the elicitation process all agreed that the risk represented by a hatchery was greatly increased if Salmonella spp. was identified through monitoring programs, such as the CFIA's regulatory fluff sampling program. The detection of Salmonellae indicates a potential defect in the food safety management system and should trigger appropriate prevention and control activities, as it has been clearly shown that the implementation of control measures at the hatchery level after early identification of contamination can prevent further transmission of Salmonella spp. (White et al., 1997; Sivaramalingam et al., 2013).

It should be emphasized that the objective of this study was to elicit expert opinion to estimate the RR of each criterion rather than determine whether the RR values differed significantly. Confidence intervals were used to quantify the degree of uncertainty around each median. Although some of the criteria were deemed by the experts to be more important than others, the relative importance of each criterion should be interpreted with care, as confidence intervals sometimes overlapped. Expert elicitations can be used as a first line of evidence, in the absence of other data, to assess the importance and weighting of criteria. However, inherent limitations of this type of study need to be considered and should trigger further studies to confirm the importance of these criteria.

Conclusions

This study aimed to quantify the assessment criteria that will be included in the ERA-H model, based on their impact on human health, with a specific focus on Salmonella spp. The median RR value assigned to each criterion and cluster during the second round of the expert elicitations will be used in this new model to assess the food safety risk of Canadian hatcheries under CFIA's jurisdiction to judiciously allocate inspection resources for the protection of public health.