Expert Elicitation for Estimation of the Proportion Foodborne for Selected Microbial Pathogens in New Zealand

Abstract

Objectives:

To estimate the proportions of human cases of nine specific microbial diseases in New Zealand that were due to transmission by food and the proportion of the foodborne burden that was due to transmission by some specific foods.

Materials and Methods:

Subjective probability distributions were elicited from 10 food safety experts using a modified Delphi approach. In addition to uniform weighting of experts' opinions, two techniques were used to measure individual's expertise; self-assessment and performance-based weighting using Cooke's classical method. Aggregate estimates were derived by simulation.

Results:

Food was estimated to be the primary route of transmission for infections due to Campylobacter spp., Listeria monocytogenes, nontyphoid Salmonella spp., Vibrio parahaemolyticus, and Yersinia enterocolitica. Uncertainties were lowest for organisms where the self-assessed expertise level was highest.

Conclusions:

Foodborne proportion estimates were more “polarized” than for a similar elicitation in 2005. That is, where food was the primary transmission route the estimated proportion on account of food was higher (62.1–90.6% in the current study for self-assessed expertise weighted estimates, compared to 56.2–89.2% in 2005); where food was not the primary transmission route the estimated proportion because of food was lower (27.6–34.0% in the current study compared to 31.5–39.5% in 2005). These estimates represent an essential resource for determining the burden of foodborne disease in New Zealand.

Introduction

Enteric microbial disease is a frequent cause of illness in New Zealand (Pattis et al., 2017) and globally (Havelaar et al., 2015). The societal and financial cost of this illness can be significant (Lake et al., 2010). For many of the causative organisms, transmission can occur through a range of routes, including food.

In New Zealand, health professionals are required to report any suspected or diagnosed notifiable disease (Ministry of Health, 2019). Since December 2007, laboratories have also been required to report notifiable disease cases. Resource constraints mean that not all notifiable disease cases are investigated. Consequently, the information obtained is often not of sufficient quality to identify the route of transmission. These and other constraints suggested that expert elicitation would be a suitable approach to estimate the foodborne proportion of enteric diseases.

Expert elicitation refers to a systematic approach to obtaining and synthesizing subjective judgments from experts on a subject where there is uncertainty caused by insufficient data. It seeks to make explicit and usable the accumulated knowledge of the experts. Expert elicitation has previously been used in New Zealand (Cressey and Lake, 2005) and internationally (Hoffmann et al., 2007, 2008, 2017; Havelaar et al., 2008; Ravel et al., 2010; Davidson et al., 2011; Vally et al., 2014; Hald et al., 2016) to elicit opinions on the proportion of disease incidence that is due to transmission by food in general or by specific foods.

Source attribution estimates are important for risk ranking and policy development. In 2005, a New Zealand expert elicitation was carried out (Gallagher et al., 2002; Lake et al., 2010) for the Ministry for Primary Industries (MPI); the regulatory body with responsibility for food safety. The 2005 estimates have contributed to MPI's ranking of food safety issues for further action (Lake et al., 2010) and monitoring of progress against their performance targets for campylobacteriosis, salmonellosis, and listeriosis (Lim et al., 2012; Pattis et al., 2017).

The current study aimed to provide updated estimates for the proportions of specific enteric diseases in New Zealand that were due to transmission by food and the proportion of the foodborne burden that was due to transmission by some specific foods. Approaches for determining expertise and incorporating estimates into aggregate judgments were also investigated.

Materials and Methods

Recruitment

Research has shown little or no improvement in the aggregate opinions of panels with more than 10 experts (Shirazi, 2009; Knol et al., 2010; United States Environmental Protection Agency, 2011). Given the modest size of the New Zealand scientific community related to foodborne disease (∼100 scientists), an expert panel of this size was considered appropriate.

An initial list of experts was selected by the project oversight group (n = 3) based on the following criteria; New Zealand-based, evidence of expertise (e.g., publications, field of employment), reputation in the required area of expertise, and impartiality (no conflict of interest). Reputation and impartiality were assessed as the consensus opinion of the oversight group. Each expert on this initial list was asked to suggest three other experts who they believed met the selection criteria (a “snowball” technique). Snowball techniques for expert identification have been used in a number of expert elicitation studies (Garabed et al., 2009; Cross et al., 2011; Davidson et al., 2011; Butler et al., 2015; Hald et al., 2016). This process quickly reached a situation where no new names were suggested. The final panel was selected to provide a range of organizational affiliations.

Assessment of participant expertise

In addition to uniform weighting of experts' opinions, two techniques were used to measure individual's expertise; self-assessment and performance-based weighting.

Participants were asked to assess their own level of expertise with respect to each of the nine organisms included in the elicitation. Assessment was reported on a five-point Likert scale, as used in a U.S. attribution expert elicitation (Hoffmann et al., 2006), with three defined scale points:

1 = low expertise–no direct experience, anecdotal knowledge only

3 = medium expertise–some direct experience, wide reading

5 = high expertise–primary focus of professional work

Participants were also given the opportunity to refrain from answering questions on specific organisms, if they felt that their level of expertise was negligible. Participants' self-assessed expertise rating (1–5) was used directly as a weighting factor.

Performance-based weights were derived using Cooke's “Classical Model” (Cooke and Goossens, 2000, 2008). This approach uses “calibration” or “seed” questions to develop performance weights used in aggregating experts' judgments. A series of seven questions were prepared relating to foodborne disease risk or prevalence of foodborne hazards in Australia and New Zealand (Supplementary Material S3). The parameters were chosen on the basis that, although their value was known, the experts chosen would be unlikely to know the exact value at the time of the elicitation.

Performance in estimating the known values of seed variables under uncertainty was used to derive two quantitative measures of performance, calibration, and information, which are analogous to the accuracy and precision of the expert opinions (Cooke and Goossens, 2000, 2008). Experts are scored using a proper scoring rule. The calculated calibration and information scores are used to aggregate experts' opinions on target variables, with the aggregation being optimized to maximize the aggregate calibration and information scores. The optimization process uses a cutoff, with experts whose opinion score is below the cutoff excluded from the optimized aggregate opinion. The cutoff value for (un)weighting experts is determined by optimizing the performance of the combination in estimating the seed variables (Cooke and Goossens, 2000). Performance-based weights were derived using the expert elicitation software Excalibur (www.lighttwist.net/wp/excalibur).

Elicitation method and participant interaction

The elicitation was conducted as a two-round Delphi (Helmer, 1967; Gallagher et al., 2002), allowing feedback of first round aggregated results and revision of estimates, but with no attempt made to generate a consensus value for estimates. Training material was provided with the first round questionnaire (Supplementary Material S1). Each round consisted of completion of a questionnaire (Supplementary Material S2). Attribution proportions were elicited as natural frequencies (“Out of 100 cases how many….”), using a four-point method (Speirs-Bridge et al., 2010). The four-point method elicits a minimum, most likely and maximum estimate of the proportion foodborne and a percentage estimate of the experts' confidence in their subjective probability distribution (SPD).

The first round was conducted by e-mail in May 2013, with the second round conducted as a facilitated face-to-face meeting in June 2013. During the e-mail phase, participants were able to direct questions concerning the completion of the questionnaire to the study coordinators. The questions and associated answers were distributed to all participants.

During the face-to-face meeting, participants were given the opportunity to discuss the outputs from the first round. Participants were particularly encouraged to discuss any extreme results from the first round. In this context, “extreme” refers to results that differ markedly from the other panel results for a particular question. Following facilitated discussions, participants were given an opportunity to update their opinions by repeating the elicitation questionnaire.

The reference year for the estimates was 2012.

Ranking of relative contributions of transmission routes

Experts were asked to rank the relative contribution of five potential transmission routes for each pathogen. The routes were food, water, environment, animal contact, and person-to-person (Supplementary Material S2 for definitions). The definitions were mostly aligned with those used in two other studies (Havelaar et al., 2008; Vally et al., 2014). The definition of water in the current study only included drinking water, while for the study of Vally et al. (2014) water also included recreational water contact. Havelaar et al. (2008) included drinking and recreational water exposures under environmental. Transmission routes were ranked from 1 (greatest contribution to disease incidence) to 5 (least contribution to disease incidence).

Proportion of foodborne transmission and proportion of foodborne transmission due to selected foods

The experts' opinions of the proportion of cases of selected microbial diseases that was due to transmission by food was determined using the method outlined under “elicitation method.” The questionnaire used is included in the Supplementary Material (Supplementary Material S2).

In addition to estimating the proportion of disease due to foodborne transmission, the proportion of the foodborne transmission attributable to commonly cited food vehicles was estimated. Specific foods were selected based on MPI priorities. Pathogen/food vehicle combinations included Campylobacter (poultry, red meat), Listeria monocytogenes (ready-to-eat meats), norovirus (seafood), non-Typhi Salmonella (poultry, red meat), Shiga toxin–producing Escherichia coli (STEC) O157 (red meat), STEC non-O157 (red meat), Vibrio parahaemolyticus (seafood), and Yersinia enterocolitica (pork).

Data analysis

The final aggregate opinions were a mathematical combination of SPDs from experts who considered themselves well enough informed, with respect to the particular pathogen/disease. Opinions of individual experts were encoded as a Pert distribution, with parameters minimum, most likely and maximum. The Pert distribution is a version of the beta distribution, which can be parameterized in a manner similar to the triangular distribution, but has the advantage that the mean is less sensitive to the extremes of the distribution (Vose, 2008).

Individual SPDs were combined by Monte Carlo simulation using the Excel add-in @Risk (Palisades Corporation). The simulation selected from each experts SPD at a probability proportion to their weighting. Simulations were run for 10,000 iterations, using Latin hypercube sampling and a Mersenne twister generator.

Results and Discussion

Recruitment

The final panel included 10 individuals. Participants were from universities (1), regulatory agencies (1), research institutes (4), public health units (2), and private consultancy (2) and represented expertise in the areas of human health, foodborne disease epidemiology, surveillance and investigation, food microbiology, and risk assessment. Of the 10 participants, 4 had participated in the 2005 New Zealand expert elicitation (Cressey and Lake, 2005).

The composition of the panel differed from those in similar studies in North America, where the majority of the panelists were from government organizations (Hoffmann et al., 2006; Ravel et al., 2010; Butler et al., 2015). The panel for the current study did not include any representatives from industry. While inclusion of experts from industry would have introduced a valuable extra perspective, this also appears to have been an issue for other studies, with the proportion of experts from industry being low compared with the proportions from government and academia (Hoffmann et al., 2006; Ravel et al., 2010; Butler et al., 2015). The impact of industry experts on aggregate opinions and comparison of industry opinions with those of other expert groups would be a useful addition to research in this area.

Measurement of expertise

Participants' self-assessed expertise levels are summarized in Table 1.

Table 1.

Participant Self-Assessed Expertise

Pathogen	No. of experts in expertise category ^a						Mean expertise score ^b
Pathogen	NR	1	2	3	4	5	Mean expertise score ^b
Campylobacter	0	0	1	2	4	3	3.9
Listeria monocytogenes	0	0	3	3	3	1	3.2
Norovirus	0	1	3	4	1	1	2.8
Salmonella	0	0	3	1	5	1	3.4
STEC O157	0	0	2	4	1	3	3.5
STEC non-O157	1	1	3	2	2	1	2.6
Toxoplasma gondii	4	0	5	0	1	0	1.4
Vibrio parahaemolyticus	3	0	4	2	1	0	1.8
Yersinia enterocolitica	1	0	3	4	2	0	2.6

The expertise categories, 1–5, were a five-point Likert scale, with 1 = low expertise and 5 = high expertise.

For the purpose of calculating a mean expertise score, “NR” responses were assigned a value of zero.

NR, no response. The participant felt they did not have sufficient expertise to provide opinions on questions related to this pathogen.

The panel had its maximum strength, as measured by mean expertise score, relating to pathogens that are of current or recent interest in New Zealand (Campylobacter, STEC O157, Salmonella, and L. monocytogenes).

A U.S. study used the same approach for participant self-assessment (Hoffmann et al., 2006). The highest mean expertise scores were obtained for STEC O157 (3.89), Salmonella (3.73), and L. monocytogenes (3.65), while the lowest mean expertise score was for Toxoplasma gondii (1.98).

Havelaar et al. (2008) employed a simpler system, in which participants could choose which pathogen they would provide opinions for. The pathogens with the highest number of responding participants were Salmonella, Campylobacter, L. monocytogenes, and STEC O157.

Performance-based weights were derived based on estimates for a series of seven seed variables (Supplementary Material S3). The optimized weighting scheme, resulted in three participants being calibrated, with normalized weights of 0.65, 0.21, and 0.14. The weights for the remaining seven experts fell below the cutoff weight and were excluded from the final performance-based weighting scheme.

Ranking of relative contributions of transmission routes

This exercise was included in the elicitation to support estimation of the foodborne proportion, by stimulating certain cognitive processes. Experts would need to consider questions such as “If not food, what is the transmission route?” and “If a proportion of the transmission is due to other routes, how much is reasonably due to food?.”

The mean rank positions for each of five potential transmission routes, using each of the three weighting schemes, are shown in Table 2.

Table 2.

Mean Rank Position for Each of Five Transmission Routes for Selected Pathogens

Pathogen	Mean rank position (uniform/self-assessed expertise/performance-based weighting) ^a
Pathogen	Food	Water	Environment	Animal contact	Person-to-person
Campylobacter	1.0/1.0/1.0	3.3/3.3/2.6	3.3/3.3/2.7	2.5/2.4/3.3	4.4/4.5/4.7
Listeria monocytogenes	1.0/1.0/1.0	3.9/3.9/4.2	2.4/2.4/3.3	2.8/3.0/2.2	3.2/3.2/2.9
Norovirus	2.3/2.3/2.3	3.3/3.4/3.3	3.0/2.9/3.1	4.9/4.9/5.0	1.1/1.2/1.2
Salmonella	1.1/1.1/1.0	4.0/4.0/4.0	3.7/3.6/4.6	2.2/2.1/2.8	3.6/3.7/2.6
STEC O157	2.0/2.3/1.2	3.5/3.5/3.7	2.5/2.4/2.3	1.9/1.8/2.1	4.2/4.3/5.0
STEC non-O157	2.1/2.5/1.6	3.6/3.3/4.4	3.0/3.1/3.0	1.8/1.6/2.2	3.7/3.8/3.8
Toxoplasma gondii	2.0/1.9/1.0	4.0/4.1/5.0	2.0/2.1/3.0	1.8/1.9/2.0	4.7/4.6/4.0
Vibrio parahaemolyticus	1.0/1.0/1.0	2.7/2.7/2.2	2.7/2.7/2.0	3.4/3.4/2.5	2.6/2.7/2.7
Yersinia enterocolitica	1.0/1.0/1.0	2.4/2.3/2.0	4.1/4.2/4.6	2.6/2.6/3.0	4.3/4.3/4.4

The individual estimates of the rank position (1 = transmission route contributing most cases, to 5 = transmission route contributing least cases, ties allowed) were combined as a weighted mean using either uniform weighting, weighting based on self-assessed expertise, or performance-based weights. Mean rank positions close to 1.0 indicate a high level of consensus that the transmission route is the main contributor to total cases, while a mean rank position close to 5.0 indicates consensus that the transmission route is the lowest contributor, of the transmission routes.

Foodborne transmission was considered to be the primary source of infection for five of the nine pathogens; Campylobacter, L. monocytogenes, Salmonella, V. parahaemolyticus, and Y. enterocolitica. Person-to-person transmission was considered to be the primary route of transmission for norovirus. Both animal contact and food were considered to be important for STEC transmission, while these routes plus environment were considered to be of approximately equal importance for transmission of T. gondii.

A Dutch study considered five transmission routes (food, environment, human, animal, and travel) and 17 pathogens by expert elicitation and probabilistic inversion (Havelaar et al., 2008). Of the pathogens included in the New Zealand study, the Dutch study assessed that food was the primary route of transmission for Campylobacter, L. monocytogenes, Salmonella, STEC (O157 and non-O157), and T. gondii. Human (person-to-person) was considered to be the primary transmission route for norovirus. V. parahaemolyticus and Y. enterocolitica were not considered in the Dutch study. Of 12 experts included in the Dutch study, as few as two experts offered opinions for some pathogens.

A Canadian study also used expert elicitation to apportion cases of 28 pathogens to five potential transmission routes; foodborne, waterborne, animal contact, person-to-person, and other (Butler et al., 2015). The Canadian study considered food to be the primary transmission route for Campylobacter, L. monocytogenes, Salmonella, T. gondii, STEC (O157 and non-O157), V. parahaemolyticus, and Y. enterocolitica. Person-to-person contact was considered most important for norovirus.

Quantitative estimates of the proportion of illness due to each of five transmission routes (food, environmental, water, human-to-human, and zoonotic) were determined from an Australian expert elicitation (Vally et al., 2014). Primary routes of transmission for nine pathogens were largely consistent with the other studies reported here.

Proportion of foodborne transmission

New Zealand estimates of the proportion of certain enteric illnesses due to foodborne transmission and the proportion of foodborne transmission due to specific foods, from the current study and from an earlier New Zealand expert elicitation are shown in Table 3.

Table 3.

Comparison of Aggregate Opinions from the Current Study with Those from the 2005 New Zealand Study

Pathogen	Quantity estimated	Mean aggregate estimate (%), based on weighting scheme (95th percentile credible interval)			Mean aggregate estimate from 2005 study ^a (95th percentile credible interval)
Pathogen	Quantity estimated	Uniform	Self-assessed	Performance-based
Campylobacter	% Foodborne	62.6 (43.4–82.5)	63.8 (44.1–83.2)	61.0 (49.3–68.8)	56.2 (26–82)
Listeria monocytogenes	% Foodborne	86.3 (52.8–98.5)	87.8 (57.9–98.5)	92.5 (83.2–98.9)	85.0 (48–100)
Norovirus	% Foodborne	33.8 (9.1–65.7)	32.7 (10.0–66.4)	20.9 (8.4–32.0)	39.2 (8–64)
Salmonella	% Foodborne	61.2 (34.2–86.0)	62.1 (35.2–86.4)	69.2 (36.5–83.9)	59.6 (18–83)
STEC O157	% Foodborne	31.0 (3.8–60.1)	29.9 (3.5–60.7)	40.1 (12.7–57.8)	39.5^c (6–95)
STEC non-O157	% Foodborne	36.4 (3.8–64.0)	34.0 (3.5–63.5)	41.4 (13.1–63.8)	39.5^c (6–95)
Toxoplasma gondii	% Foodborne	25.3 (3.6–55.6)	27.6 (3.8–57.1)	41.7 (25.2–60.2)	31.5 (3–82)
Vibrio parahaemolyticus	% Foodborne	89.0 (55.7–99.8)	90.6 (56.9–99.9)	97.3 (84.2–100)	89.2 (64–100)
Yersinia enterocolitica	% Foodborne	62.1 (27.2–91.4)	63.2 (29.0–91.5)	75.8 (58.1–93.2)	56.2 (32–92)
Campylobacter	% Poultry	74.1 (49.0–91.2)	75.4 (51.6–91.2)	62.9 (45.8–84.7)	52.9 (14–75)
Campylobacter	% Red meat	13.8 (3.8–31.4)	12.7 (3.9–30.6)	19.1 (6.0–33.4)	NC
L. monocytogenes	% RTE Meat	54.1 (27.7–86.2)	55.2 (29.9–87.7)	49.6 (32.2–70.7)	53.9 (16–80)
Norovirus	% Seafood	24.6 (3.7–54.8)	24.4 (3.9–54.7)	17.1 (3.6–26.5)	40.0^b (11–78)
Salmonella	% Poultry	18.7 (3.1–55.8)	19.2 (3.0–56.5)	19.4 (6.4–34.6)	35.7 (16–73)
Salmonella	% Red meat	18.8 (1.8–46.0)	19.2 (2.1–46.1)	17.3 (7.1–36.9)	NC
STEC O157	% Red meat	33.2 (5.6–63.9)	33.5 (4.4–64.6)	33.3 (14.8–46.9)	30.6^c (3–60)
STEC non-O157	% Red meat	27.9 (1.4–64.5)	27.1 (1.2–65.9)	25.5 (11.9–48.6)	30.6^c (3–60)
V. parahaemolyticus	% Seafood	92.9 (68.3–100)	93.8 (70.3–100)	98.4 (88.3–100)	89.2 (57–100)
Y. enterocolitica	% Pork	71.7 (36.6–93.2)	71.1 (36.8–93.1)	56.7 (34.0–75.2)	52.9 (30–74)

Note that the percentage of transmission attributed to specific foods is a proportion of the foodborne attribution.

Cressey and Lake (2005), also summarized in Lake et al. (2010).

Estimate relates to shellfish only.

The 2005 study did not consider STEC O157 and non-O157 genotypes separately.

NC, not considered; RTE, ready-to-eat; STEC, Shiga toxin–producing Escherichia coli.

Only minor differences in estimates are apparent between the application of uniform and self-assessed weighting schemes. Mean estimates using performance-based weights differ markedly from estimates using the other two weighting schemes for some pathogens (norovirus and T. gondii), while for others the three estimates are well aligned (Campylobacter, L. monocytogenes, and V. parahaemolyticus).

There is still much debate concerning the use of the “classical method” for deriving performance-based weighting schemes (Cooke and Goossens, 2000). The method uses a series of questions to “calibrate” the performance of the experts involved. In practice this often results in only a proportion of the experts being weighted and contributing to the aggregate judgments. This was the case for the current study, with only 3 of 10 experts being weighted. It has been argued that it is difficult to construct reliable and valid measures of expertise on which to base performance-based weights (Bolger and Rowe, 2015). The authors of the current study concur with this opinion. While performance-based weights are scientifically desirable, the identification of seed variables that meet the necessary conditions (within the domain of the consultation, not known to the experts at the time of the elicitation, but known to the study coordinators within the study lifetime) for all pathogens is virtually impossible in practice.

For pathogens for which food was considered to be the primary transmission route (>50%) in 2005, estimates of the proportion of illness that is due to foodborne transmission have generally increased in the current study. For example, the 2005 study estimated that 56% of Campylobacter was foodborne, while 61–64% of Campylobacter transmission was estimated to occur through the food supply in 2012. Conversely, where the 2005 estimates were of <50% transmission by food, the proportion estimated in the current exercise are lower than those made in 2005. It is unknown to what extent the opinions of the panel may have been influenced by the results of similar expert elicitations carried out in other countries.

Regulatory and industry measures implemented in the New Zealand poultry industry in 2006–2007 resulted in an approximate 50% reduction in notified cases of campylobacteriosis, from a rate 302.2 per 100,000 in 2007 to 156.8 per 100,000 in 2008 (Sears et al., 2011). Molecular epidemiological evidence suggests that poultry is still the primary source of human campylobacteriosis in New Zealand (French and Marshall, 2014).

Table 4 summarizes estimates of the proportion foodborne from the current study and other studies carried out worldwide.

Table 4.

New Zealand and Overseas Estimates of the Food Attributable Proportion of Selected Illnesses Due to Microbial Hazards

Hazard	Proportion foodborne (%)
Hazard	New Zealand (Current study) ^a	New Zealand (2005)	WHO (2016) ^b	USA (2011)	Canada (2015)	Australia (2014)	England and Wales (2002)	Netherlands ^c (2008)
Campylobacter spp.	64	56	51–76	80	62	77	80	42
Listeria monocytogenes	88	85	100	99	77	98	99	69
Norovirus	33	39	12–26	26	18	18	11^d	17
Salmonella, nontyphoidal	62	60	46–76	94	63	72	92	55
STEC O157:H7	30	40^e	40–60^e	68	61	56^e	63	40
STEC non-O157	34	40^e	40–60^e	82	60	56^e	63	42
Toxoplasma gondii	28	32	42–61	50	51	NE	NE	56
Vibrio parahaemolyticus	91	89	NE	86	85	NE	NE	NE
Yersinia enterocolitica ^f	63	56	NE	90	83	NE	90	NE

Sources: New Zealand: current study, Cressey and Lake (2005); WHO: Hald et al. (2016); USA: Scallan et al. (2011); Canada: Butler et al. (2015); Australia: Vally et al. (2014); England and Wales: Adak et al. (2002); Netherlands: Havelaar et al. (2008).

Estimates are those derived using self-assessed expertise weights.

The WHO study estimated proportions for 14 international regions. Figures presented here are the range of those estimates.

The Dutch study also collected opinions on the proportion of disease due to travel. A proportion of this will also be foodborne.

In this older document, norovirus is referred to as Norwalk-like virus.

Estimates were derived for total STEC.

For England and Wales the estimate refers to Yersinia spp., for all other countries the estimate refers to Yersinia enterocolitica.

NE, not estimated; STEC, Shiga toxin–producing Escherichia coli.

New Zealand estimates for Campylobacter spp., L. monocytogenes, and nontyphoidal Salmonella are within the range of estimates derived for other countries or regions. However, the New Zealand estimates for STEC (O157 or non-O157), T. gondii, and Y. enterocolitica are all lower than any of the estimates for other countries, while the estimates for V. parahaemolyticus and norovirus are higher than the other estimates. It is not currently possible to say whether these differences reflect true differences in the etiology of these diseases in New Zealand or simply a difference in opinions.

For STEC O157, it is likely that this New Zealand-specific view was informed, at least in part, by a prospective case–control study (Jaros et al., 2013). Experts involved in the elicitation were also involved with the case–control study and the findings of the case–control study were discussed between the first and second pass questionnaires. The case–control study identified mainly animal contact and water quality factors (recreational water contact, drinking water from nonreticulated source) as risk factors for STEC infection, while food-related factors were generally protective (adjusted odds ratios <1). The apparent impact of a single study on the aggregate expert opinion for this pathogen represents a potential limitation.

For norovirus, the study suggests a higher proportion foodborne (33%) than reported for the other attribution studies (12–26%). A recent collaborative study that derived estimates of the proportion foodborne for norovirus outbreaks, based on genotypic analysis, estimated that 13% of norovirus outbreaks in New Zealand were caused by foodborne transmission (Verhoef et al., 2015). It is possible that the higher foodborne proportion estimated through the expert elicitation was influenced by a number of well-publicized outbreaks linked to consumption of oysters that have occurred in New Zealand (Simmons et al., 2001, 2007).

The non-New Zealand estimates of the foodborne proportion for T. gondii are quite consistent (42–61%), while the New Zealand estimate was substantially lower (28%). Given the low level of self-assessed expertise among the New Zealand expert cohort (mean = 1.4), it is unlikely that these differences are based on objective information.

Conclusions

Expert elicitation provides a useful mechanism for estimating parameters that are not directly measurable. The estimates of the proportion of cases of nine microbial diseases caused by foodborne transmission presented here are more “polarized” than similar estimates made in 2005. That is, for pathogens where food was considered to be the major transmission route, estimates were higher, and for pathogens for which other routes were considered to be more important, estimates were lower.

While three separate weighting schemes were trialed to represent the expertise of the participant experts, it is not possible to say which of these schemes results in more accurate estimates. Performance-based weighting resulted in final combined estimates being heavily weighted to the opinion of one expert. This seems unlikely to be valid across a range of pathogens. The experts' self-assessed levels of expertise were well aligned with the study coordinators' views and it was felt that the experts' self-assessments represented an honest appraisal. Therefore, the estimates in this study, weighted by self-assessed expertise, are expected to be at least as accurate as the performance-weighted estimates.

New Zealand estimates of foodborne proportions are broadly similar to estimates from other countries, with the possible exception of STEC, T. gondii, and Y. enterocolitica. It is uncertain whether these differences represent true differences in disease etiology or just differences in the opinions of the expert groups.

Footnotes

Acknowledgments

The authors acknowledge the New Zealand Ministry for Primary Industries for funding the expert elicitation study. We also wish to thank the participating experts, whose valuable time and contributions made this study possible.

Disclosure Statement

No competing financial interests exist.

Supplementary Material

Supplementary Material S1

Supplementary Material S2

Supplementary Material S3

References

Adak

, Long

, O'Brien

. Trends in indigenous foodborne disease and deaths, England and Wales: 1992 to 2000. Gut, 2002; 51:832–841.

Bolger

, Rowe

. The aggregation of expert judgment: Do good things come to those who weight?. Risk Anal, 2015; 35:5–11.

Butler

, Thomas

, Pintar

KDM

. Expert elicitation as a means to attribute 28 enteric pathogens to foodborne, waterborne, animal contact, and person-to-person transmission routes in Canada. Foodborne Pathog Dis, 2015; 12:335–344.

Cooke

, Goossens

LHJ

. Procedures guide for structured expert judgement in accident consequence modelling. Radiat Prot Dosim, 2000; 90:303–309.

Cooke

, Goossens

LLHJ

. TU Delft expert judgment data base. Reliab Eng Syst Safe, 2008; 93:657–674.

Cressey

, Lake

. Ranking Food Safety Risks. Development of NZFSA Policy 2004–2005. Christchurch: Institute of Environmental Science and Research (ESR), 2005.

Cross

, Rigby

, Edwards-Jones

. Eliciting expert opinion on the effectiveness and practicality of interventions in the farm and rural environment to reduce human exposure to Escherichia coli O157. Epidemiol Infect, 2012; 140:643–654.

Davidson

, Ravel

, Nguyen

, Fazil

, Ruzante

. Food-specific attribution of selected gastrointestinal illnesses: Estimates from a Canadian expert elicitation survey. Foodborne Pathog Dis, 2011; 8:983–995.

French

, Marshall

. Source Attribution Studies for Campylobacteriosis in New Zealand. Wellington: Ministry for Primary Industries, 2014.

10.

Gallagher

, Ryan

, Kelly

, Leforban

. Wooldridge M. Estimating the risk of importation of foot-and-mouth disease into Europe. Vet Rec, 2002; 150:769–772.

11.

Garabed

, Perez

, Johnson

, Thurmond

. Use of expert opinion for animal disease decisions: An example of foot-and-mouth disease status designation. Prev Vet Med, 2009; 92:20–30.

12.

Hald

, Aspinall

, Devleesschauwer

, Cooke

, Corrigan

, Havelaar

, Gibb

, Torgerson

, Kirk

, Angulo

, Lake

, Speybroeck

, Hoffmann

. World Health Organization estimates of the relative contributions of food to the burden of disease due to selected foodborne hazards: A structured expert elicitation. PLoS One, 2016; 11:e0145839.

13.

Havelaar

, Galindo

, Kurowicka

, Cooke

. Attribution of foodborne pathogens using structured expert elicitation. Foodborne Pathog Dis, 2008; 5:649–659.

14.

Havelaar

, Kirk

, Torgerson

, Gibb

, Hald

, Lake

, Praet

, Bellinger

, de Silva

, Gargouri

, Speybroeck

, Cawthorne

, Mathers

, Stein

, Angulo

, Devleesschauwer

. World Health Organization global estimates and regional comparisons of the burden of foodborne disease in 2010. PLoS Med, 2015; 12:e1001923.

15.

Helmer

. Analysis of the Future. The Delphi Method. Santa Monica, CA: The RAND Corporation, 1967.

16.

Hoffmann

, Devleesschauwer

, Aspinall

, Cooke

, Corrigan

, Havelaar

, Angulo

, Gibb

, Kirk

, Lake

, Speybroeck

, Torgerson

, Hald

. Attribution of global foodborne disease to specific foods: Findings from a World Health Organization structured expert elicitation. PLoS One, 2017; 12:e0183641.

17.

Hoffmann

, Fischbeck

, Krupnick

, McWilliams

. Eliciting Information on Uncertainty from Heterogeneous Expert Panels. Washington, DC: Resources for the Future, 2006.

18.

Hoffmann

, Fischbeck

, Krupnick

, McWilliams

. Using expert elicitation to link foodborne illnesses in the United States to foods. J Food Protect, 2007; 70:1220–1229.

19.

Hoffmann

, Fischbeck

, Krupnick

, McWilliams

. Informing risk-mitigation priorities using uncertainty measures derived from heterogeneous expert panels: A demonstration using foodborne pathogens. Reliab Eng Syst Safe, 2008; 93:687–698.

20.

Jaros

, Cookson

, Campbell

, Besser

, Shringi

, Mackereth

, Lim

, Lopez

, Dufour

, Marshall

, Baker

, Hathaway

, Prattley

, French

. A prospective case-control and molecular epidemiological study of human cases of Shiga toxin-producing Escherichia coli in New Zealand. BMC Infect Dis, 2013; 13:450.

21.

Knol

, Slottje

, Van Der Sluijs

, Lebret

. The use of expert elicitation in environmental health impact assessment: A seven step procedure. Environ Health, 2010; 9:19.

22.

Lake

, Cressey

, Campbell

, Oakley

. Risk ranking for foodborne microbial hazards in New Zealand: Burden of disease estimates. Risk Anal, 2010; 30:743–752.

23.

Lim

, Lopez

, Borman

, Cressey

, Pirie

. Annual Report Concerning Foodborne Disease in New Zealand 2011. Wellington: Ministry for Primary Industries, 2012.

24.

Ministry of Health. Notifiable diseases. Available at: www.healthgovt.nz, accessed March 18, 2019 .

25.

Pattis

, Lopez

, Cressey

, Horn

, Roos

. Foodborne Disease in New Zealand 2016. Wellington: Ministry for Primary Industries, 2017.

26.

Ravel

, Davidson

, Ruzante

, Fazil

. Foodborne proportion of gastrointestinal illness: Estimates from a Canadian expert elicitation survey. Foodborne Pathog Dis, 2010; 7:1463–1472.

27.

Scallan

, Hoekstra

, Angulo

, Tauxe

, Widdowson

, Roy

, Jones

, Griffin

. Foodborne illness acquired in the United States-major pathogens. Emerg Infect Dis, 2011; 17:7–15.

28.

Sears

, Baker

, Wilson

, Marshall

, Muellner

, Campbell

, Lake

, French

. Marked campylobacteriosis decline after interventions aimed at poultry, New Zealand. Emerg Infect Dis, 2011; 17:1007–1015.

29.

Shirazi

. Data-Informed Calibration and Aggregation of Expert Opinion in a Bayesian Framework. College Park: University of Maryland, 2009.

30.

Simmons

, Garbutt

, Hewitt

, Greening

. A New Zealand outbreak of norovirus gastroenteritis linked to the consumption of imported raw Korean oysters. N Zeal Med J, 2007; 120:U2773.

31.

Simmons

, Greening

, Gao

, Campbell

. Raw oyster consumption and outbreaks of viral gastroenteritis in New Zealand: Evidence for risk to the public's health. Aust N Z J Public Health, 2001; 25:234–240.

32.

Speirs-Bridge

, Fidler

, McBride

, Flander

, Cumming

, Burgman

. Reducing overconfidence in the interval judgments of experts. Risk Anal, 2010; 30:512–523.

33.

United States Environmental Protection Agency. Expert Elicitation Task Force White Paper. Washington, DC: USEPA, 2011.

34.

Vally

, Glass

, Ford

, Hall

, Kirk

, Shadbolt

, Veitch

, Fullerton

, Musto

, Becker

. Proportion of illness acquired by foodborne transmission for nine enteric pathogens in Australia: An expert elicitation. Foodborne Pathog Dis, 2014; 11:727–733.

35.

Verhoef

, Hewitt

, Barclay

, Ahmed

, Lake

, Hall

, Lopman

, Kroneman

, Vennema

, Vinje

, Koopmans

. Norovirus genotype profiles associated with foodborne transmission, 1999–2012. Emerg Infect Dis, 2015; 21:592–599.

36.

Vose

. Risk Analysis. A Quantitative Guide, 3rd edition. Chichester: John Wiley & Sons, Ltd., 2008.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB

0.04 MB

0.02 MB