Abstract
Background:
Estimates of the burden of illness acquired from food inform public health policy and prioritize interventions. A key component of such estimates is the proportion of illnesses that are acquired by foodborne transmission. In view of the shortage of requisite data, these proportions are commonly obtained through a process known as expert elicitation. We report findings from an elicitation process used to assess the importance of the foodborne transmission route for nine pathogens in Australia, circa 2010.
Materials and Methods:
Eleven experts were asked to estimate the proportion of illness acquired by five transmission routes: food, environmental, water, person, and zoonotic, together with a 90% certainty interval for foodborne transmission. Foodborne estimates and intervals from each expert were combined using both modified triangular and Program Evaluation and Review Technique (PERT) distributions, in @Risk version 6, to generate final distributions from which median estimates and 95% Credible Intervals (CrI) were calculated.
Results:
Shiga toxin
Conclusions:
Foodborne proportions for most pathogens in this study were the same or lower than those estimated circa 2000 in Australia, with the greatest decline for non-STEC pathogenic E. coli. Inclusion of certainty intervals from experts helps to quantify the precision of foodborne proportions. A decline in estimates of the foodborne proportion for common pathogens will influence final estimates of the burden of illness acquired from food.
Introduction
E
This article contributes to the second component of the Mead et al. (1999) approach: that of estimating the proportion of illnesses that are acquired by foodborne transmission. A common strategy for this comparison is to ask individual experts to estimate the proportion of illnesses that are due to foodborne transmission for each of the pathogens under consideration. These individual estimates can then be combined statistically to generate an overall estimate reflecting both the available data and the knowledge of the collective body of experts, a process known as expert elicitation (Burgmann et al., 2006; Havelaar et al., 2008; Pires et al., 2009).
Expert elicitation has been used to estimate the proportion of illness due to food and other transmission routes in Australia (Hall et al., 2005; Kirk et al., in press), the Netherlands (Havelaar et al., 2008), New Zealand (Lake et al., 2010), and the United States (Mead et al., 1999; Hoffmann et al., 2007). This article reports the results of an expert elicitation used to estimate the proportion of illnesses that are domestically acquired and due to foodborne transmission for nine pathogens that were considered important for the overall estimate of the burden of foodborne disease in Australia: Campylobacter spp., Clostridium perfringens, hepatitis A virus, Listeria monocytogenes, norovirus, nontyphoidal Salmonella spp., Shigella spp., Shiga toxin–producing Escherichia coli (STEC), and non-STEC pathogenic E. coli.
Materials and Methods
Selection of experts
Twelve individuals from a range of disciplines with expertise in foodborne disease in Australia were invited to participate in this elicitation in order to encompass differing perspectives on each pathogen. All invitees accepted. The panel included two public health physicians, two microbiologists, one food safety officer, two public health veterinarians, three foodborne disease epidemiologists, and one research scientist. One expert (a public health physician) had to withdraw after the first (of three) rounds of data collection. Ethics approval for the study was given by the Australian National University Human Research Ethics Committee.
Data collection
Data were collected in three rounds. Initial estimates were elicited by e-mail from experts based solely on their prior knowledge (Round 1). Subsequent estimates were obtained (again, by e-mail) 2 weeks later after providing experts with additional information as described below (Round 2). Final estimates were elicited following discussion and reflection at a day-long workshop 1 week after the second round of elicited estimates (Round 3). In this final round, each pathogen was discussed by the whole group, average distributions (calculated as described below) for each pathogen were shown to the group for discussion, and final distributions were e-mailed to the experts 3 weeks after the workshop for comment.
The objective of providing additional information to experts prior to round 2 was to provide up-to-date information from the literature and Australian data sources for the experts to incorporate into their thinking and, after reflection, to take into consideration in their second round of estimates. This additional information consisted of systematic reviews compiled by the project team for each pathogen on the proportion of illness due to foodborne, zoonotic (direct animal-to-person contact), environmental, and waterborne transmission. Due to the lack of definitive scientific evidence on the topic, these reviews included scientific published literature and government and institutional reports published in English between 1980 and 2009.
In addition to these systematic reviews, data on the incidence of notifications from the National Notifiable Diseases Surveillance System and outbreaks from the National Outbreak Register in Australia (Kirk et al., 2008) were collated and provided to experts. For each pathogen, this comprised the number of cases and outbreaks between 2004 and 2008, the proportion of outbreaks by each possible transmission route, and for those considered foodborne, the proportion of different food vehicles and locations responsible for outbreak. An overall opinion or interpretation of the information was deliberately not provided to the experts, with the aim of having each expert synthesize the data and come to their own conclusion.
In each round, experts were given an identical questionnaire for each of the nine pathogens inviting the experts to estimate the percentage of illness acquired in Australia through five transmission routes, excluding overseas acquired illness. Experts were provided with detailed instructions on how to express estimates and uncertainties, and were asked to answer two “warm-up” questions involving scenarios not relating to foodborne disease prior to the first round. For the purpose of this study, major transmission pathways (foodborne, environmental, waterborne, zoonotic, and person-to-person) were defined at the point of ingestion as summarized in Box 1. Experts provided a number out of 100 for each estimate. To encourage experts to consider the relative likelihood of events, they were instructed to ensure that the estimates summed to 100%. When estimating the percentage of foodborne transmission for each of the pathogens, experts were additionally asked to give 90% certainty bounds to their estimate to capture the perceived precision of the estimated percentage. When referring to experts' opinions, we will describe their point estimate as the “best estimate” and the interval as the “certainty bounds.”
Experts were advised that major transmission pathways are defined at the point of ingestion.
Data analysis
As a necessary step in deriving pooled median estimates and CrI, each foodborne estimate and 90% certainty bound was converted into an uncertainty distribution. Two appropriate distributions for expert elicitation data are the modified triangular and the Program Evaluation and Review Technique (PERT) distributions (Johnson, 1997). Modified triangular distributions represent the best estimate as the mode of the distribution and use the 90% certainty bounds to describe the inner triangle, with the remaining probability mass in the outer tails of the distribution (Fig. 1a). The PERT distribution is a form of β-distribution designed for expert elicitation data (Fig. 1b), with the best estimate as the mode of the distribution and the 5 and 95 percentiles matching the uncertainty intervals. Throughout the expert elicitation process, modified triangular distributions were used and presented to experts for comment. However, given the increased use of PERT distributions in recent foodborne burden-of-disease studies (Lake et al., 2010; Scallan et al., 2011a, b; Thomas et al., 2013), we chose to recalculate all estimates using this distribution to allow a direct comparison between studies. In some instances, the mode and 90% certainty bound could not be modeled using a 3-parameter PERT distribution because values went outside the [0,1] interval, in which case, the certainty bound was taken as maximum and minimum values.

A comparison of
Once uncertainty distributions had been generated for each expert and each pathogen, combined distributions were generated for each pathogen using @Risk version 6 (Palisade Corporation, 2012) by simulating a distribution for each expert, and then computing a pointwise average of the individual uncertainty distributions. Median and 95% CrI could then be calculated for the combined distribution using @Risk, which uses a Monte Carlo simulation approach. Figure 2 provides a graphical picture of this approach using PERT distributions for nontyphoidal Salmonella. Throughout, we use the term “uncertainty interval” for interval estimates provided by experts, and describe combined distributions using the median and the 95% CrI, which is the 2.5% percentile to the 97.5% percentile. In all cases, we report estimates as percentages.

Calculation of the foodborne proportion for nontyphoidal Salmonella using the PERT distribution. The graph shows expert opinions at the top, the combined distribution below this, and the summary measures at the bottom. Circles show the “best estimate” for the expert opinions and the median of the summary measure; solid rectangles indicate the 90% certainty intervals for the expert opinion, and the 90% Credible Interval for the summary measure; and black lines show the 95% Credible Interval of the modeled PERT distribution.
Results
Table 1 shows the percentage of domestically acquired illness due to each of five transmission pathways by pathogen, presenting the mean of the “best estimates” provided by the team of experts in the final round of data collection, together with the minimum and maximum of these estimates. Pathogens that experts believe to be largely foodborne, such as Campylobacter, C. perfringens, nontyphoidal Salmonella, and L. monocytogenes, all have correspondingly low estimates for the other transmission routes. Although the zoonotic transmission route was estimated to be important for STEC, it was not a major transmission route for any other pathogen. The person-to-person transmission route was estimated as an important source of infection for non-STEC pathogenic E. coli, Shigella, norovirus, and hepatitis A virus.
In some instances percentages may not sum to 100 due to rounding.
Table 2 shows the median and 95% CrI for the foodborne proportion derived from the combined uncertainty distribution for each pathogen using both the PERT and the modified triangular distribution. The results from prior expert elicitations conducted in Australia circa 2000 are provided for comparison. Comparing the estimates of the current study using the two distributions, we see small differences in the median estimate, and more noticeable differences in the 95% CrI, which are wider under the modified triangular distribution. The cause of these differences in width is clearly apparent from the example in Figure 1: The outer tails of the modified triangular distribution extend linearly to the extreme values of 0 and 1, while the tails of the PERT distribution become thinner more rapidly.
Current study results based on combined “uncertainty distributions” from experts, who provided “best estimates” and 90% certainty intervals.
Previous study based on “best estimate” only from experts.
Hall et al., 2005.
Abelson P, Forbes MP, Hall G. The Annual Cost of Foodborne Illness in Australia. Canberra: Commonwealth of Australia, 2006.
Estimates of the foodborne proportions circa 2010 are similar to, or lower than, those circa 2000. Notably, for non-STEC pathogenic E. coli, the estimate of the proportion of foodborne transmission decreased from 50% to 23–24%, while estimates for nontyphoidal Salmonella, STEC, and norovirus also decreased. CrI for foodborne transmission of these four pathogens were wider than those that changed relatively little between the two studies.
Discussion
These expert elicitation estimates identify key transmission routes for nine major gastrointestinal pathogens in Australia circa 2010, highlighting Campylobacter, C. perfringens, nontyphoidal Salmonella, and L. monocytogenes as chiefly foodborne, and norovirus, Shigella, and hepatitis A virus as primarily spread from person to person. Estimates of the proportion of transmission that is foodborne for each pathogen changed somewhat from those estimated circa 2000, with the greatest changes being a decrease in estimates for nontyphoidal Salmonella, STEC, other pathogenic E. coli, and norovirus, together with wider CrI for these four pathogens.
The inclusion of certainty bounds from experts in this study has allowed us to better quantify the precision of foodborne estimates, and so incorporate this knowledge into burden-of-disease calculations. In order to produce combined estimates, the “best estimates” and 90% certainty bounds were used to parameterize probability distributions, which were then averaged. During primary data analysis, we used modified triangular distributions for this process, as a relatively simple and transparent representation of the data. Given the use of PERT distributions in a number of recent burden-of-disease studies (Lake et al., 2010; Scallan et al., 2011a, b; Thomas et al., 2013), we later repeated these calculations using PERT distributions, and all final estimates are reported for both distributions. Median estimates and CrI from these two distributions differ, with wider intervals resulting from the use of the modified triangular distribution due to differences in the tails of the distributions. A future study of this kind should include a judgment by the experts on which form of distribution best reflects their personal uncertainty.
Estimates of the proportion of transmission that is foodborne for specific pathogens form an integral part of burden-of-disease studies for foodborne disease. Given the weight that policy makers place on these studies, it is important that they be aware of the implications of changes in estimates of foodborne transmission for common enteric pathogens. In recent revisions to the US estimates of foodborne disease (Scallan et al., 2011b), a decrease in the proportion of norovirus that was estimated to be foodborne from 40% in 1996 to 25% in 2010 had a marked impact on the estimated burden of overall foodborne disease. The decrease in norovirus was also influential in the foodborne proportion for unspecified gastrointestinal agents, which declined from 36% (Mead et al., 1999) to 23% (Scallan et al., 2011a). We anticipate similar effects in Australia's updated estimates, with decreases in the foodborne proportion estimated for nontyphoidal Salmonella, non-STEC pathogenic E. coli, and norovirus: all moderate-to-high-incidence pathogens.
Comparisons with other national studies highlight a trend toward lower estimates of foodborne transmission compared to earlier estimates conducted in Australia, the United Kingdom, and the United States (Mead et al., 1999; Adak et al., 2002; Hall et al., 2005). While this trend is consistent with our findings, estimates from the Netherlands (Havelaar et al., 2008) for Campylobacter (42%), nontyphoidal Salmonella (55%) and L. monocytogenes (69%) are all notably lower than ours. Although these differences are in part due to their inclusion of “travel” as a pathway, our relatively low rates of travel-associated cases for most pathogens (see Supplementary Table S1; Supplementary Data are available online at
It is notable that the pathogens with the greatest change in the foodborne proportion between studies circa 2000 and circa 2010 are also those with the widest CrI in this study. This is most certainly a reflection either of the lack of evidence, or the presence of conflicting evidence, relating to the degree of foodborne transmission for these pathogens. Clearly where the transmission route for a pathogen is dominated by one pathway and there is strong evidence to reflect this, one would expect the CrI to be narrower. Supporting this is a similar expert elicitation study in the Netherlands, which identified that uncertainty was smallest for pathogens with dominant transmission routes (Havelaar et al., 2008). Our results are consistent with these findings: the pathogens with the greatest uncertainty—between and within studies—are those with two or more pathways each responsible for 15% or more of transmission.
Estimates and uncertainty distributions in this study are intended to represent the whole country and the whole bacterial or viral genus, or species and subtype. The epidemiology of enteric pathogens in Australia differs within subgroups of the population (such as indigenous people), and subtypes of some pathogens (such as Salmonella serotypes) are known to occupy specific geographic and epidemiological niches. These geographic and epidemiological differences likely impact on transmission pathways. In their reflections prior to giving their overall estimates, experts were asked to conceptually “weight” their estimates according to the population distribution and predominant pathogen types in Australia. A refinement in methods might be to ask experts to estimate proportions for subtypes and population groups separately, then to more formally weight the expert estimates to the overall population.
In conclusion, ideally estimates of transmission routes would be made using “hard scientific data,” and expert elicitations would not be necessary to estimate the total burden of foodborne disease. However, since source attribution and epidemiological studies (Pires et al., 2009; Pires et al., 2010) are not available for all pathogens, these expert elicitations form an important component of studies of the burden of foodborne illness. Given the reliance on data obtained in this way, we have endeavored to obtain the best possible estimates by eliciting uncertainty intervals from all experts that are included in our combined estimate. We also did a trial of a three-stage process for gathering estimates, together with estimates of confidence from each expert, which we feel adds further credence to our findings. Consequently, we believe that our estimates of the proportion of illness acquired by foodborne transmission for each pathogen in Australia reflects both the best available knowledge and the current levels of uncertainty in these estimates.
Footnotes
Acknowledgments
This expert elicitation was part of a larger project called Estimating Foodborne Illness in Australia circa 2010 funded by the Department of Health, Food Standards Australia New Zealand, and NSW Food Authority. The steering committee for this project was instrumental in organizing the expert elicitation. Members of the steering committee were Katie Fullerton (Department of Health), Gillian Hall (National Centre for Epidemiology and Population Health, Australian National University), Martyn Kirk (National Centre for Epidemiology and Population Health, Australian National University), Jennie Musto (Health Protection, NSW), Craig Shadbolt (NSW Food Authority), Hassan Vally (School of Public Health and Human Biosciences, La Trobe University), and Mark Veitch (Department of Health and Human Services; formerly Microbiological Diagnostic Unit, The University of Melbourne).
Much effort went into the excellent literature reviews that were conducted by Ainslie Butler, Barry Coombs, Alison Dann, Rennie D'Souza, Martyn Kirk, Chawalit Kocharunchitt, Zheng Liu, Olivia McQuestin, Cathy Moir, Jennie Musto, Bruce Nelan, Rolf Nilsson, April Roberts, and Polly Wallace.
Experts, who participated in the elicitation and gave generously of their time and expertise, were John Bates, Duncan Craig, Patricia Desmarchelier, Katie Fullerton, Joy Gregory, David Jordan, Martyn Kirk, Tony Merritt, Andrew Pointon, Jane Raupach, Lisa Szabo, and Mark Veitch.
Mark Burgman gave insightful comments and feedback on the design of the study.
Katie Fullerton is currently with the National Center for Emerging and Zoonotic Infectious Diseases at the Centers for Disease Control and Prevention, Atlanta, GA.
The expert elicitation was funded by the Department of Health and the New South Wales Food Authority.
Disclosure Statement
No competing financial interests exist.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
