The Effect of Hazard Analysis Critical Control Point Programs on Microbial Contamination of Carcasses in Abattoirs: A Systematic Review of Published Data

Abstract

Hazard analysis critical control point (HACCP) programs have been endorsed and implemented globally to enhance food safety. Our objective was to identify, assess, and summarize or synthesize the published research investigating the effect of HACCP programs on microbial prevalence and concentration on food animal carcasses in abattoirs through primary processing. The results of microbial testing pre- and post-HACCP implementation were reported in only 19 studies, mostly investigating beef (n=13 studies) and pork (n=8 studies) carcasses. In 12 of 13 studies measuring aerobic bacterial counts, reductions were reported on beef (7/8 studies), pork (3/3), poultry (1/1), and sheep (1/1). Significant (p<0.05) reductions in prevalence of Salmonella spp. were reported in studies on pork (2/3 studies) and poultry carcasses (3/3); no significant reductions were reported on beef carcasses (0/8 studies). These trends were confirmed through meta-analysis of these data; however, powerful meta-analysis was precluded because of an overall scarcity of individual studies and significant heterogeneity across studies. Australia reported extensive national data spanning the period from 4 years prior to HACCP implementation to 4 years post-HACCP, indicating reduction in microbial prevalence and concentration on beef carcasses in abattoirs slaughtering beef for export; however, the effect of abattoir changes initiated independent of HACCP could not be excluded. More primary research and access to relevant proprietary data are needed to properly evaluate HACCP program effectiveness using modeling techniques capable of differentiating the effects of HACCP from other concurrent factors.

Introduction

T he hazard analysis critical control point (HACCP) concept was first applied to the prevention of foodborne illness during manned space flight by identifying potential hazards and applying preventive measures for their control (Delazari et al., 2006). Since then, the use of HACCP programs in food processing, including abattoirs, has been endorsed by the National Academy of Sciences in the United States (1985), the World Health Organization (1993), and the Codex Alimentarius Commission (1994) as a transparent, science-based methodology to improve food safety (Fearne et al., 2004). In 1996, the Food Safety and Inspection Service of the U.S. Department of Agriculture introduced the Final Rule: pathogen reduction and HACCP systems, requiring the implementation of HACCP programs in meat and poultry abattoirs in the United States and their verification through microbial sampling and testing (Phillips et al., 2001). Thereafter, HACCP programs were similarly mandated in meat and poultry abattoirs in Australia (1997), the European Union (2001), and Canada (2004), major trading partners of the United States (Phillips et al., 2001; Fearne et al., 2004).

Challenges to implementing HACCP programs in abattoirs include cost, manpower, and training requirements (Fearne et al., 2004; Jevsnik et al., 2006; Lupin et al., 2010). The choice of critical control points (CCPs) is crucial to the effectiveness of the HACCP program, and baseline microbial sampling of carcasses at multiple points in the slaughter process is required to identify optimal CCPs (Lammerding and Todd, 2006). A comparison of national HACCP programs' effectiveness is difficult, given the range of CCPs selected in different settings and the lack of specified microbial performance standards or baseline industry studies conducted prior to HACCP adoption and implementation in many jurisdictions (Fearne et al., 2004).

The widespread implementation of HACCP programs over the past decade has sparked numerous expert and stakeholder debates on the benefits of HACCP in terms of reducing microbial contamination at the processing level. Systematic review methodology follows a transparent, structured, replicable approach to review research on intervention effectiveness and is well suited for providing informed, evidence-based inputs to policy makers (Waddell et al., 2009). If sufficient data are reported in the existing literature, meta-analysis (MA) may be applied to data from multiple similar studies, resulting in more precise quantitative overall estimates of effect. Our objective, therefore, was to identify, evaluate, and summarize any published primary research investigating the effect of HACCP or similar structured programs (please refer to definitions below) on microbial contamination, including but not restricted to bacteria and viruses, on carcasses of food animals, fish, shellfish, or molluscs in abattoirs or processing plants through primary processing, using systematic review methodology and, where appropriate, MA.

Materials and Methods

Definitions and inclusion criteria

A list of definitions used in this review is included in Table 1.

Table 1.

Definitions Used in This Review

Descriptor	Definition	Reference/source
Abattoir	Premises used for the slaughter of animals for human consumption or animal feeding.	Terrestrial Animal Health Code (2006)
CCP	A preidentified point in a process where failure is likely to occur.	Delazari et al. (2006)
Food animal carcass	Any animal/fish/mollusc/crustacean consumed by humans as food, including beef, pork, or poultry carcasses.	This review
HACCP program	A program specifically designated a HACCP program by a federal regulatory agency that routinely measures identified CCPs in a process, in which predetermined corrective measures are taken, should measurement of CCPs exceed a predetermined level.	Delazari et al. (2006); This review
Indicator organism	Microbe that usually does not cause disease in a healthy animal (e.g., generic/nonpathogenic Escherichia coli); however, its presence may be used as a convenient proxy indicator for a potentially pathogenic microbe (e.g., Salmonella spp.).	Gill (1999)
Potential pathogen	A microbe often capable of producing clinical disease in a healthy animal in some circumstances (e.g., Salmonella spp.).	Quinn et al. (2002)
Primary processing	Slaughter, evisceration, and chilling of an animal carcass, until subdivision into subprimal cuts, that is, separation of legs, from back, and from thinner body-wall areas of the carcass.	Romans et al. (1994)
Primary research	Experiments, investigations, or tests carried out to acquire data observed or collected directly from first-hand experience.	This review
Structured control program	A program in which critical points in a process are identified and controls are put in place to ensure that food safety hazards are eliminated. Descriptors include “food safety control systems,” “validated system,” “program for enhanced control,” or “hygiene assessment scheme.”	This review

CCP, critical control point; HACCP, hazard analysis critical control point.

Inclusion criteria for this review

• Population: Carcasses of food animals, fish, shellfish, or molluscs in abattoirs or processing plants of any size or location through primary processing.

• Intervention: HACCP or similar structured programs.

• Comparison: Same or similar population unexposed to HACCP.

• Outcomes: Microbial contamination, including but not restricted to bacteria and viruses, as measured by prevalence or concentration on carcasses.

• Study design: Any published primary research using a design employing the use of a comparison group was deemed relevant.

Studies that reported one of either the “before” or “after” measurements and compared with measurements reported elsewhere were also included.

MA inclusion criteria

Datasets consisting of two or more independent studies investigating the effect of HACCP implementation were stratified by microbial outcome and carcass species. Where more than one study sampled similar populations, in similar or overlapping time periods, both studies were included in data extraction; however, only the study reporting the most complete set of data was utilized for MA. For example, four studies reported potentially relevant data before and after HACCP implementation in Australia and all are included in this review. However, for each microbe/carcass species combination, only the study reporting the most complete data required (e.g., number of animals sampled, bacterial counts, and standard deviation) were included in MA.

Review team, search strategy, and screening for relevance

The research team included four epidemiologists and a research librarian, with extensive experience in systematic review methodology and/or food safety. A review protocol was drafted and is presented as Supplementary Material (Supplementary Material is available online at www.liebertonline.com/fpd). An electronic search consisting of terms pertaining to intervention (e.g., HACCP) and population settings (e.g., abattoir) was implemented in June 2008, without restriction on date of publication, in the following bibliographic databases: Agricola (1970–2008), Commonwealth Agricultural Bureau Abstracts (1900–2008), Food Science and Technology Abstracts (1969–2008), and Medline (1800–2009). No restriction was placed upon language of publication. The following terms were searched in Agricola: (“monitoring,” “verification,” “hygienic,” “hygienic assessment,” “enumeration of indicator organisms,” “critical control point,” “food safety system,” “food safety control,” “HACCP,” “food safety objective,” or “pathogen reduction”) AND (“slaughterhouse,” “slaughterhouses,” “abattoir,” “abattoirs,” “abatoir,” “abatoirs,” “abbatoir,” “abbatoirs,” “processing plant,” “processing plants,” “processing establishment,” “processing establishments,” “processing facility,” “processing facilities,” “inspected plant,” “inspected plants,” “inspected establishment,” “inspected establishments,” “inspected facility,” “inspected facilities,” “slaughter plant,” “slaughter plants,” “slaughter establishment,” “slaughter establishments,” “slaughter facility,” “slaughter facilities,” “carcass,” “carcasses,” or “food processing industry”). A complete list of the search terms is also included in the Supplementary Material. An updated search was executed in October 2010 using the same terms and databases as the original search.

Reference lists of each relevant primary research article were checked to identify additional potentially relevant work. The five most-published topic experts whose studies were captured by the search were contacted to request any work close to being submitted or in press, and nonresponders were contacted once more. An internet search was also conducted using the Google and Yahoo search engines, employing the same search terms used in the electronic databases, in September 2010. When several national baseline surveys were identified by this search, given the scarcity of relevant studies, the decision was made post hoc to include these paired baseline studies (i.e., pairs of studies available on the internet, with one investigating microbial contamination on carcasses in abattoirs “pre-HACCP” and a complementary study sampling “post-HACCP”) as a source of evidence in addition to the peer-reviewed studies captured.

All levels of screening, from first-level relevance screening to data extraction, were conducted independently by two epidemiologists using tools designed a priori and pretested, with reviewing at each level being initiated only when between-reviewer agreement yielded a kappa statistic of 0.8 or greater. First-level relevance screening (RS1), wherein each abstract was appraised using the RS1 screening tool, served to exclude irrelevant citations. Citations not excluded at RS1 advanced to RS2, where appraisal of the full article using the RS2 tool categorized studies by population sampled, bacteria measured, and type of program implemented as well as confirming relevance.

Methodological assessment and data extraction

All relevant studies were assessed for methodological soundness using the following criteria: whether the sampling scheme was random or systematic (yes/no); whether the sample size was justified (yes/no); whether steps were taken to ensure pre- and post-HACCP sample populations were exchangeable, meaning the exposure status of the two groups could be switched without impacting the study results (Dohoo et al., 2009) (yes/no); and whether indirectness, or lack of comparability, existed between sample and target population, or outcome measure employed versus outcome measure of interest (e.g., sampling calves when the population of interest is finished steers; or measuring meat juice ELISA to demonstrate Salmonella exposure, when the objective is to measure carcass Salmonella contamination), given the review objective and inclusion criteria. These criteria were captured for discussion purposes. However, no studies were excluded from the review based on methodological assessment criteria. Data extracted from all studies advancing from level 2 screening to methodological assessment and data extraction included parameters such as descriptors of the population sampled (e.g., species and age of carcass animals sampled, country in which investigation was conducted), methods of sampling (e.g., excision vs. swab), microbial outcomes measured (e.g., aerobic plate counts; prevalence or concentration of Campylobacter spp., coliforms, Escherichia coli, Enterobacteriaceae, Salmonella spp., Staphylococcus aureus), and estimates of effect (e.g., mean logarithmic differences in counts; odds ratios of prevalence post- and pre-HACCP). The tool used for data extraction, with a complete list of variables extracted, is included in the review protocol as Supplementary Material.

Review management and data analysis

All electronic citations were downloaded and deduplicated in Procite 5.0 (Thomson ResearchSoft, Philadelphia, PA) and uploaded into SRS 4.0 electronic systematic review management system (Systematic Reviews SRS 4.0; Trialstat Corporation, Ottawa, Canada).

Data were entered and cleaned in Microsoft Office Excel 2003 (Microsoft Corporation, Redmond, WA) and descriptively summarized. Random effects MA was performed based on the a priori assumption that heterogeneity existed across studies. Odds ratios of prevalence or logarithmic mean differences in bacterial counts between exposed (post-HACCP) and unexposed (pre-HACCP) groups and their respective 95% confidence intervals were calculated from reported raw data. A pooled estimate of the odds ratio or log mean difference in bacterial counts was calculated using the method of DerSimonian and Laird (1986). Cochran's Q statistic and I ² (the percentage of total variation among studies due to heterogeneity) were used to evaluate heterogeneity (Higgins et al., 2003). Evidence for the presence of publication bias as indicated by evidence of small study bias in each MA dataset was appraised using visual examination of the funnel plot (a plot of standard error by logarithmic odds ratio, or logarithmic mean difference in bacterial counts, for each study) for asymmetry. Where suspected, its extent was estimated using the trim-and-fill method of Duval and Tweedie (2000). Post hoc, the decision was made that asymmetry tests for publication bias could only be appropriately applied to MA datasets meeting the following criteria: more than 10 studies were included in MA; significant (p<0.1) heterogeneity was not demonstrated across studies and at least one study in the dataset reported a significant result (Ionnidis and Trikalinos, 2007). Descriptive statistical analyses were performed in Stata Intercooled 11 (Stata Corporation, College Station, TX); meta-analyses were performed in Comprehensive Meta-analysis 2 (Biostat, Inc., Englewoood, NJ).

Results

From 752 reviewed abstracts, 53 were identified as potentially relevant at RS1, and 19 articles were confirmed relevant at RS2. Of 19 relevant studies, 15 and 4 studies were obtained from four bibliographic databases and through simple internet searches, respectively.

Four studies reported relevant data collected after HACCP implementation, which were then compared with data obtained pre-HACCP that had been reported elsewhere (Phillips et al., 2001, 2006; Sumner et al., 2003, 2004). No relevant articles published in languages other than English were identified, precluding the need for translation.

The main characteristics of 19 studies included in this review and published from 1996 to 2009 are shown in Tables 2 and 3. These reported investigations were conducted in Canada (n=1), Australia (4), Belgium (1), Greece (1), Ireland (2), Serbia (1), South Africa (1), and the United States (8). Species sampled included beef (13), swine (8), poultry (3), sheep (2), and crawfish (1). Microbes studied included indicator organisms suggestive of general process hygiene such as total aerobic counts (13) and coliform bacteria (2), which are indicators of fecal contamination, as well as potential pathogens: Campylobacter spp. (1), E. coli (13), Listeria spp. (1), Salmonella spp. (14), and coagulase-positive Staphylococcus aureus (3). Thirteen of 19 studies investigated multiple outcomes, and 5 of 19 investigated multiple carcass species.

Table 2.

Descriptive Characteristics of 19 Studies Evaluating the Effectiveness of Hazard Analysis Critical Control Point in Reducing Total Aerobic Bacterial Counts and Escherichia coli on Beef, Pork, Poultry, and Sheep Carcasses

Reference	Location of sampling (sampling period)	Setting (method of sampling)	Outcome type	Population	Outcome measured “before HACCP” estimate (standard deviation) [number sampled]	Outcome measured “after HACCP” estimate (standard deviation) [number sampled]	Significance (statistical test)
Dormedy (1999)	United States (6 weeks)	Single abattoir (sponge)	Aerobic log mean counts	Beef	3.35 (0.42) [30]	2.54 (0.74) [10]	p<0.05 (Duncan's multiple range test)
Nastasijevic et al. (2009)	Serbia (10 months)	Single abattoir (sponge)	Aerobic log mean counts	Beef	3.03 (0.62) [240]	2.70 (0.37) [240]	p<0.05^a
Phillips et al. (2006)^b	Australia (1998–2004)	National program (sponge)	Aerobic log mean counts	Beef	2.4 [1085]	1.3 (0.8) [1147]	NR^c
Phillips et al. (2001)^b	Australia (1993/4–1998)	National program (excision, sponge)	Aerobic log mean counts	Beef	3.17 [881]	2.43 [1085]	p<0.001
Sumner et al. (2004)^b	Australia (1994–98)	National program (sponge)	Aerobic log mean counts	Beef	3.02 [881]	2.43 [1275]	NR
Sumner et al. (2003)^b	Australia (1998–2002)	National program, small abattoirs (sponge)	Aerobic log mean counts	Beef	3.17 [40]	1.72 [95]	NR
Tergney and Bolton (2006)	Ireland (6 months)	Single abattoir (sponge)	Aerobic log mean counts	Beef	3.0 [10]	3.0 [10]	NR
Wagude (1999)	South Africa (5 weeks)	Single abattoir (sponge)	Aerobic log mean counts	Beef	3.77 [50]	3.74 [50]	NR
Nastasijevic et al. (2009)	Serbia (10 months)	Single abattoir (sponge)	Aerobic mean log counts	Pork	3.98 (0.61) [120]	1.94 (0.12) [120]	p<0.05^a
Bolton (1999)	United States (1993–95)	Single abattoir (sponge)	Aerobic mean log counts	Pork	4.8 [NR]	2.0 [NR]	NR
Bryant et al. (2003)	Canada (NR)	Single abattoir (sponge)	Aerobic mean log counts	Pork	1.49 [25]	1.01 (0.73) [20]	NR
Tsola et al. (2008)	Greece (2 years)	Single abattoir (carcass rinse)	Aerobic mean log counts	Poultry	5.72 (0.08) [NR]	4.60 (0.06) [78]	p<0.001
Sumner et al. (2004)	Australia (1994–2002)	National program (sponge)	Aerobic mean log counts	Lamb	3.92 [470]	1.68 [3695]	NR
Dormedy (1999)	United States (6 weeks)	Single abattoir (sponge)	Coliforms	Beef	0.32 (0.54) [30]	1.19 (0.62) [10]	p<0.05^a,d
Tergney and Bolton (2006)	Ireland (6 months)	Single abattoir (sponge)	Coliforms	Beef	0.35 [10]	−0.55 [10]	NR
Phillips et al. (2001)^b	Australia (1993/4–1998)	National program (sponge)	E. coli	Beef	22% [881]	11.1% [1285]	p<0.05
Phillips et al. (2006)^b	Australia (1998–2004)	National program (sponge)	E. coli	Beef	10.3% [1285]	8% (0.7) [1147]	p>0.05^a
Sumner et al. (2004)^b	Australia (1994–98)	National program (sponge)	E. coli	Beef	19.1% [881]	10.3% [1275]	p<0.05^a
Sumner et al. (2003)^b	Australia (1998–2002)	National program (sponge)	E. coli	Beef	2.5% [40]	8.4% [64]	p<0.01^a,d
USDA-FSIS (1998a)	United States (1992–98)	National program (sponge)	E. coli	Beef	1.0% [2089]	1.2% [1881]	p>0.05^a
Dormedy (1999)	United States (6 weeks)	Single abattoir (sponge)	E. coli	Beef	0.247 (0.501) [30]	0.76 (0.76) [10]	p>0.05 (Duncan's multiple range test)
Phillips et al. (2006)	Australia (1998–2004)	National program (sponge)	E. coli	Beef	−0.41 (NR) [1275]	−0.8 (0.7) [1147]	p<0.01^a (t-test)
Tergney and Bolton (2006)	Ireland (6 months)	Single abattoir (sponge)	E. coli	Beef	0.24 [180]	−0.59 [180]	NR
Wagude (1999)	South Africa (5 weeks)	Single abattoir (sponge)	E. coli	Beef	−0.06 [50]	−0.20 [50]	p=0.214
USDA-FSIS (1998b)	United States (1995–98)	National program (sponge)	E. coli	Pork	20% [1296]	8.7% [1225]	p<0.01^a
USDA-FSIS (2000)	United States (1994–2000)	National program (carcass rinse)	E. coli	Poultry	8.7% [2112]	6.9% [2127]	p<0.05^a
Cates et al. (2001)	United States (2000)	Eight abattoirs (carcass rinse)	E. coli	Poultry	77.9%≤100 cfu/mL [2462]	93.4%≤100 cfu/mL [2588]	NR
Phillips et al. (2006)	Australia (1994–2002)	National program (sponge)	E. coli	Lamb	75% [470]	25% [18480]	p<0.05^a

Calculated post hoc.

Studies accessed similar datasets and are therefore not independent.

Not reported.

Levels of microbial contamination were higher post-HACCP.

Table 3.

Descriptive Characteristics of 14 Studies Evaluating the Effectiveness of Hazard Analysis Critical Control Point on Reduction of Salmonella spp., Staphylococcus aureus, and Listeria spp. on Beef, Pork, Poultry, Sheep, and Crawfish Carcasses

Reference	Location of sampling (sampling period)	Setting (method of sampling)	Outcome type	Population	Outcome measured “before HACCP” estimate (standard deviation) [number sampled]	Outcome measured “after HACCP” estimate (standard deviation) [number sampled]	Significance (statistical test)
Phillips et al. (2001)^a	Australia (1993/4–1998)	National program (sponge sampling)	Coagulase-positive Staphylococcus aureus	Beef	22% [881]	3.9% [1285]	p<0.05^b
Phillips et al. (2006)^a	Australia (1998–2004)	National program (Sponge sampling)	Coagulase-positive Staphylococcus aureus	Beef	3.9% [1285]	28.7% [1147]	p<0.05^b,c
Tsola et al. (2008)	Greece (2 years)	Single abattoir (carcass rinse samples)	Coagulase-positive Staphylococcus aureus	Poultry	2.60 (0.09) [NR¹]	0.95 (0.18) [NR]	NR^d
Dormedy (1999)	United States (6 weeks)	Single abattoir (sponge sampling)	Salmonella spp.	Beef	0 [30]	0 [30]	p>0.05^b
Ghafir et al. (2005)	Belgium (1998–2002)	National program (sponge sampling)	Salmonella spp.	Beef	2.5% [120]	0% [191]	p>0.05^b
Phillips et al. (2001)^a	Australia (1993/4–1998)	National program (Sponge sampling)	Salmonella spp.	Beef	0.50% [881]	0.60% [1275]	p>0.05^b
Phillips et al. (2006)^a	Australia (1998–2004)	National program (sponge sampling)	Salmonella spp.	Beef	0.60% [1275]	0 [1147]	p>0.05^b
Rose et al. (2002)	United States (1998–2000)	National program (swab sampling)	Salmonella spp.	Beef	0 [214]	0.4% [1092]	p>0.05^b
Sumner et al. (2004)	Australia (1998–2002)	National program (sponge sampling)	Salmonella spp.	Beef	0.34% [881]	0.26% [4687]	p>0.05^b
USDA-FSIS (1998a)	United States (1992–98)	National program (sponge sampling)	Salmonella spp.	Beef	1.2% [2089]	1.0% [1881]	p>0.05^b
Wagude (1999)	South Africa (5 weeks)	Single abattoir (sponge)	Salmonella spp.	Beef	2.0% [50]	0% [50]	p>0.05^b
Ghafir et al. (2005)	Belgium (1998–2002)	National program (sponge sampling)	Salmonella spp.	Pork	25% [464]	14.6% [287]	p>0.01^b
Rose et al. (2002)	United States (1998–2000)	National program (sponge sampling)	Salmonella spp.	Pork	5.8% [1390]	6.2% [5170]	p>0.05^b
Tamplin et al. (2001)	United States (2000)	Single abattoir (sponge sampling)	Salmonella spp.	Pork	0.8% [10]	0.7% [100]	NR
USDA-FSIS (1998b)	United States (1995–98)	National program (sponge sampling)	Salmonella spp.	Pork	8.7% [2112]	6.9% [2117]	p<0.05^b
Cates et al. (2001)	United States	Eight abattoirs (carcass rinse sample)	Salmonella spp.	Poultry	5.66% [2438]	5.91% [2587]	p>0.05^b
Ghafir et al. (2005)	Belgium (1998–2002)	National program (excision sampling)	Salmonella spp.	Poultry	43.1% [411]	12.1% [290]	p<0.01^b
Rose et al. (2002)	United States (1998–2000)	National program (carcass rinse sampling)	Salmonella spp.	Poultry	10.8% [5659]	9.1% [10057]	p<0.05^b
USDA-FSIS (2000)	United States (1994–2000)	National program (carcass rinse sampling)	Salmonella spp.	Poultry	20.1% [1296]	8.7% [1225]	p<0.05^b
Sumner et al. (2003)	Australia (1994–2002)	National program (sponge sampling)	Salmonella spp.	Lamb	5.7% [470]	0.30% [3695]	p<0.01^b
Lappi et al. (2004)	United States (2001–2002)	Single facility (tissue homogenates)	Listeria spp.	Crawfish	29.5% [78]	52.5% [101]	p<0.05^b

Studies accessed similar datasets and are therefore not independent.

Calculated post hoc.

Levels of microbial contamination were higher post-HACCP.

Not reported.

Our search captured investigations of two types of HACCP programs in abattoirs, applying HACCP principles to either regulatory health inspections in the abattoir or slaughter/primary processing itself. The former type of HACCP program was investigated in two studies of HACCP-based health inspection programs for identification of defects such as bruising or inflammatory lesions (Cates et al., 2001; Tamplin et al., 2001). Both reported no significant difference (p>0.05) in prevalence of Salmonella spp. on pork and poultry, respectively, using HACCP-based or non–HACCP-based health inspection systems. The balance of studies captured (17) examined the effects of HACCP (n=15 studies) or similar structured control programs (n=2) on microbial carcass contamination from slaughter through primary processing.

The results of individual studies indicated that HACCP or similar structured control programs were more consistently effective in reducing indicator organisms, for example, aerobic bacterial counts, compared with pathogens such as Salmonella spp. on carcasses (Tables 2 and 3).

MA of three studies investigating aerobic bacterial counts on beef carcasses before and after HACCP implementation (Dormedy, 1999; Phillips et al., 2001; Nastasijevic et al., 2009) yielded a significant (p<0.001) logarithmic mean difference of −0.75 cfu/cm² (−log 0.94 cfu/cm², −log 0.55 cfu/cm²) with insignificant heterogeneity (Q=2.3, p=0.31, I ²=14) across the studies (Fig. 1). This dataset, as with all of the MA datasets captured, did not meet our criteria for application of asymmetry tests to investigate the presence of publication bias (Ionnidis and Trikalinos, 2007).

FIG. 1.

Reduction in logarithmic mean aerobic bacterial counts on beef carcasses before and after hazard analysis critical control point implementation.

MA of three studies investigating E. coli prevalence on beef carcasses before and after HACCP implementation (USDA-FSIS 1998a; Wagude, 1999; Phillips et al., 2001) yielded an insignificant (p=0.13) odds ratio of 0.59 (0.30, 1.2) with significant heterogeneity (Q=10, p=0.006, I ²=80) across the studies (Forest plot not illustrated for brevity).

MA of five studies investigating Salmonella spp. prevalence on beef carcasses before and after HACCP initiation (USDA-FSIS, 1998a; Wagude, 1999; Phillips et al., 2001; Rose et al., 2002; Ghafir et al., 2005) yielded an insignificant (p=0.65) odds ratio of 0.89 (0.53, 1.48), with insignificant heterogeneity (Q=3.4, p=0.50, I ²=0) across studies (Fig. 2).

FIG. 2.

Odds of Salmonella spp. contamination on beef carcasses before and after hazard analysis critical control point implementation.

MA of three studies investigating Salmonella spp. prevalence on pork carcasses before and after HACCP initiation (USDA-FSIS, 1998b; Rose et al., 2002; Ghafir et al., 2005) yielded an insignificant (p=0.17) odds ratio of 0.78 (0.54, 1.1) and significant heterogeneity (Q=10, p=0.007, I ²=80) across studies (Forest plot not illustrated for brevity).

MA of three studies investigating Salmonella spp. on poultry (USDA-FSIS, 2000; Rose et al., 2002; Ghafir et al., 2005) yielded a significant (p=0.02) odds ratio of 0.39 (0.17, 0.87), with significant heterogeneity (Q=76, p<0.001, I ²=97) across studies (Forest plot not illustrated for brevity).

Problems with study design were frequently noted in the literature captured and are presented online as Supplementary Material. Ten of 19 studies reported use of convenience sampling schemes, and only 5 justified sample size. However, all investigators utilized directness of comparisons as defined by the Cochrane Collaboration (2010), that is, sampling a population comparable to the target population, using outcome measures of direct interest as opposed to a proxy measure. Also, each investigation made some effort to ensure that pre- and post-HACCP sample populations were exchangeable (e.g., sampling in multiple seasons to adjust for seasonal shedding).

Discussion

Overall, a small body of publicly available data on the effect of HACCP on microbial contamination of food animal carcasses were identified; more consistent reductions were observed for indicator bacteria associated with process hygiene (e.g., aerobic bacteria and coliforms) than for pathogens (e.g., Salmonella spp.). The relatively small volume of published literature on this topic is striking, considering the widespread endorsement and initiation of HACCP programs in food animal abattoirs over the past decade. This may be due to the proprietary nature of the data collected by abattoirs in countries with specified pathogen performance standards, such as the United States, or the fact that microbial monitoring is conducted to verify HACCP, not as part of the HACCP program itself, as well as the lack of baseline microbial studies conducted prior to HACCP initiation in many jurisdictions (Phillips et al., 2001; Fearne et al., 2004). Three of the largest studies captured were obtained not from peer-reviewed articles but from the internet search, highlighting the importance of this source of publicly available data.

MA can be a powerful tool to estimate the effect of an intervention (e.g., HACCP implementation). However, robust MA was not possible in this review, given the overall small body of data, as well as the wide range of populations and outcomes studied, and heterogeneity in diagnostic and sampling methods. Using a random effects model, power for MA depends on the total number of subjects and also on the number of studies included; if study numbers are small and variance across studies is substantial, MA power may be low (Borenstein et al., 2009). Although MA for some carcass species–bacterial species combinations (E. coli on beef; Salmonella spp. on beef and pork) yielded insignificant pooled effect estimates, nevertheless, interpretation of their respective wide confidence intervals suggests that, in some settings, HACCP implementation had a protective effect on carcass contamination. Therefore, given the small number of studies available for MA in these datasets, it is likely more accurate to conclude that, given the number of studies captured and their respective sample sizes, there is currently insufficient evidence to judge whether HACCP has a significant effect (Valentine et al., 2010). Similarly, publication bias was difficult to determine in each of the datasets undergoing MA, given the small number of studies captured and the heterogeneity detected across some datasets.

Possible sources of heterogeneity across studies include variable intensity of changes in personnel hygiene and training, improvements in cleaning and disinfection through the use of zoning or designation of high/low traffic areas, air filtration, improved process control, incorporation of antimicrobial interventions, and increased microbial process control monitoring; similarly, HACCP components such as variable choice of CCPs and differences in HAACP prerequisites could also be sources of heterogeneity (I. Jensen and R. Usborne, pers. comm.) (Rose et al., 2002; Delazari et al., 2006).

Additionally, the relationship of each of these factors with HACCP implementation may vary with jurisdiction. For example, Canada's Food Safety Enhancement Program (FSEP) includes seven prerequisite programs covering areas such as personnel hygiene and ventilation, as well as HACCP itself, and also HACCP validation and maintenance, and each part of FSEP has contributed to enhanced food safety (Canadian Food Inspection Agency, 2010) (R. Usborne, pers. comm.).

All studies included in the review were before/after studies in which the outcome of interest (i.e., microbial contamination on a given carcass species) was measured pre- and postimplementation of the intervention (HACCP or similar structured programs). Appropriate analysis and interpretation of such data must take into account measurable factors other than HACCP that might affect the outcome during the same time interval, such as those listed earlier. For example, we performed an informal analysis of Australian data on E. coli prevalence on beef carcasses before the time of national HACCP implementation and for the subsequent 4 years (Sumner et al., 2004). We divided the overall time period into two blocks, pre-HACCP (1994–1998) and post-HACCP (1998–2002), and performed linear regression on the annual prevalence for both time periods. The annual incremental reduction in E. coli prevalence did not change significantly across blocks of time (i.e., pre- vs. post-HACCP). This supports the hypothesis that other factors were also reducing the level of microbial contamination of carcasses and could have been exerting an effect around the time of HACCP implementation (I. Jenson, pers. comm.).

Another factor that makes interpretation of before–after trials difficult is the phenomenon of regression to the mean: the measure of interest may have been unusually elevated immediately before the intervention and randomly returned to a more typical level after the intervention, creating the appearance of a treatment effect, because of chance. Without analysis to identify this phenomenon if present and adjust for its effect, it is possible that some of the change in microbial contamination reported post-HACCP may in fact be due to this phenomenon. Modeling techniques such as Empirical Bayes have been used in other disciplines to interpret before–after trial data, allowing both the separation of effects from concurrent confounders and regression-to-the-mean phenomena from the underlying treatment effect (Casella, 1986; Persaud and Lyon, 2007). One reason modeling has not been more extensively used in interpreting HACCP before–after trial data may be due to a lack of large datasets that can be required for this type of analysis (Persaud and Lyon, 2007). In the absence of modeling, interpreting raw mean differences in microbial prevalence or concentration as resulting solely from the effect of HACCP implementation will inevitably yield an inaccurate estimate of treatment effect. Therefore, the extent to which reported reductions in microbial prevalence or concentration on carcasses can be attributed to HACCP implementation is currently unknown.

Currently, limited data are available to identify the effectiveness of HACCP implementation in reducing the rate of attributable foodborne illness. Notifiable disease registries in both the United States and Australia similarly report a decline in the human incidence of Campylobacter cases, with little substantive change in the incidence of reported cases of Salmonella, since the time of implementation of HACCP in each country (CDC-DHHS, 2010; GWADH, 2010). Canada's 2006 report of national laboratory surveillance data for enteric pathogens reports a decline in the reported cases of Campylobacter and also in Salmonella, but an increase in reported cases of pathogenic E. coli (Public Health Agency of Canada, 2009). These before–after data contain the same challenges in interpretation as outlined earlier for before–after trials and, therefore, should be interpreted with caution. Data such as these, obtained through passive surveillance, also reflect a geographically variable degree of underreporting, which could bias comparisons between time periods of locations (Wheeler et al., 1999; Voetsch et al., 2004). Comparing the degree of concurrence between the commonly identified animal serovars of pathogens and the commonly reported human serovars of the same pathogens, since HACCP implementation, is another method of investigating the effect of HACCP on public health outcomes. Two U.S. studies investigating Salmonella spp. prevalence prior to, and during, the time of HACCP implementation, respectively, in humans and on meat both reported that the most common serotypes on tested meat products were not the most common causes of human illness (Schlosser et al., 2000; Sarwari et al., 2001). Similarly, an Australian study reported little concurrence between prevalence of Salmonella serovars derived from red meat and human cases, either before or after HACCP, although some concurrence existed between human serovars and those on poultry (Sumner et al., 2004). Comparing concurrence of serovars, at best, may identify an association, but not the direction of movement of microbes (i.e., animal-to-man, vs. man-to-animal, or mutual exposure to a third, independent, source of the pathogen). Molecular epidemiology, particularly DNA sequence analysis and phylogenetic reconstruction, offers the potential to trace the spread of microbes from food animal carcasses to human populations and thus shed more light on the effectiveness of HACCP in reducing human illness attributable to food animal consumption, as recent research has demonstrated for both Campylobacter spp. (Mullner et al., 2009, 2010; Scottish Food Standards Agency, 2009) and Salmonella spp. (Patel, 2007) infections. As more data become available over time, it might be possible to evaluate temporal trends both in bacterial contamination on carcasses in abattoirs and their potential correlation to attributable human cases of foodborne illness.

Several knowledge gaps were identified by our review. Some combinations of bacterial species and food animal species were studied more frequently than others. In general, more large studies measuring pathogenic bacteria such as Campylobacter spp., Salmonella spp., and shiga-like E. coli are needed, given the importance of these pathogens in foodborne disease. None of the studies captured measured nonbacterial outcomes (e.g., viruses) and studies including nonbacterial outcomes would be useful in informing agrifood public health policy. Estimation of the effects of other interventions occurring in abattoirs, such as staff training, on microbial contamination on carcasses would be a useful prerequisite to modeling the effects of HACCP initiation. Overall, given the documented financial burden that HACCP implementation imposes, and the trend to implement HACCP even in small abattoirs over time, there is a need to verify its effectiveness in reducing microbial contamination on food animal carcasses. Future research accessing both proprietary and government surveillance data, employing analysis such as Empirical Bayes to adjust for potential confounders inherent in before/after trials, is needed to more accurately estimate the microbiological effectiveness of HACCP. This would be in keeping with HACCP's history of private–public sector collaboration.

Conclusion

The existing publicly available data on the effectiveness of HACCP on microbial contamination of carcasses are limited. Consistent trends in reduction of bacterial contamination were observed only for indicator organisms (e.g., aerobic bacteria). Research accessing both proprietary and government surveillance data is needed to more accurately estimate the microbiological effectiveness of HACCP, and large-scale prevalence studies measuring pathogenic bacteria and nonbacterial outcomes would be useful.

Footnotes

Acknowledgments

The authors gratefully acknowledge the complementary reviewing efforts of Oliver Bucher, Ashley Farrar, and Dr. Sarah Parker as well as the financial support of the University of Saskatchewan.

Disclosure Statement

No competing financial interests exist.

References

Bolton

, Oser

, Cocoma

, Palumbo

, Miller

. Integrating HACCP and TQM reduces pork carcass contamination. Food Technol, 1999; 53:40–43.

Borenstein

, Hedges

, Higgins

, Rothstein

. Introduction to Meta-Analysis. West Sussex, UK: John Wiley and Sons, Ltd, 2009.

Bryant

, Brereton

, Gill

. Implementation of a validated HACCP system for the control of microbiological contamination of pig carcasses at a small abattoir. Can Vet J, 2003; 44:51–55.

Casella

. An introduction to Empirical Bayes data analysis. Am Stat, 1986; 39:83–87.

Cates

, Anderson

, Karns

, Brown

. Traditional versus hazard analysis and critical control point-based inspection: results from a poultry slaughter project. J Food Prot, 2001; 64:826–832.

[CDC-DHHS] Centers for Disease Control and Prevention, Department of Health and Human Services. FoodNet facts and figures—number of infections and incidence per 100,000 persons. 2010. www.cdc.gov/foodnet/factsandfigures/AllSites19962008_Incidence.pdf. 2011 January 20(Online.)

Cochrane Collaboration. Publication bias: Interpreting funnel plots. 2010. www.cochrane-net.org/openlearning/html/mod15-3.htm. 2011 January 20(Online.)

Delazari

, Riemann

, Hajmeer

. Food safety. Foodborne Infections and Intoxications, 3 ^rd. Reimann

, Cliver

. San Diego, CA: Academic Press, 2006; 833–884.

DerSimonian

, Laird

. Meta-analysis in clinical trials. Control Clin Trials, 1986; 7:177–188.

10.

Dohoo

, Martin

, Stryhn

. Veterinary Epidemiologic Research, 2 ^nd. Charlottetown, Canada: VER, Inc, 2009.

11.

Dormedy

. Implementation and microbial verification of HACCP (hazard analysis and critical control point) systems and HACCP intervention methods in meat processing establishments for the reduction of foodborne pathogens [dissertation] Lincoln: University of Nebraska, 1999; 99–118.

12.

Duval

, Tweedie

. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 2000; 56:455–463.

13.

Fearne

, Garcia

, Henson

, Collins

, Khatri

, McEvoy

, Nganje

, le Roux

. Global HACCP Implementation in Meat Producing Countries: Final Report. London, UK: Food Standards Agency, 2004MO1030pdf.www.food.gov.uk/science/research/foodborneillness/meathygieneresearch/m01prog/m01list/m01030/. 2011 January 20(Online.)

14.

Ghafir

, China

, Korsak

, Dierick

, Collard

, Godard

, de Zutter

, Daube

. Belgian surveillance plans to assess changes in Salmonella prevalence in meat at different production stages. J Food Prot, 2005; 68:2269–2277.

15.

Gill

. HACCP by guesswork or by the numbers? Food Qual, 1999; 6:28–32.

16.

[GWADH] Government of Western Australia Department of Health. OzFoodNet (enteric infections) reports. 2010. www.public.health.wa.gov.au/3/605/3/ozfoodnet_enteric_infections_reports.pm. 2011 January 20(Online.)

17.

Higgins

, Thompson

, Deeks

, Altman

. Measuring inconsistency in meta-analyses. Br Med J, 2003; 327:557–560.

18.

Ionnidis

JPA

, Trikalinos

. The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. Can Med Assoc J, 2007; 176:1091–1096.

19.

Jevsnik

, Hlebec

, Raspor

. Meta-analysis as a tool for barrier identification during HACCP implementation to improve food safety. Acta Aliment, 2006; 35:319–353.

20.

Lammerding

, Tood

. Microbial food safety risk assessment. Foodborne Infections and Intoxications, 3 ^rd. Reimann

, Cliver

. San Diego, CA: Academic Press, 2006; 833–884.

21.

Lappi

, Thimothe

, Walker

, Bell

, Gall

, Moody

, Wiedmann

. Impact of intervention strategies on Listeria contamination patterns in crawfish processing plants: a longitudinal study. J Food Prot, 2004; 67:1163–1169.

22.

Lupin

, Parin

, Zagarramurdi

. HACCP economics in fish processing plants. Food Control, 2010; 21:1143–1149.

23.

Mullner

, Shadbolt

, Collins-Emerson

, Midwinter

, Spencer

SEF

, Marshall

, Carter

, Campbell

, Wilson

, Hathaway

, Pirie

, French

. Molecular and spatial epidemiology of human campylobacteriosis: source association and genotype-related risk factors. Epidemiol Infect, 2010; 138:1372–1383.

24.

Mullner

, Spencer

SEF

, Wilson

, Jones

, Noble

, Midwinter

, Collins-Emerson

, Carter

, Hathaway

, French

. Assigning the source of human campylobacteriosis in New Zealand: a comparative genetic and epidemiological approach. Inf Gen Evol, 2009; 9:1311–1319.

25.

Patel

. Comparison of antibiotic sensitivity profiles, molecular typing patterns, and attribution of Salmonella enterica serotype Newport in the US, 2003–2006. [dissertation] Atlanta, GA: Georgia State University, 2007.

26.

Persaud

, Lyon

. Empirical Bayes before-after safety studies: lessons learned from two decades of experience and future directions. Accid Anal Prev, 2007; 39:546–555.

27.

Phillips

, Jordan

, Morris

, Jenson

, Sumner

. A national survey of the microbiological quality of beef carcasses and frozen boneless beef in Australia. J Food Prot, 2006; 69:1113–1117.

28.

Phillips

, Sumner

, Alexander

, Dutton

. Microbiological quality of Australian beef. J Food Prot, 2001; 64:692–696.

29.

Public Health Agency of Canada. Canadian integrated surveillance report: Salmonella, Campylobacter, verotoxigenic E. coli and Shigella, from 2000 to 2004. 2009. www.phac-aspc.gc.ca/publicat/ccdr-rmtc/09vol35/35s3/human-humain-eng.php. 2011 January 20(Online.)

30.

Quinn

, Markey

, Carter

, Donnelly

, Leonard

. Veterinary Microbiology and Microbial Disease. Cornwall, UK: Blackwell Science, 2002.

31.

Romans

, Costello

, Carlson

, Greaser

, Jones

. The meat we eat. Danville, IL: Interstate Publishers, Inc, 1994; 187–211.

32.

Rose

, Hill

, Umholtz

, Ransom

, James

. Testing for Salmonella in raw meat and poultry products collected at federally inspected establishments in the United States, 1998 through 2000. J Food Prot, 2002; 65:937–947.

33.

Sarwari

, Magder

, Levine

, McNamara

, Knower

, Armstrong

, Etzel

, Hollingsworth

, Morris

. Serovar distribution of Salmonella isolates from food animals after slaughter differs from that of isolates found in humans. J Infect Dis, 2001; 183:1295–1299.

34.

Schlosser

, Hogue

, Ebel

, Rose

, Umholtz

, Ferris

, James

. Analysis of Salmonella serovars from selected carcasses and raw ground product sampled prior to implementation of the pathogen reduction: Hazard Analysis and Critical Control Point Final Rule in the US. Int J Food Microbiol, 2000; 58:107–111.

35.

Scottish Food Standards Agency. The molecular epidemiology of Scottish Campylobacter isolates from human cases of infection using multilocus sequence Typing. 2009. www.food.gov.uk/multimedia/pdfs/publication/fullreportcamps.pdf. 2011 January 20(Online.)

36.

Sumner

, Petrenas

, Dean

, Dowsett

, West

, Wiering

, Raven

. Microbial contamination on beef and sheep carcasses in South Australia. Int J Food Microbiol, 2003; 81:255–260.

37.

Sumner

, Raven

, Givney

. Have changes to meat and poultry food safety regulation in Australia affected the prevalence of Salmonella or of salmonellosis? Int J Food Microbiol, 2004; 92:199–205.

38.

Tamplin

, Feder

, Palumbo

, Oser

, Yoder

, Luchansky

. Salmonella spp. and Escherichia coli biotype I on swine carcasses processed under the Hazard Analysis and Critical Control Point-based inspection models system. J Food Prot, 2001; 64:1305–1308.

39.

Tergney

, Bolton

. Validation studies on an online monitoring system for reducing faecal and microbial contamination on beef carcasses. Food Control, 2006; 17:378–382.

40.

Tsola

, Drosinos

, Zoiopoulos

. Impact of poultry slaughter house modernisation and updating of food safety management systems on the microbiological quality and safety of products. Food Control, 2008; 19:423–431.

41.

[USDA-FSIS] United States Department of Agriculture, Food Safety and Inspection Service. Baseline data. 1998a. www.fsis.usda.gov/PDF/Baseline_Data_Cattle.pdf; www.fsis.usda.gov/OPHS/baseline/steer1.pdf. 2011 January 20(Online.)

42.

[USDA-FSIS] United States Department of Agriculture, Food Safety and Inspection Service. Baseline data. 1998b. www.fsis.usda.gov/OPHS/baseline/market_hog_1996.pdf; www.fsis.usda.gov/PDF/Baseline_Data_Swine.pdf. 2011 January 20(Online.)

43.

[USDA-FSIS] United States Department of Agriculture, Food Safety and Inspection Service. Baseline data. 2000. www.fsis.usda.gov/OPHS/baseline/broiler1.pdf; www.fsis.usda.gov/PDF/Baseline_Data_Young_Chicken.pdf. 2011 January 20(Online.)

44.

Valentine

, Pigott

, Rothstein

. How many studies do you need. A primer on statistical power for meta-analysis. J Ed Behav Stat, 2010; 35:215–247.

45.

Voetsch

, Van Gilder

, Angulo

, Farley

, Shallow

, Marcus

, Cieslak

, Deneen

, Tauxe

. Emerging Infections Program FoodNet Working Group. FoodNet estimate of the burden of illness caused by nontyphoidal Salmonella infections in the United States. Clin Inf Dis, 2004; 38,Suppl 3:S127–S134.

46.

Waddell

, Rajić

, Sargeant

, Parker

, Deckert

, McEwen

. The methodological soundness of literature reviews addressing three potential zoonotic public health issues. Zoonoses Public Health, 2009; 56:477–489.

47.

Wagude

. HACCP in a red meat abattoir. [dissertation] Praetoria, South Africa: University of Praetoria, 1999; 64–76.

48.

Wheeler

, Sethi

, Cowden

, Wall

, Rodrigues

, Tompkins

, Hudson

, Roderick

. Study of infectious intestinal disease in England: rates in the community, presenting to general practice, and reported to national surveillance. The Infectious Intestinal Disease Study Executive. Br Med J, 1999; 318:1046–1050.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.04 MB

0.00 MB