A Guide for Developing Plain-Language and Contextual Summaries of Systematic Reviews in Agri-Food Public Health

Abstract

The application of systematic reviews is increasing in the agri-food public health sector to investigate the efficacy of policy-relevant interventions. In order to enhance the uptake and utility of these reviews for decision-making, there is a need to develop summary formats that are written in plain language and incorporate supporting contextual information. The objectives of this study were (1) to develop a guideline for summarizing systematic reviews in one- and three-page formats, and (2) to apply the guideline on two published systematic reviews that investigated the efficacy of vaccination and targeted feed and water additives to reduce Salmonella colonization in broiler chickens. Both summary formats highlight the key systematic review results and implications in plain language. Three-page summaries also incorporated four categories of contextual information (cost, availability, practicality, and other stakeholder considerations) to complement the systematic review findings. We collected contextual information through structured rapid reviews of the peer-reviewed and gray literature and by conducting interviews with 12 topic specialists. The overall utility of the literature searches and interviews depended on the specific intervention topic and contextual category. In general, interviews with topic specialists were the most useful and efficient method of gathering contextual information. Preliminary evaluation with five end-users indicated positive feedback on the summary formats. We estimate that one-page summaries could be developed by trained science-to-policy professionals in 3–5 days, while three-page summaries would require additional resources and time (e.g., 2–4 weeks). Therefore, one-page summaries are more suited for routine development, while three-page summaries could be developed for a more limited number of high-priority reviews. The summary guideline offers a structured and transparent approach to support the utilization of systematic reviews in decision-making in this sector. Future research is necessary to evaluate the utility of these summary formats for a variety of end-users in different contexts.

Introduction

Systematic reviews are a method of knowledge synthesis used to identify, appraise, and integrate global research about a topic (European Food Safety Authority [EFSA], 2010; Higgins and Green, 2011; Young et al., 2014). Meta-analysis is the statistical combination of data from individual primary research studies, and it is often conducted following a systematic review to obtain overall estimates of effect on sufficiently comparable quantitative outcomes (Higgins and Green, 2011; Young et al., 2014). Systematic reviews are recommended as more reliable sources of knowledge than individual primary research studies because they have a lower risk of bias and, if meta-analysis is conducted, they can increase precision of quantitative effect estimates (Lavis et al., 2005). In addition, they use more transparent and replicable methodologies than traditional, narrative literature reviews (Sargeant et al., 2006; Waddell et al., 2009). For these reasons, the application of systematic review and meta-analysis methods is increasing in the agri-food public health sector to address various topics (Rajić and Young, 2013; Rajić et al., 2013; Young et al., 2014).

One limitation of systematic reviews is that traditionally they have not been published in a format that facilitates their accurate and timely interpretation among potential end-users such as decision-makers, policy and research analysts, and practitioners (Lavis et al., 2005; Wallace et al., 2012; Rajić and Young, 2013; Rajić et al., 2013). Another criticism of systematic reviews that investigate questions of intervention efficacy is that they typically do not address contextual factors relevant to the review question, such as implementation costs, local applicability and availability, and other important stakeholder considerations (Lavis et al., 2005; Lavis, 2009; Young et al., 2014). As a result, various groups in the health sector have developed systematic review summary documents that are ideal for rapid scanning, presented in an appealing format, written in plain language, have clearly formulated “take-home” messages, and in some cases, also incorporate supporting contextual information (Lavis, 2009; Rosenbaum et al., 2011; Khangura et al., 2012; Opiyo et al., 2013; Health Evidence, 2013; Rajić and Young, 2013).

There is a need to improve the uptake and utility of systematic reviews for end-users in the agri-food public health sector, which have specific needs not met by summary formats developed in other sectors. The objectives of this study were (1) to develop a guideline for summarizing systematic reviews of agri-food public health intervention questions using one- and three-page summary formats, and (2) to apply the guideline on two recently published systematic reviews.

Materials and Methods

Summary guideline, targeted end-users, and selection of reviews

We outline in Figure 1 a guideline for summarizing systematic reviews of agri-food public health intervention questions in two plain-language formats: (1) a one-page summary of the key systematic review results and implications, and (2) a three-page summary of the review findings with supporting contextual information. We developed the guideline and summary formats based on a review and evaluation of existing plain-language and contextual summaries in health and other sectors (Canadian Foundation for Healthcare Improvement, 2010; Rosenbaum et al., 2011; Khangura et al., 2012; Health Evidence, 2013; Opiyo et al., 2013; ResearchImpact, 2014). The one- and three-page formats were prioritized based on consultation with end-users who indicated preference for graded-entry summaries (i.e., multiple layers of short and longer summaries) and the addition of contextual information to highlight the relevance of the systematic review findings for decision-making (Rosenbaum et al., 2011; Khangura et al., 2012; Opiyo et al., 2013). The target end-users for the summaries includes professionals working at the science-to-policy interface in the agri-food public health sector, which we define as policy-makers and those who support them (e.g., policy and research analysts), practitioners, extension personnel, program officers and specialists, and other government and industry decision-makers (Rajić and Young, 2013). We selected two recently published systematic reviews as examples for initial application of the summary development guideline. The reviews investigated the efficacy of vaccination and targeted feed and water additives to control Salmonella in broiler chickens on farms (Totton et al., 2012a, 2012b).

FIG. 1.

Guideline for developing one- and three-page plain-language and contextual summaries of systematic reviews addressing intervention efficacy questions in the agri-food public health sector. The first step is to select the review topic and then to summarize the key review results and implications. To develop three-page summaries (white boxes), contextual information should be gathered through rapid reviews of the peer-reviewed and gray literature and interviews with topic specialists. The final steps include writing and formatting the summaries with the information collected and validating the final products with end-users.

One-page summary development

The first step in developing the one-page summaries involved reviewing the published systematic review articles to create summary points under the following headings: “background,” “findings,” “what does this mean?”, and “next steps” (Fig. 2). Information to populate these sections was collected and summarized by one trained research technician. Supporting information for the “background” section was also obtained via ad hoc gray literature searches in Google. The “findings” section was populated by extracting key results and discussion points from the systematic review articles. Meta-analysis results that were reported in the article as relative effective measures (e.g., odds ratios) were converted to absolute measures (e.g., percentage of Salmonella-positive birds with and without treatment) to facilitate interpretation (Higgins and Green, 2011). For the “what does this mean?” and “next steps” sections, we composed concise conclusion and recommendation statements and underscored the relevance of these statements for our target end-users (e.g., are the conclusions strong enough to act on based on the efficacy, risk of bias, and certainty of the results?) The summaries were then iteratively reviewed and revised by the authors to improve their brevity and clarity.

FIG. 2.

Overview of the one-page summary for the vaccination intervention.

Three-page summary development

To develop the three-page summaries, we summarized the systematic review articles in plain language as described above for one-page summaries (Fig. 1). We also obtained contextual information to complement the systematic review findings for the following prespecified categories that are known to be important considerations in agri-food public health decision-making: intervention implementation costs, local availability of the intervention, practicality of implementation, and other stakeholder considerations (e.g., intervention acceptability) (Fazil et al., 2008; Rajić and Young, 2013; Young et al., 2014). Contextual information was identified through structured rapid reviews of the peer-reviewed and gray literature and by conducting interviews with industry specialists (Ganann et al., 2010; Rajić and Young, 2013). Specific headings were created for each contextual category and the overall “take-home” messages and “next steps” (Fig. 3).

FIG. 3.

Overview of the three-page contextual summary for the vaccination intervention.

We conducted a targeted search of peer-reviewed literature using the bibliographic database Scopus on November 8, 2011 (Fig. 1). The search algorithm is available as Supplementary Data (Supplementary Data are available online at www.liebertpub.com/fpd). Search results were exported into an Excel spreadsheet (Microsoft Corporation, Redmond, WA), and the citation titles and abstracts were screened for relevance by two independent reviewers. Those that did not discuss broiler chickens, Salmonella, or any relevant intervention types were excluded. Full articles of potentially relevant citations were then reviewed and relevant information was extracted by one reviewer.

We then conducted searches for gray literature (e.g., research reports and policy documents) in Google from March to July 2012, to identify additional sources of contextual information (Fig. 1). Targeted search algorithms were developed and implemented for each intervention and contextual category (see Supplementary Data for details). For each search, one reviewer assessed the first 100 hits sorted by relevance by the search engine. Relevant documents were procured and contextual information was extracted for use in summary development.

For search verification, the reference lists of all relevant articles were examined and any additional, potentially relevant articles were obtained and reviewed. We also documented any identified literature review articles and used them to obtain additional background information about the issue.

Once the initial literature searches were completed, 12 topic specialists were contacted and interviewed to obtain additional information and insights for each contextual category (Fig. 1). The specialists represented veterinarians, extension personnel, poultry producers, feed mill nutritionists, hatchery officials, and researchers. Semistructured interviews with topic specialists were conducted in person (n=7) or via telephone (n=5) and followed a prespecified question guide consisting of 16 questions (Supplementary Data). We also followed up with selected specialists via e-mail (n=12) and telephone (n=9) to obtain additional clarification about any identified information gaps. All interviews were conducted by A.K. and were held from January to April 2012. During the interviews, we documented any information identified by specialists that was not considered during the initial gray literature searches (e.g., relevant products or practices), and additional, targeted gray literature searches were conducted to elaborate on these contextual categories as necessary.

Summary template design and evaluation

All information obtained in the previous steps was compiled, and draft summaries were developed using a template developed in the graphic-design program InDesign 5.0 (Adobe Systems Incorporated, San Jose, CA). We based our design of the summary template on similar formats developed in the health and other sectors (Canadian Foundation for Healthcare Improvement, 2010; Rosenbaum et al., 2011; Health Evidence, 2013; ResearchImpact, 2014). Eight end-users were selected to evaluate the summaries to ensure that the information presented was reliable, credible, appealing, and easily understandable. The end-users represented veterinarians, extension personnel, and other industry specialists. End-users were sent a copy of the summary documents via e-mail and were asked to provide detailed comments and suggestions for improvement. Follow-up e-mails were sent as a reminder if responses were not received after 2–3 weeks from the initial contact. End-user feedback was reviewed and incorporated into each summary document.

Results

Rapid review results

An overview of the rapid review process to obtain contextual information from the peer-reviewed literature is shown in Figure 4. From 684 initial citations, 347 were considered potentially relevant at the abstract level, and 96 full articles were obtained and assessed for relevant contextual information. Only seven articles provided relevant information to populate the summaries (four for the vaccination intervention and three for the feed and water additive intervention). In addition, 26 and 30 potentially relevant documents were identified from gray literature searches for each respective intervention, of which 8 and 14, respectively, provided relevant information used to populate the summaries.

FIG. 4.

Overview of the rapid review process for obtaining contextual information from peer-reviewed literature.

Utility of information sources for summary development

The relative utility of each information source to provide contextual information for the final three-page summaries is shown in Table 1. On average, approximately 4–5 h and 6–8 h per contextual category were needed to identify, screen, and extract relevant information from peer-reviewed and gray literature, respectively, and approximately 1–1.5 h were needed to interview each topic specialist.

Table 1.

Level of Utility and Time Required to Obtain Contextual Information from Different Sources to Populate Three-Page Summaries of Systematic Reviews Investigating Two Interventions to Reduce Salmonella Colonization in Broiler Chickens on Farms

		Information sources and their level of utility in providing relevant information to inform the summaries ^a
Intervention	Contextual category	Peer-reviewed literature	Gray literature	Topic specialists
Vaccination	Cost	Low	Low	High
	Availability	Low	High	Moderate
	Practicality	Low	Low	High
	Other stakeholder considerations	Low	Low	High
Feed and water additives	Cost	Low	Low	High
	Availability	Moderate	High	High
	Practicality	Low	Moderate	High
	Other stakeholder considerations	Low	Moderate	High
Time required to obtain information from each source:		∼4–5 h/contextual category	∼6–8 h/contextual category	∼1–1.5 h/specialist

Level of utility was rated according to the following criteria: High=most or all information used within the contextual category was populated from the indicated source; Moderate=approximately half of the information within the contextual category was populated from the indicated source. Low=little to no information within the contextual category was populated from the indicated source.

The peer-reviewed literature search provided information about various experimental vaccines to control Salmonella in broiler chickens, but few studies stated the commercial vaccine names. During interviews, we noted that some industry specialists were not familiar with the currently available Salmonella vaccines, so interviewing several individuals was necessary. Gray literature searches yielded the most useful information about vaccine product availability. Overall, the most useful source of information about vaccine costs, practicality, and other stakeholder considerations was interviews with topic specialists (Table 1).

Information concerning the availability of feed and water additives was captured in all three information sources; however, gray literature and topic specialist interviews provided information of higher utility (Table 1). Similar to the vaccination summary, interviews with topic specialists provided the most useful contextual information.

End-user evaluation of summaries

Five of the eight end-users responded with comments and recommendations for the summaries; the other three initially agreed to participate but did not respond with feedback after multiple contact attempts. Feedback was largely positive, with end-users favoring the plain-language formats and the additional contextual information in three-page summaries to provide them with a well-rounded overview of the issue. Minor suggestions were also received to clarify some concepts and terminology. Copies of the final one- and three-page summaries for each intervention are available as Supplementary Data.

Discussion

We have presented and applied a guideline to assist professionals working at the science-to-policy interface in developing summaries of systematic reviews to better inform and support decision-making in the agri-food public health sector (Fig. 1). The guideline uses a structured and transparent approach to summarize key systematic review findings and implications in plain language to enhance their interpretation among end-users. In addition, it outlines a rapid review and interview process to supplement these findings with relevant contextual information. Although developed primarily for systematic reviews addressing questions of intervention efficacy, the guideline could be applied to reviews of other topics (e.g., risk factors for infection and disease) with necessary adjustments to the contextual categories. It could also be modified and applied to other published sources of synthesized research knowledge addressing agri-food public health topics, such as risk and exposure assessments (EFSA, 2010; Bucher et al., 2012; Rajić and Young, 2013).

In our application of the guideline to develop summaries for two systematic reviews investigating the efficacy of interventions to control Salmonella in broiler chickens, we found that the majority of relevant contextual information was obtained from interviews with topic specialists (Table 1). However, the overall utility of each information source depended largely on the contextual category and intervention type being investigated. For example, while there are some Salmonella vaccines available and approved for use in the Canadian poultry industry, they are still not widely used by broiler chicken producers (Keery, 2010; Young et al., 2010). In contrast, feed and water additives are more common within the industry (Young et al., 2010; Totton et al., 2012a), and as a result, contextual information for this intervention was more readily available and accessible throughout peer-reviewed and gray literature sources.

We were able to develop the one-page summaries more rapidly than the three-page formats due to the incorporation of supporting contextual information in the latter. As a result, we estimate that in practice the one-page summaries could be developed by trained science-to-policy professionals within 3–5 days, depending on the complexity of the topic, competing priorities, need for translation into other languages, and the extent of end-user evaluation and editing, revisions, and formatting required, while the three-page summaries would require more time (e.g., 2–4 weeks) and resources to develop. For these reasons, the one-page summaries are more suited to being developed on a routine basis to inform decision-making, while the three-page summaries could be developed for a limited number of high-priority issues. If additional time and resources were available for summary development, more rigorous systematic review procedures could be used to obtain contextual information and more comprehensive approaches of integrating knowledge from topic specialists could also be explored, such as the Delphi technique and multicriteria decision-analysis (Fazil et al., 2008; United States Environmental Protection Agency and United States Department of Agriculture [USEPA/USDA], 2012).

Essential skills for efficient and effective development of the summaries should include expertise in plain-language writing of scientific information, familiarity with systematic reviews and meta-analysis and how to interpret these methods, and background knowledge of the topic being addressed (Rajić and Young, 2013; Rajić et al., 2013). Additional technical skills in structured rapid review and interview methods are required for development of the three-page summaries. Some graphical design expertise may be needed to format the summaries into a standard template, and translation services will be required in countries or regions with multilingual populations. While these activities could be conducted by sufficiently trained science-to-policy professionals such as research and policy analysts, dedicated knowledge brokers with both research and policy expertise are uniquely equipped to facilitate this process (Ward et al., 2009; Rajić et al., 2013). Future research should explore the potential utility of knowledge brokers to support the development of research summaries in this sector.

The guideline presented in this article could be used to incorporate one- and three-page summaries into future systematic reviews of agri-food public health topics to enhance knowledge transfer and exchange of the review findings (Rajić and Young, 2013; Rajić et al., 2013). For example, researchers are increasingly required by funding and commissioning agencies to disseminate their findings in formats that can reach a wider range of targeted end-users (Rajić and Young, 2013; Rajić et al., 2013). End-users could use the summaries for a variety of purposes to better inform decision-making. For example, research and policy staff could use the summaries to develop and enhance policy briefings, while extension personnel and practitioners could use the summaries to inform their practice and guidance to industry and other stakeholders.

We identified some challenges in our application of the summary development guideline in this study, mainly in the acquisition of pertinent contextual information. Knowledge gaps existed in the peer-reviewed and gray literature for the contextual categories of interest (e.g., intervention cost-effectiveness and stakeholder acceptability). This makes selection of topic specialists for interviews a critical activity. We suggest that those developing future summaries aim to interview a sufficiently diverse group of topic specialists with different backgrounds, roles, and affiliations (e.g., government, academia, and industry) and in relevant geographic regions to incorporate a range of opinions and perspectives on the issue (USEPA/USDA, 2012). We encourage professionals to use and adapt the guideline to suit their context, which may include modified emphasis on different methods of collecting contextual information, depending on the complexity and polarization of the topic, availability of research, and timelines. For example, in situations where an urgent response is required, the rapid review process may not be as efficient or feasible as interviewing key topic specialists.

One of the limitations of this study is that we used only one database to search for peer-reviewed literature and we used targeted and focused search algorithms. Therefore, it is possible that we could have missed some relevant literature. In addition, only one reviewer screened and extracted relevant information from gray literature documents. We were also not able to assess the risk of bias of relevant articles used to provide contextual information for the three-page summaries due to the wide and diverse nature of included information sources. However, despite these limitations, we are confident that the process yielded accurate information for summary development, as we cross-verified the included information with topic specialists.

Another limitation of this study was the small number of end-users who evaluated the summaries. However, a complementary survey was conducted by Pham et al. in 2012 to determine the perceived utility of these and other summary formats among policy-makers and policy analysts and advisors in the Canadian agri-food public health sector. The authors found that the three-page summary format with supporting contextual information was the most preferred source of systematic review knowledge to inform policy-making (49% of respondents) as compared to the one-page summaries (23%), the full journal article (15%), and a quantitative “summary-of-findings” table format (13%) (Pham et al., 2013). These results correspond to similar findings in the health sector, indicating that end-users prefer to receive systematic review knowledge in a contextualized plain-language format (Lavis et al., 2005; Rosenbaum et al., 2011; Opiyo et al., 2013). Additional research is necessary to evaluate the utility of these summary formats further among different populations of end-users in this sector.

Since conducting this study, we have developed further one- and three-page summaries for other systematic reviews, including other interventions (e.g., competitive exclusion, biosecurity) against Salmonella in broiler chickens and interventions against E. coli during beef processing (Greig et al., 2012; Totton et al., 2012b; Kerr et al., 2013). Copies of these summaries can be obtained from the corresponding author upon request. The summary guideline could be applied to any topic relevant to the broad field of agri-food public health, including food safety, veterinary public health, zoonoses, and “One Health.” Further research is needed to adapt and apply this guideline on nonintervention systematic reviews and other research in this sector and to evaluate how best to disseminate and distribute the summaries to end-users.

Conclusions

We developed and applied a guideline for the creation of one- and three-page plain-language and contextual summaries of systematic reviews in the agri-food public health sector. For the topics investigated in this study, interviews with topic specialists were the most useful and efficient source of contextual information to inform the three-page summaries. While the literature searches provided some additional insights, their utility was limited due to research knowledge gaps for relevant contextual factors. The summary development guideline provides a structured and transparent approach to facilitate the uptake and utilization of systematic review evidence in agri-food public health decision-making.

Footnotes

Acknowledgments

We thank Dr. Sheila Keay and Farzeen Foda for assistance developing and editing the summary templates; Dr. Sheila Keay and Robin Bates for assistance gathering contextual information; Dr. Teresa Cereno, Dr. William (Bill) Cox, Dr. Agnes Agunos, Tim Nelson, and Dr. Csaba Varga for their review and evaluation of the summaries; and all individuals who participated in the interviews. Funding for this study was received from the Laboratory for Foodborne Zoonoses, Public Health Agency of Canada, and the University of Guelph and Ontario Ministry of Agriculture and Food/Ministry of Rural Affairs' Agri-Food and Rural Link “Knowledge Translation and Transfer Funding Program.”

Disclosure Statement

No competing financial interests exist.

References

Bucher

, Fazil

, Rajić

, Farrar

, Wills

, McEwen

. Evaluating interventions against Salmonella in broiler chickens: Applying synthesis research in support of quantitative exposure assessment. Epidemiol Infect, 2012; 140:946–950.

Canadian Foundation for Healthcare Improvement. Communication notes: Reader-friendly writing—1:3:25. 2010. Available at: http://www.cfhi-fcass.ca/SearchResultsNews/10-06-01/d497a465-5398-4ec8-addf-d7cbf86b1e43.aspx, accessed July 2, 2014 .

[EFSA] European Food Safety Authority. Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA J, 2010; 8:1637.

Fazil

, Rajić

, Sanchez

, McEwen

. Choices, choices: The application of multi-criteria decision analysis to a food safety decision-making problem. J Food Prot, 2008; 71:2323–2333.

Ganann

, Ciliska

, Thomas

. Expediting systematic reviews: Methods and implications of rapid reviews. Implement Sci, 2010; 5:56.

Greig

, Waddell

, Wilhelm

, Wilkins

, Bucher

, Parker

, Rajić

. The efficacy of interventions applied during primary processing on contamination of beef carcasses with Escherichia coli: A systematic review-meta-analysis of the published research. Food Control, 2012; 27:385–397.

Health Evidence. Briefing note: Decisions, rationale and key findings summary. 2013. Available at: http://www.healthevidence.org/documents/practice-tools/HETools_BriefingNote_18.Mar.2013.doc, accessed July 2, 2014 .

Higgins

JPT

, Green

(eds.). Cochrane Handbook for Systematic Reviews of Interventions: Version 5.1.0. The Cochrane Collaboration, 2011. Available at: http://www.cochrane-handbook.org, accessed October 19, 2014 .

Keery

. Salmonella Enteritidis control programs in the Canadian poultry industry. 2010. Available at: http://www.agf.gov.bc.ca/lhmr/pubs/se_control_programs0910.pdf, accessed July 2, 2014 .

10.

Kerr

, Farrar

, Waddell

, Wilkins

, Wilhelm

, Bucher

, Wills

, Bailey

, Varga

, McEwen

, Rajić

. A systematic review-meta-analysis and meta-regression on the effect of selected competitive exclusion products on Salmonella spp. prevalence and concentration in broiler chickens. Prev Vet Med, 2013; 111:112–125.

11.

Khangura

, Konnyu

, Cushman

, Grimshaw

, Moher

. Evidence summaries: The evolution of a rapid review approach. Syst Rev, 2012; 1:10.

12.

Lavis

. How can we support the use of systematic reviews in policymaking?. PLoS Med, 2009; 6:e1000141.

13.

Lavis

, Davies

, Oxman

, Denis

, Golden-Biddle

, Ferlie

. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy, 2005; 10(Suppl 1):35–48.

14.

Opiyo

, Shepperd

, Musila

, Allen

, Nyamai

, Fretheim

, English

. Comparison of alternative evidence summary and presentation formats in clinical guideline development: A mixed-method study. PLoS ONE, 2013; 8:e55067.

15.

Pham

, Rajić

, Young

, Sargeant

, Papadopoulos

, McEwen

. The utility of systematic reviews for informing agri-food public health policy: A survey of Canadian policy makers. 21st Cochrane Colloquium 2013, September 19–23, Quebec City, Canada, P2.074.

16.

Rajić

, Young

. Knowledge synthesis, transfer and exchange in agri-food public health: A handbook for science-to-policy professionals. 2013. Available at: https://atrium.lib.uoguelph.ca/xmlui/handle/10214/7293, accessed July 2, 2014 .

17.

Rajić

, Young

, McEwen

. Improving the utilization of research knowledge in agri-food public health: A mixed-method review of knowledge translation and transfer. Foodborne Pathog Dis, 2013; 10:397–412.

18.

ResearchImpact. Knowledge mobilization is turning research into action. 2014. Available at: http://www.researchimpact.ca/home/, accessed July 2, 2014 .

19.

Rosenbaum

, Glenton

, Wiysonge

, Abalos

, Mignini

, Young

, Althabe

, Ciapponi

, Marti

, Meng

, Wang

, la Hoz Bradford

, Kiwanuka

, Rutebemberwa

, Pariyo

, Flottorp

, Oxman

. Evidence summaries tailored to health policy-makers in low- and middle-income countries. Bull World Health Organ, 2011; 89:54–61.

20.

Sargeant

, Torrence

, Rajić

, O'Connor

, Williams

. Methodological quality assessment of review articles evaluating interventions to improve microbial food safety. Foodborne Pathog Dis, 2006; 3:447–456.

21.

Totton

, Farrar

, Wilkins

, Bucher

, Waddell

, Wilhelm

, McEwen

, Rajić

. The effectiveness of selected feed and water additives for reducing Salmonella spp. of public health importance in broiler chickens: A systematic review, meta-analysis, and meta-regression approach. Prev Vet Med, 2012a;106:197–213.

22.

Totton

, Farrar

, Wilkins

, Bucher

, Waddell

, Wilhelm

, McEwen

, Rajić

. A systematic review and meta-analysis of the effectiveness of biosecurity and vaccination in reducing Salmonella spp. in broiler chickens. Food Res Int, 2012b;45:617–627.

23.

[USEPA/USDA] United States Environmental Protection Agency and United States Department of Agriculture. Microbial risk assessment guideline: Pathogenic microorganisms with focus on food and water. 2012. Available at: http://www.epa.gov/raf/files/mra-guideline-july-final.pdf, accessed July 2, 2014 .

24.

Waddell

, Rajić

, Sargeant

, Parker

, Deckert

, McEwen

. The methodological soundness of literature reviews addressing three potential zoonotic public health issues. Zoonoses Public Health, 2009; 56:477–489.

25.

Wallace

, Nwosu

, Clarke

. Barriers to the uptake of evidence from systematic reviews and meta-analyses: A systematic review of decision makers' perceptions. BMJ Open, 2012; 2:e00122.

26.

Ward

, House

, Hamer

. Knowledge brokering: The missing link in the evidence to action chain?. Evid Policy, 2009; 5:267–279.

27.

Young

, Gropp

, Pintar

, Waddell

, Marshall

, Thomas

, McEwen

, Rajić

. Experiences and attitudes towards evidence-informed policy-making among research and policy stakeholders in the Canadian agri-food public health sector. Zoonoses Public Health, 2014;doi: 10.1111/zph.12108.

28.

Young

, Rajić

, Letellier

, Cox

, Leslie

, Sanei

, McEwen

. Knowledge and attitudes towards food safety and use of good production practices among Canadian broiler chicken producers. J Food Prot, 2010; 73:1278–1287.

29.

Young

, Waddell

, Sanchez

, Wilhelm

, McEwen

, Rajić

. The application of knowledge synthesis methods in agri-food public health: Recent advancements, challenges and opportunities. Prev Vet Med, 2014; 113:339–355.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.95 MB