Scientific Reporting

Abstract

This article is based on a presentation that was made at the 2014 annual meeting of the editorial board of Health Education & Behavior. The article addresses critical issues related to standards of scientific reporting in journals, including concerns about external and internal validity and reporting bias. It reviews current reporting guidelines, effects of adopting guidelines, and offers suggestions for improving reporting. The evidence about the effects of guideline adoption and implementation is briefly reviewed. Recommendations for adoption and implementation of appropriate guidelines, including considerations for journals, are provided.

Keywords

dissemination and implementation evaluation measurement issues reporting standards research design

There have long been concerns expressed about the quality of research and reporting in health and health-related journals. For example, almost five decades ago, Schor and Karten (1966) reviewed reporting quality in 67 of the most read medical journals. The study identified 12 types of statistical errors, including lack of control groups when necessary, inappropriate use of statistical techniques, improper study designs, unsubstantiated findings, and too much confidence in negative results with small sample sizes (Schor & Karten, 1966). The authors found that only 28% of articles categorized as analytic studies acceptably adhered to quality standards in scientific reporting (Schor & Karten, 1966). More recently, Chan and Altman (2005) reviewed all 519 English and French reports of randomized controlled trials (RCTs) published in PubMed in 2000. They reported that 79% of reports did not describe the method of randomization and 82% did not report if allocation was concealed (Chan & Altman, 2005).

While it may be tempting to dismiss these failures in reporting as simple mistakes or errors, it is also important to recognize that accurate, comprehensive, and transparent reporting of study methods and results is needed in order to assess adherence to research standards and the quality of research being conducted. However, many studies do not report sufficient information to determine if they were conducted correctly. The literature has demonstrated that this problem is common across medicine, public health, clinical psychology, social work, criminology, and other social science disciplines (Grant, Mayo-Wilson, Melendez-Torres, & Montgomery, 2013; Han et al., 2009; Naleppa & Cagle, 2010; Perry, Weisburd, & Hewitt, 2010; Stinson, McGrath, & Yamada, 2003; Torgerson, Torgerson, Birks, & Porthouse, 2005). While limited evidence suggests that reporting quality may have improved somewhat over time (Reveiz et al., 2015), concerted action by the scientific community is needed to facilitate large and sustained improvements in reporting health research (L. Turner, Shamseer, Altman, Schulz, & Moher, 2012).

This article describes some of the more important issues in research reporting that have been documented in the literature, discusses the role and importance of reporting guidelines, provides some evidence for guideline effectiveness, and lists some of the suggestions for journal implementation of guidelines and other approaches to more transparent reporting.

Issues in Research Reporting

Problems in research reporting reflect more than just a failure to report results and poor statistical methodology. Many RCTs, as well as other types of studies, frequently do not provide adequate information on threats to internal validity or the potential for bias. Threats to validity include inadequate randomization and allocation of concealment, no clear definition or explanation of interventions, inappropriate and misinterpreted subgroup analyses, selective reporting of positive findings, underreporting of harms, inadequate reporting of participant characteristics, and poor reporting of intervention confounders (Glasziou, Meats, Heneghan, & Shepperd, 2008; Kasenda et al., 2014; Montgomery, Underhill, Gardner, Operario, & Mayo-Wilson, 2013; Saini et al., 2014).

Poor reporting can lead to serious problems when findings are used to inform future research, policy, and practice. For instance, it hinders critical appraisal of research, which can cause stakeholders to implement irrelevant findings in their professional settings and may undermine subsequent research agendas, practice guidelines, and policy development. Systematic reviewers are not able to include poorly reported research results in their reviews or meta-analyses. Readers may not be able to fully understand the rationale, methods, and applicability of studies and practitioners may not be able to replicate interventions. As a result, the translation of scientific research into practice may be ineffective, the adoption of new practices or continued use of ineffective/harmful practices can occur, and scarce resources for research and interventions may be wasted. As a result, accurately reporting the essential details of research is both a scientific and ethical responsibility (Glasziou et al., 2014).

Internal and External Validity

While transparent reporting of study designs and research results is critical to the scientific enterprise and researchers’ ability to judge study quality, it is also important to view the issue of research reporting within a larger perspective. Much of the concern with poor reporting of scientific findings has focused on internal validity. However, as represented by the work of Green and Glasgow (2006), there is also increasing concern with the lack of information needed to judge the external validity of research (Glasgow, Vogt, & Boles, 1999) and to translate research findings into practice. For example, a 2008 report in the American Journal of Preventive Medicine found that controlled interventions to prevent childhood obesity did not provide sufficient details to allow readers to judge the generalizability of findings (Klesges, Dzewaltowski, & Glasgow, 2008). Of the 24 dimensions of external validity expected in the review, only 4 were reported more than 90% of the time, and 6 of the dimensions were reported less than 10% of the time (Klesges et al., 2008).

Reporting Biases

Increasingly, systematic reviews are used to determine the effectiveness of treatments or to summarize results across different types of interventions (E. Turner, Matthews, Linardatos, Tell, & Rosenthal, 2008). The results of systematic reviews, however, depend on the completeness and transparency of included studies. Publication bias and selective outcome reporting have been documented in cohorts of studies (e.g., registered trials or trials approved by research ethics committees; Dwan et al., 2008; Schmucker et al., 2014). Overall, the selective reporting of study results (and the failure to publish small or nonsignificant results) leads to the overestimation of intervention effects in systematic reviews, a phenomenon called meta-bias (Goodman & Dickersin, 2011). Reporting bias may also be related to funding and sponsorship, such as the influence of tobacco and drug companies on published research results. In a review of drug trials for gabapentin, researchers found that trials with nonsignificant results were not published and the primary outcomes found in company reports with nonsignificant findings did not match the primary outcomes in related publications (Vedula, Bero, Scherer, & Dickersin, 2009; Vedula, Li, & Dickersin, 2013). Academic researchers may also fail to publish nonsignificant results because of intellectual allegiances or a belief that nonsignificant results are uninteresting and will not lead to results being published in high-impact journals.

The selective reporting of research results remains difficult to detect. This can be in the form of reporting only positive findings, post hoc data mining and subgroup analyses, or changing definitions of primary outcomes. In one study of selective reporting bias, for example, the authors reviewed 26 reports on 22 studies for reporting bias including RCTs, cluster RCTs, controlled before and after studies, and 5 uncontrolled before and after studies (Pearson & Peters, 2012). The authors found 19 instances of missing or incompletely reported outcomes with 12 of the 19 leading to a high risk of bias in the study results. Another study reported that up to 50% of RCTs had not been reported in the literature (Scherer, Langenberg, & von Elm, 2005).

Publication bias has long been documented in research funded by the National Institutes of Health (NIH) and remains prevalent even among recent trials that are prospectively registered (Mathieu, Boutron, Moher, Altman, & Ravaud, 2009). Mandatory reporting of conflicts of interest has increased transparency about research funding, but it does not address reporting biases directly.

This issue of mis-, under-, and selective reporting, particularly in randomized trials, has led to an emphasis on registration of RCTs and proposals to begin registering other types of studies. Trial registration encourages clear specification of study design, outcome measures, recruitment and assignment procedures, and analytic approaches including proposed subgroup analyses. Preregistration increases researchers’ ability to identify unreported studies and to compare study methods and results with planned study designs (Dickersin, 1992; Meinert, 1988; Simes, 1986).

Trials can be registered online in 20 minutes (Zarin & Keselman, 2007). Training materials for authors are available from ClinicalTrials.gov (https://clinicaltrials.gov/ct2/manage-recs/present). To make best use of trial registrations, editors can use tool kits available from the Enhancing the QUAlity and Transparency of health Research (EQUATOR) network (http://www.equator-network.org/) to instruct authors and reviewers about their expectations for reporting transparency and completeness. As part of the peer review process, authors should submit relevant reporting checklists and trial registrations, and peer reviewers should check that reports are complete and accurate. Editors and peer reviewers may have the ultimate responsibility to enforce the requirement that reports include descriptions of deviations from the trial registration.

Reporting Guidelines and the Effect of Adoption

In recent years, many public health, medical, and social science researchers have participated in the development of reporting standards for different types of study designs and content areas. These reporting guidelines frequently include a list and definition of required items for reporting (e.g., a checklist and a flow diagram) and have recently included explanation and elaboration documents that provide detailed definitions of terms and examples of good reporting. The proximal goal of reporting guidelines is to improve research transparency, rather than to judge study quality or to prescribe research conduct. Guidelines are tools to help authors, editors, and reviewers communicate the information needed to understand research. As noted in their adoption of the Transparent Reporting of Evaluations With Nonrandomized Designs (TREND) guidelines, the American Journal of Public Health editors stated, “The purpose of guidelines is to improve the transparency, or clarity, of reporting on individual studies and that the TREND checklist was not intended as a criterion for evaluating articles for publication” (Des Jarlais, Lyles, & Crepaz, 2004, p. 362). This was subsequently reiterated in their adoption of other guidelines, including Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Moher, Liberati, Tetzlaff, & Altman, 2009).

Currently, the Consolidated Standards of Reporting Trials (CONSORT) statement is the most widely used reporting guideline. The CONSORT group led the development of reporting guidelines for RCTs in response to a growing recognition of problems with the reporting of RCTs and the desire to improve the quality and transparency of published work (Moher et al., 2009). Two groups of journal editors, methodologists, and specialists in RCTs met in the early 1990’s to develop and publish recommendations on the reporting of RCTs (Begg et al., 1996). The two groups subsequently met to develop a common set of recommendations, the outcome of which was the original CONSORT Statement (Begg et al., 1996; Rennie, 1995). The CONSORT Statement was originally published in 1996, then revised in 2001 and 2010. Extensions of the CONSORT Statement have been developed to offer guidance about reporting RCTs for interventions. An upcoming extension to CONSORT for social and psychological interventions promises to include up-to-date guidance for the reporting of trials in public health (Montgomery, Grant, et al., 2013).

Since the CONSORT Statement was developed in 1996, there have been over 276 guidelines developed for health-related research. Several relatively recent guidelines may be particularly useful for the social and behavioral sciences within public health. Guidelines related to quantitative studies include reporting nonrandomized designs (TREND; Vlahov, 2004), guidelines for reporting systematic reviews (PRISMA; Moher et al., 2009), guidelines for reporting on interventions (Template for Intervention Description and Replication checklist and guide; Hoffmann et al., 2014), guidelines for reporting on observational studies in epidemiology (Strengthening the Reporting of Observational Studies in Epidemiology; Von Elm et al., 2007), and the guidelines under development for the reporting of social and psychological interventions (CONSORT-SPI). In addition, there are three new guidelines for reporting qualitative studies—Standards for Reporting Qualitative Research (SRQR; Tong, Sainsbury, & Craig, 2007), Consolidated Criteria for Reporting Qualitative Research (COREQ; Tong et al., 2007), and Enhancing Transparency in the Synthesis of Qualitative Research (Tong, Flemming, McInnes, Oliver, & Craig, 2012)—which, while not covering all types of qualitative research, have considerable potential for improving reporting. Several quantitative and qualitative guidelines related to social and behavioral sciences within public health are provided in Table 1.

Table 1.

Selected Reporting Guidelines for Research in the Health-Related Social and Behavioral Sciences.

Study design	Guideline	Description
Randomized control trials	CONSORT	Guideline for reporting parallel group randomized trials; includes 25-item checklist and flowchart to help authors prepare reports covering title, abstract, introduction, methods, results, and discussion
Interventions	TIDieR	Extension of CONSORT for describing interventions; includes a 12-item checklist related specifically to who, what, when, and where interventions take place
Nonrandomized evaluations	TREND	Guideline for reporting nonrandomized studies; similar to CONSORT; includes 22-item checklist; specific items include theories used, descriptions of treatment and comparison groups, study design, and methods of adjusting for potential biases
Observational studies	STROBE	Guideline for reporting observational studies including specific guidance for three main study designs: cohort, case-control, and cross-sectional; includes 22-item checklist covering the manuscript title, abstract, introduction, method, result, and discussion sections of articles
Systematic reviews and meta-analyses	PRISMA	Guideline for reporting of systematic reviews and meta-analyses; includes 27-item checklist and flow diagram covering title, abstract, methods, results, discussion and funding
Qualitative studies	COREQ	Guideline for reporting qualitative studies, specifically interviews and focus groups; includes 32-item checklist related to the research team, study methods, context of the study, findings, analysis, and interpretation
	SRQR	Guideline for reporting qualitative research; includes a 21-item checklist covering title, abstract, introduction, methods, results/findings, and discussion
	ENTREQ	Guideline for reporting syntheses of qualitative research studies; includes 21-item checklist covering introduction, methods and methodology, literature search and selection, appraisal, and synthesis of findings

Note. CONSORT = Consolidated Standards of Reporting Trials; TIDieR = Template for Intervention Description and Replication; TREND = Transparent Reporting of Evaluations With Nonrandomized Designs; STROBE = Strengthening the Reporting of Observational Studies in Epidemiology; PRISMA = Preferred Reporting Items for Systematic Reviews and Meta-Analyses; COREQ = Consolidated Criteria for Reporting Qualitative Research; SRQR = Standards for Reporting Qualitative Research; ENTREQ = Enhancing Transparency in Reporting the Synthesis of Qualitative Research. Citations for each guideline are provided in the article narrative.

In addition to guidelines for writing research reports, advocates of research transparency have developed guidelines that go beyond the traditional journal article to include prospective registration (i.e., of studies and analytic plans) and the sharing of study materials, data, and analytic code (Nosek, Spies, & Motyl, 2012). To emphasize methods rather than results, several journals now review preanalysis plans and agree to publish studies (or not) based on their design rather than their outcomes (https://osf.io/8mpji/wiki/home/). NIH has proposed to require all NIH-funded clinical trials be preregistered and post their results, and the International Committee of Medical Journal Editors will soon require posting of individual participant data following publication of clinical trials (NIH, 2014). Increasingly, reporting guidelines are one part of a much larger effort to increase scientific transparency and to improve the usability of research.

There is accumulating evidence that the adoption and implementation of guidelines, at least with regard to CONSORT, are associated with improved quality of research reporting within refereed journals in the health and public health fields (Moher, Schulz, Simera, & Altman, 2010). However, as with any behavioral intervention, the effectiveness of reporting guidelines depends on the extent to which the standards are adequately implemented by authors, journal editors, and peer reviews. In a recent systematic review of the completeness of reporting by journals adopting CONSORT, there were statistically significant associations between completeness of published reports and CONSORT endorsement by journals (L. Turner et al., 2012), though reporting quality was still bad overall.

A recent systematic review of the effects of journals endorsing guidelines, other than CONSORT, found that only 9 of 101 guidelines had been evaluated for effectiveness and that insufficient evidence exists for reporting guidelines on completeness of reporting (Stevens et al., 2014). The authors noted,

No standard way exists in which journals endorse reporting guidelines. . . . Furthermore, other than including recom-mendations in their “Instructions to Authors” little is known about what else is being done by individual journals to ensure adherence to the guidelines. (Stevens et al., 2014, p. 5)

Journals vary considerably in whether they refer authors to guidelines, require the submission of flow diagrams and other documentation, and ask editors and peer reviewers to monitor compliance with guidelines. Obviously, the effectiveness of guidelines will be related to the extent to which they are used by authors and enforced by journals. There is some evidence that active implementation is associated with higher quality reports (Pandis, Shamseer, Kokich, Fleming, & Moher, 2014).

Reporting Guidelines in Journals

Reporting in research is a critical part of dissemination. Reporting allows best practices to be shared, knowledge to be generated, and research to be replicated. Both scientists and participants in research put themselves at risk and accept delays in treatment to generate information that will help other people and advance science. To fulfill ethical obligations, researchers must not only collect data but journals must also disseminate information accurately and completely (Strom, Buyse, Hughes, & Knoppers, 2014). Research requires considerable human and economic resources. When research is not disseminated, these resources are wasted, evidence that could be used to inform policy and practice goes unused, and studies are repeated unnecessarily (Chan et al., 2014; Macleod et al., 2014).

Requiring trial and study registration may require some changes in how journals review and report studies so that reviewers, editors, and readers can access the original study designs and compare them with what is being reported. International Committee of Medical Journal Editors has required prospective trial registration as a condition for publication since 2004 (De Angelis et al., 2005); this has not yet solved the problem of reporting bias in its member journals, but it has increased the ability to detect bias.

Of course, much of the discussion on under reporting also requires that journals pay close attention to their obligations and ethical responsibilities in publishing based on its quality and clinical importance rather than the magnitude or significance of results. There is evidence that journals tend to reject null study findings, thus selecting what is published based on the study results (Ioannidis, 2014). Furthermore, authors themselves fail to submit manuscripts with small and nonsignificant results and most commonly cite “lack of time” as the reason for nonpublication (Dickersin, Min, & Meinert, 1992; Scherer, Ugarte-Gil, Schmucker, & Meerpohl, 2015). In addition to technical solutions, overcoming reporting biases will require a cultural shift in our attitudes about the importance and dissemination of research, particularly results with null findings.

The EQUATOR Network

The EQUATOR network works to improve reporting in health-related literature by promoting reporting guidelines (Simera, Moher, Hirst, et al., 2010). The network consists of researchers, journal editors, peer reviewers, and funding bodies including a number of the representatives who helped develop the CONSORT guidelines, as well as the Association of Editors of Public Health and Medicine Journals.

The EQUATOR network encourages journals to not only adopt specific reporting standards but also implement them in a systematic way. They have published Guidelines for Reporting Health Research: How to Promote Their Use in Your Journal, and the EQUATOR website includes tool kits for introducing guidelines into the journal, selecting reporting guidelines, and suggesting how and when to use reporting guidelines in a journal (http://www.equator-network.org/toolkits/). Of particular importance are EQUATOR’s suggestions:

Referring to the EQUATOR network in instructions to authors

Alerting authors to selected reporting guidelines and motivating their use through clearly describing the purpose of guidelines, linking to appropriate guidelines, and providing clear instruction on how authors are expected to use appropriate guidelines, including statements on expected degree of compliance

Considering how to best remind authors about reporting guidelines in the manuscript submission process

Alerting peer reviewers about the importance of good reporting and the use of guidelines during their reviews

Sharing knowledge about reporting guidelines with colleagues

Writing an editorial support transparent research reporting and supporting the use of selected guidelines

Considerations

In deciding to adopt reporting guidelines, journals should consider the quality of the available guidelines and their ability to implement those guidelines effectively. Once editors determine which guidelines are appropriate for the types of studies reported in a journal, they must ask the following questions:

Are the items in the guidelines clear and evidence-based?

Were guidelines developed transparently?

Were guidelines developed following best practices (e.g., a systematic review of existing guidelines and a consensus process)?

Do guidelines include adequate documentation to use them effectively (e.g., an “Explanation and Elabo-ration” document)?

Are the guidelines up-to-date?

Are there plans to review the guidelines regularly and to keep them up-to-date?

Have the guidelines been evaluated and shown to be effective?

Growing attention to guidelines for reporting completed research has led to initiatives to develop guidelines for other study stages. For instance, researchers have recently created guidance for improving the content, quality, and comprehensive reporting of study protocols. For example, the SPIRIT 2013 statement recommends items of information to include in protocols for clinical trials (Chan et al., 2013), whereas the PRISMA–Protocol 2015 statement recommends items for systematic review protocols (Shamseer et al., 2015). The recent Transparency and Openness Promotion (TOP) Guidelines provide concrete, actionable strategies for journals to implement in order to enhance the transparency of the research that they publish (Nosek et al., 2015). Practices recommended in TOP to improve the transparency of the research process and its products include sharing data, analytic codes, and study materials, as well as the preregistration of study protocols and analysis plans. Taken together, guidelines related to research conduct are inextricable with guidelines for publishing research products in efforts to improve the utility of research publications.

Discussion

This article argues for the importance of adopting reporting standards and implementing reporting guidelines to reduce the long-standing threats to the credibility of health research posed by poor reporting of studies. The majority of health research is reported incompletely, leading to waste in both the time and resources invested in conducting research (Glasziou et al., 2014). Unbiased and usable research reports are crucial components in increasing the value of and reducing waste in health research (Macleod et al., 2014). Reporting guidelines facilitate more useful research reports by collating reporting standards for a particular study design or disciplinary area.

Reporting guidelines advise authors on the information required in research reports to provide a transparent account of the design, conduct, analysis, and interpretation of a study (Simera, Moher, Hirst, et al., 2010). Reporting guidelines for the most commonly used study designs in health-related research include the CONSORT Statement for RCTs (Schulz, Altman, & Moher, 2010), the PRISMA Statement for systematic reviews and meta-analyses (Moher et al., 2009), the TREND statement for nonrandomized study designs (Vlahov, 2004), COREQ for qualitative studies (Tong et al., 2007), SRQR (O’Brien, Harris, Beckman, Reed, & Cook, 2014), and COREQ (Tong et al., 2007). Authors should consult the EQUATOR Network’s catalogue of reporting guidelines for health research to ensure they are using the most appropriate and updated reporting guideline in each instance (Simera, Moher, Hoey, Schulz, & Altman, 2010).

Various stakeholders can benefit from the adoption of reporting guidelines in their work in five major ways (Shamseer et al., 2015). First, researchers can use reporting guidelines at the outset of study design to ensure they consider the essential study aspects that they will have to describe in a study report. Use of reporting guidelines throughout the study (from design to reporting) will improve the accuracy, completeness, and transparency of final manuscript content. Second, journal editors implementing reporting guidelines will improve the manuscripts that they publish. Journals endorsing reporting guidelines should enforce policies (rather than only providing nominal support) to ensure that guidelines are actually used by authors, editors, and peer reviewers (Stevens et al., 2014). Third, research funders adopting reporting guidelines will increase the quality of submitted applications and thus facilitate the commissioning of the most important and rigorous research. Fourth, policy makers and practitioners encouraging the use of reporting guidelines by researchers could lead to higher quality publications that these stakeholders can then use to appraise study rigor, apply findings to study populations and settings of interest, and replicate study procedures such as intervention implementation. Finally, faculty and students could use reporting guidelines as a means to improve the next generation of researchers’ understanding of research methods.

Reporting guidelines are part of a growing and lively movement to conduct “research on research” and study behaviors involved with scientific publication (Ioannidis, 2014; Moher & Srivastava, 2015). Stakeholders in health education and behavior should look to be involved in these efforts aimed at improving the transparency. For instance, authors of this article are involved with developing an extension of the CONSORT Statement for the reporting of trials of social and psychological interventions, known as CONSORT-SPI (Grant et al., 2013; Mayo-Wilson et al., 2013; Montgomery, Grant, et al., 2013). Given the pertinence of CONSORT-SPI to interventions evaluated in Health Education & Behavior, the readership is welcome to get involved in its dissemination and implementation. In addition, readers can look to implement the TOP Guidelines in their own research and recommend to colleagues that they do the same (Nosek et al., 2015). Such efforts to increase research transparency comprise a critical element in improving the quality and utility of the scientific enterprise.

Conclusion

For nearly 20 years, medical researchers have made a concerted effort to improve the reporting of health research by developing explicit standards for transparently describing the essential features of research studies. These reporting standards are not requirements that determine the format for writing or designing a study but rather are recommendations on the content that study reports should consistently and transparently include to allow for critical appraisal. In other words, standards do not prescribe research conduct but suggest the items of information that are necessary to understand how a study was conducted. While researchers may be aware of the important components of a study, reporting standards serve as a safeguard to prevent seemingly mundane yet extremely important information from being overlooked or forgotten. Adherence to these standards helps ensure that research can effectively inform practice and policy accountable to robust evidence.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Begg

Cho

Eastwood

Horton

Moher

Olkin

. . . Stroup

(1996). Improving the quality of reporting of randomized controlled trials: The CONSORT statement. Journal of the American Medical Association, 276, 637-639.

Chan

Altman

(2005). Identifying outcome reporting bias in randomised trials on PubMed: Review of publications and survey of authors. British Medical Journal, 330(7494), 753.

Chan

Song

Vickers

Jefferson

Dickersin

Gøtzsche

. . . Van der Worp

(2014). Increasing value and reducing waste: Addressing inaccessible research. The Lancet, 383(9913), 257-266.

Chan

Tetzlaff

Gøtzsche

Altman

Mann

Berlin

. . . Moher

(2013). SPIRIT 2013 explanation and elaboration: Guidance for protocols of clinical trials. British Medical Journal, 346, e7586.

De Angelis

Drazen

Frizelle

Haug

Hoey

Horton

. . . Weyden

. (2005). Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors. New England Journal of Medicine, 352, 2436-2438.

Des Jarlais

D. C.

Lyles

Crepaz

. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND statement. American Journal of Public Health, 94, 361-366.

Dickersin

(1992). Why register clinical trials? Revisited. Controlled Clinical Trials, 13, 170-177.

Dickersin

Min

Meinert

(1992). Factors influencing publication of research results: Follow-up of applications submitted to two institutional review boards. Journal of the American Medical Association, 267, 374-378.

Dwan

Altman

Arnaiz

Bloom

Chan

Cronin

. . . Williamson

(2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One, 3(8), e3081.

10.

Glasgow

Vogt

Boles

(1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322-1327.

11.

Glasziou

Altman

D. G.

Bossuyt

Boutron

Clarke

Julious

. . . Wager

(2014). Reducing waste from incomplete or unusable reports of biomedical research. The Lancet, 383(9913), 267-276.

12.

Glasziou

Meats

Heneghan

Shepperd

(2008). What is missing from descriptions of treatment in trials and reviews? British Medical Journal, 336(7659), 1472.

13.

Goodman

Dickersin

(2011). Metabias: A challenge for comparative effectiveness research. Annals of Internal Medicine, 155, 61-62.

14.

Grant

Mayo-Wilson

Melendez-Torres

Montgomery

(2013). Reporting quality of social and psychological intervention trials: A systematic review of reporting guidelines and trial publications. PLoS One, 8(5), e65442.

15.

Green

Glasgow

(2006). Evaluating the relevance, generalization, and applicability of research issues in external validation and translation methodology. Evaluation & the Health Professions, 29, 126-153.

16.

Han

Kwak

Marks

Pae

Bhatia

. . . Patkar

(2009). The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry. Contemporary Clinical Trials, 30, 116-122.

17.

Hoffmann

Glasziou

Boutron

Milne

Perera

Moher

. . . Michie

(2014). Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. British Medical Journal, 348, g1687.

18.

Ioannidis

(2014). How to make more published research true. PLoS Medicine, 11(10), e1001747.

19.

Kasenda

Schandelmaier

Sun

von Elm

You

Blümle

. . . Briel

(2014). Subgroup analyses in randomised controlled trials: Cohort study on trial protocols and journal publications. British Medical Journal, 349, g4539.

20.

Klesges

Dzewaltowski

Glasgow

(2008). Review of external validity reporting in childhood obesity prevention research. American Journal of Preventive Medicine, 34, 216-223.

21.

Macleod

Michie

Roberts

Dirnagl

Chalmers

Ioannidis

J. P.

. . . Glasziou

(2014). Biomedical research: Increasing value, reducing waste. The Lancet, 383(9912), 101-104.

22.

Mathieu

Boutron

Moher

Altman

D. G.

Ravaud

(2009). Comparison of registered and published primary outcomes in randomized controlled trials. Journal of the American Medical Association, 302, 977-984.

23.

Mayo-Wilson

Grant

Hopewell

Macdonald

Moher

Montgomery

(2013). Developing a reporting guideline for social and psychological intervention trials. Trials, 14(1), 242.

24.

Meinert

C. L.

(1988). Toward prospective registration of clinical trials. Controlled Clinical Trials, 9, 1-5.

25.

Moher

Liberati

Tetzlaff

Altman

(2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151, 264-269.

26.

Moher

Schulz

Simera

Altman

(2010). Guidance for developers of health research reporting guidelines. PLoS Medicine, 7, 2.

27.

Moher

Srivastava

(2015). You are invited to submit . . . BMC Medicine, 13, 180.

28.

Montgomery

Grant

Hopewell

Macdonald

Moher

Michie

Mayo-Wilson

(2013). Protocol for CONSORT-SPI: An extension for social and psychological interventions. Implementation Science, 8(1), 99.

29.

Montgomery

Underhill

Gardner

Operario

Mayo-Wilson

(2013). The oxford implementation index: A new tool for incorporating implementation data into systematic reviews and meta-analyses. Journal of Clinical Epidemiology, 66, 874-882.

30.

Naleppa

Cagle

(2010). Treatment fidelity in social work intervention research: A review of published studies. Research on Social Work Practice, 20, 674-681.

31.

National Institutes of Health. (2014). NIH request for public comments on the draft NIH policy on dissemination of NIH-funded clinical trial information. Retrieved from grants.nih.gov/grants/guide/notice-files/NOT-OD-15-019.html

32.

Nosek

Alter

Banks

Borsboom

Bowman

Breckler

. . . Yarkoni

(2015). Promoting an open research culture. Science, 348(6242), 1422-1425.

33.

Nosek

Spies

J. R.

Motyl

(2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7, 615-631. doi:10.1177/1745691612459058

34.

O’Brien

Harris

Beckman

Reed

Cook

(2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89, 1245-1251.

35.

Pandis

Shamseer

Kokich

V. G.

Fleming

P. S.

Moher

(2014). Active implementation strategy of CONSORT adherence by a dental specialty journal improved randomized clinical trial reporting. Journal of Clinical Epidemiology, 67(9), 1044-1048. doi:10.1016/j.jclinepi.2014.04.001

36.

Pearson

Peters

(2012). Outcome reporting bias in evaluations of public health interventions: Evidence of impact and the potential role of a study register. Journal of Epidemiology & Community Health, 66, 286-289.

37.

Perry

Weisburd

Hewitt

(2010). Are criminologists describing randomized controlled trials in ways that allow us to assess them? Findings from a sample of crime and justice trials. Journal of Experimental Criminology, 6, 245-262.

38.

Rennie

(1995). Reporting randomized controlled trials: An experiment and a call for responses from readers. Journal of the American Medical Association, 273, 1054-1055.

39.

Reveiz

Chapman

Asial

Munoz

Bonfill

Alonso-Coello

(2015). Risk of bias of randomized trials over time. Journal of Clinical Epidemiology, 68, 1036-1045.

40.

Saini

Loke

Y. K.

Gamble

Altman

Williamson

Kirkham

(2014). Selective reporting bias of harm outcomes within studies: Findings from a cohort of systematic reviews. British Medical Journal, 349, g6501.

41.

Scherer

Langenberg

von Elm

(2005). Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews, (2), MR000005.

42.

Scherer

Ugarte-Gil

Schmucker

Meerpohl

(2015). Authors report lack of time as main reason for unpublished research presented at biomedical conferences: A systematic review. Journal of Clinical Epidemiology, 68, 803-810.

43.

Schmucker

Schell

L. K.

Portalupi

Oeller

Cabrera

Bassler

. . . Meerpohl

J. J.

(2014). Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS ONE, 9(12), e114023.

44.

Schor

Karten

(1966). Statistical evaluation of medical journal manuscripts. JAMA, 195, 1123-1128.

45.

Schulz

Altman

Moher

(2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340, c332.

46.

Shamseer

Moher

Clarke

Ghersi

Liberati

Petticrew

. . . Stewart

L. A.

(2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. British Medical Journal, 349, g7647.

47.

Simera

Moher

Hirst

Hoey

Schulz

K. F.

Altman

D. G.

(2010). Transparent and accurate reporting increases reliability, utility, and impact of your research: Reporting guidelines and the EQUATOR Network. BMC Medicine, 8, 24. doi:10.1186/1741-7015-8-24

48.

Simera

Moher

Hoey

Schulz

K. F.

Altman

(2010). A catalogue of reporting guidelines for health research. European Journal of Clinical Investigation, 40, 35-53.

49.

Simes

R. J.

(1986). Publication bias: The case for an international registry of clinical trials. Journal of Clinical Oncology, 4, 1529-1541.

50.

Stevens

Shamseer

Weinstein

Yazdi

Turner

Thielman

. . . Moher

(2014). Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: Systematic review. British Medical Journal, 348, g3804.

51.

Stinson

J. N.

McGrath

P. J.

Yamada

J. T.

(2003). Clinical trials in the Journal of Pediatric Psychology: Applying the CONSORT statement. Journal of Pediatric Psychology, 28(3), 159-167. doi:10.1093/jpepsy/jsg001

52.

Strom

Buyse

Hughes

Knoppers

(2014). Data sharing, Year 1—access to data from industry-sponsored clinical trials. New England Journal of Medicine, 371, 2052-2054.

53.

Tong

Flemming

McInnes

Oliver

Craig

(2012). Enhancing Transparency in Reporting the Synthesis of Qualitative Research: ENTREQ. BMC Medical Research Methodology, 12(1), 181.

54.

Tong

Sainsbury

Craig

(2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19, 349-357.

55.

Torgerson

Birks

Porthouse

(2005). A comparison of RCTs in health and education. British Educational Research Journal, 31, 761-785.

56.

Turner

Matthews

Linardatos

Tell

Rosenthal

(2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine, 358, 252-260.

57.

Turner

Shamseer

Altman

D. G.

Schulz

K. F.

Moher

(2012). Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Systematic Reviews, 1(1), 60. doi:10.1186/2046-4053-1-60

58.

Vedula

Bero

Scherer

Dickersin

(2009). Outcome reporting in industry-sponsored trials of gabapentin for off-label use. New England Journal of Medicine, 361, 1963-1971.

59.

Vedula

Dickersin

(2013). Differences in reporting of analyses in internal company documents versus published trial reports: Comparisons in industry-sponsored trials in off-label uses of gabapentin. PLoS Medicine, 10(1), e1001378.

60.

Vlahov

(2004). Transparent Reporting of Evaluations With Nonrandomized Designs (TREND). Journal of Urban Health, 81, 163-164.

61.

Von Elm

Altman

D. G.

Egger

Pocock

S. J.

Gøtzsche

P. C.

Vandenbroucke

J. P.

, & Strobe Initiative. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Preventive Medicine, 45, 247-251.

62.

Zarin

Keselman

(2007). Registering a clinical trial in ClinicalTrials.gov. Chest, 131, 909-912.