Abstract
BACKGROUND:
The distortion of truth or “spin” has become a major concern in various fields as it can affect the accuracy of research dissemination. Recognizing the extent of this issue in physiotherapy research is essential for promoting good-quality research and raising awareness.
OBJECTIVE:
This study aimed to determine the prevalence of spin in randomized controlled trials (RCTs) published by Indian physiotherapists.
METHODS:
A Medline search was conducted to identify RCTs published by Indian physiotherapists from January 2000 to November 2018. We included studies with nonsignificant primary outcomes and reviewed them for spin in both abstracts and full texts using the Boutron checklist. Three reviewers were involved in the process.
RESULTS:
Of the 44 RCTs with nonsignificant outcomes, 40 (90%) showed spin in their abstracts and 39 (88%) in their main text conclusions.
CONCLUSIONS:
Our findings reveal a significant prevalence of spin in nonsignificant RCTs published by physiotherapy researchers. Improving research knowledge, enhancing scientific writing skills, and providing mentorship may help reduce spin in result reporting.
Keywords
Introduction
Spin is the practice of specific reporting strategies to highlight that the experimental treatment is beneficial if the results are insignificant [1, 2]. Authors intentionally or unintentionally deceive nonsignificant effects as valuable by focusing on secondary outcomes, omitting data within-group analysis, or even using the language used to interpret the information. As a result of such inappropriate interpretations, healthcare practitioners deliver non-beneficial or harmful interventions to the patients by reading spun findings [3–5]. Spin may remain subtle and unnoticed by novice readers, even sometimes unseen by authors of systematic reviews and clinical practice guidelines [6]. Furthermore, this might lead to science hype where researchers will inappropriately overstate unproven treatment, and commercialization of such interventions could result in economic loss [7].
During the past decade, spin in published research reports has been extensively researched and known to be highly prevalent in many disciplines of biomedical research [3, 8–12]. Presence of spin in biomedical literature is associated with numerous factors. Grants and funding from industries seem to influence the interpretation of nonsignificant results in the funder’s favor [13]. In physiotherapy, a study analyzed the cohort of low back pain trials abstracts revealed that there is a consistent amount of misinterpretation in results and conclusion [14]. Lack of endorsement of reporting guidelines, journal impact factors, and reduced word count in abstracts are some aspects reported being related with spin in physiotherapy [15]. In addition to these factors, researchers’ lack of research expertise and experience, especially from low and middle-income countries, might also contribute to an inappropriate or misleading interpretation of results. However, few studies have systematically identified and evaluated the influence of researchers’ expertise on the prevalence of spin [16].
Research in the field of physiotherapy has been well developed around the world. There is a significant increase in the number of articles published by physiotherapists in recent years [17]. Novice researchers often undergo limited and substandard research training and do not have access to experienced research mentors. They compensate for lack of expertise and mentorship in writing research papers by emulating reporting strategies commonly adopted in published studies. Hence, there is a need to analyze spin in nonsignificant trials published where there are insufficient mentors to guide researchers [18]. Also, good quality randomized controlled trials with appropriate interpretation and reporting of the results pertaining to each country is vital because it enhances generalizability of results across the local population. Further, physical therapy researchers might benefit from a report that analyzes misinterpretation of study results done. Therefore, we aimed to estimate the prevalence of spin in randomized control trials (RCT) published by physiotherapists from India. We focused on all the sections of the article from abstract to main text to analyze the level of spin present in each article.
Methods
Prevalence of spin in abstracts and main texts
A sample of randomized study reports published in Medline-indexed journals in the field of physiotherapy was selected to estimate the prevalence of spin in abstracts and main texts. The selection was based on the popularity of PubMed among clinicians and novice researchers. The prevalence of studies having spin in abstracts and main texts were estimated in percentage from the selected sample of reports.
Search strategy
A MEDLINE search was conducted on 5th December 2018 to identify randomized controlled trials published by Indian physiotherapists between January 2000 and November 2018. The search was performed using the keywords “India OR Indian AND physiotherapy OR physical therapy” with filtered only RCTs. The initial screening was done by one independent author (NK) by going through the title, abstract, and authorship. Full texts of articles were retrieved if necessary. The search identified 2953 articles, of which 2439 were excluded for being out of the scope of physiotherapy and having foreign affiliation. After the initial screening, eligible studies were parallel-group RCTs published by having a statistical nonsignificant primary outcome (P > 0.05). Studies with more than two groups, cross-over, cluster trials, and protocols were excluded. Among 514 articles after initial screening, 376 articles were excluded for not being a clinical trial or had a significant improvement in primary outcomes (Fig. 1). A second independent author (SK) assessed all the screened reports to ensure quality, and disagreements were discussed and resolved. Full-text articles were retrieved for all the included articles and processed for data extraction.
Review flowchart.
The strategies of spin were scored using the pre-tested and standardized form defined by Boutron and colleagues. A difference in the primary outcome greater than 5% was considered statistically nonsignificant. Authors may selectively report the study by focusing on secondary outcomes if there is no significant change in the primary outcome but positive changes in the secondary outcomes. Results can also be misrepresented by focusing on within-group changes rather than between-group changes. When there is no significant difference between the groups, authors report both treatments as equally effective rather than reporting it as “no difference between the groups". The detailed content of the checklist for strategies of spin is given in Table 1. Presence of spin in all sections of the full text was identified by manually reading through the article and marked as 1,0, denoting yes and no for all strategies in the list. Two authors, NK & SK, independently trained themselves on the methodology of coding spin using the adopted score. A sample of two reports from included articles were coded independently by both authors, and the results were similar.
Summary of study characteristics
Summary of study characteristics
The presence of spin in abstract conclusion was classified as high, moderate, and low. High spin was considered as claiming efficacy for nonsignificant primary outcome measure and recommendation of treatment in clinical practice. Moderate spin was considered as claiming efficacy for nonsignificant primary outcome measure. Low spin was considered as uncertainty in acknowledging nonsignificant primary measure outcome and ambiguous about the treatment effects for nonsignificant outcome.
Data extraction
Data from each included article were extracted independently by NK and SK and recorded in an Excel sheet. Disagreements were resolved through discussion with the third author, HK. Once data extraction was complete, HK reviewed all the included studies along with the checklists completed by NK and SK. General characteristics of each study, such as the number of authors, funding sources, impact factor 2018, PEDro score, therapeutic intervention, comparator, sample size, and clinical trial registration, were extracted. After coding the included studies using the predefined checklist, descriptive analysis was performed. The frequency and percentage of each strategy of spin present in the included articles were reported. The level of spin was estimated in three categories by percentage.
Results
Characteristics of included studies
Out of 138 studies identified, 44 studies met the inclusion criteria and were included for review. The reasons for exclusion of 94 studies are presented in Fig. 1. Table 1 summarizes the general characteristics of the included studies, including the number of authors, funding sources, impact factor 2018, PEDro score, therapeutic intervention, comparator, sample size, and clinical trial registration.
Spin in included studies
From the representative sample of included studies, spin was identified in 90% (n = 40) of the included RCTs. The frequency of spin was assessed in the abstract, results section, and conclusion section of the studies. Among the 44 included studies, 32 (72%) had spin in the results section, and 40 (90%) had spin in the conclusion section in the abstract. In the main text, 28 (63%) and 39 (88%) studies had spin in the results and conclusion section, respectively. Table 2 provides a detailed breakdown of the frequency of different strategies of spin in the included trials. The journals that published articles with spin are listed in Table 3.
Frequency of spin in main text and abstracts of included studies
Frequency of spin in main text and abstracts of included studies
List of Journals containing articles published with spin
Based on the percentage of spin present in each study, the level of spin was categorized into high, medium, and low levels. Among the included studies, 54.5% were categorized as having a high level of spin, 25% were categorized as having a medium level of spin, and 20.5% were categorized as having a low level of spin.
Discussion
Our study reveals that the problem of spin in Indian physiotherapy research publications is widespread, with approximately 90% of the included studies containing some form of spin in various sections of the manuscript. Interestingly, almost half of the studies included in our review concluded that the treatment with nonsignificant outcomes was beneficial and recommended its use in clinical practice, which is categorized as a high level of spin. This high prevalence and level of spin indicate the gravity of the issue and raise concerns about the credibility and transparency of research reporting in Indian physiotherapy [19]. Furthermore, the high prevalence and level of spin identified in our study may have implications for the authors of systematic reviews and clinical practice guidelines, especially if they are not aware of the issue. The inclusion of studies with spin in these reviews and guidelines could result in inaccurate recommendations, potentially impacting patient care. Therefore, it is essential for researchers, reviewers, and guideline developers to be aware of the presence of spin and its potential impact on clinical decision-making [6]. Studies with spin that are published in high-impact journals are often highly cited, leading to a potential misdirection of research and resulting in significant research waste [20].
Our study found that high prevalence of spin was present in the abstracts of our included studies. Given that clinicians and consumers may have limited time or skills to critically read the entire manuscript, they may rely on the abstract or conclusion for decision making. However, the impact of spin on reader’s interpretation has been assessed and it has been shown that consumers reading abstracts with spin are more likely to perceive a treatment as effective even if the primary outcomes are not significant [21]. Similarly, press releases based on articles with spin can impact the interpretation of the findings by the general public and healthcare policy makers. This can potentially lead to inappropriate adoption of a treatment, resulting in harmful or ineffective outcomes [4, 22].
In comparison to the relatively low prevalence of spin reported in other medical disciplines, our study found a high prevalence of spin in the included physiotherapy research articles. For instance, spin was reported in 46% of oncology studies [3], 31% of diagnostic studies [10], 56% of psychiatric studies [23], and 50% of cardiovascular studies [9] in previous studies. However, our findings are consistent with the results reported by Nascimento et al., who surveyed 200 abstracts of nonsignificant low back pain trials and 66 systematic reviews, and found spin in 98% of the trials and 80% of the systematic reviews [14, 24]. These results suggest that the field of physiotherapy research may be more susceptible to spin compared to other medical disciplines.
There are several potential reasons for the high prevalence of spin observed in our study. Firstly, some instances of spin may occur unintentionally, particularly among novice researchers and postgraduate students who may imitate and reproduce spin in their articles after reading spun articles in the literature. Secondly, one possible explanation for the observed prevalence of spin is that approximately one third of our included studies interpreted within-group significance as indicative of treatment effectiveness, potentially due to poor understanding of statistical methods or a lack of experienced statisticians [6]. Finally, the high proportion of positive result publications in journals and the rejection of negative results can create pressure on authors to find positivity out of nonsignificant outcomes, potentially contributing to the observed prevalence of spin in our study [25].
Despite the expectation that high quality journals have effective peer review processes to detect and prevent spin, our study found that spin was still present in some of these journals (Table 2). This is consistent with previous reports of spin in high quality surgical journals, which suggests that the peer review process may not always be sufficient to detect and prevent spin [26, 27]. To gain further insights, we attempted to obtain explanations from journal editors regarding published articles with spin by sending emails, but unfortunately, we did not receive reliable responses from most of them.
The high prevalence of spin in 90% of published clinical trials in Indian physiotherapy research is a significant problem that affects the dissemination and interpretation of research findings. The issue may be due to a lack of research expertise and mentorship for novice researchers. To address this problem, it is essential to enhance research knowledge, writing skills, and mentorship in low-resource settings, and researchers must prioritize transparent reporting. To reduce the impact of spin, it is recommended that confidence intervals are used to report results instead of P-values, and selective publication by journals should be avoided [28–30]. Additionally, readers should critically evaluate research papers, even those published in high-impact journals. The study has some limitations, such as its focus on articles indexed in Medline and published by Indian authors, and the use of P-values as a cut-off for inclusion. To address these limitations, training researchers in scientific writing with appropriate mentors, transparent reporting, and a willingness to publish negative results can help reduce spin. Consumers should be aware of spin and consider the data while reading research papers before making clinical decisions.
In conclusion, this study highlights a significant problem of spin in Indian physiotherapy research publications, with a high prevalence of spin in abstracts and full-text articles of nonsignificant randomized controlled trials. This spin reduces the credibility and transparency of research reporting and can lead to claims of treatment effectiveness where there is none. The high level of spin may indicate the need for enhancing research knowledge, writing skills, and mentorship in low-resource settings. It is essential for physiotherapy researchers to be aware of this problem and to take responsibility for transparent reporting. Selective publication by journals should be avoided to reduce the effect of spin and also reduce false positive results. Readers should critically evaluate the studies they read, even if published in high-impact journals. This study emphasizes the importance of transparency in reporting and the need for journals to publish negative results.
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.