Abstract
Objective:
This review aimed to describe the quality and comprehensiveness of adverse event (AE) reporting in clinical trials incorporating manual therapy (MT) as an intervention for extremity conditions using the Consolidated Standards of Reporting Trials (CONSORT)-Harms extension as the benchmark. The secondary aim was to determine whether the quality of AE reporting improved after the availability of the CONSORT reporting checklist.
Design:
Systematic review.
Methods:
A literature search was conducted using multiple databases to identify trials where MT was used to treat extremity conditions. Studies that reported AEs were identified and evaluated using the CONSORT-Harms extension. The frequency of trials reporting study AEs before and after the publication of the updated 2010 CONSORT statement was calculated, along with the categorization of how study AEs were reported.
Results:
Of the 55,539 studies initially identified, 220 trials met all inclusion criteria. Eighty trials (36.4%) reported AE occurrence. None of the studies that reported AEs adhered to all 10 criteria proposed by the 2010 CONSORT-Harms extension. The most commonly reported criterion was number four, which clarified how AE-related information was collected (30% of trials). The least reported criterion was number six, which describes the participant withdrawals for each arm due to AEs and the experience with the allocated treatment (1.3% of trials). The nomenclature used to describe AEs varied substantially. Fifty-nine of 76 trials (33.3%) were published after the updated CONSORT Harms-checklist was available, compared to 21 of 44 trials (46.7%) published before it was available.
Conclusion:
Reporting of AEs in trials investigating MT for extremity conditions is poor. Every included trial lacked adherence to all 10 criteria proposed by the CONSORT-Harms Extension. The quality and comprehensiveness of AE reporting did not improve after the most recent CONSORT update recommending AE reporting. Clinicians must obtain informed consent before performing any intervention, including MT, which requires disclosing potential risks, which could be better known with improved tracking, analyzing, and reporting of AEs. The authors recommend improved adherence to best practices for adequately tracking and reporting AEs in future MT trials.
Introduction
Manual therapy (MT),
The original CONSORT statement, first published in 1996, dedicated only one item to unintended AEs. The reporting of AEs-related data in RCTs remains low. 23 To address this critical need, the CONSORT-Harms extension was added to the CONSORT statement in 2004. The update of the CONSORT statement published in 2010 specifically refers to the additional CONSORT-Harms extension. Still, it remains unclear if the availability of this tool has improved the reporting of AEs. 24
There is a paucity of information about how well AEs are reported in clinical trials investigating the use of MT for extremity conditions. 25 This systematic review aimed to describe the quality and comprehensiveness of AE reporting in trials using MT for individuals presenting with an extremity condition using the CONSORT-Harms extension. The secondary aim was to determine if AE reporting improved after the availability of the 2010 CONSORT statement update.
Methods
Protocol and registration
This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines 26 and was prospectively registered (PROSPERO CRD42021252227).
Search strategy
A literature search from the database's inception to June 2022 was conducted in PubMed, CINAHL, and OVID Medline. With the assistance of a medical librarian-controlled vocabulary, MeSH terms (or database-appropriate terms), Boolean operators “AND” and “OR,” and keyword-combined terms were used (Supplementary Table S1).
Eligibility criteria
Eligible studies included RCTs published from the inception of the database to June 2022 that used thrust manipulation (TM), nonthrust manipulation (NTM), or STM to any body region, including the spine, either as the sole intervention or as part of a multimodal management approach for an individual presenting with an extremity disorder. For this review, the upper extremity was defined as the shoulder, elbow, wrist, and hand, and the lower extremity was defined as the hip, knee, ankle, and foot.
Studies included any intervention provided by a licensed clinician (e.g., physical therapist, chiropractor, or osteopathic physician). Owing to a lack of consensus on the definition of MT and the fact that MT includes a variety of treatments, the focus of this review was specifically on TM, NTM, and STM to maintain a homogeneous group (see Supplementary Table S2 for definitions). 27,28 The National Institute of Health (NIH) guideline for reporting AEs was used to categorize severity into mild, moderate, or severe due to the vast differences used to define AEs. Studies were excluded (1) if they were not RCTs, (2) if the full text was not available or translated to English, (3) if they had been retracted, (4) where the intervention was self-administered, (5) where participants were healthy, or (6) where it was unclear if the MT intervention applied was TM, NTM, or STM (see Supplementary Table S3). Figure 1 shows the study selection process.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.
Data management
Covidence (Veritas Health Innovation Ltd, Melbourne, Australia) was used for screening and data extraction. 29 Two independent reviewers (J.F., A.F.) performed the title/abstract screening, full-text screening, risk of bias scoring, data extraction, and analysis. Disagreements were resolved by a third reviewer (J.L.Y.).
Data extraction
To determine the comprehensiveness and quality of AE reporting, two reviewers (J.F., A.F.) independently assessed each trial for the presence of the 10 recommended criteria from the CONSORT-Harms extension (Table 1). The reviewers recorded a “yes/no” for each criterion in each trial. Each reviewer also independently extracted data from each trial, including the total number of participants, the country where the study was conducted, the specific body region for which the MT intervention was directed, the type of MT used (i.e., TM, NTM, and STM), and the practitioner applying the MT intervention. Each reviewer extracted the authors' a priori definition(s) of AEs, whether AEs were acknowledged, what section of the articles AEs were located (i.e., abstract, results, methods, etc.), the individual number of AEs in the MT and control arm of the trial, the number of dropouts from the trial, and the reason for dropouts.
Consolidated Standards of Reporting Trials-Harms Extension
The year of publication was extracted to determine the effects of the 2010 CONSORT guidelines on the reporting of AEs (studies published in 2010 or earlier were considered before the CONSORT, and studies published in 2011 or later were considered after the CONSORT). Information related to the reporting of AEs was searched in the title and abstract. If no information was found there, the full article was searched. The participant flow diagram and appendices were also reviewed for AE acknowledgment. The two reviewers recorded the nomenclature used to describe AEs. Due to the vast differences in descriptions, 30 AEs were categorized into mild, moderate, or severe based on the NIH guidelines for reporting AEs (Table 2). 13
Classification of Adverse Events (Adapted from the National Institute of Health Guidelines) 13
AE, adverse event; MT, manual therapy; NIH, National Institute of Health.
The risk of bias assessment was performed using the Cochrane RoB-2 assessment tool (RoB-2), 31 which provides a framework for considering the risk of bias in any type of RCT. Cochrane RoB-2 has five risk domains: random sequence generation, allocation concealment, selective reporting, and other sources of bias. Studies are scored as having “low,” “some concerns,” or “high” risk of bias.
Data analysis
Inter-rater reliability was assessed for the title/abstract, full-text screening, and RoB-2 using Cohen's kappa. If a study reported that an AE occurred, the authors used the 10 recommended criteria from the CONSORT-Harms extension to evaluate the quality and comprehensiveness of AE reporting (Fig. 2). 23 Counts and percentages were calculated for the total number of AEs that occurred in the MT arm and/or comparator arm, the total number of AEs in each NIH category (mild, moderate, and severe), amount of AEs reported per body region, and the number of CONSORT-Harms extension criteria reported in each trial (Supplementary Table 4). The percentage of studies reporting AEs after the availability of the 2010 CONSORT statement was compared to the percentage of studies published before its availability. After the risk of bias assessment was completed, counts and percentages were calculated for the total number of trials that were rated “low,” “some concerns,” or “high” risk of bias.

Consolidated Standards of Reporting Trials-Harms extension criteria reported across included randomized controlled trials.
Results
Study characteristics
The authors identified 55,539 studies from the initial literature search. A total of 33,284 unique records remained after the removal of duplicates. After screening titles and abstracts, 494 studies remained for full-text review. The inter-rater reliability for title/abstract and full-text screening was 0.37, indicating fair agreement. After full-text review, 220 studies were included in the final review.
Reporting of the 10 criteria from the 2010 CONSORT-HARMS extension
Eighty 32 –110 (36.4%) of the 220 included trials reported whether or not an AE occurred in subjects participating in the trial. For the 80 studies that reported AEs, none of the studies reported all 10 criteria from the CONSORT-Harms extension. The most reported criterion was number four, which clarified how AE-related information was collected (30% of trials). The least reported criterion was number six, which describes the participant withdrawals due to AEs in each arm and the experience with the allocated treatment (1.3% of trials). Figure 2 provides a summary of the percentage of RCTs that adhered to each criterion in the CONSORT-Harms extension.
AE reporting in shoulder trials
Twenty 32 –51 of the 73 trials (27.4%) investigating shoulder disorders reported AE occurrence in subjects in the MT intervention arm. Sixteen 32 –34,36 –42,45 –49,51 (80.0%) included no AEs, two 35,50 (10.0%) described mild AEs in subjects, and one 43 (5.0%) noted a serious AE in one subject. One trial 44 (5.0%) specifically clarified that the AE was not attributed to the MT intervention.
AE reporting in elbow trials
Seven 53 –58,111 of the 15 trials (46.7%) investigating elbow disorders detailed the AE occurrence. Three 55,57,58 reported no AEs, and four 53,54,56,111 noted mild AEs in subjects in the MT arm.
AE reporting in wrist and hand trials
Five 59 –63 of the 13 trials (38.5%) assessing wrist and hand disorders discussed AEs, with all five studies reporting that no AEs occurred in subjects in either trial arm.
AE reporting in hip trials
Six 64 –69 of the 12 trials (50%) on hip disorders reported AE occurrence. Three 65,67,69 reported no AEs, two 64,66 noted a mild AE occurred in subjects in the MT arm, and one 68 revealed a mild-and-moderate AE occurred in subjects in the MT arm.
AE reporting in knee trials
Twenty-two 70 –91,139 of the 49 trials (44.9%) investigating knee disorders reported the occurrence of AEs. Nine reported no AEs occurred, six revealed a mild AE occurred in subjects in the MT arm, four 73,77,78,90 reported how AEs were documented but did not disclose if an AE occurred, and three 70,76,91 reported that the AEs that occurred were not attributed to MT.
AE reporting in foot/ankle trials
Twenty 92 –110,112 of the 58 trials (34.5%) assessing ankle and foot disorders reported AE occurrence. Fourteen 92,93,95 –98,101 –103,105,106,108,109 noted no AEs, and four 94,99,100,104 revealed a mild AE occurred in subjects in the MT arm, one 110 reported a moderate AE in subjects in the MT arm, and one identified that the AE that occurred was not attributed to MT. 107
Overall comprehensiveness of reporting of classification of AEs before and after the updated 2010 CONSORT statement
Of the 220 included trials, 176 32,33,37 –39,41 –49,51,54,55,58 –65,67 –71,75,76,80,82 –85,87 –90,92 –96,98 –101,103,105 –111 (80%) were published after the 2010 CONSORT statement was available, while 44 34 –36,40,50,53,56,57,66,72 –74,77 –79,81,86,91,97,102,104,113 –134 were published before. Of those published after the 2010 CONSORT, 59 (33.3%) reported AE occurrence, while 18 (40.9%) reported AE occurrence before the 2010 CONSORT.
Risk of bias assessment
Using the RoB-2 criteria, just over half of the included trials were determined to have a “high” risk of bias (n = 113; 51.4%), about one-third had “some concerns” (n = 73; 33.2%), and the remaining trials had a “low risk” of bias (n = 34; 15.5%). The inter-rater reliability was fair (κ = 0.39; p < 0.001).
Discussion
This review aimed to describe the quality and comprehensiveness of AE reporting in RCTs using MT for extremity conditions, using the 2010 CONSORT-Harms criteria as the benchmark. Based on the findings of this review, adherence to the recommendations for reporting AEs in trials that investigate the use of MT for musculoskeletal extremity conditions can be improved. None of the studies referenced the 2010 CONSORT statement, and most of the criteria recommended by the CONSORT-Harms extension were underreported. In addition, an update to the CONSORT statement in 2010 did not improve AE reporting in RCTs.
The CONSORT-Harms extension was developed with the goal of enhancing the reporting of safety data in RCTs. 135 This review showed that the 36.4% of RCTs using MT as a sole intervention or part of a multimodal management approach for extremity conditions are not acknowledging AEs or abiding by the criteria put forth by the 2010 CONSORT statement, which includes the CONSORT-Harms extension. Zero trials reported all 10 items of the CONSORT-Harms extension, and items that were reported, such as describing detailed plans of coding or statistical analyses or describing how AEs were recorded, were reported superficially. Failure to adhere to the CONSORT-Harms extension guidelines when conducting an RCT may have significant consequences related to the validity, transparency, and interpretability of the trial results. 135 Better compliance with recommended checklists, such as the CONSORT-Harm extension, is needed. Inadequate adherence to AE reporting standards, particularly in the context of MT interventions, can lead to inaccurate assumptions about the safety of the intervention. 136 An informed assessment of intervention risks begins with an explicit knowledge of the nature of the AE, its categorization based on NIH guidelines, and, when relevant, the specific technique or aspect of the intervention that led to the event. Comprehensive tracking and reporting of AEs in the literature can enhance discussions during informed consent of the treatment in clinical practice, helping to disclose potential risks. 137 However, challenges in AE reporting procedures, such as word count limits, editorial adjustments, investigator awareness, checklist use, and the primary trial emphasis on benefits rather than harms, may result in incomplete information available to readers. A higher percentage of trials was published before the updated 2010 CONSORT complied with AE reporting than those published after the update. The CONSORT-Harms extension was released 17 years ago, and the CONSORT statement was updated 12 years ago. This time frame has allowed researchers adequate time for guideline implementation. Based on the findings from this review, greater strategies for increased endorsement and adoption of the CONSORT statement, including the Harms extension, are necessary. Researchers conducting MT trials must improve AE reporting, reference standardized and validated definitions, and create new definitions when appropriate. 23 The authors recommend reporting AE data according to the CONSORT-Harms extension to aid in the proper tracking of AEs during the trial planning phase. This may also lead to a better understanding of the relationships between MT and AEs, including if the AEs are related to the MT received or if they can be attributed to some other factor. Specifically, AEs within the context of subgroups (i.e., age, sex, race, etc.) may also help clinicians to better apply this information to individual patients.
Limitations
The findings of this review must be considered with acknowledged limitations. First, the findings could be influenced by a lack of standardization in the literature surrounding MT definitions. 1,138 Second, considering the substantial variability in the terminology used to describe AEs, we opted to adopt the NIH criteria as a reference standard. However, it is important to acknowledge the possibility of misclassifying certain AEs using these criteria. Third, non-English studies may have been missed. Finally, when comparing the influence of the updated 2010 CONSORT statement, studies were dichotomized into studies published in 2010 or earlier were considered before the CONSORT, and those published in 2011 or later were considered after the CONSORT. The year 2011 may have been too soon since the CONSORT statement was updated to ensure adequate implementation. Future work in this area will need to consider the 2023 update to the CONSORT-Harms extension.
Conclusion
The quality and comprehensiveness of AE reporting and adherence to reporting per the recommendations in the CONSORT-Harms extension checklist are underreported in RCTs. Without proper tracking, analyzing, and reporting of AEs, determining the risk associated with interventions for managing musculoskeletal extremity conditions, which may include MT, will not be possible.
Footnotes
Acknowledgments
The authors would like to thank the librarian at Bellin College, Cindy Reinl, and the rest of the staff for their support and contributions to this work.
Authors' Contributions
J.F. and A.F. made substantial contributions to the conception and design of the study, data collection, and analysis, and contributed to the writing of the article. J.L.Y. and D.I.R. made substantial contributions to the conceptualization of the work and editing of the article. All authors have approved the submitted version of this article. All authors have agreed both to be personally accountable for the author's contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.
Disclaimer
The views expressed herein are those of the author(s) and do not necessarily reflect the official policy or position of the Uniformed Services University, the Defense Health Agency, the Department of Defense, or the U.S. Government.
Author Disclosure Statement
No competing financial interests exist.
Funding Information
No funding was received for this article.
Supplementary Material
Supplementary Table S1
Supplementary Table S2
Supplementary Table S3
Supplementary Table S4
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
