Adherence to CONSORT Items in Randomized Controlled Trials of Integrative Medicine for Colorectal Cancer Published in Chinese Journals

Abstract

Background:

The majority of studies of integrative treatment for colorectal cancer (CRC) have been published in Chinese journals. These studies indicate potential benefits, but concerns have been raised over the quality of trials published in Chinese journals. The CONSORT statement provides a guide for study reporting that has been endorsed by more than 400 international journals. Previous studies have used the CONSORT checklist to assess the quality of randomized controlled trials (RCTs).

Objectives:

This study focused on RCTs of integrative and traditional medicine for CRC published in Chinese journals and assessed: (1) the overall quality of reporting with a focus on methodological aspects; (2) change over time; and (3) the influence of study funding, level of institution conducting the trial, rank of the journal, and the length of the article.

Design:

Searches of seven databases identified RCTs. Quality was assessed using CONSORT 2010 with adaptations to facilitate scoring. Additional codes were added for publication year, hospital rank, report length, and status of the journal. Scores of each checklist item, total scores, and scores for eight items associated with RCT methodology were calculated.

Results:

Eighty-one studies were included in the main analyses. The RCT methodology subgroup scores were significantly higher in studies: with public funding, conducted by authors from university hospitals, published in higher ranked journals, and in longer articles.

Conclusions:

Few Chinese journals mention CONSORT in their author guidelines. In these RCTs on CRC better reporting of RCT methodology was associated with ranking of the journal as “core,” public funding of the RCT, and first or correspondent author from a university hospital but the quality of reporting had not significantly improved in 15 years. As the volume of scientific information produced in China grows, it is imperative that there is growth in the quality of this information.

Introduction

In the field of medicine and health, 1055 Chinese journals published 644,950 articles in 2013. Of these, the 901 journals dealing with clinical research published 518,016 articles.¹ A substantial proportion of these articles reported the results of clinical trials, but data on the exact number are not available. Based on current trends the number of journal articles published in China is projected to continue to increase.² It has been estimated that 146 Chinese journals have been indexed in MEDLINE and PubMed^3,4 and about 205 Chinese journals have been indexed in MEDLINE or EMBASE⁵ amounting to about 8% of the estimated 2529 journals included in the five main Chinese biomedical and scientific databases.⁴ Consequently much of the medical and healthcare research conducted in China is not available to international medical researchers due to language barriers and the relatively small numbers of reports that are abstracted in international databases.

In addition, there is the issue of whether the conduct of randomized controlled trials (RCTs) reported in Chinese journals was in accord with international standards. Wu et al. conducted telephone interviews with the authors of 2235 articles published from 1994 to 2005 that reported results of RCTs and found that only 6.8% adhered to established RCT methodology. There was no difference between trials of conventional and traditional medicine in terms of methodological quality. In their stratification analysis, all premarket drug trials were authentic RCTs, and 51.6% of government funded studies were considered authentic. In China, hospitals are categorized into three grades according to their level and size, the highest level being level 3 which are hospitals affiliated to medical universities and provincial hospitals, other hospitals are classified as level 2 or below. The authors found that hospitals affiliated to medical universities were better in conducting RCTs than other level 3 hospitals and level 2 hospitals.⁶ A study of 70 Cochrane systematic reviews of studies of Chinese traditional medicine found that most reviews produced inconclusive outcomes due to poor methodology and heterogeneity of the included studies.⁷

Assessment of the quality of a clinical trial depends on whether the design, conduct, and analysis are adequately reported in the publication. Although there is no uniform international guideline for reporting clinical trials, the CONSORT statement which was introduced in 1996 has been endorsed by more than 400 international journals and several editorial groups. The current version includes checklists that provide guidance for authors to report their trials clearly and completely.⁸ In China, there is a relatively low level of endorsement. Li et al. reported that 6 of 195 high-impact medical journals (3.08%) mentioned CONSORT in the author guidelines⁹ and Song et al. reported that 7 of 1221 (0.57%) medical journals endorsed CONSORT.¹⁰ Only 1 of 90 journals of Chinese traditional medicine (English version only) and one specialist oncology journal endorsed CONSORT.

The CONSORT checklist⁸ has been used as measure of quality of reporting in RCTs. A study of 76 RCTs of integrative medicine published in two Chinese journals before and after adoption of CONSORT found that adoption and length of the article both improved the quality of reporting.¹¹ An earlier study of the reporting of CONSORT 2001 items in studies of traditional medicine published in Chinese journals from 1999 to 2004 found inadequate reporting with only 7.9% providing a description of the method of generation of randomization sequence.¹² A subsequent study of 6994 articles published from 2005 to 2012 that used CONSORT 2010 items for scoring found better reporting with 16.26% describing the randomization sequence.¹³

A study of adherence to CONSORT in 357 RCTs published from 2005 to 2009 in 10 major international oncology journals found that 29% adequately reported the method for generation of randomization sequence, and other items were inadequately reported. More recent publication, higher journal impact factor (IF), and industry funding were all predictors of higher quality of reporting.¹⁴ In comparison, 34% of 616 general medicine RCTs indexed in PubMed in 2006 adequately reported this item.¹⁵

Most Chinese language medical journals have no IF or a low IF. For example, the Chinese medical journal, which is the oldest medical journal in China, had an IF of 1.05 in 2015. Chinese journals are ranked by the library of Beijing University based on nine criteria to produce lists of “core” journals in each field, including 37 journals in the field of general medicine and healthcare, 24 journals specializing in internal medicine, and 19 journals specializing in Chinese traditional and integrative medicine.¹⁶

Integrative approaches are recommended in Chinese traditional medicine guidelines for cancer. Diagnosis is based on biomedical criteria and treatment involves surgery, chemotherapy, Chinese traditional therapies, and supportive or palliative care.¹⁷ The number of RCTs of Chinese and integrative medicine for cancer has substantially increased over the last 20 years.¹⁸ Systematic reviews of integrative medicine indicate potential benefits for colorectal cancer (CRC).^19

–22 Most of the RCTs on this topic were conducted in China and published in Chinese journals, but the adequacy of reporting can impact on confidence in the outcomes. In the present study the authors assess the quality of reporting based on CONSORT of RCTs of CRC treated with traditional and integrative medicine that were published in China. The study aims to assess: (1) the overall quality of reporting with a focus on methodological aspects; (2) whether there was change over time; and (3) whether study funding, level of institution conducting the trial, rank of the journal, and the length of the article influenced the quality of reporting.

Materials and Methods

Searches were conducted of the databases PubMed, Cochrane CENTRAL, CINAHL, Science Direct, PsycINFO, China Academic Journals (CNKI), and Chongqing VIP Information (CQVIP) in the category of Chinese Science and Technology Journals (CSTJ) from their inceptions to December 2013. Three groups of search terms were used: (1) Study design: RCT and related terms; (2) Disorder: CRC and related terms; (3) Intervention: herbal medicine, complementary medicine, Chinese medicine, and related terms (Supplementary Data; Supplementary Data are available online at www.liebertpub.com/acm). Hand searches were conducted of article reference lists to identify additional studies.

Included studies were designated RCT by the author(s), enrolled participants who were all diagnosed with CRC based on pathologic tests, and used traditional herbal medicine (with or without concurrent chemotherapy) in the test group. Control groups could be chemotherapy, no treatment, or placebo.

The quality of reporting in each study was assessed using the CONSORT 2010 checklist.⁸ CONSORT 2010 contains 25 items but a number of these include subitems so there are 37 items in total. Items 1 to 22 were included in CONSORT 2001,²³ and items 23 to 25 (other information) were added in CONSORT 2010.⁸ Following a trial run using the unmodified version, the authors found that adding two subitems facilitated assessment and scoring as follows: item 6a was split into 6a(1) Primary and secondary outcomes and item 6a(2) Endpoints (time of data measured); and item 13a Flow of participants was divided into item 13a(1) Flow diagram as figure, and item 13a(2) Flow of participants (verbal) (Table 1). These same modifications appear in the assessment of RCTs of medical oncology by Peron et al.¹⁴

Table 1.

Scores on CONSORT 2010 Checklist (Modified) with Comparisons to Previous Studies

CONSORT checklist item no., criteria ⁸	This review (n = 81) 1998–2013, n (%)	Li et al.¹³ (n = 6994) 2005–2012, n (%)	Peron et al.¹⁴ (n = 357) 2005–2009, n (%)
Title/Abstract
1a. Identification as a RCT in the title	2 (2.47)	56 (0.80)	189 (53)
1b. Structured summary of trial design, methods, results, and conclusions	69 (85.19)	5852 (83.67)	323 (90)
Introduction
2a. Background and objectives	67 (82.72)	2028 (29.00)	340 (95)
2b. Specific objectives or hypotheses	10 (12.35)	410 (5.86)	131 (37)
Methods
Trial design
3a. Description of trial design	6 (7.41)	277 (3.96)	120 (34)
3b. Changes to methods after commencement	0 (0)	0	NA
Participants
4a. Eligibility criteria for participants	58 (71.60)	3925 (56.12)	338 (95)
4b. Setting and locations of data collections	80 (98.77)	4938 (70.60)	246 (69)
Interventions
5. Details of interventions in each group	67 (82.72)	5699 (81.48)	321 (90)
Outcomes
6a(1) Primary and secondary outcomes	81 (100)	6176 (88.30)	267 (75)
6a(2) Endpoints (time of data measured)	79 (97.53)	NA	260 (73)
6b. Any changes to trial outcomes after the trial commenced, with reasons	0 (0)	0	NA
Sample size
7a. Sample-size determination	0 (0)	20 (0.29)	241 (67)
7b. Explanation of any interim analyses and stopping guidelines (if applicable)	0 (0)	0	320 (90)
Randomization
8a. ^* Method used to generate the random allocation sequence	24 (29.63)	1137 (16.26)	109 (31)
8b. ^* Type of randomization with details of any restriction (such as blocking)	2 (2.47)	117 (1.67)	298 (83)^a
9. ^* Allocation concealment	0 (0)	48 (0.69)	182 (51)
10. ^* Implementation of random-allocation sequence	0 (0)	36 (0.51)	NA
11a. ^* Blinding of participants (and others) to group assignment	2 (2.47)	268 (3.83)	146 (41)
11b. ^* Description of the similarity of interventions (if relevant)	1 (1.23)	8 (0.11)	NA
12a. Statistical methods	75 (92.59)	4686 (67.00)	323 (90)
12b. Methods for additional analyses (e.g., subgroup analyses)	0 (0)	0	227 (64)
Results
13a(1). Flow diagram as figure	0 (0)	NA	215 (60)
13a(2). Flow of participants (verbal)	2 (2.47)	20 (0.29)	212 (59)
13b. ^* For each group, losses and exclusions after randomization, with reasons	10 (12.35)	246 (3.52)	NA
14a. Dates of recruitment and follow-up	78 (96.3)	5198 (74.32)	243 (68)
14b. Why the trial ended or was stopped	0 (0)	0	NA
15. Baseline and demographic clinical characteristics (table)	13 (16.05)	445 (6.36)	354 (99)
16. ^* For each group, number of participants (denominator) included in each analysis and whether used intent to treat	4 (4.94)	127 (1.82)	220 (62)
17a. Outcomes and estimated effect size with 95% confidence interval	0 (0)	89 (1.27)	236 (66)
17b. For binary outcomes, presentation of both absolute (RD) and relative (RR) effect sizes	0 (0)	21 (0.30)	NA
18. Subgroup analyses/ancillary analyses	1 (1.23)	0	291 (81)
19. Adverse events/harms	5 (6.17)^b	1830 (26.17)	282 (79)
Discussion
20. Trial limitations, sources of potential bias, imprecision, etc.	9 (11.11)	572 (8.18)	NA
21. Generalization (external validity) of findings	2 (2.47)	187 (2.67)	NA
22. Interpretation consistent with results, balancing benefits and harms, and other relevant evidence	81 (100)	6473 (92.55)	NA
Other information
23. Registration number, name of trial registry	0 (0)	1 (0.01)	NA
24. Protocol	0 (0)	1 (0.01)	NA
25. Funding (public)	18 (22.22)	413 (5.91)	305 (86)
Mean score (±SD) of each study (scale)	10.44 (±2.40) (max score = 39)	NA	19.3 (max score = 27)
Additional items
1. Publication year: 2009 or after/before 2009	38 (46.91)/43 (53.09)	NA	NA
2. Author from university	42 (51.85)	NA	NA
3. Length of article ≥3 pages	35 (43.21)	NA	NA
4. Published in core journal	21 (25.93)	NA	NA
5. Syndrome according to CM theory	27 (37.03)	NA	NA

Eight items closely related to RCT methodology.

Data as reported in Table 2 of Peron et al.¹⁴

AEs are related to traditional interventions, excluding chemotherapy-related AEs reported as outcome measures.

AE, adverse event; CM, Chinese medicine; NA, not available; RCT, randomized controlled trial; RD, risk difference; RR, risk ratio; SD, standard deviation.

A subgroup of eight items directly related to RCT methodology based on Jadad's scale²⁴ and Cochrane risk of bias²⁵ was identified as follows: item 8a sequence generation method; item 8b type of randomization; item 9 allocation concealment; item 10 implementation of random allocation sequence; item 11a blinding of participants (and others) to group assignment; item 11b description of the similarity of interventions (if relevant); item 13b for each group, losses and exclusions after randomization, with reasons; and item 16 for each group, number of participants (denominator) included in each analysis and whether intent-to-treat analysis was used.⁸

In addition, data were entered for publication year; rank of the hospital in China to which the first or corresponding author was affiliated (1) university hospital, (2) other type of hospital (provincial hospital, municipal, or county-level hospital); length of the report based on A4 size pages rounded to the nearest half page; and whether or not the report was published in a leading (core) journal based on the ranking by Beijing University library¹⁶; reporting of syndrome according to Chinese medicine theory²⁶; and whether or not the journal endorsed CONSORT in the author guidelines.

Data were independently extracted into a predesigned excel table by M.C. and J.C. The data were cross-checked by the two data extractors. Differences were resolved through discussion. Disagreements were mediated by BHM. Each CONSORT checklist item was given a “yes,” “no,” or “not applicable” response depending on whether the item was included in the study report. “Yes” was scored 1, “no” and “not applicable” were scored 0.

Data analysis

For each study the number of items reported was calculated (items 1–25, maximum score = 39), and average scores were obtained for each publication year to investigate any trend over time. Comparisons with reviews of Chinese journals¹³ and international oncology journals¹⁴ were planned.

Studies were equally divided into two groups according to: publication year; length of the article (in pages); “core” journal (or not), ranking of the author's hospital, and public funding of the study (or not). Between-group analyses were based on: (1) items 1–22 (maximum score of 36) since these are the items concerned with the quality of the study report that were included in both CONSORT 2001²³ and 2010⁸ and (2) the subgroup of eight items directly related to RCT methodology.

Means and standard deviations (SDs) were calculated in Excel. RevMan5.3 was used to evaluate differences between groups. Risk ratio (RR) was used for dichotomous data and mean difference (MD) for continuous data with 95% confidence interval (95% CI) in a fixed effect model. Our hypothesis was that recency of publication, length of article, “core” journal, ranking of the author's hospital, and public funding would affect reporting quality. Significant differences were indicated by RR more than 1 or MD not less than 0 with p-value not more than 0.05. I ² measured the proportion of heterogeneity. If I ² was more than 50% this indicated substantial heterogeneity.

Results

Following screening, 88 full-text studies were evaluated. Four studies written in English published in international journals were excluded leaving 84 studies that were published in China. One of these studies was written in English in a Chinese journal, and two studies were Chinese theses published in CNKI. These three studies were excluded from the main analysis which focused on articles in medical journals written in Chinese (Fig. 1 flow chart). The 81 articles were published from 1998 to 2013 in 53 different Chinese journals. None of the earlier studies met the inclusion criteria. All were written in Chinese and ranged in length from one to four pages (mean 2.44, SD ±0.75). Short articles were defined as one to two pages (n = 46), and longer articles were three or more pages (n = 35). For publication year, the most equal division was 1998 to 2008 (43 articles) and 2009 to 2013 (38 articles). Eighteen studies were funded by national, provincial, or local governments (22.2%). The first or corresponding author of the study was from a university or affiliated hospital in 42 studies (51.9%). Twenty-one studies (25.9%) were published in core medical journals according to the ranking by the library of Beijing University.¹⁶ No endorsement of CONSORT in the author guidelines was found in any of the 53 journals.

FIG. 1.

Flow diagram of the search and selection process of RCTs of integrative and TM for CRC published in China. CRC, colorectal cancer; CT, clinical trial of TM without randomization; DU, duplicate publication; MT, multi-cancer CT; Other, not a controlled trial, not a CT of TM; RCT, randomized controlled trial; RE, review; TM, traditional medicine.

In the 81 studies, the total mean score of the checklist items was 10.44 (SD ±2.40) out of a maximum of 39. Two items were reported by all studies (6a, 22), and 12 items were not reported at all (items 3b, 6b, 7a, 7b, 9, 10, 12b, 13a(1), 14b, 17a, 17b, 23, 24) (Table 1). Reporting rates were above 70% for 10 items (1b, 2a, 4a, 4b, 5, 6a(1), 6a(2), 12a, 14a, 22).

In the subgroup of eight items on RCT methodology, item 9 allocation concealment and item 10 implementation of randomization were not reported in any study. Item 8a sequence generation was reported by 29.6% studies, item 13b losses and exclusions by 12.35%, item 16 intent-to-treat analysis by 4.94%, and item 11a blinding by 2.47%; so these four items were the main contributors to the analyses. Eighteen studies mentioned a form of public funding, none mentioned commercial funding, and no studies mentioned published protocols or trial registration. Chinese medicine syndromes were reported in 37.03% of studies (Table 1).

In the single study published in English²⁷ the total score was 29 and the two theses written in Chinese had an average score of 14.^28,29 None of these mentioned the three other information items.

Change in reporting over time

There was no significant difference in total scores or in the eight items for RCT methodology for studies published during 1998–2008 (n = 43) and 2009–2013 (n = 38). No items were significantly lower. Two items (2a, 12a) were significantly improved in 2009–2013 (Table 2).

Table 2.

Results of Group Comparisons for 22 Items from CONSORT 2010 Checklist and Subgroup of 8 items on Methodological Quality

Group	Year of publication: 1998–2008 (n = 43) vs. 2009–2013 (n = 38)	Length of article: ≥3 pages (n = 35) vs. <3 pages (n = 46)	Ranking of journal: core (n = 21) vs. noncore (n = 60)	Funding: public funding (n = 18) vs. no funding (n = 63)	Rank of author's hospital: university (n = 42) vs. non-university (n = 39)
22 items (0–36 points)	RR 1.02 [1.00, 1.05], I ² = 7%	RR 1.02 [0.99, 1.04], I ² = 28%	RR 1.00 [0.97, 1.04], I ² = 29%	RR 1.01 [0.97, 1.05], I ² = 33%	RR 1.00 [0.97, 1.02], I ² = 0%
8 item subgroup (0–8 points)	RR 1.03 [0.59, 1.79], I ² = 0%	RR 1.72^a [1.01, 2.92], I ² = 0%	RR 1.73^a [1.01, 2.98], I ² = 0%	RR 2.78^a [1.64, 4.74], I ² = 13%	RR 1.90^a [1.07, 3.38], I ² = 0%
Item 8a	1.58 [0.80, 3.14]	1.55 [0.79, 3.04]	1.43 [0.72, 2.84]	1.75 [0.90, 3.41]	1.55 [0.77, 3.12]
Item 8b	Not estimable	Not estimable	Not estimable	Not estimable	Not estimable
Item 9	Not estimable	Not estimable	Not estimable	Not estimable	Not estimable
Item 10	Not estimable	Not estimable	Not estimable	Not estimable	Not estimable
Item 11a	1.13 [0.07, 17.48]	1.31 [0.09, 20.29]	2.86 [0.19, 43.67]	3.50 [0.23, 53.23]	0.93 [0.06, 14.34]
Item 11b	Not estimable	Not estimable	Not estimable	Not estimable	Not estimable
Item 13b	0.48 [0.13, 1.74]	1.97 [0.60, 6.46]	1.90 [0.60, 6.10]	5.25^a [1.66, 16.61]	3.71 [0.84, 16.43]
Item 16	0.38 [0.04, 3.48]	1.31 [0.19, 8.88]	0.95 [0.10, 8.66]	10.50^a [1.16, 94.93]	2.79 [0.30, 25.67]

Significant difference at p < 0.05; when items have zero data RR is not estimable.

Effect of length of article

When articles of three pages or more were compared to those with less than three pages, there was no difference in total scores, but the scores for the eight-item subgroup were significantly higher in the longer articles (RR 1.72 [1.01, 2.92], I ² = 0%). Longer articles were more likely to mention RCT in the title (item 1) and state specific objectives or hypotheses (item 2b) (Table 2).

Effect of core versus noncore journal

There was no significant difference in total scores between core journals and noncore journals but the eight-item subgroup scores were significantly higher in the core journal group (RR 1.73 [1.01, 2.98], I ² = 0%). Articles in core journals were more likely to state specific objectives or hypotheses (item 2b) (Table 2).

Effect of public funding

There was no statistical difference between studies with public funding and studies without for total scores but studies with public funding were significantly better on the eight-item subgroup (RR 2.78 [1.64, 4.74], I ² = 13%). In addition, there were significant improvements in items: item 1a RCT in the title, item 2a scientific background and study rationale, item 13b losses and exclusions after randomization, and item 16 analysis of intent to treat (Table 2).

Rank of author's hospital

The total reporting rate was not different between articles from authors affiliated with university hospitals and articles from lower ranking hospitals, but the eight-item subgroup scores were significantly higher in articles from university hospitals (RR 1.90 [1.07, 3.38], I ² = 0%), although there were no differences for specific items (Table 2).

Discussion

In this review the quality of reporting of 81 RCTs of the management of CRC published in Chinese journals was analyzed using items from the CONSORT 2010 checklist and a subgroup of eight items that described key aspects of RCT methodology. Most of the included studies were brief (two to three pages) consisting of title, abstract, text body, tables, and references. This style of bulletin-like reporting limited the amount of included information. Thirteen items were not reported in any study, including sample size determination (items 7a, 7b), allocation concealment (item 9), and implementation of random-allocation sequence (item 10). In addition, no study included a flow diagram and only 2.47% verbally described participant flow (items 13a(1) and 13a(2)). In Li et al.,¹³ these items were reported in less than 1% of studies of traditional medicine, but flow diagram was not scored separately. Overall, the pattern of results of this study was similar to Li et al.¹³ Five items (3b, 6b, 7b, 12b, 14b) had zero reporting rates in both studies, and six items (1b, 4b, 5, 6a(1), 14a, 22) were reported by over 70% of studies in both Li et al.¹³ and the present study (Table 1). Compared to previous studies of Chinese journals, generation of randomization sequence was higher in this study than in Li et al. (16.26%),¹³ He et al. (9%),³¹ and Wang et al. (7.9%).¹² Reporting of Chinese medicine syndromes was higher than in Wang et al. (16.7%).

A Cochrane review on the effect on CONSORT endorsement on reporting quality in medical journals found that three items (7a sample size calculation, 8a generation of random allocation sequence, and 9 allocation concealment) were significantly better reported in articles from CONSORT endorsing journals.³⁰ In the study of oncology RCTs in 10 well-known international journals, sample size determination was reported in 67% of articles, allocation concealment in 51%, 60% provided a flow diagram, while 31% adequately described randomization sequence generation.¹⁴ In the present analysis, generation of random allocation sequence was reported in 29.63% of studies, while no studies reported on the other three items. These differences indicate gaps in the quality of reporting of oncologic RCTs between Chinese journals and international journals.

Previous studies reported improvements in reporting of sequence generation over time.^13,31 In this study there was a 13% increase in the proportion of studies adequately describing random sequence generation between 1998–2008 (23.3%) and 2009–2013 (36.8%), but this was not statistically significantly different. It is possible that our sample of 81 studies did not have sufficient power to detect a significant effect for this item.

The subgroup analysis results for length of article, publication in a core journal, public funding, and rank of first author/correspondent author's hospital found nonsignificant differences in total scores, but there were significant differences in the subgroup of eight items closely related to RCT methodology. Overall, articles that were longer, published in a core journal, publicly funded, and had a first or correspondent author from a university hospital showed greater adherence to these CONSORT items. This result was consistent with previous Chinese studies.^5,11,13 These aspects may be useful additions in assessments of the quality of RCTs published in Chinese journals and could be used as criteria for sensitivity analyses in meta-analyses.

Limitations of the study

The 81 RCTs reported results for CRC treated with integrative medicine, each included Chinese herbal treatment, and all studies were published during the 15-year period after 1997. Therefore, the results cannot be generalized to all cancer trials published in Chinese medical journals. Nevertheless, the overall results were similar to those found in other surveys.^12,13,31

Some of the checklist items presented scoring and interpretation issues. In the title and abstract section (items 1a, 1b), only 2.47% of studies mentioned RCT in the title, but 64 studies (76.2%) stated randomization in the abstract. In the remaining studies, the term RCT was found in the introduction or method section. Therefore screening of title and abstract alone was not sufficient for the Chinese studies to be identified as RCTs. Item 15 specified the use of a table of baseline information which was provided in only 13 studies (16.05%), but 77 studies verbally described the baseline information and only one study didn't report baseline information. Therefore reporting of baseline information was much better than the scoring system suggests. No study reported outcome data as specified by CONSORT 2010 (items 17a, 17b), but all provided numerical data suitable for use in meta-analysis, including mean with SD, rate, and p-value. Consequently, data reporting was not as poor in these Chinese studies as the ratings suggest. Ten studies reported losses and exclusions (item 13b) and four of these imputed losses back in the final analysis (item 16), but there were no losses in the other studies. Since there were no available pretrial protocols for these studies, the authors could not confirm that the final analysis number was the original randomization number. Therefore these studies were judged “NA” and received a zero score. The oncology studies tended to be of inpatients and of short duration so few dropouts were plausible. Li et al.¹³ also found low attrition rates in Chinese trials. Few studies provided clear statements of trial design (item 3a) such as specifying that the study was prospective with parallel groups. However, 79 out of 81 included studies were two armed controlled studies, 37 had equal numbers of participants in both arms, while another two studies were three armed controlled trials. Considering the brevity of the articles it is likely that the authors considered it apparent that a parallel design was used. Adverse event (AE) reporting was difficult to assess in these oncology studies since most included specific categories of chemotherapy-related events as outcome measures. To provide a comparison with Li et al., the authors only scored reports of AEs associated with the traditional medicine interventions. These showed a low level of reporting but most intervention groups combined traditional medicine with chemotherapy so the chemotherapy-related events were not distinguished from those of the traditional medicines.

These examples illustrate difficulties in using the CONSORT 2010 checklist in assessing reporting quality. Incomplete explicit reporting of an item leads to a score of zero although the item may have been partially reported and some items are not applicable in certain studies. Of the eight-item subgroup, items 8b and 10 presented scoring difficulties and 11b may be applicable to certain types of studies only. Nevertheless, the other five items could provide a useful subscore for RCT methodology.

Future directions

There have been a number of calls to improve the quality of reporting in Chinese journals of clinical trials of traditional^{11,26,31

–37} and general medicine.^{9,10,38
–40} As Moher et al. have noted, clinical trial reporting needs to be clear, complete, and transparent; poorly conducted clinical trials waste resources and could mislead decision makers.⁸ This issue is especially important for trials of traditional medicine because the majority of these trials are conducted in China and reported in Chinese journals. The following suggestions may help to improve reporting of clinical trials in China:

(1) Personnel education. According to Wu et al., up to 85% of trial authors in China have not correctly interpreted the methodological principles of randomized trials.⁶ So all personnel who participate in a clinical trial should be properly trained before conducting a RCT.

(2) Require trial registration and a written protocol as a precondition of publication. This can help prevent selective reporting of outcomes, data mistreatment, and duplicated publication.⁸ Registration of a clinical trial is becoming a requirement for publication in medical journals worldwide. A survey of the instructions to authors found that 44 out of 165 “high impact factor medical journals” stated that all recent clinical trials must be registered as a requirement of submission to that journal.⁴¹

(3) Improve publishing guidelines. DerSimonian et al.⁴² pointed out that editors could improve the reporting of clinical trials by providing authors with a list of items to be strictly reported. Empirical evidence has proven that checklists can improve the quality of reporting of clinical trials internationally^15,43 and in China.^10,11,44,45 Thus, Chinese journals should require that CONSORT and the extensions for traditional medicine²⁶ be followed as a requirement for submission.

(4) Balance the need for short articles with the need for complete reporting. Brief reporting has long been the norm in Chinese medical articles and it is a challenge to completely report an RCT according to the CONSORT checklist in two to three pages in a hard copy journal. However, with the use of modern technology greater detail can be included in Supplementary Data and online resources to provide the interested reader and researchers with the data required to assess the quality of the study.

These reforms are all feasible and their implementation would likely lead to greater international impact of clinical research conducted in China.

Conclusions

Relatively few Chinese journals mention CONSORT in their author guidelines, and the quality of reporting of RCTs in Chinese journals remains inadequate. Features of Chinese articles that were associated with better reporting of RCT methodology included ranking of the journal as “core,” public funding of the RCT, and first or correspondent author from a university hospital. Some studies have found improvements in reporting over the past decade, but this sample of RCTs on CRC found little improvement in 15 years. As the volume of scientific information produced in China continues to grow, it is imperative that there is growth in the quality of this information and its international impact.

Footnotes

Acknowledgments

M.C. is supported by an Australian Postgraduate Award at RMIT University, Australia. This project is partially funded by the China-Australia International Research Centre for Chinese Medicine.

Author Disclosure Statement

No competing financial interests exist.

References

Zhou

Ed. Chinese Scientific & Technology Citation Reports, 2013 edition. Beijing: Scientific & Technological Documentation Publishing House, 2014 [in Chinese].

Adams

, King

, Ma

. Global Research Report China. Research and collaboration in the new geography of science. 2009. Online document at: http://sciencewatch.com/sites/sw/files/sw-article/media/grr-china-nov09.pdf 08022017 accessed January 6, 2017 .

Fung

. Chinese journals: A guide for epidemiologists. Emerg Themes Epidemiol, 2008; 5:20.

Xia

, Wright

, Adams

. Five large Chinese biomedical bibliographic databases: Accessibility and coverage. Health Info Libr J, 2008; 25:55–61.

Xiao

, Hu

, Zhang

, et al. Endorsement of CONSORT by Chinese medical journals: A survey of instruction to authors. Chin J Integr Med, 2014; 20:510–515.

, Li

, et al. Randomized trials published in some Chinese journals: How many are randomized?. Trials, 2009; 10:46.

Manheimer

, Wieland

, Kimbrough

, et al. Evidence from the Cochrane Collaboration for traditional Chinese medicine therapies. J Altern Complement Med, 2009; 15:1001–1014.

Moher

, Hopewell

, Schulz

, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. BMJ, 2010; 340:c869.

, Tao

, Zhou

, et al. Endorsement of the CONSORT statement by high-impact medical journals in China: A survey of instructions for authors and published papers. PLoS One, 2012; 7:e30683.

10.

Song

, Leng

, Zhong

, et al. CONSORT in China: Past development and future direction. Trials, 2015; 16:243.

11.

Liu

, Zhang

, Wen

, et al. Impact of the Consolidated Standards of Reporting Trials (CONSORT) checklist on reporting of randomized clinical trials in traditional Chinese medicine. J Evid Based Med, 2015; 8:192–208.

12.

Wang

, Mao

, et al. The quality of reporting of randomized controlled trials of traditional Chinese medicine: A survey of 13 randomly selected journals from mainland China. Clin Ther, 2007; 29:1456–1467.

13.

, Liu

, Chen

, et al. The quality of reports of randomized clinical trials on traditional Chinese medicine treatments: A systematic review of articles indexed in the China National Knowledge Infrastructure Database from 2005 to 2012. BMC Complement Altern Med, 2014; 14:362.

14.

Peron

, Pond

, Gan

, et al. Quality of reporting of modern randomized controlled trials in medical oncology: A systematic review. J Natl Cancer Inst, 2012; 104:982–989.

15.

Hopewell

, Dutton

, Yu

, et al. The quality of reports of randomised trials in 2000 and 2006: Comparative study of articles indexed in PubMed. BMJ, 2010; 340:c723.

16.

Peking University Library Ed. Catalogue of Chinese Core Journals, Sixth Edition. Beijing: Peking University Press, 2014 [in Chinese].

17.

Chinese Society of Traditional Chinese Medicine. Guideline for diagnosis and treatment of tumor in TCM. Beijing: China Traditional Chinese Medicine Press, 2008 [in Chinese].

18.

Liang

HLM

, Chang

DH-T

. Clinical trials of Chinese medicine for the treatment of cancer. In: Cho

WCS

, ed. Supportive Cancer Care with Chinese Medicine. Dordrecht: Springer Netherlands; 2010; 271–291.

19.

Chen

, May

, Zhou

, et al. FOLFOX 4 combined with herbal medicine for advanced colorectal cancer: A systematic review. Phytother Res, 2014; 28:976–991.

20.

Chen

, May

, Zhou

, et al. Integrative medicine for relief of nausea and vomiting in the treatment of colorectal cancer using oxaliplatin-based chemotherapy: A systematic review and meta-analysis. Phytother Res, 2016; 30:741–753.

21.

Chen

, May

, Zhou

, et al. Meta-analysis of oxaliplatin-based chemotherapy combined with traditional medicines for colorectal cancer: Contributions of specific plants to tumor response. Integr Cancer Ther, 2016; 15:40–59.

22.

Chen

, May

, Zhu

, et al. Oxaliplatin-based chemotherapy combined with traditional medicines for neutropenia in colorectal cancer: A meta-analysis of the contributions of specific plants. Crit Rev Oncol Hematol, 2016; 105:18–34.

23.

Moher

, Schulz

, Altman

. The CONSORT Statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials 2001. Explore (NY), 2005; 1:40–45.

24.

Jadad

, Moore

IRA

, Carroll

, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary?. Control Clin Trials, 1996; 17:1–12.

25.

Higgins

, Green

. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. Available from www.handbook.cochrane.org accessed October 18, 2017 .

26.

Bian

, Liu

, Moher

, et al. Consolidated standards of reporting trials (CONSORT) for traditional Chinese medicine: Current situation and future development. Front Med, 2011; 5:171–177.

27.

Cao

, Li

, et al. Yiqizhuyu decoction combined with FOLFOX-4 as first-line therapy in metastatic colorectal cancer. Chin J Integr Med, 2011; 17:593–599.

28.

Yang

. The comparative study of Kang'ai injection combined with FOLFOX4 regimen for advanced colorectal cancer. Fuzhou, Fujian University of Traditional Chinese Medicine. Master Thesis, 2008 [in Chinese].

29.

Zheng

Colorectal cancer postoperative chemotherapy of integrative medicine. Guangzhou University of TCM. Master Thesis 2011 [in Chinese].

30.

Turner

, Shamseer

, Altman

, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev, 2012; 11:MR000030.

31.

, Du

, Liu

, et al. Quality assessment of reporting of randomization, allocation concealment, and blinding in traditional Chinese medicine RCTs: A review of 3159 RCTs identified from 260 systematic reviews. Trials, 2011; 12:122.

32.

Gagnier

, Boon

, Rochon

, et al. Reporting random controlled trials of herbal medicines. Explore, 2006; 2:143–149.

33.

Dagenais

, Tricco

, Bian

, et al. Critical appraisal of clinical studies in Chinese herbal medicine. Zhong Xi Yi Jie He Xue Bao, 2006; 4:455–466.

34.

Zhong

, Fu

, Liu

, et al. The reporting quality, scientific rigor, and ethics of randomized placebo-controlled trials of traditional Chinese medicine compound formulations and the differences between Chinese and non-Chinese trials. Curr Ther Res Clin Exp, 2010; 71:30–49.

35.

, Hu

, Liu

. The reporting quality assessment of complex interventions articles in traditional Chinese medicine. Evid Based Complement Alternat Med, 2013; 250690.

36.

Wang

, Xu

, Sun

, et al. Assessment of the reporting quality of randomized controlled trials on the treatment of diabetes mellitus with traditional Chinese medicine: A systematic review. PLoS One, 2013; 8:e70586.

37.

Fung

, Linn

. Developing traditional Chinese medicine in the era of evidence-based medicine: Current evidences and challenges. Evid Based Complement Alternat Med, 2015; 425037.

38.

, Li

, Zhang

, et al. Chinese authors do need CONSORT: Reporting quality assessment for five leading Chinese medical journals. Contemp Clin Trials, 2008; 29:727–731.

39.

, Ke

, Chen

, et al. Does the reporting of randomized clinical trials published in Chinese pediatrics journals improve after the CONSORT statement is adopted?. Contemp Clin Trials, 2012; 33:889–894.

40.

Shi

, Tian

, Ren

, et al. Methodological reporting of randomized trials in five leading Chinese nursing journals. PLoS One, 2014; 9:e113002.

41.

Hopewell

, Altman

, Moher

, et al. Endorsement of the CONSORT Statement by high impact factor medical journals: A survey of journal editors and journal ‘Instructions to Authors’. Trials, 2008; 9:20.

42.

DerSimonian

, Charette

, et al. Reporting on methods in clinical trials. N Engl J Med, 1982; 306:1332–1337.

43.

Plint

, Moher

, Morrison

, et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust, 2006; 185:263–267.

44.

, He

, et al. Impact evaluation of CONSORT and STRICTA guidelines on reporting quality for randomized controlled trials of acupuncture conducted in China. Chin J Integr Med, 2017; 23:10–17.

45.

, Chen

, Xu

, et al. Do the CONSORT and STRICTA checklists improve the reporting quality of acupuncture and moxibustion randomized controlled trials published in Chinese journals? A systematic review and analysis of trends. PLoS One, 2016; 11:e0147244.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB