Poor Result Reporting Rate in Cell Therapy Trials Registered at ClinicalTrials.gov

Abstract

As research associates in clinical experiments, we have an obligation to disclose clinical methodologies and findings in full transparency in ethics. However, inadequate disclosure in results reporting clinical trials registered on ClinicalTrials.gov has been revealed, with approximately half the trial results not being reported in an applicable manner. Our recent study in clinical trials of regenerative medicine for four kinds of neurological diseases revealed that the rate of result reporting to ClinicalTrials.gov is inadequate for gene and cell therapy (CT) trials. In this path, further curiosity emerged to see what the findings would be if the analysis was conducted for trials in all disease areas, and outcomes if gene therapy (GT) and CT were distinguished in terms. In this study, the scope of analysis was further expanded to include all disease areas, and the drug classification from the AdisInsight database was used for modality classification, with biologic drug trials classified as controls, CT, ex vivo GT, and in vivo GT. To begin, among all interventional clinical trials with registration in the ClinicalTrials.gov registry and with a primary completion between 2010 and 2019, we created a total of 5539 datasets corresponding to trials classified as GT and CT, while biologics (BLG) as controls in the AdisInsight drug classification. The status of reported results of these trials was identified by surveying posting status of ClinicalTrials.gov and publication in journals (PubMed), respectively. Based on the obtained dataset, multivariate analysis was performed on the data on the reporting rate of clinical trial results, aggregated by sponsor, phase, status, and modality (CT, ex vivo GT, in vivo GT, and BLG), respectively. The result shows that CT was identified as an independent factor restraining result reporting ratio in both ClinicalTrials.gov and total disclosures, whereas ex vivo GT as boosting result reporting ratio. Since the result reporting rate of CT results was notably poor, we discussed the causes and solutions in this regard.

Impact statement

In our recent study, we brought up an inadequate disclosure in results reporting clinical trials in regenerative medicine with limited disease area. In this analysis, we performed multivariate analysis by classifying regenerative medicine into cell therapy (CT), ex vivo gene therapy (GT), and in vivo GT without any limitation for targeting disease area. To the best of our knowledge, it is first time to claim that the result reporting rates for CT trials are particularly poor. Regenerative medicine holds great promise for the future, and prompt and transparent sharing of clinical trial results are essential for its proper translation.

Introduction

Globally, cell and gene therapy (GT) technologies have achieved breakthroughs in the past decades, while conducting clinical trials in broader therapeutic areas, and have rapidly progressed toward practical applications. However, have the results of their clinical trials been reported adequately?

ClinicalTrials.gov¹ and other trial registries^2,3 have established ethical and legal obligations to disclose trial protocols and outcomes promptly; therefore, the clinical research team needs to share clinical trial protocols and results with the public, with sufficient transparency. However, Anderson et al.⁴ observed that reporting rates of clinical trial results were insufficient. In addition to analyses of study results reporting rates for unlimited disease areas, analyses of result reporting for limited disease areas, such as cancer⁵ and urology,⁶ have revealed poor study result reporting rates.

Given that analysis follows a different approach, the factors affecting the under-reporting of results included sponsors,^7,8 clinical phase,⁸ and recruitment status.⁵ Amid the growing landscape of clinical research activities involving regenerative medicine areas, such as gene therapies, to promote clinical research safely and without unnecessary duplication, the results of clinical trials must be shared with experts. As part of a study aimed at the practical application of regenerative medicine, this study investigated the factors that influence the rate of reporting of clinical trial results.

Initially, we could not find such an analysis for regenerative medicine trials, so we conducted a pilot study of analyzing result reporting rates in four neurological diseases with regenerative medicine registered with ClinicalTrials.gov as our previous report.⁹ In doing so, it is inappropriate to classify treatment modalities by the terms listed in the intervention column (e.g., “drug,” “biological,” “genetic”) because ClinicalTrials.gov data are not accurately categorized by treatment modality.

Therefore, in the previous study,⁹ the treatment modality was manually determined from the listed drug names in the intervention column and analyzed. The total number of trials was small due to the limited therapeutic area, which allowed for the manual classification of therapeutic modalities such as gene and cell therapy (CT), small-molecule compounds, and biologics (BLG). Based on the treatment modality classification, and by setting up small-molecule compounds and BLG as study controls, a comparative analysis was performed. As a result, despite the dataset's small sample size, the reporting rate of regenerative medicine study results was found to be substantially poor. In this path, further curiosity emerged to see what the findings would be if the analysis was conducted for trials in all disease areas, and outcomes if GT and CT were distinguished in terms.

Taking advantage of the fact that the AdisInsight database (Springer Nature) categorizes drugs by therapeutic modality, such as BLG, GT, and CT, to extend the scope of analysis to all disease areas in this study, the AdisInsight database was used instead of manual classification. The data of 5539 clinical trials registered with primary completion dates between 2010 and 2019 at ClinicalTrials.gov were extracted. The clinical trial was divided into GT and CT, and the GT was further divided into in vivo and ex vivo, while BLG (i.e., other than CT and GT) were used as the study control.

In addition to the treatment modality, three factors that had been previously reported—sponsor, trial phase, and recruitment status—were also investigated. Moreover, an analysis was conducted to determine the result reporting rates among submissions to ClinicalTrials.gov or journal publications (PubMed). CT was identified as an independent suppressing factor for the rate of result reporting for ClinicalTrials.gov and total dissemination, while ex vivo GT was identified as an independent increasing factor. Based on findings, to facilitate result reporting in the field of regenerative medicine, some suggestions to improve the issue were also examined.

Methods

Design

This is a retrospective cohort study using the ClinicalTrials.gov database and is not human-subject research.

Data collection

(1)

We used AdisInsight “Drug Class Hierarchy” with treatment modalities categorized as drug classes to classify the treatment modality. The dataset of “Cell therapies trial set” as well as “Gene therapies trial set” were downloaded from the AdisInsight website using “Cell therapies” and “Gene therapies” as keywords, respectively. Since CT and GT are subjects of this analysis, as well as subject to the Biologics License Application, we used BLG trial data as a control. That is, the field of “Drug Class” was searched in the AdisInsight database using the following search terms: “Biological intercellular signaling peptides and proteins,” “Biological peptides,” “Biological proteins,” “Bacterial proteins,” “Blood proteins,” “Bone morphogenetic proteins,” “Cytokines,” “Globulins,” “Glycoproteins,” “Recombinant proteins,” “Silver proteins,” “Viral proteins” or “Blood products and substitutes.” We obtained a dataset that includes at least one of the search terms and named it the “Biologics” trial set.

(2)

We extracted all registered data that contained NCT numbers in the Trial Identifier field from each “Biologics,” “Cell therapies,” and “Gene therapies” trial set and we labeled each of them as a candidate number of drug class for each trial set.

(3)

We downloaded a dataset from the ClinicatTrials.gov website across the intervention trials in Phase I/II to Phase IV with a primary completion date from 2010 to 2019. The “Intervention” column was searched with the following keywords: “Drug:,” “Biological:,” “Genetic:,” and “Combination:,” and trials that did not correspond were excluded, and further, trials with “withdrawn” in the “status” column were excluded.

(4)

The collected dataset was designated as SearchResults 2010_2019. We selected clinical trials categorized into one of three treatment modalities by searching the NCT number field in the dataset of “SearchResults 2010_2019” with the obtained candidate number. Next, we selected duplicate trials between gene therapies and cell therapies, and thereafter designated them as ex vivo GT. We also defined the treatment modality classifications as follows: in vivo GT (excluding trials where trials were duplicating CT), CT (excluding trials where trials were duplicating GT), and BLG (excluding GT and CT).

(5)

Finally, we selected trials with “Actual Study Completion Date” as “completed” trials for this analysis.

Data of result reporting

We applied trials with “Has Results” in the “Study Results” field of the ClinicalTrials.gov registry as reported trials to ClinicalTrials.gov. We defined the publication of the research article by searching the PubMed database using the NCT number assigned to each clinical trial as an index, and articles that had published the results of the clinical trial were included in this analysis. We manually reviewed each article whether the results that correspond to at least one primary outcome measure were reported in PubMed with confirmed results, were considered “result reported.” Articles without corresponding results, with the study designs or protocols only, and withdrawn were considered no result reported. The publication date was adopted as the date published on the PubMed or the electronic publication date, if available, whichever is earlier. Articles published before the “Primary Completion Date” and secondary sources, such as reviews, were excluded.

Sponsor classification

To clarify the relationship between reporting the results and study sponsors, we classified sponsoring organizations as big pharmaceuticals, small pharmaceuticals, or academia. The top 66 companies,¹⁰ with revenues exceeding $2 billion in the global pharmaceutical industry in 2018, were classified as “big pharma”; smaller companies in terms of revenue other than big pharma were designated as “small pharma”; and noncommercial organizations, that is, universities, hospitals, and government organizations, were designated as “academia.”

Phase classification

The higher phase was adopted as the clinical phase for analysis and Phase I/II and Phase II/III were merged into Phases II and III, respectively.

Decreasing the time course of the unreported result ratio

The decreasing curves of the unreported result ratio due to time passed from the primary completion were analyzed using Kaplan–Meier estimator analysis. We defined result reporting dates as either the dates posted as “Result First Posted” in ClinicalTrials.gov or the dates published in journals (PubMed). For total dissemination, if results were reported in ClinicalTrials.gov and PubMed, the earlier report was adopted. Furthermore, if results were not reported by the data collection date (December 1, 2021), this data collection date was used as the last update, thus counting the number of days from the “actual primary completion date.”

Software

The downloaded CSV file was subjected to data cleaning and tagging using Microsoft Excel. IBM SPSS Statistics v28 (IBM Corp., Armonk, NY) was used for statistical analysis.

Statistical analysis

IBM SPSS Statistics v.28 (IBM Corp.) was used for statistical analysis. The significance level was set as p = 0.05, unless otherwise specified. The Clopper–Pearson method was used to calculate the 95% confidence intervals (95% CIs) for the population rate. The unreported result ratios in the time course for each of the four groups (BLG, CT, ex vivo GT, and in vivo GT) were illustrated by Kaplan–Meier curves. Please be aware that Figure 4 only depicted those results reported within 5 years from the primary completion date. In addition, a log-rank test was performed to test combinations between two groups of six patterns for each of the four groups' curves followed by the Bonferroni method (using an altered significance level of 0.05/6 = 0.008333). These statistical analyses were performed across the entire period.

Experiment

Data collection

Figure 1 illustrates the Venn diagram of modality classification used in this study. And, summary of the data collection is shown in Figure 2. The dataset of SearchResults 2010_2019 was downloaded from the ClinicatTrials.gov website on December 1, 2021. The dataset consists of 52,003 trials. Moreover, on December 2, 2021, the trial module datasets were downloaded from the AdisInsight website with a given treatment modality for a drug class. Then, from the downloaded trial datasets, ClinicalTrials.gov registration numbers (NCT number starting with NCT0) were selected in the trial identifier column. Only trial numbers registered with ClinicalTrials.gov were extracted from the AdisInsight data.

FIG. 1.

Schematic representation of abbreviations for treatment modalities. CT: clinical trials evaluating cell therapies, not including gene therapies, in vivo GT: clinical trials evaluating gene therapies, not including cell therapies, ex vivo GT: duplication of trials queried by gene therapies and cell therapies by AdisInshgiht database, BLG: clinical trials evaluating BLG, not including gene therapies and cell therapies. Using the drug class (speech balloons) in the AdisInsight database, trials that evaluated therapeutic drug candidates subject to the BLA were classified as BLG, CT, in vivo GT, and ex vivo GT and used for analysis. BLA, Biologics License Application; BLG, biologics; CT, cell therapy; GT, gene therapy.

FIG. 2.

Schematic representation of data collection and processing. n = 25,583 BLG, 1303 cell therapies, and 1771 gene therapies were obtained as NCT numbers of trials to evaluate drug candidates of each modality based on the information obtained from the Drug Class of AdisInsight. Using these as references, 52,003 SearchResults 2010_2019 trials were surveyed to select 6703 matched BLG, cell therapies, and gene therapies. Data cleaning was conducted, and trials without primary completion dates were excluded, resulting in 5101 BLG, 253 CT, 133 in vivo GT, and 52 ex vivo GT, for a total of 5539 trials. These were used for subsequent analyses.

Using these categorized NCT numbers, 6703 trials corresponding to the three treatment modalities were identified by screening the SearchResults 2010_2019 dataset. In this dataset, eight trials did not meet our criteria for BLG, extracted as errors, and 6695 trials were classified as verified. Furthermore, 1156 trials with no primary completion date were excluded, and 5539 trials with actual primary completion dates were used for the analysis. The final breakdown for BLG, CT, in vivo GT, and ex vivo GT were 5101, 253, 133, and 52, respectively.

Data analysis

In addition to treatment modality, sponsors, trial phases, and the status of recruitment, which were previously reported in other studies as affecting factors, were analyzed to compare the result reporting rates. Figure 3a shows the result reporting rate of trials classified by the sponsors. Sponsors with big pharma had a total result dissemination rate of 95.5% (95% confidence interval: 94.6%–96.4%), and the reporting rate of ClinicalTrials.gov was 92.5% (95% CI: 91.3–93.6). In contrast, for small pharma and academia, the total reporting rate was <70%, and the reporting rate to ClinicalTrials.gov was 56.6% (95% CI: 54.2–59.1) and 49.0% (95% CI: 46.8–51.3), respectively. Therefore, for the trial sponsors with big pharma, the reporting of study results is nearly complete, while for small pharma and academia, the reporting of results is still insufficient.

FIG. 3.

(a) Result reporting rate of clinical trials classified by sponsor. (b) Result reporting rate of clinical trials classified by phase. (c) Result reporting rate of clinical trials classified by treatment modality. (d) Result reporting rate of clinical trials classified by recruitment status. We did not include “Active, not recruiting,” “Unknown Status,” “Enrolling by invitation,” “Recruiting,” and “Suspended” from this analysis, as we considered these as error. Please refer main text.

Figure 3b shows the result reporting rate classified by phases, with a total dissemination rate of 84.8% (95% CI: 83.1–86.4) for Phase III, compared to 75.7% (95% CI: 74.2–77.2) and 71.2% (95% CI: 67.2–75.0) for Phase II and Phase IV, respectively. The reporting rate to ClinicalTrials.gov was 75.6% (95% CI: 73.6–77.5) for Phase III, 64.2% (95% CI: 62.5–65.9) for Phase II, and only 55.3% (95% CI: 51.0–59.5) for Phase IV. The comparison revealed that compared to the reporting rate for Phase III, Phase II and Phase IV had a lower result reporting rate.

Figure 3c shows reporting rates by modality. The total dissemination rate by modalities was 79.6% (95% CI: 78.4–80.7) for BLG, 74.4% (95% CI: 66.2–81.6) for in vivo GT, and 88.5% (95% CI: 76.6–95.6) for ex vivo GT, while for CT, it was only 53.8% (95% CI: 47.4–60.0). The same trend of reporting rates to ClinicalTrials.gov was 68.9% (95% CI: 67.7–70.2) for BLG, 55.6% (95% CI: 46.8–64.2) for in vivo GT, and 75.0% (95% CI: 61.1–86.0) for ex vivo GT, compared to 36.4% (95% CI: 30.4–42.6) for CT. This indicated that CT, in vivo GT, and ex vivo GT did not have uniformly lower reporting rates compared to BLG, but CT had the lowest reporting rate.

Figure 3d shows the reporting rates by recruitment status. There were no major differences in patient recruitment status in terms of total dissemination rate and ClinicalTrials.gov reporting rate, but the “Terminated” trials (in which studies have stopped early and will not start again) had a significantly lower publication disclosure (PubMed) rate. Although Liu et al.⁵ reported that an oncology field analysis indicated that “Terminated” trials had lower result reporting rates than “Completed” trials (in which studies have ended normally), in this study, despite the poor result reporting rate in PubMed, the total reporting rate was comparable.

Multivariate analysis was conducted across the sponsor, phase, recruitment status, and modality, highlighting the factors that affect result reporting rates (Tables 1–3). Multivariate analysis with completed trials as a reference in recruitment status identified “Terminated” as an increasing factor, affecting result reporting rate at ClinicalTrials.gov (Table 1), and conversely a decreasing factor in publication in journals (PubMed) (Table 2); however, none of the factors was found in the total dissemination (Table 3).

Table 1.

Multivariate Analyses on the Results Posted at ClinicalTrials.gov

Variables	Trials with actual completion date	Reported to Clinical Trials.gov	Clopper-Pearson	Multivariate analysis
Variables	Trials with actual completion date	Reported to Clinical Trials.gov	Proportion (%) (95% CI)	Crude OR (95% CI)	p	Adjusted OR (95% CI)	p
Total	5539	3722	67.2 (65.9–68.4)
Status:
Completed	4222	2800	66.3 (64.9–67.7)	Reference		Reference
Terminated	894	660	73.8 (70.8–76.7)	1.43 (1.22–1.69)	<0.001	1.82 (1.53–2.17)	<0.001
Unknown^*	423	262	61.9 (57.1–66.6)	0.83 (0.67–1.02)	0.07	0.95 (0.75–1.20)	0.671
Phase:
II	3119	2003	64.2 (62.5–65.9)	0.58 (0.51–0.66)	<0.001	1.14 (0.98–1.33)	0.101
III	1874	1417	75.6 (73.6–77.5)	Reference		Reference
IV	546	302	55.3 (51.0–59.5)	0.40 (0.33–0.49)	<0.001	0.60 (0.47–0.75)	<0.001
Sponsor:
Academia	1934	948	49.0 (46.8–51.3)	0.74 (0.64–0.84)	<0.001	0.70 (0.61–0.81)	<0.001
Big pharma	2041	1888	92.5 (91.3–93.6)	9.44 (7.79–11.45)	<0.001	9.47 (7.76–11.56)	<0.001
Small pharma	1564	886	56.6 (54.2–59.1)	Reference		Reference
Modality:
BLG	5101	3517	68.9 (67.7–70.2)	Reference		Reference
CT	253	92	36.4 (30.4–42.6)	0.26 (0.20–0.34)	<0.001	0.39 (0.30–0.52)	<0.001
Ex vivo GT	52	39	75.0 (61.1–86.0)	1.35 (0.72–2.54)	0.349	2.04 (1.08–3.89)	0.029
In vivo GT	133	74	55.6 (46.8–64.2)	0.57 (0.40–0.80)	0.001	0.81 (0.56–1.16)	0.252

Unknown^* includes “Unknown status,” “Enrolling by invitation,” “Recruiting,” and “Suspended.”

ORs <1.0 with significance (p < 0.05) are highlighted in bold.

ORs >1.0 with significance (p < 0.05) are highlighted in ital.

95% CI, 95% confidence interval; BLG, biologics; CT, cell therapy; GT, gene therapy.

Table 2.

Multivariate Analyses on the Results Published in Journals (PubMed)

Variables	Trials with actual completion date	Reported in journals (PubMed)	Clopper-Pearson	Multivariate analysis
Variables	Trials with actual completion date	Reported in journals (PubMed)	Proportion (%) (95% CI)	Crude OR (95% CI)	p	Adjusted OR (95% CI)	p
Total	5539	2512	45.4 (44.0–46.7)
Status:
Completed	4222	2094	49.6 (48.1–51.1)	Reference		Reference
Terminated	894	170	19.0 (16.5–21.7)	0.24 (0.20–0.29)	<0.001	0.25 (0.21–0.30)	<0.001
Unknown^*	423	248	58.6 (53.8–63.4)	1.44 (1.18–1.76)	<0.001	1.59 (1.29–1.97)	<0.001
Phase:
II	3119	1169	37.5 (35.8–39.2)	0.40 (0.36–0.45)	<0.001	0.53 (0.47–0.61)	<0.001
III	1874	1125	60.0 (57.8–62.3)	Reference		Reference
IV	546	218	39.9 (35.8–44.2)	0.44 (0.36–0.54)	<0.001	0.49 (0.40–0.60)	<0.001
Sponsor:
Academia	1934	673	34.8 (32.7–37.0)	0.90 (0.78–1.03)	0.119	1.04 (0.89–1.21)	0.624
Big pharma	2041	1255	61.5 (59.3–63.6)	2.68 (2.34–3.07)	<0.001	2.47 (2.14–2.86)	<0.001
Small pharma	1564	584	37.3 (34.9–39.8)	Reference		Reference
Modality:
BLG	5101	2368	46.4 (45.0–47.8)	Reference		Reference
CT	253	73	28.9 (23.4–34.9)	0.47 (0.36–0.62)	<0.001	0.77 (0.57–1.03)	0.074
Ex vivo GT	52	23	44.2 (30.5–58.7)	0.92 (0.53–1.59)	0.753	1.70 (0.95–3.03)	0.075
In vivo GT	133	48	36.1 (27.9–44.9)	0.65 (0.46–0.93)	0.019	1.01 (0.69–1.48)	0.954

Unknown^* includes “Unknown status,” “Enrolling by invitation,” “Recruiting,” and “Suspended.”

ORs <1.0 with significance (p < 0.05) are highlighted in bold.

ORs >1.0 with significance (p < 0.05) are highlighted in ital.

Table 3.

Multivariate Analyses on the Results Disclosed in Total Dissemination

Variables	Trials with actual completion date	Total dissemination	Clopper-Pearson	Multivariate analysis
Variables	Trials with actual completion date	Total dissemination	Proportion (%) (95% CI)	Crude OR (95% CI)	p	Adjusted OR (95% CI)	p
Total	5539	4340	78.4 (77.2–79.4)
Status:
Completed	4222	3322	78.7 (77.4–79.9)	Reference		Reference
Terminated	894	697	78.0 (75.1–80.6)	0.96 (0.81–1.14)	0.634	1.09 (0.90–1.31)	0.371
Unknown^*	423	321	75.9 (71.5–79.9)	0.85 (0.67–1.08)	0.183	0.91 (0.71–1.17)	0.482
Phase:
II	3119	2362	75.7 (74.2–77.2)	0.56 (0.48–0.65)	<0.001	0.94 (0.79–1.11)	0.448
III	1874	1589	84.8 (83.1–86.4)	Reference		Reference
IV	546	389	71.2 (67.2–75.0)	0.44 (0.36–0.56)	<0.001	0.58 (0.46–0.75)	<0.001
Sponsor:
Academia	1934	1320	68.3 (66.1–70.3)	0.99 (0.86–1.15)	0.918	1.03 (0.88–1.20)	0.752
Big pharma	2041	1950	95.5 (94.6–96.4)	9.89 (7.82–12.52)	<0.001	9.59 (7.54–12.20)	<0.001
Small pharma	1564	1070	68.4 (66.0–70.7)	Reference		Reference
Modality:
BLG	5101	4059	79.6 (78.4–80.7)	Reference		Reference
CT	253	136	53.8 (47.4–60.0)	0.30 (0.23–0.39)	<0.001	0.49 (0.37–0.63)	<0.001
Ex vivo GT	52	46	88.5 (76.6–95.6)	1.97 (0.84–4.62)	0.12	3.17 (1.34–7.47)	0.008
In vivo GT	133	99	74.4 (66.2–81.6)	0.75 (0.50–1.11)	0.149	1.15 (0.77–1.73)	0.493

Unknown^* includes “Unknown status,” “Enrolling by invitation,” “Recruiting,” and “Suspended.”

ORs <1.0 with significance (p < 0.05) are highlighted in bold.

ORs >1.0 with significance (p < 0.05) are highlighted inital.

For sponsors, setting up small pharma as a reference, multivariate analysis was conducted and it was observed that the big pharma is an increasing factor of result reporting rate for posting at ClinicalTrials.gov, publication in a journal (PubMed), and total dissemination (Tables 1–3), while academia is a decreasing factor of result reporting rate for posting at ClinicalTrials.gov (Table 1).

In the clinical phase, Phase IV was identified as a decreasing factor for posting at ClinicalTrials.gov, publication, and total dissemination in the multivariate analysis when Phase III was set up as a reference (Tables 1–3). Phase II was identified as the decreasing factor only for publication (PubMed) (Table 2).

For treatment modalities, CT was identified as an independent factor causing a decrease in the result reporting rates for posting at ClinicalTrials.gov (Table 1) and total dissemination (Table 3) in multivariate analysis when BLG was set up as a reference. In contrast, ex vivo GT was identified as an increasing factor in the result reporting rates for posting at ClinicalTrials.gov (Table 1) and total dissemination (Table 3).

Table 4 shows the result reporting rates by cross-analysis of sponsor and modality. The number of eligible trials (and proportion) sponsored by major pharmaceutical companies in each modality was 2165 (35.6%) for BLG, compared to 8 (2.3%), 11 (6.2%), and 2 (2.1%) for CT, in vivo GT, and ex vivo GT, respectively.

Table 4.

Result Reporting Rate of Clinical Trials Classified by Sponsor × Modality

Variables	Trials with actual completion date	ClinicalTrials.gov		PubMed		Total dissemination
Variables	Trials with actual completion date	Trials Reported to Clinical Trials.gov	Clopper-Pearson Proportion (%) (95% CI)	Trials Published in journals (PubMed)	Clopper-Pearson Proportion (%) (95% CI)	Total dissemination Trials	Clopper-Pearson Proportion (%) (95% CI)
BLG × Big Pharma	2165	1874	86.6 (85.0–88.0)	1297	59.9 (57.8–62.0)	1985	91.7 (90.4–92.8)
BLG × Small Pharma	1631	774	47.5 (45.0–49.9)	600	36.8 (34.4–39.0)	1009	61.9 (59.5–64.2)
BLG × Academia	2277	869	38.1 (36.2–40.2)	720	31.6 (29.7–34.0)	1314	57.7 (55.6–59.7)
CT × Big Pharma	8	3	37.5 (8.5–75.5)	2	25.0 (3.2–65.0)	5	62.5 (24.5–91.5)
CT × Small Pharma	233	52	22.3 (17.1–28.2)	64	27.5 (21.8–34.0)	100	42.9 (36.5–49.5)
CT × Academia	106	37	34.9 (25.9–44.8)	28	26.4 (18.3–36.0)	52	49.1 (39.2–59.0)
In vivo GT × Big Pharma	11	9	81.8 (48.2–97.7)	6	34.5 (23.4–83.0)	10	90.9 (58.7–99.8)
In vivo GT × Small Pharma	122	40	32.8 (24.6–41.9)	35	28.7 (20.9–38.0)	63	51.6 (42.4–60.8)
In vivo GT × Academia	45	25	55.6 (40.0–70.4)	17	37.8 (23.8–54.0)	36	80.0 (65.4–90.4)
Ex vivo GT × Big Pharma	2	2	100 (15.8-)	2	100 (15.8-)	2	100 (15.8-)
Ex vivo GT × Small Pharma	57	20	35.1 (22.9–48.9)	18	31.6 (19.9–45.0)	31	54.4 (40.7–67.6)
Ex vivo GT × Academia	38	17	44.7 (28.6–61.7)	13	34.2 (19.6–51.0)	23	60.5 (43.4–76.0)

Cells discussed in the main text are highlighted in bold.

Despite the large gap, the reporting rates of big pharma consistently exceed those of small pharma and academia in terms of result reporting rates to ClinicalTrials.gov and total dissemination, indicating that large pharmaceutical companies are a strong factor in increasing result reporting rates. In terms of total reporting rate, in vivo GT was 51.6% (95% CI: 42.4–60.8) and 42.9% (95% CI: 36.5–49.5) in small pharma compared to CT, respectively, which is similar, whereas in terms of academia reporting rate, in vivo GT was significantly higher at 80.0% (95% CI: 65.4–90.4), which is significantly higher compared with CT at 49.1% (95% CI: 39.2–59.0).

Table 5 shows the result reporting rates by cross-analysis of phase and modality. There was no Phase IV studies in CT and ex vivo GT to meet the criteria of this analysis. CT × Phase III had the lowest total dissemination rate with 45% and the reporting rate to ClinicalTrials.gov with 25% was the lowest among the 10 cross groups.

Table 5.

Result Reporting Rate of Clinical Trials Classified by Phase × Modality

Variables	Trials with actual completion date	ClinicalTrials.gov		PubMed		Total dissemination
Variables	Trials with actual completion date	Trials Reported to Clinical Trials.gov	Clopper-Pearson Proportion (%) (95% CI)	Trials Published in journals (PubMed)	Clopper-Pearson Proportion (%) (95% CI)	Total dissemination Trials	Clopper-Pearson Proportion (%) (95% CI)
BLG × Phase II	2737	1817	66.4 (64.6–68.0)	1051	38.4 (36.6–40.0)	2114	77.2 (75.6–78.8)
BLG × Phase III	1819	1398	76.9 (74.8–79.0)	1100	60.4 (58.2–63.0)	1557	85.6 (83.9–87.2)
BLG × Phase IV	545	302	55.4 (51.1–60.0)	217	39.8 (35.7–44.0)	388	71.2 (67.2–75.0)
CT × Phase II	222	84	37.8 (31.4–45.0)	61	27.5 (21.7–58.0)	122	55.0 (48.2–61.6)
CT × Phase III	31	8	25.8 (11.9–45.0)	12	38.7 (21.9–58.0)	14	45.2 (27.3–64.0)
CT × Phase IV	0	0		0		0
In vivo GT × Phase II	110	64	58.2 (48.4–68.0)	35	31.8 (23.3–41.0)	82	74.5 (65.4–82.4)
In vivo GT × Phase III	22	10	45.5 (24.4–68.0)	12	54.5 (32.2–76.0)	16	72.7 (49.8–89.3)
In vivo GT × Phase IV	1	0	0 (−98.0)	1	100 (2.5-)	1	100 (2.5-)
Ex vivo GT × Phase II	50	38	76.0 (61.8–87.0)	22	44.0 (30.0–59.0)	44	88.0 (75.7–95.5)
Ex vivo GT × Phase III	2	1	50.0 (1.3–99.0)	1	50.0 (1.3–99.0)	2	100 (15.8-)
Ex vivo GT × Phase IV	0	0		0		0

Cells discussed in the main text are highlighted in bold.

The number of eligible trials (and proportion) of Phase III or higher phase for each modality was 2364 (46.3%) for BLG, 31(12.3%) for CT, 23 (17.3%) for in vivo GT, and 2 (3.8%) for ex vivo GT. Based on the percentage of Phase III and Phase IV trials for each modality, it is necessary to consider that CT, in vivo GT, and ex vivo GT are more recent modalities compared to the BLG studies. In addition, the reporting rates to ClinicalTrials.gov and total dissemination of Phase II × CT were significantly lower than those of Phase II × in vivo GT and those of Phase II × ex vivo GT.

Finally, the trends in the rate of unreported results were analyzed for modality using the Kaplan–Meier estimator over the 5 years from the date of primary completion (Fig. 4). Although an analysis of the entire 2010–2019 period was conducted, Figure 4 illustrates a 5-year period from the primary completion date of each trial. In terms of result reporting within ClinicalTrials.gov, the clinical trial results must be reported within 1 year from the date of completion according to regulations. However, these results were seldom reported on time in all modalities (Fig. 4a). Therefore, an analysis was conducted at 2 years with more practicality, as described by Liu et al.⁵ It revealed a decrease in the rate of unreported results in ex vivo GT at 52%, BLG at 55%, and in vivo GT at 70%. CT was the worst at 80%.

FIG. 4.

Unreported result ratio trucked by modality for 5 years from the primary completion date. (a) Results posted at ClinicalTrials.gov. (b) Result published in journals (PubMed). (c) Total dissemination. *p < 0.05. The red vertical line indicates 2 years after the primary completion date.

The test of significance in these modalities was performed and the significance was found in five combinations, excluding the combination of ex vivo GT and BLG. In terms of total dissemination in the rate of unreported results (Fig. 4c), a similar trend was observed with ex vivo GT at 44%, BLG at 45%, in vivo GT at 55%, and CT at 73%. This indicates that a significant difference was observed in CT compared to the other modalities. In terms of results published in journals (PubMed) (Fig. 4b), unreported results in CT were 89% and other modalities were 77%–79%, indicating that a more significant difference was observed in CT compared with BLG. Based on the Kaplan–Meier estimator analysis, the rates of result reporting in CT were lower than in others.

Discussion

To achieve practical applications in the field of regenerative medicine, the objective of this work was to determine whether the results of clinical trials are adequately reported, and if not, to determine the causes, and identify opportunities for improvement. For treatment modalities, CT was identified as an independent factor causing a decrease in the result reporting rates on ClinicalTrials.gov and total dissemination in multivariate analysis when BLG was set up as a reference, whereas ex vivo GT was identified as an increasing factor in the result reporting rates.

To our knowledge, this is the first time it has been identified. Furthermore, Kaplan–Meier analysis was used to compare treatment modalities and revealed that CT reporting rates were significantly lower on ClinicalTrials.gov, among journal publications (PubMed), and in total dissemination. In addition, they were less than half the reporting rates of BLG and ex vivo GT compared to the unreported results 2 years after the studies' primary completion dates, demonstrating that the result reporting of CT is inadequate compared with BLG and ex vivo GT.

The impact of treatment modality, as well as sponsor, was examined on the outcomes of our previous study,⁹ which focused on only four different types of neurological diseases, while CT and GT trials were summed together as GCT (gene and/or CT), and it revealed that GCT had lower result reporting rates than small-molecule drugs and BLG.

A breakdown of the previous results showed that only two trials were classified as ex vivo GT, both sponsored by big pharma, but no report of their results was found. In contrast, on the other hand, the remaining trials classified as GCT were mostly CT equivalent, with 5 in vivo GT and 35 CT equivalents. The observed result differences were thought to be due to the insufficient sample size of ex vivo GT equivalents in four neurological diseases, being greatly influenced by the CT majority factor. In this work, this is thought to have been solved by and classifying CT, in vivo GT, and ex vivo GT separately and performing a complete survey.

We would now like to discuss the reasons for the low result reporting rate of CT in this study. Despite the fact that ex vivo GT had a similar small proportion of big pharma sponsors as CT, and despite the fact that ex vivo GT had a smaller proportion of higher phase clinical trials, the reporting rate of results for ex vivo GT was excellent (Table 4).

However, assuming that ex vivo GT is due to the particular success of CAR-T commercialization, we would provisionally exclude ex vivo GT and consider in vivo GT versus CT here. In vivo GT did not differ from CT in terms of the total dissemination rate among small pharma, but was significantly higher in terms of the reporting rate in academia. In other words, in vivo GT does not suppress the reporting rate even in trials conducted by academia. In contrast, CT × academia suppressed the reporting rate. This suggests that there is some difference between CT × academia and GT × academia.

On the other hand, in a cross-analysis of Phase × modality (Table 5), the reporting rates to ClinicalTrials.gov and total dissemination of Phase II × CT were significantly lower than those of Phase II × in vivo GT and those of Phase II × ex vivo GT. As noted in our previous report,⁹ this is suspected to be related to the tendency of investigators to be reluctant to report “unfavorable” results, which accounts for the low reporting rate of Phase II trials. It is also interesting to note that Phase III × CT, the next step after Phase II × CT, had the lowest reporting rate to ClinicalTrials.gov. However, further discussion of this issue needs to be done separately in conjunction with an analysis of the success or failure of each trial. In any case, the results suggest that CT translations lag behind those of GT.

In general, BLG is already a mature modality,^11–13 with genetically modified products having been in practical use since the 1980s, whereas GT and CT are still in the process of development. Over time, GT has been experimented for practical applications, mainly using adenoviruses and retroviruses since the 1990s,^14,15 and product development mainly using AAVs as vectors has been active since the 2010s.¹⁶ Recently, CAR-T products^17–20 for ex vivo GT (classified under ex vivo GT category in this study) have become a breakthrough in the hematology/oncology field. In contrast, CT has been investigated using bone marrow cells since the 1980s,^21,22 with the actual commercialization of CT products beginning in the 2000s, and products using multipotent stem cells have been commercialized since the 2010s.^23–25

However, the translation of embryonic stem cells and induced pluripotent stem cells, which are attracting attention as breakthrough products, is yet in the stage of clinical evaluation.^26–30 In light of the above factors and observing the trials completed between 2010 and 2019, GT may be about a decade ahead of CT or applying practical usage, which may have encouraged the participation of large companies and the implementation of higher phases, resulting in a positive impact on the reporting of trial results. In addition, an early-stage GT had experienced a serious adverse event,³¹ which was thought to have fostered a culture of compliance in reporting trial results and had a positive impact.

Since the 90s, rather than developing initial drug seeds independently, big pharma bought seeds discovered by small pharma and academia as soon as their POCs (proofs of concept) were achieved, and developed them as their own products, terming them as “open innovations.”^32,33 In other words, big pharma does not take on the risks of failure to achieve POCs on its own; instead, small pharma and academia are assumed to shoulder the risk.

If those POCs have essentially been already achieved, big pharma can conduct high-phase clinical trials with minimal risk, and thus face a relatively low barrier in reporting trial results. The reason why the result reporting rate of big pharma was relatively higher may be due to such a scheme,³⁴ and their attitude toward legal compliance contrasts with that of small pharma and academia, who take the potential risk of not achieving. This may explain the low reporting rate of Phase II × CT.

In contrast to big pharma, the result reporting rate in clinical trials conducted by small pharma and academia is lower, which may be affected by the following factors: the risk of achieving POC, inadequate reporting systems within the organization, lack of funding and human resources, and relatively insufficient investigator's knowledge about legal compliance requirements.³⁵ There will be a need for public support and educational efforts to improve those result reporting rates. One of those moves is reported, such as setting up a specialized unit inside the organization.³⁶ We proposed in our previous report⁹ that the reporting of clinical trial results to registries should be given a rational rating as valuable as journal publication. This was a measure focused on improving the reporting of results, especially by small pharma and academia. It was hoped that such a proposal would increase the motivation of small pharma and academia to report results.

Since the poor result reporting of CT was revealed in this analysis, we claim that improvement focused on this issue also needs to be done. First, it is necessary to widely publicize the fact that the result reporting rate of CT is inadequate compared with BLG, ex vivo GT, and in vivo GT, and to educate the public about the improvement of CT. Discussion and cooperation with CT-related academic societies on this issue will also be necessary. Furthermore, Powell-Smith and Goldacre reported a unique monitoring system called TrialsTracker,³⁷ which automatically identifies the sponsors who fail to publish clinical trial results, is in operation. Similar monitoring system focused on CT sponsors may be needed. We hope this article is a first step for the improvement.

Following factors are potential limitations of this analysis: ClinicalTrials.gov was the only source database adopted for this analysis, which means we excluded trials that have not registered on ClinicalTrials.gov. The reasons why other clinical registries were not used include the following: (1) for example, the International Clinical Trials Registry Platform (ICTRP) contains clinical trials with no “Actual Primary Completion Data” causing incomplete data, (2) it was unrealistic to remove vast numbers of duplicate entities, and (3) we considered ClinicalTrials.gov a perfectly suitable database in terms of comprehensiveness because it contains the largest number of clinical trials.

For labor- and time-saving reasons, AdisInsight was used for modality classification; therefore, the accuracy of modality classification relies on the drug class classification. In addition, the AdisInsight database does not include clinical trials of drug candidates in which profit companies are not involved in either R&D or marketing. Therefore, such trials are omitted from this review. However, Adisinsight covers about 64% of clinical trials on ClinicalTrials.gov (Personal communication; [Parag Budukh], August 16, 2022). The identification of journals reporting results from trials registered at ClinicalTrials.gov relied on the presence of the NCT number in the PubMed abstract or metadata.

Footnotes

Acknowledgments

The authors express their appreciation to Ms. Katsuko Tanaka for excellent assistance in this research work.

Authors' Contributions

T.N., H.O., and A.M. conducted the study. T.N., T.A., S.H., and H.O. analyzed the data. All authors wrote the article and contributed to article editing.

Data Availability

The data that support the findings of this study are available from the corresponding author upon request.

Disclosure Statement

No competing financial interests exist.

Funding Information

This work was supported by The Japan Agency for Medical Research and Development (AMED) under Grant Number JP22bm0904001.

References

National Institutes of Health

, Department of Health and Human

Services

. Clinical trials registration and results information submission. Final Rule. Fed Regist, 2016; 81:64981–65157.

European Medical Agency. Call for all sponsors to publish clinical trial results in EU database. 2019. Available from: https://www.ema.europa.eu/en/news/call-all-sponsors-publish-clinical-trial-results-eu-database [Last accessed: June 16, 2022].

Joint Statement on public disclosure of results from clinical trials. 2017. Available from: https://www.who.int/news/item/18-05-2017-joint-statement-on-registration [Last accessed: June 16, 2022].

Anderson

, Chiswell

, Peterson

, et al. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med, 2015; 372(11):1031–1039; doi: 10.1056/NEJMsa1409364

Liu

, Zhang

, Li

, et al. Evaluation of oncology trial results reporting over a 10-year period. JAMA Netw Open, 2021; 4(5):e2110438; doi: 10.1001/jamanetworkopen.2021.10438

Magnani

. Steinberg JR, Harmange CI, et al. Clinical trial outcomes in urology: Assessing early discontinuation, results reporting and publication in ClinicalTrials.gov registrations 2007–2019. J Urol, 2021; 205(4):1159–1168; doi: 10.1097/JU.0000000000001432

Zwierzyna

, Davies

, Hingorani

, et al. Clinical trial design and dissemination: Comprehensive analysis of ClinicalTrials.gov and PubMed data since 2005. BMJ, 2018; 361:k2130; doi: 10.1136/bmj.k2130

Goldacre

, DeVito

, Heneghan

, et al. Compliance with requirement to report results on the EU clinical trials register: Cohort study and web resource. BMJ, 2018; 362:k3218; doi: 10.1136/bmj.k3218

Negoro

, Okura

, Hayashi

, et al. A pilot study on result reporting rates from clinical trials of regenerative medicines. Tissue Eng Part B Rev, 2023 [Epub ahead of print]; doi: 10.1089/ten.TEB.2022.0126

10.

Ito

List of the Top 66 Companies with Revenues Exceeding $2 Billion in the Global Pharmaceutical Industry in 2018. Nikkei Biotechnol Bus Nikkei: Tokyo; 2019.

11.

The

. Human insulin: DNA technology's first drug. Am J Hosp Pharm, 1989; 46(11 Suppl 2):S9–S11.

12.

Longstaff

, Whitton

, Stebbings

, et al. How do we assure the quality of biological medicines?. Drug Discov Today, 2009; 14(1–2):50–55; doi: 10.1016/j.drudis.2008.09.010

13.

Schulz

, Bonig

. Update on biosimilars of granulocyte colony-stimulating factor—When no news is good news. Curr Opin Hematol, 2016; 23(1):61–66; doi: 10.1097/MOH.0000000000000204

14.

Flotte

. Gene therapy: The first two decades and the current state-of-the-art. J Cell Physiol, 2007; 213(2):301–305; doi: 10.1002/jcp.21173

15.

Kotterman

, Chalberg

, Schaffer

. Viral vectors for gene therapy: Translational and clinical outlook. Annu Rev Biomed Eng, 2015; 17:63–89; doi: 10.1146/annurev-bioeng-071813-104938

16.

Keeler

, Flotte

. Recombinant adeno-associated virus gene therapy in light of luxturna (and Zolgensma and Glybera): Where are we, and how did we get here?. Annu Rev Virol, 2019; 6(1):601–621; doi: 10.1146/annurev-virology-092818-015530

17.

Sermer

, Brentjens

. CAR T-cell therapy: Full speed ahead. Hematol Oncol, 2019; 37 Suppl 1:95–100; doi: 10.1002/hon.2591

18.

van de Donk

NWCJ

, Usmani

, Yong

. CAR T-cell therapy for multiple myeloma: State of the art and prospects. Lancet Haematol, 2021; 8(6):e446–e461; doi: 10.1016/S2352-3026(21)00057-0

19.

Larson

, Maus

. Recent advances and discoveries in the mechanisms and functions of CAR T cells. Nat Rev Cancer, 2021; 21(3):145–161; doi: 10.1038/s41568-020-00323-z

20.

Ali

, Kjeken

, Niederlaender

, et al. The European Medicines Agency review of Kymriah (Tisagenlecleucel) for the treatment of acute lymphoblastic leukemia and diffuse large B-cell lymphoma. Oncologist, 2020; 25(2):e321–e327; doi: 10.1634/theoncologist.2019-0233

21.

Hong

. Bone marrow transplantation. Adv Pediatr, 1993; 40:101–124.

22.

Lenarsky

, Parkman

. Bone marrow transplantation for the treatment of immune deficiency states. Bone Marrow Transplant, 1990; 6(6):361–369.

23.

Kon

, Filardo

, Di Martino

, et al. ACI and MACI. J Knee Surg, 2012; 25(1):17–22; doi: 10.1055/s-0031-1299651

24.

Pellegrini

, Ardigò

, Milazzo

, et al. Navigating market authorization: The path holoclar took to become the first stem cell product approved in the European Union. Stem Cells Transl Med, 2018; 7(1):146–154; doi: 10.1002/sctm.17-0003

25.

Kurtzberg

, Prockop

, Teira

, et al. Allogeneic human mesenchymal stem cell therapy (remestemcel-L, Prochymal) as a rescue agent for severe refractory acute graft-versus-host disease in pediatric patients. Biol Blood Marrow Transplant, 2014; 20(2):229–235; doi: 10.1016/j.bbmt.2013.11.001

26.

Qiu

. Transplantation of human embryonic stem cell-derived retinal pigment epithelial cells (MA09-hRPE) in macular degeneration. NPJ Regen Med, 2019; 4:19; doi: 10.1038/s41536-019-0081-8

27.

Mandai

, Watanabe

, Kurimoto

, et al. Autologous induced stem-cell-derived retinal cells for macular degeneration. N Engl J Med, 2017; 376(11):1038–1046; doi: 10.1056/NEJMoa1608368

28.

Takahashi

. iPS cell-based therapy for Parkinson's disease: A Kyoto trial. Regen Ther, 2020; 13:18–22; doi: 10.1016/j.reth.2020.06.002

29.

Bloor

AJC

, Patel

, Griffin

, et al. Production, safety and efficacy of iPSC-derived mesenchymal stromal cells in acute steroid-resistant graft versus host disease: A phase I, multicenter, open-label, dose-escalation study. Nat Med, 2020; 26(11):1720–1725; doi: 10.1038/s41591-020-1050-x

30.

Schulz

. Concise review: Manufacturing of pancreatic endoderm cells for clinical trials in Type 1 diabetes. Stem Cells Transl Med, 2015; 4(8):927–931; doi: 10.5966/sctm.2015-0058

31.

Hacein-Bey-Abina

, von Kalle

, Schmidt

. A serious adverse event after successful gene therapy for X-linked severe combined immunodeficiency. N Engl J Med, 2003; 348(3):255–256; doi: 10.1056/NEJM200301163480314

32.

Schuhmacher

, Gassmann

, Bieniok

, et al. Open innovation: A paradigm shift in pharma R&D?. Drug Discov Today, 2022; 27(9):2395–2405; doi: 10.1016/j.drudis.2022.05.018

33.

Martinez-Grau MA Alvim-Gaston

. Powered by open innovation: Opportunities and challenges in the pharma sector. Pharmaceut Med, 2019; 33(3):193–198; doi: 10.1007/s40290-019-00280-5

34.

Miller

, Wilenzick

, Ritcey

, et al. Measuring clinical trial transparency: An empirical analysis of newly approved drugs and large pharmaceutical companies. BMJ Open, 2017; 7(12):e017917; doi: 10.1136/bmjopen-2017-017917

35.

Mayo-Wilson

, Heyward

, Keyes

, et al. Clinical trial registration and reporting: A survey of academic organizations in the United States. BMC Med, 2018; 16:60; doi: 10.1186/s12916-018-1042-6

36.

Keyes

, Mayo-Wilson

, Nuamah

, et al. Creating a Program to support registering and reporting clinical trials at Johns Hopkins University. Acad Med, 2021; 96(4):529–533; doi: 10.1097/ACM.0000000000003806

37.

Powell-Smith

, Goldacre

. The TrialsTracker: Automated ongoing monitoring of failure to share clinical trial results by all major companies and research institutions. F1000Res, 2016; 5:2629; doi: 10.12688/f1000research.10010.1