“Substantial Evidence” from a Replicated Secondary Analysis,Followed by a Single Prospective Confirmatory Study

In the context of a New Drug Application in the United States, the term “substantial evidence” of clinical efficacy is usually understood to mean two well-controlled confirmatory studies in which a statistically significant difference is shown on the primary endpoint(s) in both studies. We examine whether this usual understanding can be safely extended to include data packages based on other combinations of studies. For instance, it is not uncommon in drug development for two simultaneous phase 3 confirmatory studies to be designed on basis of limited phase 2 data and then for those studies to fail to show a statistically significant difference on the primary endpoint, but there is a difference seen in both studies on a secondary endpoint or in a particular subgroup. The standard paradigm would suggest that this “finding” would need to be replicated in two further confirmatory studies with the revised endpoint/study population as primary. However, given the evidence from the original two studies, replication of this “finding” in just one further study may provide “substantial evidence” for the purposes of registering an efficacy claim. We examine this and other similar scenarios in terms of whether accepting such registration packages by the Food and Drug Administration would inflate the statistical risk of accepting a false efficacy claim.