An analysis of the controversy over classical one-sided tests

Abstract

Background When applying classical tests of the null hypothesis in clinical trials, there has been considerable controversy over the choice between a one-sided versus a two-sided test. The choice between a one-sided and two-sided test still impacts on sample size calculations, assessment of study results by regulatory authorities, and publication of study results in academic journals.

Purpose To analyze the main elements in the controversy, and examine the procedures from both a Bayesian and classical viewpoint.

Methods and Results Using a Bayesian decision framework, it is shown that there is no reason to double the p-value when moving from a one-sided to a two-sided test. Within the classical framework, it is shown that the doubling of the p-value results from a discontinuity due to testing a point null hypothesis. A three-decision rule, credited to Neyman or Wald, is presented that does not require the doubling of the p-value, and is consistent with a Bayesian approach.

Conclusions For most comparative clinical trials the three-decision rule is appropriate, and its use would abolish the controversy over one-sided tests. Clinical Trials 2008; 5: 635—640. http://ctj.sagepub.com

Get full access to this article

View all access options for this article.

References

Gardner MJ , Altman DG Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J 1986; 292: 746-50.

Borenstein M. The case for confidence intervals in controlled clinical trials. Controlled Clin Trials 1994 ; 15: 411-28.

Braitman LE Statistical estimates and clinical trials. J Biopharm Stat 1993; 3: 249-56.

Piantadosi S. Clinical Trials: A Methodologic Perspective. Wiley, New York, 1997, pp. 148-81.

Senn S. Statistical Issues in Drug Development. Wiley, Chichester, UK, 1997, pp. 169-85.

Goodman SN P values, hypothesis tests, and likelihood: implications for epidemiology of a neglected historical debate. Am J Epidemiol 1993; 137: 485-96.

Armitage P. Statistical Methods in Medical Research (1stedn). Blackwell , Oxford, 1971, p. 104.

Pocock SJ Clinical Trials: A Practical Approach. Wiley, Chichester, UK, 1984, pp. 205-06.

Piantadosi S. Clinical Trials: A Methodologic Perspective. Wiley, New York, 1997, p. 112.

10.

Senn S. Statistical Issues in Drug Development. Wiley, Chichester, UK, 1997, pp. 161-67.

11.

Bland JM , Altman DG One and two sided tests of significance. Br Med J 1994; 309: 208.

12.

Wolterbeek R. One and two sided tests of significance: Statistical hypothesis should be brought into line with clinical hypothesis. Br Med J 1994; 309: 873.

13.

Enkin MW One sided tests should be used more often. Br Med J 1994; 309: 873-74.

14.

Bland JM , Altman DG One and two sided tests of significance: Authors' reply. Br Med J 1994; 309: 874.

15.

Fleiss JL Some thoughts on two-tailed tests. Controlled Clin Trials 1987; 8: 394.

16.

Boissel JP Some thoughts on two-tailed tests (and two-sided designs). Controlled Clin Trials 1987; 8: 385-86.

17.

Goodman SN

One-sided or two-sided p-values?

Controlled Clin Trials 1987; 8: 387-88.

18.

Peace KE Some thoughts on one-tailed tests. Controlled Clin Trials 1987; 8: 383-84.

19.

Fleiss JL ``Statistical methods for Rates and Proportions``, (1stedn). Wiley, New York, 1973, p. 21.

20.

Journal of the National Cancer Institute. Instructions for Authors, at http://www.oxfordjournals.org/our_journals/jnci/for_authors/index.html; date accessed 10/1/2008.

21.

Bohrer R. Multiple three-decision rules for parametric signs. J Am Stat Assoc 1979; 74: 432-37.

22.

Kaiser HF Directional statistical decisions. Psychol Rev 1960; 67: 160-67.

23.

Berger JO , Sellke T. Testing a point null hypothesis: the irreconcilability of P values and evidence . J Am Stat Assoc 1987; 82: 112-22.

24.

Casella G. , Berger RL Reconciling Bayesian and Frequentist Evidence in the One-sided Testing Problem . J Am Stat Assoc 1987; 82: 106-11.

25.

Mood AM , Graybill FA (1963). Introduction to the Theory of Statistics, McGraw-Hill, New York, Section 12.5.

26.

Cox DR , Hinkley DV Theoretical Statistics. Chapman and Hall/ CRC, Boca Raton, 1974, p. 65.

27.

Anscombe FJ The summarizing of clinical experiments by significance tests. Stat Med 1990; 9: 703-08.

28.

Tukey JW The philosophy of multiple comparisons. Stat. Sci. 1991; 6: 100-16.

29.

Jones LV , Tukey JW A sensible formulation of the significance test. Psych Methods 2000; 5: 411-14.

30.

Roy SN On a heuristic method of test construction and its use in multivariate analysis . Ann Math Stat 1953; 24: 220-38.

31.

Pratt JW Length of confidence intervals. J Am Stat Assoc 1961; 56: 549-67.

32.

Benjamini Y. , Hochberg Y. , Stark PB Confidence intervals with more power to determine the sign: two ends constrain the means. J Am Stat Assoc 1998; 93: 309-17.

33.

International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guidelines: Statistical Principles for Clinical Trials E9, Step 4 Version, 1988.