Using proportion of similar response to evaluate correlates of protection for vaccine efficacy

Abstract

A question of interest in many vaccine clinical development programmes is whether vaccine-induced serum antibody level can be used as a correlate of vaccine efficacy; that is, whether serum antibody levels induced by a candidate vaccine can reliably predict the risk of breakthrough disease. Traditionally, analyses to answer this question have been based on modelling the incidence of breakthrough disease as a function of antibody level, among vaccinated subjects in clinical trials. The Proportion of Similar Response (PSR) method will be described and explored, and compared to the Receive Operator Characteristics (ROC) curve as a graphical tool and the area under the ROC (AUROC) as a summary measure in the context of evaluating correlates of protection. A way to use PSR analysis as complementary to Youden's index as a simple and elegant method to determine the discriminatory ability of a test and to set an optimal threshold value will be presented. In addition, the relationships among PSR and other measures of overlap and discrimination will be described. An example based on a clinical trial from a development programme for a vaccine against human papillomavirus (HPV) will be presented.

Keywords

effect size receiver operating characteristics optimal cut point sensitivity specificity nonparametric estimation

Get full access to this article

View all access options for this article.

References

Bradley

. Overlapping coefficient. Encycl of Stat Sci 1985; 6: 546–547.

Rom

Hwang

. Testing for individual and population equivalence based on the proportion of similar responses. Stat Med 1996; 15: 1489–1505.

Cohen

. Statistical power analysis for the behavioral sciences, New York, NY: Academic Press, 1969.

Inman

Bradley

. Hypothesis tests and confidence interval estimates for the overlap of two normal distributions with equal variances. Environmetrics 1994; 5: 167–189.

Inman

. Behavior and properties of the overlapping coefficient as a measure of agreement between distributions (Doctoral dissertation) , University of Alabama: Birmingham, 1984.

Hodges

Lehman

. Estimates of location based on rank tests. Ann Math Stat 1963; 34: 598–611.

Royston

Altman

. Visualizing and assessing discrimination in the logistic regression model. Stat Med 2010; 29: 2508–2520.

Stine

Heyse

. Non-parametric estimates of overlap. Stat Med 2001; 20: 215–236.

Böhning

Hempfling

Schelp

F-P

Schlattmann

. The area between curves (ABC) measure in nutritional anthropometry. Stat Med 1992; 11(10): 1289–1304.

10.

Mann

Whitney

. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 1947; 18: 50–60.

11.

Birnbaum

. On a use of the Mann-Whitney statistic. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability 1956; 1: 13–17.

12.

Hauck

Hyslop

Anderson

. Generalized treatment effects for clinical trials. Stat Med 2000; 19: 887–899.

13.

Acion

Peterson

Temple

Arndt

. Probabilistic index: an intuitive non-parametric approach to measuring the size of treatment effects. Stat Med 2006; 25: 591–602.

14.

Gastwirth

. Statistical measures of earnings differentials. Amer Statistician 1975; 29: 32–35.

15.

Pepe

. The statistical evaluation of medical tests for classification and prediction, New York, NY: Oxford University Press, 2003.

16.

Hanley

McNeil

. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29–36.

17.

Youdens

. An index for rating diagnostic tests. Cancer 1950; 3: 32–35.

18.

Böhning

Holling

. Revisiting Youden's index as a useful measure of the misclassification error in meta-analysis of diagnostic studies. Stat Methods Med Res 2008; 17: 543–554.

19.

. A solution to the most basic optimization problem associated with an ROC curve. Stat Methods Med Res 2006; 15: 571–584.

20.

Schisterman

Perkins

Liu

Bondell

. Optimal cut-point and its corresponding Youden's index to discriminate individuals using pooled blood samples. Epidemiology 2005; 16: 73–81.

21.

Böhning

Holling

Patilea

. A limitation of the diagnostic-odds ratio in determining an optimal cut-off value for a continuous diagnostic test. Stat Methods Med Res 2010; 00: 1–10.

22.

Greiner

Sohr

Göbel

. A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests. J Immunol Methods 1995; 185: 123–132.

23.

Greiner

Pfeiffer

Smith

. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med 2000; 45: 23–41.

24.

Dias

Van Doren

Schlottmann

Kelly

Puchalski

Ruiz

. Optimization and validation of a multiplexed luminex assay to quantify antibodies to neutralizing epitopes on human papillomavirus 6, 11, 16 and 18. Clin Diagn Lab Immunol 2005; 12: 959–969.

25.

Garland

Hernandez-Avila

Wheeler

Perez

Harper

Leodolter

. Quadrivalent vaccine against human papillomavirus to prevent anogenital diseases. New Eng J Med 2007; 356: 1929–1943.

26.

Shapiro

Wilk

. An analysis of variance test for normality (complete samples). Biometrika 1965; 52(3–4): 591–611.

27.

Lambert

Lipkovich

. A macro for getting more out of your ROC curve. SAS Global Forum 2008; 231.

28.

Browne

. The t-test p value and its relationship to the effect size and P(X > Y). Amer Statistician 2010; 64(1): 30–33.

29.

Chan

ISF

Matthews

Chan

Vessey

Sadoff

. Use of statistical models for evaluating antibody response as a correlate of protection against varicella. Stat Med 2002; 21: 3411–3430.

30.

Dunning

. A model for immunological correlates of protection. Stat Med 2006; 25: 1485–1497.