Choosing screening instrument and cut-point on screening instruments. A comparison of methods

Abstract

Aims: This study analyzes decisive measures of efficiency of a test, receiver operating characteristic (ROC) analysis and QROC analysis combined with considerations about clinical, health-economic, and ethical aspects when choosing screening instruments. Methods: Analyses of Common Mental Disorders Screening Questionnaire (CMD-SQ) and its subscales SCL-SOM, Whiteley-7, SCL-ANX4, SCL-DEP6, SCL-8, plus combinations, for early detection of psychiatric disorders, are the subject for this analysis. In all, 46.4% of 2,414 new people with continuous sickness absence for more than eight weeks over one year in a well-defined Danish population of 120,000 inhabitants participated in the study. The study was performed as a two phase study. All 1,121 persons in Phase 1 filled in the CMD-SQ. In Phase 2, a random subsample of Phase 1 on 337, the people were further examined by a psychiatrist using SCAN as gold standard. The analyses were performed as weighted analyses on Phase 2. Results: From 17 analyses it was shown that the efficiency of a test, ROC analyses, and QROC analyses resulted in different optimal scales and cut-points. The random possibility of a positive test or negative test in the population is discussed for efficiency and ROC analyses. QROC analyses correct for this by the relative κ-values as decisive measures. However, QROC analyses may discard tests of value, all depending on the purpose of the test. Conclusions: In supplement to test statistics the capacity of services to follow up on screening, ethics, and health economy are issues that should be considered in deciding what rating scale and cut-point should be adopted.

Keywords

Efficiency mental disorder screening sensitivity specificity sickness absence QROC analysis ROC analysis validation

Get full access to this article

View all access options for this article.

References

Andlin-Sobocki P. , Jonsson B. , Wittchen HU , Olesen J. Cost of disorders of the brain in Europe. Eur J Neurol 2005 (Suppl 1):127.

Kessler RC , Ormel J. , Demler O. , Stang PE Comorbid mental disorders account for the role impairment of commonly occurring chronic physical disorders: results from the National Comorbidity Survey. J Occup Environ Med 2003; 45:1257-66.

Munk-Jorgensen P. , Fink P. , Brevik JI , Dalgard OS , Engberg M. , Hansson L. , et al. Psychiatric morbidity in primary public health care: a multicentre investigation. Part II. Hidden morbidity and choice of treatment. Acta Psychiatr Scand 1997;95:6-12.

Goldberg D. , Bridges K. Screening for psychiatric illness in general practice: the general practitioner versus the screening questionnaire . J R Coll Gen Pract 1987;37:15-18.

Soegaard HJ Frequency of mental disorders and comorbidity in a population of long term sick-leave patients. Scand J Public Health 2008(submitted).

Toft T. , Fink P. , Oernboel E. , Christensen K. , Frostholm L. , Olesen F. Mental disorders in primary care: prevalence and co-morbidity among disorders. results from the functional illness in primary care (FIP) study . Psychol Med 2005; 35:1175-84.

Hensing G. , Spak F. Psychiatric disorders as a factor in sick-leave due to other diagnoses. A general population-based study. Br J Psychiatry 1998;172:250-6.

Jenkins R. Minor psychiatric morbidity in employed young men and women and its contribution to sickness absence. Br J Ind Med 1985;42:147-54.

Christensen KS , Fink P. , Toft T. , Frostholm L. , Ornbol E. , Olesen F. A brief case-finding questionnaire for common mental disorders: the CMDQ. Fam Pract 2005 ; 22:448-57.

10.

Christensen KS , Toft T. , Frostholm L. , Ornbol E. , Fink P. , Olesen F. Screening for common mental disorders: who will benefit? Results from a randomised clinical trial. Fam Pract 2005;22:428-34.

11.

Sackett D. , Haynes RB , Guyatt GH , Tugwell P. Clinical epidemiology. A basic science for clinical medicine. Boston: Little, Brown & Co.; 2000.

12.

Metz CE Basic principles of ROC analysis. Semin Nucl Med 1978;8:283-98.

13.

Kraemer HC Evaluating medical tests: objective and quantitative guidelines. Newbury Park, CA: SAGE Publications; 1992.

14.

Søgaard HJ , Bech P. Design involving screening and psychiatric examination for detection of undetected mental disorders in long-term sickness absence (submitted).

15.

Scan Schedules for Clinical Assessment in Neuropsychiatry (computer program). Geneva: World Health Organization; 1994.

16.

Derogatis LR SCL-90-R. Administration, scoring, and procedures. MANUAL-II. Towson, MD: Clinical Psychometric Research; 1983.

17.

Fink P. , Ewald H. , Jensen J. , Sorensen L. , Engberg M. , Holm M. , et al. Screening for somatization and hypochondriasis in primary care and neurological in-patients: a seven-item scale for hypochondriasis and somatization . J Psychosom Res 1999;46:261-73.

18.

Fink P. , Ornbol E. , Hansen MS , Sondergaard L. , De Jonge P. Detecting mental disorders in general hospitals by the SCL-8 scale . J Psychosom Res 2004;56:371-5.

19.

Christensen KS , Toft T. , Frostholm L. , Ornbol E. , Fink P. , Olesen F. The FIP study: a randomised, controlled trial of screening and recognition of psychiatric disorders. Br J Gen Pract 2003;495:758-63.

20.

Fiellin DA , Reid MC , O’Connor PG Screening for alcohol problems in primary care: a systematic review . Arch Intern Med 2000;160:1977-89.

21.

Dunn G. , Pickles A. , Tansella M. , Vazquez-Barquero JL Two-phase epidemiological surveys in psychiatric research. Br J Psychiatry 1999;174:95-100.

22.

Bisoffi G. , Mazzi MA , Dunn G. Evaluating screening questionnaires using Receiver Operating Characteristic (ROC) curves from two-phase (double) samples. Int J Methods Psychiatr Res 2000;9:121-33.

23.

SPSS 14.0 [computer program]. Chicago: SPSS Inc.; 2006.

24.

STATA 10.0 [computer program]. Texas: Stata Corp.; 2006.

25.

Hickie IB , Davenport TA , Ricci CS Screening for depression in general practice and related medical settings. Med J Aust 2002 ;177(Suppl):S111-S116.