Using Administrative Databases in the Surveillance of Depressive Disorders

Abstract

The objective of this study was to assess the usefulness of provincial administrative databases in carrying out surveillance on depressive disorders. Electronic medical records (EMRs) at 3 family practice clinics in St. John's, NL, Canada, were audited; 253 depressive disorder cases and 257 patients not diagnosed with a depressive disorder were selected. The EMR served as the “gold standard,” which then was compared to these same patients investigated through the use of various case definitions applied against the provincial hospital and physician administrative databases. Variables used in the development of the case definitions were depressive disorder diagnoses (either in hospital or physician claims data), date of diagnosis, and service provider type [general practitioner (GP) vs. psychiatrist]. Of the 120 case definitions investigated, 26 were found to have a kappa statistic greater than 0.6, of which 5 case definitions were considered the most appropriate for surveillance of depressive disorders. Of the 5 definitions, the following case definition, with a 77.5% sensitivity and 93% specificity, was found to be the most valid ([ ≥1 hospitalizations OR ≥1 psychiatrist visit related to depressive disorders any time] OR ≥2 GP visits related to depressive disorders within the first 2 years of diagnosis). This study found that provincial administrative databases may be useful for carrying out surveillance on depressive disorders among the adult population. The approach used in this study was simple and resulted in rather reasonable sensitivity and specificity. (Population Health Management 2012;15:372–380)

Introduction

D epressive disorders are common worldwide, often accompany other chronic diseases, and are associated with considerable socioeconomic burden.^1
–3 The partitioning of health systems across community and acute care services has fostered a fragmented approach to the collection of data on interventions specific to mental health.^4,5 As a result, there is not an accurate picture of the incidence, prevalence, or nature of mental disorders in Canada, which hinders the ability to evaluate the effectiveness of policies, programs, and services aimed at mental health. As well, the lack of a comprehensive national surveillance system specific to mental disorders impedes efforts to monitor and subsequently improve mental health care delivery. This study was undertaken to address this challenge by assessing the usefulness and validity of using administrative databases for surveillance of depressive disorders. The objective of this study was to develop valid case definitions for surveillance of depressive disorders among the adult population in Newfoundland and Labrador using administrative data.

Methods

Cases and controls as the “gold standard”

The authors audited electronic patient charts at 3 family practice clinics affiliated with Memorial University of Newfoundland in the province of Newfoundland and Labrador, Canada. The clinics were equipped with electronic medical records (EMR) systems with data collection occurring during January to July 2007. Cases of depressive disorders were all patients aged 18 years or older who had a diagnosis or a co-diagnosis of any depressive disorder including major depressive disorder, dysthymic disorder, and depressive disorders not otherwise specified (eg, minor depressive disorder, recurrent brief depressive disorder). The Diagnostic and Statistical Manual of Mental Disorders—Fourth Edition—Text Revision criteria were used wherever possible to make a clinical judgment. Bipolar disorders were excluded from the study because bipolar disorders are a different subset of mood disorders than depressive disorders. All excluded cases were reviewed independently by an experienced psychiatrist who was not involved in the study to maximize objectivity. Non-cases of depressive disorders or controls were selected from patients in the same family practice clinics by using a sex- and age-stratified random sampling approach. These patients did not have any psychiatric diagnoses including depressive disorders at any given time (January–July 2007). Charts selected underwent a full chart review and the data collection form was piloted initially on 20 electronic patient charts. Each electronic patient chart within the EMR system was pulled and viewed on a PC monitor and various sections, including demographics, history of present illness, physical, medications, past medical history, and consultations for each visit were reviewed and pertinent information was collected in the data collection form.

Administrative data sources/data linkage

Provincial hospital separation database

The provincial hospital database (1995/1996 to 2007/2008) captures demographic, clinical, and interventional information for patients admitted to all acute health care facilities and surgical day cares in the province. The coding classification system in the hospital database changed from International Classification of Diseases, Ninth Revision (ICD-9) to International Classification of Diseases, Tenth Revision—Canadian Enhancement (ICD-10-CA) in April 2001.

Provincial fee-for-service physician claims database

The Newfoundland and Labrador Medical Care Plan (MCP) was established in 1969 with a primary function of processing payments for fee-for-service physicians in the province. The MCP database (1995 to 2008) captures information on age and sex, as well as codes with information on service billed for, diagnosis, and physician involved. The classification system used in the physician MCP database is ICD-9. Appendix A presents a list of ICD-9 and ICD-10-CA codes used to identify patients with depressive disorders. It should be noted that in this study major depressive disorders were included in the ICD-9 coding.

Data linkage

Data obtained from the MCP database included MCP numbers (for data linkage purposes), date of service, diagnosis code (ICD-9), provider type (eg, general practitioner [GP], psychiatrist), and procedure code (eg, psychotherapy). Given that data in the hospital database are collected on a fiscal year basis and on a calendar year in the MCP database, to be consistent, the data were extracted from both databases for the period from April 1, 1995 until March 31, 2008.

Given the nature of depressive disorders with long latency or a chronic course, the patients captured in EMRs were linked to all 13 years of available hospital and physician data (April 1, 1995 to March 31, 2008). Considering that the nature and the severity of depressive disorders range from mild to severe depression requiring hospitalization, a number of potential case definitions were developed and compared to the results of the EMR review, which is considered to be a gold standard. It should be noted that the severity of depressive disorders was not assessed and captured in this study using the chart audit. However, from a case definition development point of view, patients with depressive disorders who required hospitalization were deemed to have severe depression, while those who did not require hospitalization were considered mild/moderate depression that most likely required a clinic visit.

Case definitions

Variables used in the development of the case definitions were depressive disorders diagnoses (hospital and/or MCP data), date of diagnosis, and service provider type. These case definitions were based on the severity of depressive disorders ranging from mild depression to debilitating levels requiring professional help, medication, and even hospitalization. In developing 120 various case definitions, we considered a psychiatrist visit and/or a hospital admission related to depressive disorder to be accurate in identifying cases of depressive disorders. Further, because depressive disorders can require several GP visits, we examined multiple GP visits related to depressive disorders (ranging from 1 to 8) in various scenarios to assess the specificity and sensitivity of case definitions (Appendix B).

Statistical analysis

Data obtained using the EMR review were compared to the results of linking the hospital and MCP databases and using various case definitions. The following measures were investigated: sensitivity, specificity, positive predictive value, and negative predictive value. Percent agreement was calculated by summing the true positives and true negatives and dividing by the total study population. Cohen's kappa coefficient was used to quantify agreement between various case definitions and the gold standard. Kappa statistics and 95% confidence limits also were calculated to determine the percent agreement attributed to chance. Analyses were performed using SPSS version 15.0 (SPSS Inc., Chicago, IL) and SAS version 9.2 (SAS Institute Inc., Cary, NC).

This study was approved by the Human Investigation Committee of Memorial University of Newfoundland.

Results

A total of 420 patients who had a diagnosis code for depressive disorder were identified from the EMR. Their charts were reviewed and 253 met the case criteria (depressed patients). Using a sex- and age-stratified random sampling technique, 318 patients who did not have a diagnosis code for depressive disorder in their electronic chart were selected, and 257 met the control criteria (nondepressed patients). The average age was almost the same for the 2 groups (51 years). Table 1 presents demographics, comorbidities, and medication pattern for the depressed and nondepressed patients. A higher proportion of the depressed patients had symptoms such as anxiety, insomnia, fatigue, and chronic pain than the nondepressed patients (P<0.05). A significantly higher proportion of the depressed cohort used antidepressant, antipsychotic, anticonvulsant, narcotic, and anxiolytic medications compared to the nondepressed cohort (P<0.05). Respiratory and gastrointestinal medications as well as antibiotics were used more frequently by the nondepressed cohort (P<0.05).

Table 1.

Demographic and Clinical Characteristics of the Depressed and Nondepressed Patients

	Depressed (n=253)		Nondepressed (n=257)
Factors	Number	%	Number	%	P value
Sex (male)	69	27.3	70	27.2	0.993
Comorbidities
Cancer	19	7.5	13	5.1	0.254
Endocrinological disorders	55	21.7	55	21.4	0.926
Mental disorders excluding depression	61	24.1	5	1.9	0.000
Cardiovascular disorders	115	45.5	109	42.4	0.489
Respiratory disorders	58	22.9	46	17.9	0.159
Gastrointestinal disorders	78	30.8	69	26.8	0.321
Neurologic disorders	45	17.8	33	12.8	0.121
Musculoskeletal disorders	54	21.3	83	32.3	0.005
Other medical conditions
Insomnia	57	22.5	10	3.9	0.000
Chronic pain	67	26.5	45	17.5	0.014
Fatigue	29	11.5	12	4.7	0.005
Anxiety	113	44.7	28	10.9	0.000
Medication use
No history of medication use	0	0.0	29	11.3	0.000
Selective serotonin reuptake inhibitors	161	63.6	0	0.0	0.000
Serotonin-norepinephrine reuptake inhibitors	57	22.5	1	0.4	0.000
Monamine-oxidase inhibitors	1	0.4	0	0.0	0.313
Tricyclic antidepressants (TCAs)^*	27	10.7	2	0.8	0.000
Other antidepressants^**	229	90.5	3	1.2	0.000
Benzodiazepines	91	36.0	18	7.0	0.000
Antipsychotics	7	2.8	0	0.0	0.007
Other psychiatric medication	59	23.3	11	4.3	0.000
Narcotics	22	8.7	8	3.1	0.007
Nonsteroidal anti-inflammatory drugs	48	19.0	64	24.9	0.106
Anticonvulsive	17	6.7	3	1.2	0.001
Cardiovascular medications	90	35.6	100	38.9	0.436
Hormonal medications^***	47	18.6	47	18.3	0.933
Hematologic medications	1	0.4	0	0.0	0.313
Gastrointestinal medications	81	32.0	55	21.4	0.007
Respiratory medications	60	23.7	40	15.6	0.020
Antimicrobial agents	4	1.6	21	8.2	0.001

TCAs with a dose greater than 75 mg/day. ^**Includes zopiclone, bupropion, and mirtazapine; ^***Includes insulin, metformin, and levothyroxine, among others.

Among 253 patients with a depressive disorder, 22 had at least 1 hospitalization related to depressive disorder between April 1, 1995 and March 31, 2008 compared to none for the nondepressed patients. Five of the 253 patients in the depressed cohort did not have a record of any physician visits in the MCP database. They could be individuals who did not have valid MCP numbers, such as visitors/tourists or armed forces personnel. Conversely, 14 of 257 nondepressed patients were found not to have any physician visits in the physician database. Table 2 presents descriptive statistics on health service utilization related to depressive disorders among the depressed and nondepressed patients. Depressed patients utilized physician services more than nondepressed patients during the study period.

Table 2.

Descriptive Statistics of Health Service Utilization during the Study Period (1995/1996 to 2007/2008)

Service	Depressed Patients	Nondepressed Patients
Hospitalizations
Number of patients	22	0
Mean (±SD) number of hospitalizations	3.9 (±9.1)	–
Median number of hospitalizations	1	–
Mode number of hospitalizations	1	–
Maximum number of hospitalizations	43	–
Physician visits
Number of patients	226	45
Mean (±SD) number of visits	23.2 (±38.3)	8.2 (±19.6)
Median number of visits	10	1
Mode number of visits	3	1
Maximum number of visits	399	110
Number of visits
1–4 visits	62	32
5–8 visits	43	7
9–12 visits	28	1
≥13 visits	93	5

The depressed and nondepressed patients captured in EMRs were linked to all 13 years of available hospital and physician data (April 1, 1995 to March 31, 2008), based on the 120 case definitions (Appendix B) developed. Compared to the EMR gold standard, 26 of the 120 were found to have a kappa statistic greater than 0.6. Considering the clinical characteristics of the different types of depressive disorders, as well as the severity and chronicity, and a sensitivity threshold of greater than 70% (with kappa greater than 0.6), 5 of 26 selected case definitions were considered the most appropriate for surveillance of depressive disorders (Table 3). Specificity and false positive rate for these 5 definitions were similar.

Table 3.

Inferential Statistics for the Five Most Appropriate Case Definitions

	≥1 Hospital OR ≥2 Physician within 1 Year	≥1 Hospital OR ≥2 Physician within 2 Years	(≥1 Hospital OR ≥1 PSY) OR ≥2 GP within 1 Year	(≥1 Hospital OR ≥1 PSY) OR ≥2 GP within 2 Years	(≥1 Hospital OR ≥1 PSY) OR ≥3 GP within 3 Years
Sensitivity (%)	70.0	75.1	73.1	77.5	69.6
Specificity (%)	93.8	93.0	93.4	93.0	93.4
FP rate (%)	6.2	7.0	6.6	7.0	6.6
FN rate (%)	30.0	24.9	26.9	22.5	30.4
PPV (%)	91.7	91.3	91.6	91.6	91.2
NPV (%)	76.0	79.1	77.9	80.7	75.7
Agreement (%)	82.0	84.1	83.3	85.3	81.6
Kappa (95% CI)	0.64 (0.57–0.70)	0.68 (0.62–0.74)	0.67 (0.60–0.73)	0.71 (0.64–0.77)	0.63 (0.57–0.70)

FN, false negative; FP, false positive; GP, general practitioner visit; NPV, negative predictive value; PPV, positive predictive value; PSY, psychiatrist visit.

Given that information on medication utilization is not currently captured in the hospital or physician databases, it was not possible to consider information on drugs in the development of potential case definitions for depressive disorders. However, the authors examined the impact of including antidepressant medication information (via the EMR) on the selected 5 most appropriate case definitions (Table 4). The medication information specific to antidepressants was combined with the case definitions either as “AND” or “OR.” Addition of medication data to the 5 case definitions increased either the specificity or the sensitivity level to 100%.

Table 4.

Inferential Statistics for the Five Most Appropriate Case Definitions with the Addition of Antidepressant Medication Information

	≥1 Hospital OR ≥2 Physician within 1 Year AND Antidepressant Medication	≥1 Hospital OR ≥2 Physician within 2 Years AND Antidepressant Medication	(≥1 Hospital OR ≥1 PSY) OR ≥2 GP within 1 Year AND Antidepressant Medication	(≥1 Hospital OR ≥1 PSY) OR ≥2 GP within 2 Years AND Antidepressant Medication	(≥1 Hospital OR ≥1 PSY) OR ≥3 GP within 3 Years AND Antidepressant Medication
Sensitivity (%)	62.1	66.8	64.8	68.8	61.3
Specificity (%)	100.0	100.0	100.0	100.0	100.0
FP rate (%)	0.0	0.0	0.0	0.0	0.0
FN rate (%)	37.9	33.2	35.2	31.2	38.7
PPV (%)	100.0	100.0	100.0	100.0	100.0
NPV (%)	72.8	75.4	74.3	76.5	72.4
Percent agreement (%)	81.2	83.5	82.5	84.5	80.8
Kappa (95% CI)	0.62 (0.56–0.69)	0.67 (0.61–0.73)	0.65 (0.59–0.71)	0.69 (0.63–0.75)	0.61 (0.55–0.68)

	≥1 Hospital OR ≥2 Physician within 1 Year OR Antidepressant Medication	≥1 Hospital OR ≥2 Physician within 2 Years OR Antidepressant Medication	(≥1 Hospital OR ≥1 PSY) OR ≥2 GP within 1 Year OR Antidepressant Medication	(≥1 Hospital OR ≥1 PSY) OR ≥2 GP within 2 Years OR Antidepressant Medication	(≥1 Hospital OR ≥1 PSY) OR ≥3 GP within 3 Years OR Antidepressant Medication
Sensitivity (%)	100.0	100.0	100.0	100.0	100.0
Specificity (%)	92.7	91.9	92.3	91.9	92.3
FP¹ rate (%)	7.3	8.1	7.7	8.1	7.7
FN² rate (%)	0.0	0.0	0.0	0.0	0.0
PPV³ (%)	92.9	92.3	92.6	92.3	92.6
NPV⁴ (%)	100.0	100.0	100.0	100.0	100.0
Percent agreement (%)	96.3	95.9	96.1	95.9	96.1
Kappa (95% CI)	0.93 (0.89–0.96)	0.92 (0.88–0.95)	0.92 (0.89–0.96)	0.92 (0.88–0.95)	0.92 (0.89–0.96)

FN, false negative; FP, false positive; GP, general practitioner visit NPV, negative predictive value; PPV, positive predictive value; PSY, psychiatrist visit.

Discussion

In this study we examined the usefulness and validity of using provincial administrative databases to carry out surveillance of depressive disorders. One important challenge to developing a mental illness surveillance system by using administrative data is to create indicators and case definitions for the conditions.⁶ A recent study by Kisely et al⁷ used data from physician billings, hospital discharge abstracts, and community-based clinics records, and concluded that using administrative data to measure the prevalence of mental health disorders is feasible.

Administrative databases are being used increasingly for many types of research in economically developed countries, given their advantage of having larger sample sizes, lower costs, and increased generalizability; they have not been used widely for public health surveillance.⁸ Although surveys also have been used for surveillance of mental disorders,^9

–13 their validity depends on many factors, such as the sampling strategy (which can impact generalizability) and the accuracy of self-reporting. Agreement between administrative databases and population surveys varies with the chronic conditions studied.¹⁴ Okura et al¹⁵ suggested that conditions that are chronic and require ongoing repeated health services utilization have an increased likelihood of identification in administrative data.

Our findings showed that the depressed cohort had a much higher utilization rate compared to the nondepressed group. They also had more symptoms, such as anxiety, insomnia, fatigue, and chronic pain than the nondepressed patients. These findings are consistent with other studies.^1

–4 Although the depressed patients were shown to have a higher rate of medical comorbidities, no significant differences were found between the 2 cohorts. This finding should be viewed with caution given the rather small sample size for this cohort.

Further, our results showed that only a small proportion of depressed patients (11.6%, 22/190) were hospitalized for the primary reason of depressive disorder. The large discrepancy between hospital records and the gold standard in this study may be related in part to patients being hospitalized for reasons other than depression, yet subsequently receiving a comorbid diagnosis of depression. Nonetheless, none of the nondepressed cohort patients were found to have been hospitalized as a result of a depressive disorder, suggesting high specificity of the hospital data for identifying patients diagnosed with a depressive disorder. It is worth noting that depressions that require hospitalizations are often severe and protracted, with potential multiple outpatient treatment failures. Identifying and diagnosing such patients may not be as challenging as identifying those who have mild depression, who often are treated in an outpatient setting.

Of all patients with a depressive disorder who were found by chart review, 89.3% (226/253) were identified in the provincial fee-for-service physician claims database as having an outpatient visit with an associated code of depressive disorder. The discrepancy between the charts and the physician claims database could be explained partially by individuals who do not have a valid MCP number and, as such, would not be included in the physician claims database. In this study, we could not identify the disease severity because of a lack of clinical information in the medical charts and administrative databases. In addition, patients with mild depressive disorders may not be diagnosed easily in the clinic. Although it is arguable that those patients who had a hospitalization or psychiatric visit may have had severe depression, not all patients with depressive disorders require hospital admissions or psychiatric visits. The lack of objective or at least quantitative assessments of severity of depressive symptoms in this study may indicate the potential usefulness of routine and repeated assessment using simple measures such as the PHQ-9 (the 9-item depression scale of the Patient Health Questionnaire) on a routine basis, for detection as well as for monitoring.

Despite the fact that many patients with depressive disorders seek help in primary care, GPs still have difficulty diagnosing and treating depression.^16

–20 Additionally, because depressive disorders are more common among those with comorbid chronic medical disorders, depressions with significant somatic comorbidity may remain unrecognized in primary care.²¹ Moreover, studies using administrative databases to conduct research on depression often identify patients by ICD codes.²² Basing the analysis on diagnostic codes must be done with caution because mental illnesses may not be fully reported on insurance claims because of their potential for stigma. West et al²² conducted a study in Canada to evaluate the validity of using a health administrative claims database to conduct research on depressed patients who were using antidepressants. The study showed a high number of true positives and true negatives, exhibiting promise for the use of administrative databases to explore depression. Conversely, Spettell et al²³ also conducted a study to evaluate algorithms to identify physician-recognized depression using a large US managed health care organization database. The results highlighted the difficulty of identifying depressed patients from administrative data using algorithms based only on diagnostic and pharmacy codes.

In this study, considering similar chronic disease models for depressive disorders and their case definitions (eg, the Canadian Chronic Disease Surveillance System model), a large number of potential case definitions were developed. Unlike most previous studies,^{7,22, 24

–29} the case definitions used in this study incorporated physician specialty when counting the number of physician visits. This approach is unique in that different types of visits were assigned different weights depending on the role of the physician.

Surveillance case definitions must balance competing needs for sensitivity, specificity, and feasibility. Because of the need for simplicity, surveillance case definitions typically are brief.³⁰ For diseases with long latency or a chronic course (eg, depressive disorders), developing a case definition depends on decisions regarding which phase to monitor: asymptomatic, early disease, late disease, or death. Ideally, surveillance case definitions should both inform and reflect clinical practices.³⁰ In this study, incorporating physician specialty and assigning different weights based on the role of the physician enabled the capture of the range of depression, from mild depression not requiring hospitalizations or psychiatric visits, to major depression that may have required either hospitalizations or psychiatric visits.

This study found that valid case definitions to identify patients with depressive disorders require at least 1 hospitalization and at least 2 fee-for-service physician visits. A minimum of 2–3 years of retrospective data is required in order for hospitalizations and fee-for-service physician visits to identify the majority of the cases.

The time frame required to generate sufficient numbers of health care visits is crucial to develop a valid case definition. A 2- to 3-year time frame has been suggested for chronic diseases with relatively structured visiting behavior such as diabetes and hypertension.^31,32 Using a 2-year time frame for our case definition was appropriate, given that depressive symptoms often persist over a protracted time. This time frame is also consistent with other established case definitions of chronic diseases, such as diabetes, in the Canadian Chronic Disease Surveillance System. Previous research has shown that errors in prevalence estimates decrease with increased follow-up time.^31,32 For conditions that may be challenging to diagnose, such as asthma, a time frame of up to 5 years may be required.¹⁴ In other cases, using a period longer than 2 years may not be feasible for ongoing surveillance.²⁶ Other investigators used a shorter time frame of 2 years mainly because the purpose of their investigation was to identify cases rather than to estimate disease burden.³³

Of the 5 case definitions found to perform the best, the case definition, (≥1 hospitalization OR ≥1 psychiatrist visit related to depressive disorders any time) OR ≥2 GP visits related to depressive disorders within the first 2 years of diagnosis, appears to be the most appropriate, having high sensitivity (77.5%), specificity (93%), and positive predictive value (91.6%), with a kappa statistic of 0.706. We put forward this case definition as the most appropriate for studies of depressive disorders using hospital and physician administrative databases because: (1) there is congruence between administrative databases and medical charts in identifying cases, (2) it has high sensitivity and specificity, (3) it incorporates physician specialty in the case definitions, and (4) it considers a minimum of 2 years of retrospective data.

In our study, the addition of drug use data from the EMR (antidepressant medications) enhanced the accuracy of all 5 of our case definitions by improving the sensitivity level to 100% and the kappa statistic to more than 90%, while the false negative rate was reduced to zero. This finding could have important implications in the development of future case definitions by considering data from population-based pharmacy networks.

Damush et al²⁴ showed that when antidepressant medications were included, the accuracy of the case-finding algorithm for post-stroke depression using administrative data improved. In Manitoba, Lix et al¹⁴ investigated the congruence between administrative databases and surveys in studying chronic disease by examining multiple case definitions for arthritis, asthma, diabetes, heart disease, hypertension, and stroke. Those authors reported that using prescription drug data, in addition to hospital and physician databases, had mixed effects on agreement in ascertaining disease state between administrative databases and surveys. Although case ascertainment for asthma benefited from the use of both diagnostic and prescription drug information when the definition was based on 1 or 2 years of administrative data, improvement in agreement was less substantial for diabetes. For hypertension, there also was some improvement in agreement associated with using both diagnosis and prescription drug data for case ascertainment, but not for other diseases. It is important to note that a specific set of prescription drugs are used to treat asthma and diabetes; for other chronic diseases such as hypertension or arthritis, the drugs prescribed for an individual may be used to treat more than 1 chronic disease and, therefore, may not be helpful to identify cases. In our study, drug use data with medications' names and classes (eg, selective serotonin reuptake inhibitors, tricyclic antidepressants), dosages and durations were available from the EMR. We believe that for the purpose of surveillance of depressive disorders, the classes of antidepressant medications (as opposed to the name of each individual medication) along with the dosages should be included in the case definitions. The duration of treatment can be obtained via the dispensed dates. A population-based pharmacy network, such as the one being implemented in Newfoundland and Labrador (2010), will provide an opportunity to include medication information at a population level in the development of case definitions for the surveillance of many types of diseases.

The depressed cohort in this study had a higher proportion of patients with anxiety, insomnia, fatigue, and chronic pain than the nondepressed cohort. The reason for this is that such symptoms are frequently associated with a diagnosis of depressive disorders. Further, the depressed cohort appeared to have a higher proportion of narcotics, anticonvulsant, gastrointestinal, and respiratory prescriptions prescribed for them than the nondepressed cohort. The depressed patients were also more likely to be more frequent users of physician services than the nondepressed patients. It has been reported that patients with depressive disorders are at increased risk of having 1 or more comorbidities.^9,10,21,34 Moussavi et al³⁵ found that depressive disorders produced a greater decrement in health than other chronic diseases, including angina, arthritis, asthma, and diabetes, and that those with angina, arthritis, asthma, or diabetes were at increased risk for depressive disorders.

Strengths

In this study, a medical chart review was considered the gold standard, which may have decreased potential bias in identifying patients with true depressive disorders. Further, all cases excluded from this study were independently reviewed by an experienced psychiatrist who was not involved in the study in order to maximize objectivity. Other similar studies used various sources such as surveys and registries (eg, prescription drug data, National Population Health Survey, cancer registry). Unlike several previous studies, the case definitions used in this study incorporated physician specialty (GP vs. psychiatrist). This approach is unique in that different types of visits were assigned different weights depending on the role of the physician and, therefore, added a degree of certainty. Three large family practice clinics also were included in the study, which may have decreased potential selection bias.

It should be noted that the health care system in Canada is considered universal whereas there currently is no national care health plan for all citizens in the United States. Further, EMR use is not fully implemented in the United States, and the ICD-10 will not be implemented there until January 2013. Considering these differences, it may be challenging to conduct a similar study in a setting such as the United States.

Limitations

A limitation of this study is that the results may not be generalizable to practices that are not university based. The study setting included only university-based clinics in which most staff are either academic physicians or rotating residents. The majority of family practice clinics in the province are operated either by fee-for-service or salaried physicians; these clinics may have a different pattern of claims submission practices than academic settings. It is also important to note that rural areas, which most likely include non-academic physician practices, were not included in this study. The provincial fee-for-service physician claims database does not capture all patients with depressive disorders, such as those who do not have a valid MCP number, or those who visit a salaried physician. This may have had a negative impact on our estimation of the agreement between the EMR and administrative databases. Also, ICD-9 and ICD-10-CA codes were used in the administrative databases to identify patients with depressive disorders. The potential for misclassification of diagnostic codes in the medical record must be considered. Further, the coding classification system in the hospital database changed from ICD-9 to ICD-10-CA in April 2001. Given the mapping issue that exists for diagnosis codes in these 2 classification systems, potential discrepancies (between ICD-9 and ICD-10-CA) with regard to identifying patients with depressive disorders are possible. In this study it did not appear that the changeover from ICD-9 to ICD-10-CA created a significant challenge to the accuracy of the data, albeit the authors did not investigate this matter. The time period available for reviewing the medical charts in the 3 family practice clinics was approximately 6 months. This rather short time period may have had an impact on the assessment of charts in the clinics, particularly with a chronic disease such as depression that often fluctuates over time. During chart auditing the reviewer was not blinded as to which group the charts belonged to (depressed cohort vs. nondepressed cohort). This could have created a potential bias. Further, only 1 psychiatrist (although experienced with many years of practice) reviewed all excluded charts. An interview clearly is the best option for a psychiatrist to assess a depressive disorder and confirm the disease status (depressed vs. nondepressed)³⁶; however, this was not a feasible option in our study. We recognize that a psychiatrist's review of a medical record in order to confirm or refute a diagnosis of depression may not be the most valid method. However, it should be noted that most studies employed a similar approach (ie, having a psychiatrist or psychologist review medical records) in order to confirm a diagnosis of depression.^36
–38 Finally, the lack of available drug data limited our ability to enhance the development of valid case definitions for depressive disorders.

Conclusions

This study found that provincial administrative databases may be useful for undertaking surveillance of depressive disorders among the adult population. The approach used to develop case definitions for depressive disorders was simple and resulted in a reasonable degree of sensitivity, specificity, and positive predictive value. Although this study focused on depressive disorders, the methodology can be adopted for other mental disorders.

Footnotes

Acknowledgments

This work could not have been completed without the assistance of the physicians and staff at the Family Practice Unit, particularly Marshall Godwin, MD, Barbara Morrisey, Graduate Diploma in Business Education, Louise Noftall, LPN, and Barbara Dunphy, LPN. We are also grateful to Neil Gladney, MSc, Sarah Wickham, MSc, Jeff Dawn, MSc, and Khokan Sikdar, PhD from the Research and Evaluation Department of the Centre for Health Information for their support. Further, we would like to thank Terence Callanan, MD, and Terence Fogwill, MD for their support during the study.

Disclosure Statement

Drs. Alaghehbandan, MacDonald, Barrett, Collins, and Chen disclosed no conflicts of interest with respect to the research, authorship, and/or publication of this article.

This study was funded by the Public Health Agency of Canada and presented at the 43rd Annual Society for Epidemiologic Research Meeting in Seattle, WA (June 23–26, 2010).

Appendix B.

Potential Case Definitions Examined in the Study

Case Definitions #1–8	≥1 hospitalizations due to depressive disorders any time OR ≥1–8 physician visits^* due to depressive disorders any time
Case Definitions #9–16	≥1 hospitalizations due to depressive disorders any time OR ≥1–8 physician visits due to depressive disorders within the first year of diagnosis
Case Definitions #17–24	≥1 hospitalizations due to depressive disorders any time OR ≥1–8 physician visits due to depressive disorders within the first 2 years of diagnosis
Case Definitions #25–32	≥1 hospitalizations due to depressive disorders any time OR ≥1–8 physician visits due to depressive disorders within the first 3 years of diagnosis
Case Definitions #33–40	≥1 hospitalizations due to depressive disorders any time OR ≥1–8 physician visits due to depressive disorders within the first 4 years of diagnosis
Case Definitions #41–48	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) AND ≥1–8 GP visits due to depressive disorders any time
Case Definitions #49–56	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) AND ≥1–8 GP visits due to depressive disorders within the first 1 year of diagnosis
Case Definitions #57–64	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) AND ≥1 GP visits due to depressive disorders within the first 2 years of diagnosis
Case Definitions #65–72	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) AND ≥1–8 GP visits due to depressive disorders within the first 3 years of diagnosis
Case Definitions #73–80	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) AND ≥1–8 GP visits due to depressive disorders within the first 4 years of diagnosis
Case Definitions #81–88	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) OR ≥1–8 GP visits due to depressive disorders any time
Case Definitions #89–96	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) OR ≥1–8 GP visits due to depressive disorders within the first 1 year of diagnosis
Case Definitions #97–104	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) OR ≥1–8 GP visits due to depressive disorders within the first 2 years of diagnosis
Case Definitions #105–112	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) OR ≥1–8 GP visits due to depressive disorders within the first 3 years of diagnosis
Case Definitions #113–120	(≥1 hospitalizations OR ≥1 psychiatrist visit due to depressive disorders any time) OR ≥1–8 GP visits due to depressive disorders within the first 4 years of diagnosis

Physician visits indicate general practitioner (GP) visits.

References

Rihmer

, Angst

. Mood disorders: Epidemiology. Kaplan

, Sadock

. Kaplan and Sadock's Comprehensive Textbook of Psychiatry. New York: Lippincott, Williams & Wilkins, 2005; 1576–1582.

Bromet

, Andrade

, Hwang

et al. Cross-national epidemiology of DSM-IV major depressive episode. BMC Med, 2011; 9:90.

Hyman

, Chisholm

, Kessler

, Patel

, Whiteford

. Mental disorders. Jamison

, Breman

, Measham

et al. Disease Control Priorities in Developing Countries, 2nd. Washington, DC: World Bank, 2006; 605–622.

Post

, Kilbourne

, Bremer

, Solano

Jr , Pincus

, Reynolds

3rd . Organizational factors and depression management in community-based primary care settings. Implement Sci, 2009; 4:84.

Kilbourne

, Daugherty

, Pincus

. What do general medical guidelines say about depression care? Depression treatment recommendations in general medical practice guidelines. Curr Opin Psychiatry, 2007; 20:626–631.

Patel

. Toward a national strategy on mental illness and mental health. http://www.cma.ca/index.cfm/ci_id/33248/la_id/1.htm. 2011 June 4.

Kisely

, Lin

, Lesage

et al. Use of administrative data for the surveillance of mental disorders in 5 provinces. Can J Psychiatry, 2009; 54:571–575.

Harpe

. Using secondary data sources for pharmacoepidemiology and outcomes research. Pharmacotherapy, 2009; 29:138–153.

Patten

, Wang

, Williams

et al. Descriptive epidemiology of major depression in Canada. Can J Psychiatry, 2006; 51:84–90.

10.

Wang

, Williams

, Lavorato

, Schmitz

, Dewa

, Patten

. The incidence of major depression in Canada: The National Population Health Survey. J Affect Disord, 2010; 123:158–163.

11.

Patten

, Beck

. Major depression and mental health care utilization in Canada: 1994 to 2000. Can J Psychiatry, 2004; 49:303–309.

12.

Vasiliadis

, Lesage

, Adair

, Boyer

. Service use for mental health reasons: Cross-provincial differences in rates, determinants, and equity of access. Can J Psychiatry, 2005; 50:614–619.

13.

Satyanarayana

, Enns

, Cox

, Sareen

. Prevalence and correlates of chronic depression in the Canadian community health survey: Mental health and well-being. Can J Psychiatry, 2009; 54:389–398.

14.

Lix

, Yogendran

, Shaw

, Burchill

, Metge

, Bond

. Population-based data sources for chronic disease surveillance. Chronic Dis Can, 2008; 29:31–38.

15.

Okura

, Urban

, Mahoney

, Jacobsen

, Rodeheffer

. Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol., 2004; 57:1096–1103.

16.

Charbonneau

, Rosen

, Owen

et al. Monitoring depression care: In search of an accurate quality indicator. Med Care, 2004; 42:522–531.

17.

Wells

, Schoenbaum

, Unützer

, Lagomasino

, Rubenstein

. Quality of care for primary care patients with depression in managed care. Arch Fam Med, 1999; 8:529–536.

18.

Dhingra

, Kroenke

, Zack

, Strine

, Balluz

. PHQ-8 days: A measurement option for DSM-5 major depressive disorder (MDD) severity. Popul Health Metr, 2011; 9:11.

19.

Nease

Jr , Aikens

, Klinkman

, Kroenke

, Sen

. Toward a more comprehensive assessment of depression remission: The Remission Evaluation and Mood Inventory Tool (REMIT) Gen Hosp Psychiatry, 2011; 33:279–286.

20.

Katon

. Epidemiology and treatment of depression in patients with chronic medical illness. Dialogues Clin Neurosci, 2011; 13:7–23.

21.

Akiskal

. Mood disorders. Kaplan

, Sadock

. Kaplan and Sadock's Comprehensive Textbook of Psychiatry. New York: Lippincott, Williams & Wilkins, 2005; 1560–1575.

22.

West

, Richter

, Melfi

, McNutt

, Nennstiel

, Mauskopf

. Assessing the Saskatchewan database for outcomes research studies of depression and its treatment. J Clin Epidemiol, 2000; 53:823–831.

23.

Spetell

, Wall

, Allison

et al. Identifying physician-recognized depression from administrative data: Consequences for quality measurement. Health Serv Res, 2003; 38:1081–1102.

24.

Damush

, Jia

, Ried

et al. Case-finding algorithm for post-stroke depression in the Veterans Health Administration. Int J Geriatr Psychiatry, 2008; 23:517–522.

25.

Fultz

, Skanderson

, Mole

et al. Development and verification of a “virtual” cohort using the national VA Health Information System. Med Care, 2006; 44:S25–S30.

26.

Hux

, Ivis

, Flintoft

, Bica

. Diabetes in Ontario: Determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care, 2002; 25:512–516.

27.

James

, Blanchard

, Campbell

et al. A model for non-communicable disease surveillance in Canada: The prairie pilot diabetes surveillance system. Chronic Dis Can, 2004; 25:7–12.

28.

Solberg

, Engebretson

, Sperl-Hillen

, Hroscikoski

, O'Connor

. Are claims data accurate enough to identify patients for performance measures or quality improvement? The case of diabetes, heart disease, and depression. Am J Med Qual, 2006; 21:238–245.

29.

, Dell

, Dick

et al. Case verification of children with asthma in Ontario. Pediatr Allergy Immunol, 2006; 17:69–76.

30.

Buehler

. Surveillance. Rothman

, Greenland

, Lash

. Modern Epidemiology. Philadelphia: Lippincott, Williams & Wilkins, 2008; 459–480.

31.

Robinson

, Young

, Roos

, Gelskey

. Estimating the burden of disease. Comparing administrative data and self-reports. Med Care, 1997; 35:932–947.

32.

Hoogenveen

, Westert

, Dijkgraaf

, Schellevis

, de Bakker

. Disease prevalence estimations based on contact registrations in general practice. Stat Med, 2002; 21:2271–2285.

33.

Coffin

, Saunders

, Thomas

, Loewen

, Ghali

, Campbell

. Validity of ICD-9-CM administrative data for determining eligibility for pneumococcal vaccination triggers. Am J Med Qual, 2005; 20:158–163.

34.

Himelhoch

, Weller

, Wu

, Anderson

, Cooper

. Chronic medical illness, depression, and use of acute medical services among Medicare beneficiaries. Med Care, 2004; 42:512–521.

35.

Moussavi

, Chatterji

, Verdes

, Tandon

, Patel

, Ustun

. Depression, chronic disease and decrements in health: Evidence from the World Health Surveys. Lancet, 2007; 369:851–858.

36.

Sanchez-Villegas

, Schlatter

, Ortuno

et al.

Validity of a self-reported diagnosis of depression among participants in a cohort study using the Structured Clinical Interview for DSM-IV (SCID-I)

BMC Psychiatry, 2008; 8:43.

37.

Löwe

, Spitzer

, Gräfe

et al. Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians' diagnoses. J Affect Disord, 2004; 78:131–140.

38.

Stafford

, Berk

, Jackson

. Validity of the Hospital Anxiety and Depression Scale and Patient Health Questionnaire-9 to screen for depression in patients with coronary artery disease. Gen Hosp Psychiatry, 2007; 29:417–424.

Using Administrative Databases in the Surveillance of Depressive Disorders—Case Definitions