Level of agreement between medical record and ICD-10-AM coding of mental health,alcohol and drug conditions in trauma patients

Abstract

Background:

Despite the reliance on administrative data in epidemiological studies, there is little information on the completeness of co-morbidities in administrative data coded from medical records.

Objective:

The aim of this study was to quantify the agreement between the International Classification of Diseases, Tenth Revision, Australian Modification (ICD-10-AM) administrative coding of mental health, drug and alcohol co-morbidities and medical records in a severely injured patient population.

Method:

A random sample of patients (n = 500) captured by the Victorian State Trauma Registry and definitively managed at the state’s adult major trauma services was selected for the study. Retrospective medical record review was conducted to collect data about documented co-morbidities. The agreement between ICD-10-AM data generated from routine hospital coding and medical record–based co-morbidities was determined using Cohen’s κ and prevalence-adjusted bias-adjusted kappa (PABAK) statistics.

Results:

The percentage of agreement between the medical record and ICD-10-AM coding for mental health, drug and alcohol co-morbidities was 72.8%, and the PABAK showed moderate agreement (PABAK = 0.46; 95% confidence interval (CI): 0.37, 0.54). There was no difference in agreement between unintentional injury patients (PABAK = 0.52; 95% CI: 0.42, 0.62) compared with intentional injury patients (PABAK = 0.36, 95% CI: 0.23, 0.49), and no change in agreement for patients admitted before (PABAK = 0.40; 95% CI: 0.30, 0.50) and after the introduction of mandatory co-morbidity coding (PABAK = 0.46; 95% CI: 0.37, 0.54).

Conclusion:

Despite documentation in the medical record, a large proportion of mental health, drug and alcohol conditions were not coded in ICD-10-AM. Acknowledgement of these limitations is needed when using ICD-10-AM coded co-morbidities in research studies and health policy development.

Implications:

This work has implications for researchers of drug and alcohol abuse; mental health; accidents and injuries; workers' compensation; health workforce; health services; and policy decisions for healthcare, emergency services, insurance industry, national productivity and welfare costings reliant on those research outcomes.

Keywords

clinical coding medical record ICD-10-AM data quality documentation activity-based funding registries mental health co-morbid

Introduction

Pre-existing mental health, drug and alcohol co-morbidities play a large role in injury risk and outcome (O’Donnell et al., 2009). Although mental illness is most strongly associated with intentional injuries, such as self-harm or assault, recent studies have linked conditions such as depression and psychoses with unintentional falls and motor vehicle crashes (Dicker et al., 2011; Weinberg et al., 2016). In order to characterise the injured patient population, valid and reliable information about mental health, drug and alcohol co-morbidities is necessary. From a health service planning and policy perspective, these co-morbidities require consideration in clinical care, burden of injury estimates and healthcare expenditure (Gijsen et al., 2001). From an epidemiological perspective, an accurate measurement of co-morbidities is essential to adjust for confounding factors in analyses of injury risk and outcomes (Degroot et al., 2003).

Mental health, drug and alcohol co-morbidities are challenging to identify in research studies, as clinical diagnosis relies on self-reported symptoms and behavioural observation (Jablensky, 2016). Since mental health conditions are considerably more prevalent in self-harm and violence-related injuries, demonstrable signs and symptoms of a mental health condition are more commonly recognised in intentional injury groups (van der Westhuizen et al., 2014). In contrast, less severe conditions may be less commonly identified in clinical settings, making it difficult for clinicians to reliably diagnose these conditions, particularly in the absence of refined biological markers (Neumann and Spies, 2003).

Previous studies have used a variety of sources of data to characterise co-morbidities in injured populations (O’Donnell et al., 2009; Palmu et al., 2015; Turner et al., 2003; Zatzick et al., 2004). Traditionally, clinical interviews are used as a “gold” standard for capturing information about pre-existing mental health conditions (Hawker et al., 1997), but are not feasible for large studies (Spitzer, 1983). Medical record reviews provide a source of detailed clinical information, but are typically onerous and labour-intensive to conduct (Yawn and Wollan, 2005). Administrative data preclude the need for self-reported data, are convenient, readily available and collect large volumes of information (Duijsens et al., 1996; Nau et al., 2013). However, administrative data coding of co-morbid diagnoses has shown varying levels of agreement with medical record review (Henderson et al., 2006; Humphries et al., 2000; Malenka et al., 1994).

The purpose of this study was to examine the agreement between International Classification of Diseases, Tenth Revision, Australian Modification (ICD-10-AM) administrative data coding of mental health, drug and alcohol conditions and medical record documentation of these conditions in a severely injured patient population.

In Australia, patient diseases are coded from the medical record for admitted patients using ICD-10-AM, while the Australian Classification of Health Interventions (ACHI) is used to code procedures and interventions. The Australian Coding Standards (ACS) are used to assist clinical coders to assign clinical codes. The ICD-10-AM, ACHI and the ACS form the basis of the Australian Refined Diagnosis-Related Group (AR-DRG) classification system with procedure and diagnoses codes grouped to DRGs. The ACS stipulate that conditions are coded as co-morbidities only if they require investigation, increased clinical care or monitoring, altered treatment, or use resources during the admission (National Centre for Classification in Health, 1998). A recent update was implemented as part of a review of the ACS, which aimed to address under-coding of chronic conditions in administrative data sets through the incorporation of supplementary “U” codes. This led to the development, in July 2015, of a list of “clinically important” conditions, which include depression and schizophrenia, that are specifically mapped so as not to be included in the DRG allocation. These supplementary codes are to be coded when there is evidence that a condition is part of a patient’s current health status during an episode of admitted patient care, but the condition has not met the criteria for coding (National Centre for Classification in Health, 2015). The introduction of mandatory coding of chronic conditions is likely to increase the prevalence of mental health diagnoses in administrative data sets.

Method

Setting

The Victorian State Trauma Registry (VSTR) is a population-based registry collecting data about virtually all major trauma patients from every trauma-receiving hospital (n = 138 facilities) in Victoria (Gabbe et al., 2015). The VSTR captures information about all eligible major trauma patients, who are included in the registry unless they (or their next of kin in the case of deaths) request to opt out. The opt-out rate is less than 1% of all eligible cases (Gabbe et al., 2006).

Study design

Patients were eligible for inclusion in this study if they met each of the following criteria:

aged 18 years or older;

definitively managed at one of the two adult major trauma services in Victoria; and

survived to 12 months post-injury.

A random sample of 500 patients registered on the VSTR was selected for the study. The sample comprised 250 patients at each adult major trauma service, consisting of 200 patients admitted prior to the ACS changes (admitted between 1 July 2013 and 30 June 2015) and 50 patients admitted after the changes (admitted between 1 July 2015 and 31 December 2015). The sample of patients prior to and after the ACS changes from each hospital consisted of 60% unintentional injury and 40% intentional injury patients. In order to enable the comparison of agreement between injury intent groups, intentional injury patients were oversampled (Figure 1), because major trauma cases generally comprise approximately 10% of intentional injuries (Gabbe et al., 2017). This study was approved by the Human Research Ethics Committees at the Alfred Hospital, the Royal Melbourne Hospital and Monash University.

Figure 1.

Sampling method for the study (n = 500).

Medical record review

Retrospective review of the hospital medical records of the sample of trauma patients recorded on the VSTR was conducted. Manual medical record review was completed by reviewing the medical record for the major trauma admission recorded in the VSTR, which was located using identifiers including hospital unit record number, admission identification and date of injury. Information from other admissions contained within the medical record was not reviewed. Data were collected from patients’ medical records at each hospital using a systematic procedure and entered into a standardised data collection form. This included examination of the patient’s discharge summary, admission, clinical progress, transfer and pharmacist notes, in correspondence with specialists, liaison psychiatry reports, and trauma and resuscitation records. To enable comparison with the ICD-10-AM codes, co-morbid conditions were collected from the medical records based on the major categories of conditions coded in ICD-10-AM: mental disorders (organic mental disorders, schizophrenia, mood disorders, neurotic or anxiety disorders, behavioural and personality disorders), drug use disorders (opioids, cannabinoids, sedatives or hypnotics, stimulants) and alcohol use disorders.

For the data collected from the medical record, a clear distinction was made between symptoms of mental health problems and having a clinical diagnosis for investigation of pre-existing co-morbidities only. Patients reported to have symptoms of depression or anxiety, but without an explicit diagnosis were excluded from any data collection from the medical record. If there was evidence of a mental health, drug or alcohol condition present during the major trauma admission in the medical record, this had to be substantiated by other documentation. This included evidence such as the presence of an alcohol withdrawal scale, medications administered for a mental health condition in the medication chart, treatment by the addiction medicine service for a drug or alcohol use condition or a psychiatrist referral letter for ongoing treatment. This evidence must have been consistent with clinical notation. Mental health conditions that developed after the injury event, that is, post-traumatic stress disorder, retrograde or anterograde amnesia, were excluded. Clinical notation of delirium, delusions or hallucinations was excluded, unless explicitly associated with a specified mental disorder. If there was more than one mental disorder described in the record for the major trauma admission, all conditions described were recorded. If physician or other clinical diagnoses conflicted with psychiatry reports, the mental diagnoses were obtained from the notes recorded by psychiatric liaison services. In the case of multiple diagnoses of mental health and/or substance use condition, only conditions supported by substantiating evidence were recorded. The reviewer was unable to view patients’ injury intent and ICD-10-AM codes that had been assigned to patient admissions. This method was designed to minimise potential bias, as the reviewer was not influenced by preconceived agreement between the data sources. Difficulties encountered in cases which required further scrutiny were resolved by consultation with researchers with clinical and health expertise to reach consensus.

ICD-10-AM codes

Indicator variables were generated to represent the presence of the key mental health, drug and alcohol use conditions coded by ICD-10-AM, which included the supplementary “U” codes that were introduced from July 2015. The major categories were alcohol use disorders (F10), drug use disorders (F11-F15, F16, F19) and mental disorders (F00-F03, F07, F09, F20-F48, F60-F69, F90-F99, U792, U793). The subcategories of mental disorders were schizophrenia (F20-F29, U792), mood disorders (F30-F39, U793), neurotic disorders (F40-F48) and personality and behavioural disorders (F60-F69; F90-F99). Drug use disorders included opioids (F11), cannabinoids (F12), sedatives or hypnotics (F13) and stimulants (F14, F15).

Data analysis

Data abstracted from the medical records were used to assess the level of agreement between the medical record and administrative data in the form of ICD-10-AM codes provided by the participating hospitals to the VSTR. Within each group of patients admitted, pre- and post-ACS changes and within each intent group, percent agreement and κ values were calculated to assess agreement in mental health, drug and alcohol diagnoses determined by ICD-10-AM coding versus medical record review.

The prevalence of mental health, drug and alcohol conditions was calculated, and agreement was measured using Cohen’s κ. The κ indicates a numeric rating of the degree of agreement between two raters, taking into account the degree of agreement that would be expected by chance. The prevalence-adjusted bias-adjusted kappa (PABAK) was used to account for a low or high proportion of responses in a single category, as the PABAK takes into consideration the prevalence of the condition within the sample as well as the discrepancy between the diagnoses assigned by different raters (Cicchetti and Feinstein, 1990; Feinstein and Cicchetti, 1990). The positive and negative percentage agreement was calculated for each condition in the sample: the negative percentage agreement refers to the average proportionate agreement for having no mental health, drug or alcohol condition in either source and the positive percentage agreement indicates the proportionate agreement in which there is a condition diagnosed (Cicchetti and Feinstein, 1990).

The 95% confidence intervals for the κ and PABAK statistics were calculated using the 95th percentile from 200 bootstrap replications. A κ of 0.80 and above was considered excellent agreement, coefficients between 0.61 and 0.80 substantial agreement, coefficients between 0.41 and 0.60 moderate agreement and <0.41 fair to poor agreement (Cohen, 1960). All analyses were conducted using Stata 13.0 (StataCorp, College Station, Texas, USA).

Results

The clinical and demographic characteristics of the study sample are summarised in Table 1. There was an even distribution of patients among the age groups and the cohort represented a predominantly male population. The majority of patients did not have a Charlson comorbid condition recorded and were non-compensable by the Transport Accident Commission or WorkSafe Victoria. A large proportion of injuries involved a motor vehicle crash (19.0%), or motorcycle or pedal cyclist crash (14.2%). A large proportion of patients were injured being struck by an object or person (22.8%), because of the deliberate over-sampling of a high proportion of intentional injured patients.

Table 1.

Clinical and demographic characteristics of the study sample.

Demographic characteristics, n (%)	Total, n = 500
Age (years)
18–24	63 (12.6)
25–34	92 (18.4)
35–44	98 (19.6)
45–54	91 (18.2)
55–64	72 (14.4)
≥65	84 (16.8)
Gender
Male	374 (74.8)
Female	126 (25.2)
Intent
Unintentional	300 (60.0)
Intentional	200 (40.0)
Mechanism
Motor vehicle crash	95 (19.0)
Motorcycle/pedal cyclist crash	71 (14.2)
Pedestrian collision	22 (4.4)
Fall	95 (19.0)
Struck by object or person	114 (22.8)
Cutting/piercing object	55 (11.0)
Horse-related	15 (3.0)
Other	33 (6.6)
Charlson Comorbidity Index
0	351 (70.2)
1	115 (23.0)
≥2	34 (6.8)
Injury severity score
<9	25 (5.0)
9–15	163 (32.6)
16–24	202 (40.4)
>24	110 (22.0)
Compensable status^a
Compensable	180 (36.1)
Non-compensable	318 (63.9)

^aCompensable status not known for n = 2.

Based on the medical record documentation, 45% of patients in the sample had at least one mental health, drug or alcohol use co-morbidity, compared to 24% of patients having a condition coded in ICD-10-AM (Table 2). The most common co-morbidities were a mood disorder, alcohol use disorder and drug use disorder.

Table 2.

Prevalence of pre-existing mental health, drug and alcohol co-morbidities in medical record data and ICD-10-AM coded data for all patients.

Co-morbidity	Medical record data, n = 500	ICD-10-AM data, n = 500
Co-morbidity	Total, n (%)	Total, n (%)
Any mental, drug or alcohol use disorder	225 (45.0)	119 (23.8)
Any mental disorder	157 (32.0)	68 (13.6)
Schizophrenia	17 (3.4)	17 (3.4)
Mood disorders	130 (26.4)	41 (8.2)
Neurotic disorders	67 (13.4)	9 (1.8)
Behavioural and personality disorders	23 (4.6)	11 (2.2)
Not specified	8 (1.6)	–
Any drug use disorder	97 (19.4)	21 (4.2)
Opioids	30 (6.0)	11 (2.2)
Cannabinoids	50 (10.0)	5 (1.0)
Sedatives or hypnotics	6 (1.2)	4 (0.8)
Stimulants	44 (8.8)	5 (1.0)
Not specified	13 (2.6)	–
Alcohol use disorder	104 (20.8)	23 (4.6)

ICD-10-AM: International Classification of Diseases, Tenth Revision, Australian Modification.

There was no significant increase in the prevalence of any disorder in the post-ACS phase compared to the pre-ACS phase (Table 3).

Table 3.

Prevalence of mental health, drug and alcohol co-morbidities in ICD-10-AM coded data before and after changes to the ACS.

Co-morbidity	Pre-ACS, n = 400	Post-ACS, n = 100
Co-morbidity	% (95% CI)	% (95% CI)
Any mental, drug or alcohol use disorder	22.8 (18.6, 26.9)	28.0 (19.2, 36.8)
Any mental disorder	12.3 (9.0, 15.5)	19.0 (11.3, 26.7)
Schizophrenia	3.5 (1.7, 5.3)	3.0 (0, 6.3)
Mood disorders	6.8 (4.3, 9.2)	14.0 (7.2, 20.8)
Any drug use disorder	3.8 (1.9, 5.6)	6.0 (1.3, 10.7)
Alcohol use disorder	5.3 (3.1, 7.4)	2.0 (0, 4.7)

ICD-10-AM: International Classification of Diseases, Tenth Revision, Australian Modification; ACS: Australian Coding Standards.

Overall, the agreement for all mental health, drug and alcohol co-morbidities between the medical record and ICD-10-AM was moderate (Table 4). The κ was moderate for schizophrenia and behavioural and personality disorders. After adjusting for prevalence and bias, the PABAK rose to excellent agreement (Table 4). The κ and PABAK differed considerably for a number of specific sub–co-morbidities, indicating the κ statistic was vulnerable to change due to sparse data for these categories. In the adjusted κ, the agreement on mood disorders between the medical record and ICD-10-AM was moderate to substantial, but excellent for schizophrenia and behavioural and personality disorders. Agreement for diagnoses of drug and alcohol use co-morbidities was substantial, and drug use co-morbidities due to opioids, cannabinoids, sedatives or hypnotics, and stimulants showed excellent agreement. The negative agreement percentages varied from 0.79 to 0.99, indicating excellent agreement when there was no diagnosis in either source, and from 0.07 to 0.62 for positive agreement percentages.

Table 4.

Agreement between medical record and ICD-10-AM coded data for the presence of mental health, drug and alcohol co-morbidities, all patients (n = 500).

Co-morbidity	Agreement (%)	κ coefficient (95% CI)	PABAK (95% CI)	Negative agreement (%)	Positive agreement (%)
Any mental, drug or alcohol use disorder	72.8	0.43 (0.37, 0.51)	0.46 (0.37, 0.54)	79.3	60.5
Any mental disorder	80.2	0.46 (0.37, 0.54)	0.60 (0.53, 0.68)	87.2	62.6
Schizophrenia	97.2	0.57 (0.35, 0.74)	0.94 (0.92, 0.97)	98.6	58.8
Mood disorders	82.2	0.41 (0.29, 0.50)	0.64 (0.57, 0.71)	89.3	48.0
Neurotic disorders	86.0	0.05 (−0.02, 0.14)	0.72 (0.66, 0.78)	92.4	7.9
Behavioural and personality disorders	96.4	0.45 (0.21, 0.66)	0.93 (0.89, 0.96)	98.1	47.1
Any drug use disorder	84.0	0.27 (0.16, 0.38)	0.68 (0.62, 0.74)	90.9	32.2
Opioids	95.4	0.42 (0.23, 0.65)	0.91 (0.87, 0.94)	97.6	43.9
Cannabinoids	90.6	0.13 (0.03, 0.28)	0.81 (0.76, 0.86)	95.0	14.5
Sedatives or hypnotics	99.2	0.60 (0.24, 0.96)	0.98 (0.97, 1.00)	99.6	60.0
Stimulants	91.4	0.11 (0.00, 0.25)	0.83 (0.78, 0.88)	95.5	12.2
Alcohol use disorder	83.4	0.29 (0.20, 0.42)	0.67 (0.61, 0.73)	90.5	34.6

ICD-10-AM: International Classification of Diseases, Tenth Revision, Australian Modification; PABAK: prevalence-adjusted bias-adjusted kappa.

The unintentional injury group yielded poor agreement in the unadjusted κ statistics (Table 5). After adjusting for bias and prevalence, the PABAK demonstrated moderate agreement for all co-morbidities. There was no difference in the agreement in unintentional injury patients compared to the intentional injury group. The agreement between the medical record and ICD-10-AM coded data for patients admitted in the pre-ACS phase was moderate. As shown in Table 5, the introduction of mandatory coding made no difference to the agreement between the medical record and ICD-10-AM for most co-morbidities. Overall, the Alfred Hospital demonstrated similar agreement to the Royal Melbourne Hospital (Table 5).

Table 5.

Agreement between medical record and ICD-10-AM coded data for the presence of mental health, drug and alcohol co-morbidities, by injury intent, before and after mandatory coding and between trauma services.

Co-morbidity	Unintentional injury patients (n = 300)			Intentional injury patients (n = 200)
Co-morbidity	Agreement (%)	κ coefficient (95% CI)	PABAK (95% CI)	Agreement (%)	κ coefficient (95% CI)	PABAK (95% CI)
Any mental, drug or alcohol use disorder	76.0	0.28 (0.19, 0.39)	0.52 (0.42, 0.62)	68.0	0.39 (0.26, 0.49)	0.36 (0.23, 0.49)
Any mental disorder	80.3	0.10 (0.01, 0.24)	0.61 (0.51, 0.70)	80.0	0.59 (0.49, 0.70)	0.60 (0.49, 0.71)
Any drug use disorder	91.7	0.17 (0.04, 0.39)	0.83 (0.77, 0.90)	72.5	0.27 (0.17, 0.40)	0.45 (0.33, 0.57)
Alcohol use disorder	91.3	0.32 (0.15, 0.50)	0.83 (0.77, 0.89)	71.5	0.25 (0.16, 0.36)	0.43 (0.31, 0.55)
	Pre-ACS (n = 400)			Post-ACS (n = 100)
	Agreement (%)	κ coefficient (95% CI)	PABAK (95% CI)	Agreement (%)	κ coefficient (95% CI)	PABAK (95% CI)
Any mental, drug or alcohol use disorder	70.0	0.37 (0.26, 0.45)	0.40 (0.30, 0.50)	84.0	0.66 (0.48, 0.78)	0.46 (0.37, 0.54)
Any mental disorder	79.0	0.41 (0.31, 0.50)	0.58 (0.50, 0.66)	85.0	0.61 (0.43, 0.75)	0.60 (0.53, 0.68)
Any drug use disorder	83.0	0.23 (0.11, 0.33)	0.66 (0.59, 0.73)	88.0	0.45 (0.22, 0.70)	0.68 (0.62, 0.74)
Alcohol use disorder	84.0	0.33 (0.21, 0.41)	0.68 (0.61, 0.75)	81.0	0.14 (0.00, 0.39)	0.67 (0.60, 0.73)
	Alfred Hospital (n = 250)			Royal Melbourne Hospital (n = 250)
	Agreement (%)	κ coefficient (95% CI)	PABAK (95% CI)	Agreement (%)	κ coefficient (95% CI)	PABAK (95% CI)
Any mental, drug or alcohol use disorder	74.4	0.48 (0.39, 0.58)	0.49 (0.37, 0.60)	71.2	0.35 (0.26, 0.48)	0.42 (0.31, 0.54)
Any mental disorder	82.0	0.52 (0.42, 0.63)	0.64 (0.54, 0.74)	78.4	0.39 (0.28, 0.53)	0.57 (0.47, 0.67)
Any drug use disorder	83.2	0.31 (0.18, 0.46)	0.66 (0.57, 0.76)	84.8	0.22 (0.09, 0.39)	0.70 (0.60, 0.79)
Alcohol use disorder	84.0	0.43 (0.29, 0.56)	0.68 (0.59, 0.77)	82.8	0.06 (−0.02, 0.17)	0.66 (0.56, 0.76)

ICD-10-AM: International Classification of Diseases, Tenth Revision, Australian Modification; PABAK: prevalence-adjusted bias-adjusted kappa; ACS: Australian Coding Standards.

Discussion

In this study of randomly sampled trauma registry patients, routinely collected administrative coded co-morbidity data were compared with the hospital clinical documentation. Consistent with a prior study (Preen et al., 2004), we found that the prevalence of co-morbid conditions obtained from administrative data was lower than that from medical records. The results showed the medical record and ICD-10-AM agreed well when the patient did not have the co-morbidity in either source of data. In contrast, the concordance was relatively poor in cases where a mental health, drug or alcohol condition was present in the medical record, but not coded in the ICD-10-AM data, particularly for specific sub-conditions in which the prevalence was low (Table 4). Under-reporting of co-morbidity information in administrative hospital data has previously been reported for a number of co-morbidities (Januel et al., 2011; Powell et al., 2001).

Several factors could have played a role in the discrepancies between the medical record documentation and administrative data in this study. Mandatory clinical coding aside, clinical coders extract information according to strict coding guidelines and are confined to coding co-morbidities that are identified in the medical record as clinically significant for the patient’s admission, or that require evaluation, treatment, management or care (World Health Organization, 1992). It is likely that the mental health, drug and alcohol co-morbidities, despite the presence of documentation of clinical signs and symptoms during the patient’s admission within the medical record, did not meet the threshold according to the ACS for receiving treatment or resource use and subsequent ICD-10-AM coding (National Centre for Classification in Health, 2015). It is possible that the coding error may have contributed to under-coding of mental health, drug and alcohol co-morbidities in administrative data; however, in accordance with previous findings (Fowles et al., 1998; Preen et al., 2004), the occurrence of coding in the absence of medical record documentation or “false positives” was low for all co-morbidities. Notwithstanding the mandatory co-morbidity codes, this indicates a high degree of adherence to the ICD-10-AM ACS and clinical coding guidelines by hospital coders (Henderson et al., 2006).

Within the “activity-based funding” infrastructure, under-coding means the hospital may not receive all the reimbursement to which it is entitled, whereas over-coding could result in penalties and regulatory consequences (Cheng et al., 2009). Upon discharge, healthcare facilities are allocated a single AR-DRG, or predetermined financial payment expressed as a price weight, for each type of patient episode, such as same day episodes, surgery required and episodes for radiotherapy (Curtis et al., 2002). Due to the ACS, coders employed by hospitals would have not included co-morbidities that did not meet the criteria for coding and subsequent DRG allocation. In the case of major trauma, DRG allocation is among the highest price weights, as the multiple and complex injuries sustained use many and diverse hospital resources. It is therefore less likely that major trauma patients with mental health, drug and alcohol co-morbidities, that did not increase the DRG weighting, were coded in ICD-10-AM. This strengthens the argument that the under-recording of co-morbidities in administrative data reflects the coding practices (Nguyen et al., 2017).

As this is the first study, to our knowledge, to compare mental health diagnoses in hospital administrative data with medical record documentation in trauma patients, it is difficult to make direct comparisons with previous research. The agreement for mental health diagnoses in this study appears to be lower than that reported by other studies. Kashner (1998) reported 94% agreement for alcohol dependence and 95% for drug dependence between medical record and Veterans Health Affairs administrative data for a random sample of inpatients (n = 414), most likely a result of the low frequencies of these co-morbidities diagnosed in their sample. Bender and Smith (2016) studied the agreement between International Classification of Diseases, Ninth Revision, Clinical Modification coded mental health, drug and alcohol co-morbidities and clinical documentation in heart failure patients (κ = 0.69), with higher agreement than observed in our study. The variation among these studies may reflect the fact that administrative data are inconsistent across jurisdictions or influenced by inconsistency in study methods, such as variation in sample sizes and data collection methods. We employed standardised and systematic methods to abstract information from the medical records, and substantial evidence of a mental, drug or alcohol condition needed to be present. However, medical record reviews are inherently subjective and require judgements to be made by the abstractor on the evidence available, which may account for the positive classification of sub-threshold conditions in this study, in comparison with previous work.

The difference between the level of agreement before and after the changes to the ACS was minor overall. The update of the ACS in Australia in 2015 has the potential to improve concordance in administrative hospital coding relative to the medical record, and it was anticipated there would be an increase in the prevalence of diagnoses of schizophrenia and depression, which were specified sub-conditions included in the list of mandatory “U” codes. The prevalence of mood disorders, such as depression, did rise for patients admitted prior to and after the ACS changes, but there were no substantial increases in the recording of schizophrenia or other mental conditions. It appears that had a greater range of mental and substance use disorders been included in the list of mandatory “U” codes, there could have been observable discrepancies between the two phases. Accuracy of administrative coding clinical conditions can improve over time (Januel et al., 2011). It is possible that a slow uptake of the coding changes reflects insufficient time to make progress on the “learning curve” with the updated ACS. Therefore, a replication of this study at a later time point may show greater adherence to the ACS.

Limitations of the study

The results of this study should be interpreted in light of a number of considerations. The reviewer did not have formal training in health information management or a clinical background. It is possible that classification error may have limited our abstraction of the medical records. Handwritten medical and nursing progress notes were at times illegible and poor-quality original documents were sometimes difficult to read. Erroneous, incomplete or missing (3%) medical records made it difficult to establish specific diagnoses. Nonetheless, prior to the study, establishing the methodology for abstracting the mental, drug and alcohol co-morbidities from medical records in a consistent, standardised manner, and having clear research questions, reduced the risk of misclassifications.

This study was not a formal validation study whereby clinical interview was used to assess whether the condition was truly present in a patient and so not based on a “gold standard” diagnostic criterion. The purpose of the study was to assess how well ICD-10-AM administrative data agreed with medical record data for an important set of co-morbid conditions. The extent to which clinical information was missing in the medical record documentation cannot be determined without clinical diagnostic tests and additional comparison with clinical interview. Future studies with better resources should aim to include self-reported data to aid in the understanding of clinically relevant details and reduce uncertainties in unconfirmed diagnoses.

It is recognised that the sample was relatively small, particularly for patients admitted post-ACS changes. As there was only a 5-month period following the introduction of mandatory co-morbidity coding, this sample of 100 patients was extracted based on limited data. Subsequently, the study may have lacked statistical power to detect differences before and after the ACS update, which may have been a limiting factor in the comparison.

Conclusion

Despite the documentation of these co-morbidities in the medical record, a large proportion of mental health, drug and alcohol conditions are not coded in hospital administrative data, resulting in incomplete co-morbidity capture. Researchers and policymakers using the data for research studies, health policy and guideline development should exercise caution and consider the limitations of these data.

Footnotes

Acknowledgements

The authors would like to acknowledge the project team: Mimi Morgan, Melissa Hart, Sue McLellan, Jane Ford, Tani Thomas and Adrian Buzgau. The authors also acknowledge the data collectors and participating hospitals of the VSTR, and the members of the VSTR Steering Committee. The authors would like to thank Mary Lou Greer and Carol Roberts (Royal Melbourne Hospital).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Victorian State Trauma Registry (VSTR) is funded by the Department of Health and Human Services, State Government of Victoria and the Transport Accident Commission. TN was supported by an Australian Government Research Training Program Scholarship. BG was supported by an Australian Research Council Future Fellowship (FT170100048) during the preparation of this manuscript.

ORCID iD

Tu Q Nguyen, BBiomedSc, MBiomedSc

References

Bender

Smith

(2016) Using administrative mental health indicators in heart failure outcomes research: comparison of clinical records and international classification of disease coding. Journal of Cardiac Failure 22: 56–60.

Cheng

Gilchrist

Robinson

, et al. (2009) The risk and consequences of clinical miscoding due to inadequate medical documentation: a case study of the impact on health services funding. Health Information Management Journal 38: 35–46.

Cicchetti

Feinstein

(1990) High agreement but low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology 43: 551–558.

Cohen

(1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: 37–46.

Curtis

Bollard

Dickson

(2002) Coding errors and the trauma patient – is nursing case management the solution? Australian Health Review 25: 73–80.

Degroot

Beckerman

Lankhorst

, et al. (2003) How to measure comorbidity: a critical review of available methods. Journal of Clinical Epidemiology 56: 221–229.

Dicker

Mah

Lopez

, et al. (2011) Screening for mental illness in a trauma center: rooting out a risk factor for unintentional injury. Journal of Trauma Injury, Infection, and Critical Care 70: 1337–1344.

Duijsens

Bruinsma

Jansen

, et al. (1996) Agreement between self-report and semi-structured interviewing in the assessment of personality disorders. Personality and Individual Differences 21: 261–270.

Feinstein

Cicchetti

(1990) High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology 43: 543–549.

10.

Fowles

Fowler

Craft

(1998) Validation of claims diagnoses and self-reported conditions compared with medical records for selected chronic diseases. Journal of Ambulatory Care Management 21: 24–34.

11.

Gabbe

Cameron

Hannaford

, et al. (2006) Routine follow up of major trauma patients from trauma registries: what are the outcomes? Journal of Trauma Injury, Infection, and Critical Care 61: 1393–1399.

12.

Gabbe

Lyons

Fitzgerald

, et al. (2015) Reduced population burden of road transport-related major trauma after introduction of an inclusive trauma system. Annals of Surgery 261: 565–572.

13.

Gabbe

Simpson

Cameron

, et al. (2017) Long-term health status and trajectories of seriously injured patients: a population-based longitudinal study. PLoS Medicine 14: e1002322.

14.

Gijsen

Hoeymans

Schellevis

, et al. (2001) Causes and consequences of comorbidity: a review. Journal of Clinical Epidemiology 54: 661–674.

15.

Hawker

Coyte

Wright

, et al. (1997) Accuracy of administrative data for assessing outcomes after knee replacement surgery. Journal of Clinical Epidemiology 50: 265–273.

16.

Henderson

Shepheard

Sundararajan

(2006) Quality of diagnosis and procedure coding in ICD-10 administrative data. Medical Care 44: 1011–1019.

17.

Humphries

Rankin

Carere

, et al. (2000) Co-morbidity data in outcomes research: are clinical data derived from administrative databases a reliable alternative to chart review? Journal of Clinical Epidemiology 53: 343–349.

18.

Jablensky

(2016) Psychiatric classifications: validity and utility. World Psychiatry: Official Journal of the World Psychiatric Association 15: 26–31.

19.

Januel

Luthi

Quan

, et al. (2011) Improved accuracy of co-morbidity coding over time after the introduction of ICD-10 administrative data. BMC Health Services Research 11: 194.

20.

Kashner

(1998) Agreement between administrative files and written medical records: a case of the Department of Veterans Affairs. Medical Care 36: 1324–1336.

21.

Malenka

McLerran

Roos

, et al. (1994) Using administrative data to describe casemix: a comparison with the medical record. Journal of Clinical Epidemiology 47: 1027–1032.

22.

National Centre for Classification in Health (1998) International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Australian Modification (ICD-10-AM) Australian Coding Standards. Darlinghurst: University of Sydney.

23.

National Centre for Classification in Health (2015) Reference to Changes for ICD-10-AM/ACHI/ACS Ninth Edition. Darlinghurst: University of Sydney.

24.

Nau

Wutzler

Dorr

, et al. (2013) Liver cirrhosis but not alcohol abuse is associated with impaired outcome in trauma patients – a retrospective, multicentre study. Injury 44: 661–666.

25.

Neumann

Spies

(2003) Use of biomarkers for alcohol use disorders in clinical practice. Addiction 98(Suppl. 2): 81–91.

26.

Nguyen

Simpson

Gabbe

(2017) The prevalence of pre-existing mental health, drug and alcohol conditions in major trauma patients. Australian Health Review 41: 283–290.

27.

O’Donnell

Creamer

Elliott

(2009) Prior trauma and psychiatric history as risk factors for intentional and unintentional injury in Australia. Journal of Trauma Injury, Infection, and Critical Care 66: 470–476.

28.

Palmu

Partonen

Suominen

, et al. (2015) Return to work six months after burn: a prospective study at the Helsinki Burn Center. Burns 41: 1152–1160.

29.

Powell

Lim

Heller

(2001) Accuracy of administrative data to assess comorbidity in patients with heart disease: an Australian perspective. Journal of Clinical Epidemiology 54: 687–693.

30.

Preen

Holman

Lawrence

, et al. (2004) Hospital chart review provided more accurate comorbidity information than data from a general practitioner survey or an administrative database. Journal of Clinical Epidemiology 57: 1295–1304.

31.

Spitzer

(1983) Psychiatric diagnosis: are clinicians still necessary? Comprehensive Psychiatry 24: 399–411.

32.

Turner

Bombardier

Rimmele

(2003) A typology of alcohol use patterns among persons with recent traumatic brain injury or spinal cord injury: implications for treatment matching. Archives of Physical Medicine and Rehabilitation 84: 358–364.

33.

van der Westhuizen

Wyatt

Williams

, et al. (2014) Prevalence and predictors of mental disorders in intentionally and unintentionally injured emergency center patients. Journal of Nervous and Mental Disease 202: 638–646.

34.

Weinberg

Narayanan

Boden

, et al. (2016) Psychiatric illness is common among patients with orthopaedic polytrauma and is linked with poor outcomes. Journal of Bone and Joint Surgery (American Volume) 98: 341–348.

35.

World Health Organization (1992) The International Statistical Classification of Diseases and Related Health Problems, Tenth Revision. Geneva: World Health Organization.

36.

Yawn

Wollan

(2005) Interrater reliability: completing the methods description in medical records review studies. American Journal of Epidemiology 161: 974–977.

37.

Zatzick

Jurkovich

Russo

, et al. (2004) Posttraumatic distress, alcohol disorders, and recurrent trauma across level 1 trauma centers. Journal of Trauma Injury, Infection, and Critical Care 57: 360–366.