Abstract
The study examined data from the Arrestee Drug Abuse Monitoring–II (ADAM-II) program from 2007 until 2010 at 10 U.S. metropolitan jails to determine factors influencing the accuracy of self-reported drug use. The overall kappa coefficient for self-report data and urinalysis results of any type of drug use in the past 72 hr was .52, indicating a moderate level of agreement. Greater accuracy in self-reported drug use was found among arrestees who tested positive for methamphetamine and marijuana, although these results differed by age and race/ethnicity. African Americans provided less accurate self-reports of drug use than Caucasians, and younger arrestees less accurately self-reported all types of drug use except for marijuana. Persons with no prior arrests had higher accuracy of self-reported drug use than those with a history of frequent arrests, and prior involvement in substance abuse treatment was associated with more accurate self-reporting of drug use. Findings indicate moderate accuracy of self-reported drug use among new arrestees, with the accuracy influenced by demographic factors, arrest history, and substance abuse treatment history.
Jail and prison populations have risen dramatically over the past two decades, due in large part to arrests for drug-related offenses (Warren, 2008). Substance abuse problems are significantly more prevalent among offenders than in the general population, with rates of lifetime substance use disorders ranging from 70% to 74% (Boles & Miotto, 2003; Feucht & Gfroerer, 2011; Mumola & Karberg, 2006; Peters, Greenbaum, Edens, Carter, & Ortiz, 1998). Mental disorders are also more prevalent among offenders, with rates four to eight times higher than the general population (Steadman, Osher, Clark Robbins, Case, & Samuels, 2009). For instance, a 2006 survey from the Bureau of Justice Statistics found that 64% of inmates reported one or more symptoms of mental illness (Mumola & Karberg, 2006). Approximately 70% of inmates have a mental disorder (Fazel & Seewald, 2012; Feucht & Gfroerer, 2011; National GAINS Center for People With Co-Occurring Disorders in the Justice System, 2004; Sirdifield, Gojkovic, Brooker, & Ferriter, 2009; Steadman et al., 2009), and among this group, approximately three quarters have a co-occurring substance use disorder. Prevalence rates of mental disorders appear to be higher for female inmates than for male inmates (Steadman et al., 2009).
Failure to accurately identify substance use disorders among offenders can result in a range of negative consequences, particularly for those in need of treatment and with co-occurring disorders. For example, undiagnosed disorders can result in improper diagnoses, inappropriate interventions, premature dropout from treatment, and criminal recidivism. Inadequate screening and assessment of these disorders can delay triage to specialized offender treatment and supervision services, or lead to inappropriate placement in intensive services for persons who could benefit from less restrictive settings and less structured interventions. Thus, it is highly important to accurately compile substance use information, including the primary drug of choice, frequency and patterns of use, severity of substance-related symptoms, and diagnosis of specific substance use disorders.
Self-report is the primary method used to gather substance use information across diverse criminal justice settings (e.g., jails, prison, drug courts, and community treatment facilities). Collecting self-report data is fairly easy, more practical, and less intrusive than other forms of testing (e.g., laboratory tests) and allows for description of substance use patterns, such as frequency, intensity, and social context of use (Magura & Kang, 1996; Nelson et al., 1998; Rosay, Najaka, & Hertz, 2007). There are some limitations of self-reported substance use data and disadvantages of relying on this source of information. For example, negative consequences of reporting substance use, such as more severe sentences, treatment termination, or lengthier stays in jail or prison, may influence the type of information that an individual is willing to report. Furthermore, accuracy of offenders’ self-reported drug use may be affected by the desire to portray themselves positively, and/or perceived pressure to underreport socially undesirable behaviors (Johnson & Fendrich, 2005). Poor recall of events and behaviors may also affect accuracy of information, as is the case with all self-report measures.
Laboratory tests (e.g., urine, saliva, hair, blood) are also used to detect substance use. Laboratory tests using biological markers (e.g., urine, saliva, hair, blood) have been used for several decades to detect the presence of various substances. Although several methods are available, urine testing continues to be the most widely used (Martin, 2010) due to the low costs, relatively easy collection procedures, and high concentration of drug metabolites within urine specimens (Dupont & Selavka, 2008). Disadvantages of laboratory tests include the invasive nature of these procedures and the limited practicality of use in some settings. For instance, it is not always feasible to provide drug testing during intake to substance abuse treatment programs, nor is drug testing able to identify historical patterns of substance use, which is important in developing individualized treatment plans. Specific limitations of urine testing include difficulties in obtaining untampered specimens and susceptibility to false negatives (e.g., via sample dilution; Martin, 2010).
Hair testing is a promising alternative to urine testing, as it provides information regarding the level of use and trends of use over time (e.g., up to 90 days), cannot be easily falsified, and is less invasive to obtain (Martin, 2010; Wish, 1988). However, the delay in obtaining hair test results (24-72 hr) prevents immediate action in responding to drug use, and recent use is difficult to detect. Hair testing results are affected by race and by how the hair has been treated (e.g., chemical treatments, dye). Hair testing is also more expensive than urine testing, with costs ranging from US$40 to US$100 for a single test kit (Kaune & Callahan, 2006; Kidwell, Lee, & DeLauder, 2000; Martin, 2010; Wish, 1988).
Drug testing is commonly used with offenders to monitor and deter use while in prison/jail, during pretrial release, and while on probation or parole (Kleinpeter, Brocato, & Koob, 2010; Wish & Gropper, 1990). Drug testing is particularly useful for offenders because they are more likely than other groups to underreport substance use (Magura & Kang, 1996; Yacoubian, Wish, & Perez, 2001). Drug testing technologies have improved over time, with the ability to examine a broader scope of drugs and the use of more reliable procedures to collect and handle samples, which has enhanced the accuracy of results (Dupont & Selavka, 2008; Wish, 1988).
Due to the limitations of biological measures and the resulting reliance on self-report information, there is an important need to accurately identify concordance rates between self-report and laboratory tests of substance use. Unfortunately, there has not been much recent research compiled to describe these concordance rates, particularly among offenders. Much of the research on self-report information is dated by more than a decade, and several recent studies have been conducted with non-offender samples in community settings.
Due to the unique qualities and advantages related to self-reported substance use and drug testing information, research has focused on the concordance rates across these types of data. Among offender samples, overall concordance rates between self-reported substance use and urinalysis have been fairly high, with 80% to 85% agreement rates reported (Wood, 2008). Similar rates are seen in community samples, with mean concordance rates of 80% to 88% (Hamid, Deren, Beardsley, & Tortu, 1999; Schuler, Lechner, Carter, & Malcolm, 2009). Furthermore, concordance rates between self-report and urine screens were 80% to 84% among individuals with co-occurring mental health and substance use disorders (Jackson, Covell, Frisman, & Essock, 2005). Interestingly, Hamid et al. (1999) found that the rate of agreement was 58% when drug testing was performed after collection of self-report data, but increased to 93% when the drug test preceded self-report. More than 90% of discrepancies in self-report data involve underreporting of substance use.
Meta-analyses have shown varying rates of agreement across different samples (offender, treatment, community). Magura and Kang (1996) found a median kappa agreement of .42 across 24 studies, with only four studies having kappas in the range of .70 to .80. However, this meta-analysis includes fairly dated studies, diverse populations (e.g., 83% adults, 17% juveniles, 38% offenders, 63% non-offenders), and diverse drugs of choice (e.g., opiates, heroin, cocaine, marijuana), which may contribute to the varying rates of agreement. Similar variance in concordance rates has been observed in mental health settings, in which agreement between self-report and subsequent drug tests ranges from 30% to 80% (Large et al., 2012). However, a more recent meta-analysis of substance-involved populations (Rygaard Hjorthoj, Rygaard Hjorthoj, & Nordentoft, 2012) found significantly higher agreement rates ranging from 84% to 90%, with higher agreement found among samples with psychiatric comorbidity and lower agreement in randomized controlled trials.
Factors such as primary type of substance use, type of population sampled, age, race, gender, and nature of the assessment may influence the reliability of self-reported substance use, and are likely to account for some of the variability in the rates of agreement between self-reported drug use and drug test results. A recent meta-analysis involving predominantly non-offender samples indicates that average rates of agreement are 87% to 90% for marijuana, 79% to 84% for cocaine, and 94% for opiates (Rygaard Hjorthoj et al., 2012). Cocaine and marijuana are the most prevalent drugs of abuse across studies examining accuracy of self-reporting among offenders (Magura & Kang, 1996), and findings consistently show that self-reports of marijuana use among offenders are more accurate than those of cocaine (Magura & Kang, 1996). Rosay et al. (2007) suggest that offenders tend to overreport marijuana use and to underreport cocaine use.
In general, arrestees, youthful offenders, cocaine-involved adolescents, and post-treatment populations provide lower than average accuracy of self-reported substance use (Magura & Kang, 1996; Sloan, Bodapati, & Tucker, 2004). Drug Use Forecasting (DUF) data indicate that Caucasian offenders are more likely to provide valid self-reported substance use data in comparison with African American offenders, and that this latter group is particularly likely to underreport cocaine (Rosay et al., 2007). Another study indicates that females have significantly higher accuracy in self-reporting drug use than males (86% and 96%, respectively; Schuler et al., 2009). In addition, how a question is interpreted, ability to recall information related to substance use history, and willingness to report accurate information may affect the accuracy of self-reported drug use (Bradburn, 2000; Langenbucher & Merrill, 2001; Schwarz, Groves, & Schuman, 1998). For instance, misinterpretation, memory deficits, and lack of anonymity can lead to decreased accuracy of self-reporting. Question content and format of assessment administration can also influence accuracy. Some individuals have been found to provide more accurate reporting during self-administered assessment as opposed to interview-based assessment (Langenbucher & Merrill, 2001).
In summary, studies examining self-reported substance use indicate that these are generally valuable and accurate sources of information for both offender and non-offender samples. Self-report data provide unique information related to duration, frequency, intensity of substance use, route of administration, social context of use, and other patterns of use that are critically important in guiding triage and treatment planning decisions (Magura & Kang, 1996). Given the discrepancies between self-reported substance use and collateral sources of data, use of biological measures (e.g., drug testing) is needed to confirm the accuracy of self-reports whenever feasible. Several factors influence rates of concordance between self-reported drug use and drug test results, and should be considered in examining the accuracy of self-reported drug use. These include age, race/ethnicity, gender, type of drugs used, populations sampled, assessment techniques, and environmental contingencies.
The current study examined the accuracy of self-report data among arrestees, using one of the largest available criminal justice databases in the United States (Arrestee Drug Abuse Monitoring [ADAM]), consisting of new arrestees housed in metropolitan jails. The current study moved beyond the descriptive analyses provided in annual ADAM program reports to examine the accuracy of self-reported drug among new arrestees, including rates of agreement between self-reported drug use and drug test results, with particular attention to differences among arrestees who have a history of mental health and substance abuse treatment. The study also examined rates of agreement by age, race/ethnicity, frequency of prior felony arrest, offense type for the most recent index arrest, and type of drugs used as measured by Cohen’s kappa. Specifically, it was hypothesized that (a) younger arrestees will have lower rates of agreement between self-reported drug use and drug test results in comparison with older arrestees; (b) younger arrestees who use cocaine will have lower rates of agreement in comparison with those using other types of drugs, and to older arrestees; (c) rates of agreement will be higher among Caucasian arrestees than African Americans; (d) arrestees who use marijuana will have the highest rates of agreement, and arrestees using cocaine will have the lowest rates of agreement. In addition, the study explored the influence of (a) age, (b) race/ethnicity, and (c) history of mental health and/or substance abuse treatment on rates of agreement across the type of drugs used.
Method
Participants
The study involved a sample of 15,528 male participants in the ADAM program from 2007 to 2010. ADAM is the successor to the DUF program, which was implemented by the National Institute of Justice from 1987 to 1997. Both the DUF and ADAM programs compiled voluntary drug test and self-reported drug use information from new arrestees to metropolitan jails, and served as important vehicles to examine patterns of offender drug use throughout the United States, including new and emerging drugs of abuse. These databases are valuable to criminal justice professionals, researchers, and policymakers, and provide both local and national data on drug use prevalence and trends (National Institute of Justice, 1998).
The DUF program began in 12 metropolitan sites, but was extended to 24 sites from 1990 to 1997, representing diverse geographic areas of the United States. Several changes were made to the program from 1997 to 2000, and the number of sites was expanded from 24 to 35. The retitled ADAM program was initiated in 2000, but was discontinued in 2003 due to budget constraints. The program was reinstated in 2007, and is now operated by the Office of National Drug Control Policy.
The ADAM sampling procedures differed from the DUF program, in that it consisted of a probability-based design for all males who were arrested and booked at each site, thus providing a representative sample from each site, based on age, race/ethnicity, and offense type. ADAM interviews were conducted every day of the week and 24 hrs per day to eliminate bias related to the time of arrest and booking in jail. The original sampling procedure also included a diverse representation of detention facilities that varied in size, urban and rural setting, and speed of release from jail (U.S. Department of Justice, Office of Justice Programs, National Institute of Justice, 2000). However, due to budgetary considerations, only 10 of the original ADAM jail sites were active during 2007-2010, including those located in the following cities: Atlanta, Charlotte, Chicago, Denver, Indianapolis, Minneapolis, New York, Portland, Sacramento, and Washington, D.C.
The sample of 15,528 arrestees in the current study was drawn from a larger sample of 32,139 arrestees identified by the ADAM program from 2007 to 2010. The study sample consisted of arrestees who both completed the self-report interview and provided a urine sample. ADAM participants included 3,345 of 8,296 cases (40%) in 2007 for whom both an interview and urine sample were completed; 3,924 of 7,717 cases (51%) in 2008; 4,077 of 7,794 cases (52%) in 2009; and 4,182 of 8,332 cases (50%) in 2010.
Persons were excluded from the study for whom only facesheet information was available (13,511 arrestees). A facesheet is a form on which basic information about the arrestee was recorded, and that indicated the arrestee’s gender and the booking charge. In addition, the facesheet contained information describing whether a participant completed the interview and provided a urine sample, only completed the interview, or refused to participate altogether. If a selected arrestee was not available, the facesheet also stated the reason. Also excluded were those for whom facesheet and interview data were available, in the absence of drug test results (2,893 arrestees). Facesheet information from the ADAM database was limited to aggregate data describing index offenses, the total number of arrestees per year, and the number of arrestees per ADAM site. There were a number of reasons explaining why arrestees screened for the ADAM program were not interviewed or drug tested. In some cases, arrestees refused to take part in the study or were transferred to another location or jail facility. Others were released from jail immediately after arrest, were unavailable for an interview due to involvement in other jail activities (e.g., work assignments), or were not in stable condition due to mental or other health-related disorders. The most common factors preventing participation in ADAM interviews were release from jail (29%), transfer to another facility (24%), and refusal to participate (23%).
In the current study, we included only cases in which there was a completed interview and a urine sample. All partial interviews and interviews without a urine sample were removed from the analysis. Given the lack of information available for arrestees for whom only facesheet information was available, it was difficult to examine differences between arrestees included in our analysis and the 15,528 study participants. However, demographic and background information was available for the 2,893 arrestees who completed an interview but no drug test, allowing comparison with the study participants. Arrestees who provided only an interview were similar to the study sample in age, race/ethnicity, and type of index offense. However, the interview-only group significantly differed from the study sample in the number of prior arrests, χ2 = 52.16, p < .001, with the interview-only group more likely than study participants to report no prior arrests (19% and 14%, respectively), χ2 = 26.42, p < .001, and less likely to report more than 10 arrests (24% and 31%, respectively), χ2 = 31.77, p < .001. The interview-only group also differed from the study sample in the type of self-reported drug use in the 72 hr prior to the index offense, χ2 = 141.21, p < .001, with the study sample reporting higher rates of both marijuana use (33% and 26%, respectively), χ2 = 57.56, p < .001 and cocaine use (13% and 8%, respectively), χ2 = 60.28, p < .001.
Procedures
The current study examined all 3 years of ADAM program data (2007-2010) following the program’s reinstatement. Within a maximum of 24 hrs of booking in the jail, all arrestees were contacted by trained interviewers and recruited to participate in the voluntary ADAM program. Staff used structured interview protocols to query arrestees about their drug use habits and related behaviors, such as buying drugs. All responses were anonymous and by arrangements made with local prosecutors, ADAM information could not be used in legal proceedings. At the end of the interview, arrestees were asked to provide a urine sample for drug testing. Overall, 82% of arrestees who completed the interview also provided a urine sample. If a selected arrestee was not available for an interview, the interviewer completed only a facesheet with basic demographic characteristics and the index arrest charges.
Despite being commonly used in the criminal justice system and other settings, urine drug tests have several limitations. First, drugs can only be detected if they are present at a high enough level (i.e., threshold), to be picked up by the test. The threshold value is the lowest concentration at which a drug test would be classified as positive for drugs. The ADAM study established threshold levels for all substances for which drug tests were conducted. Second, some drugs remain in the body longer than other drugs (Carey, 2011; Moeller, Lee, & Kissack, 2008). For instance, marijuana typically remains for up to 7 days after periods of infrequent use and up to 30 days after chronic use (Couper & Logan, 2014; Moeller et al., 2008). Cocaine and most other substances are detectable for up to 72 hr, although the detection window varies with frequency and amount of use (Couper & Logan, 2014; Moeller et al., 2008). This has important implications for assessing the accuracy of self-reported drug use, as was discussed previously.
Measures
Several variables from the ADAM database were selected for analysis in the current study. These included background and demographic variables, information pertaining to drug test results, self-reported drug use information, history of mental health and substance abuse treatment, and criminal justice/offense history. A summary of key variables examined is described in the following section.
Self-Reported Drug Use and Drug Test Results
Although 11 drugs were assessed in the ADAM interview, analysis was provided only for drugs that were reported by more than 4% of the sample, including marijuana, cocaine, opiates, and methamphetamine. To promote validity, a conservative approach was used to assess concordance of self-reported drug use and drug test results. Concordance was evaluated during a self-report period of the previous 72 hr, even though several other longer periods of self-reported drug use (past week, month, year) were available in the ADAM database.
The 72-hr self-report period was used to align with the maximum period of detection by urinalysis for three of the four drugs examined in the study (cocaine, methamphetamine, opiates). Although the ability to detect drugs depends on the quantity, frequency, and duration of use (Kapur, 1993), the typical amount of time that these three drugs are detectable in the urine is from 48 to 72 hr (Couper & Logan, 2014; Moeller et al., 2008). Due to the fat solubility of tetrahydrocannabinol (THC), marijuana’s active ingredient, biological concentrations of this drug may also decline quickly (i.e., by 90% in the first hour after use), although the typical window of detection is 5 to 7 days for moderate use of marijuana (Moeller et al., 2008). Thus, use of a self-report period beyond 72 hr would likely diminish the accuracy of concordance between self-reported drug use and drug test results. In summary, the 72-hr self-report period is consistent with the period of time in which the drugs examined in the current study are detectable by urinalysis, and thus was determined to provide the most precise gauge of concordance.
Treatment History
The ADAM database does not provide an extensive set of variables reviewing severity of mental health or substance abuse problems. However, the database included self-reported treatment history in the past year for both mental health treatment and substance abuse treatment. This information provided some evidence of the presence of mental health problems and substance use problems. Substance abuse treatment included both inpatient and outpatient treatment episodes, whereas mental health treatment included only inpatient treatment. Treatment history is seen as providing a conservative estimate of ADAM participants who have mental or substance use problems.
Prior Arrests
The ADAM database included the frequency of prior arrests, and offense type associated with the arrest leading to the most recent incarceration (index arrest). Five categories of arrest frequency were created for purposes of analysis, based on the proportion of study participants who reported differing levels of prior arrests. The following categories were examined: (a) no prior arrests, (b) one prior arrest, (c) 2 to 5 prior arrests, (d) 6 to 10 prior arrests, and (e) more than 10 prior arrests.
Index Offense
Following common practice in criminal justice research (Harris, Smallbone, Dennison, & Knight, 2009; Miethe, Olson, & Mitchell, 2006), index offenses for study participants were categorized into four types: person (e.g., assault, murder, domestic violence, sex offense 1 ), property (e.g., arson, burglary, theft), drug (e.g., possession charges, driving under the influence), and other (e.g., trespassing, gambling, violation of probation, traffic violations).
Age
Based on the proportion of study participants who were of various different ages, the following age cohorts were examined: (a) 18 to 25, (b) 26 to 34, and (c) 35 to 89. Persons 18 and younger were not eligible for participation in the ADAM program. The age cohorts used in the study are consistent with those that are widely used in criminal justice research.
Race/Ethnicity
The following categories of race/ethnicity were identified in the ADAM database: African American, Asian, Caucasian, Hispanic or Latino, and Native American.
Data Analysis
The current study uses Cohen’s kappa to determine the accuracy of self-reported drug use and concordance rates between self-reported drug use and urine drug test results. The study also examines the impact of arrestee demographic characteristics, type of drug, type of index offense, prior drug treatment, and prior mental health treatment on the accuracy of self-reported drug use. It can be expected that some agreement between self-reported drug use and drug test results occurs by mistake or chance. Cohen’s kappa is considered a measure of “true” agreement because it evaluates only the agreement between self-reported drug use and drug test results beyond that which is expected by chance (Landis & Koch, 1977; Viera & Garrett, 2005). The kappa coefficient calculates the achieved “beyond chance of agreement” as a proportion of the possible “beyond chance agreement” between self-reported drug use and drug test data. The general equation is as follows: k = [(observed agreement) − (chance agreement)] / [(1) − (chance agreement)].
Kappa coefficients were computed to determine rates of agreement between self-reported drug use in the past 72 hr and drug test results. Kappa values between 0 and .20 indicate slight agreement, .21 and .40 fair agreement, .41 and .60 moderate agreement, .61 and .80 substantial agreement, and .81 and .99 very high agreement (Landis & Koch, 1977). Rates of agreement were evaluated across age, race/ethnicity, type of drugs used, involvement in mental health and substance abuse treatment, prior arrests, and index offense. The analyses were restricted to only those cases with a completed interview and drug test data (both positive and negative results). When comparing kappa values, a confidence interval of 95% was used to determine significant differences between kappa coefficients (Foody, 2009), with a p value of .05. Statistical significance was determined by examining whether kappa values fell within a particular confidence interval, based on the kappa values of the reference group. The positive concordance rate was also calculated, which represents individuals who had a positive drug test and who also self-reported use. Within each set of analyses, the positive concordance rate was calculated by dividing the number of individuals who self-reported drug use for a particular drug and tested positive for the drug by the total number of individuals who tested positive for the drug. For each variable, the reference group was based on previous literature in cases where evidence was available (e.g., race, age, drug type, treatment history). However, in cases where evidence was not available (e.g., prior offense, offense type), we chose the group with the largest sample size.
Results
Participant Characteristics
Descriptive statistics were used to examine demographic characteristics of the sample and can be found in Table 1. The age range of the sample was 18 to 89 years old, with a mean age of 34. The largest racial/ethnic group represented in the sample is African American (48%), followed by Caucasians (26%), and Hispanic or Latinos (21%). Most individuals had multiple prior arrests, including 31% who had more than 10 arrests and 29% who had two to five arrests, with only 14% of the sample reporting no prior arrests. Approximately 16% of the sample reported receiving previous substance abuse treatment and very few reported receiving previous mental health treatment, or receiving both substance abuse and mental health treatment. Prevalence rates of self-reported drug use and positive urinalysis results are also described in Table 1. Marijuana was the most commonly self-reported drug, and marijuana and cocaine were the two drugs for which participants were most likely to test positive.
Demographic Variables, Self-Reported Drug Use, and Drug Test Results
Accuracy of Self-Reported Drug Use
Accuracy of self-reported drug use was evaluated across variables of age, race/ethnicity, treatment history, prior arrests, and index offense, as described in Table 2. The overall kappa coefficient was .52 in assessing agreement between drug test results and self-reported use of any drug in the past 72 hr, indicating a moderate level of agreement. Rates of agreement were also similar across the 10 ADAM sites, with kappa values ranging from .46 to .59. The overall proportion of participants who were correctly classified in the study ranged from 81% to 97%, based on self-reported drug use within the 72 hr prior to arrest.
Rates of Agreement by Overall Drug Use and Background Characteristics
The sample includes those for which overall drug use information was reported.
The positive concordance rate represents those who had a positive drug test and also self-reported use.
This identifies the reference group for each category to determine significant differences between kappa values using the 95% CI.
p < .05.
Drug Type
The accuracy of self-reported drug use by drug type is described in Table 3, with kappas ranging from .53 to .70. Arrestees who used methamphetamine had the highest rates of agreement (.70), followed by those using marijuana (.61), opiates (.55), and cocaine (.53). Across all types of self-reported drug use, methamphetamine had the highest rates of agreement. The interactions between drug type and age, race/ethnicity, and treatment history are reported in subsequent sections.
Rates of Agreement by Drug Type and Demographic Characteristics
Note. Rates of agreement represent concordance rates between self-reported drug use in the past 72 hr and urinalysis results.
The positive concordance rate represents those who had a positive drug test and also self-reported use.
This identifies the reference group for each category to determine significant differences between kappa values using the 95% CI.
p < .05.
Age
As described in Table 2, there were no significant differences among the three age cohorts in the accuracy of overall self-reporting of drug use, with kappas ranging from .50 to .54. However, in examining concordance rates for specific types of drugs (see Table 3), in comparison with the other age cohorts (26-34, 35-89), the youngest age cohort (18-25) had lower agreement rates (i.e., kappas) for each of the four drugs examined in the study, and had significantly lower rates than the 35-to-89 age cohort among those who had used cocaine, opiates, and methamphetamine. The 26-to-34 age cohort also had significantly lower rates of agreement in comparison with the 35-to-89 cohort for cocaine, opiates, and methamphetamine, although these differences were not as pronounced as with the 18-to-25 cohort.
Race/Ethnicity
Self-report accuracy varied by race/ethnicity, as indicated in Tables 2 and 3. In general, African Americans had significantly lower rates of accuracy in reporting drug use (k = .47) than other racial/ethnic groups (kappas ranged from .52 to .58), with Caucasians having the highest rates of agreement. However, patterns of agreement by race/ethnicity varied tremendously according to the type of drug used. For example, the lowest rates of agreement for each drug were as follows: marijuana—Asians (.52), cocaine—Hispanic or Latinos (.41), opiates—Native Americans (.46), and methamphetamine—African Americans (.57). There was also a considerable variance in rates of agreement within particular ethnic/racial groups, across different types of drugs. Caucasians, for example, had rates of agreement of .63 for marijuana, .60 for cocaine, .55 for opiates, and .75 for methamphetamine. However, the degree of variance in the rates of agreement differed within particular ethnic/racial groups across types of drugs, with Caucasians experiencing wide variation (.55-.75, as previously noted) and African Americans experiencing much less variation (.52-.57).
Treatment History
As indicated in Table 2, arrestees who reported prior substance abuse treatment or both mental health and substance abuse treatment had significantly higher rates of agreement in comparison with those reporting no treatment. Conversely, those reporting a history of mental health treatment had a lower rate of agreement compared with those reporting no treatment. Table 4 describes the accuracy of self-reported drug use by drug type and treatment history. Across all drug types, participants who indicated a history of substance abuse treatment had significantly higher rates of agreement between self-report and drug test results in comparison with those with no prior substance abuse treatment. Arrestees who indicated a history of mental health treatment had significantly higher rates of agreement for methamphetamine, but lower rates of agreement for opiates in comparison with those with no prior mental health treatment. For arrestees reporting a history of both substance abuse and mental health treatment, rates of agreement were significantly higher for cocaine, but lower for methamphetamine in comparison with those who had not received both types of treatments.
Rates of Agreement by Drug Type and Treatment History
Note. The sample includes those who provided information on treatment history and also provided information on drug use. In the Between-Group Differences column, 1 = substance abuse treatment versus mental health treatment, 2 = substance abuse treatment versus substance abuse/mental health treatment, and 3 = mental health treatment versus substance abuse/mental health treatment.
The positive concordance rate represents those who had a positive drug test and also self-reported drug use.
Within-group differences are represented in the Cohen’s kappa column by an asterisk (*). The “no treatment” group is always the reference category for within-group differences.
Between-group differences represent those obtained across type of treatment categories. Only those “yes” categories are compared. The asterisk (*) is placed in line with the reference category.
For all drugs except methamphetamine, arrestees who reported a history of substance abuse treatment had significantly higher rates of agreement than those reporting a history of mental health treatment. Previous recipients of substance abuse treatment also had significantly higher rates of agreement for opiates and methamphetamine in comparison with those reporting previous treatment for both substance abuse and mental health problems. Arrestees reporting a history of mental health treatment had higher rates of agreement for methamphetamine use than those reporting a history of both substance abuse and mental health treatment.
Arrest History and Type of Index Offense
As indicated in Table 2, the accuracy of self-reported drug use varied by the frequency of prior arrest, with a general trend of lower rates of agreement for persons who had more prior arrests. For example, in comparison with persons with more than 10 prior arrests, those with no prior arrests had significantly higher rates of agreement. Rates of agreement were similar across index offenses, with kappas ranging from .50 to .53.
Discussion
Only a few studies have examined the accuracy of self-reported substance use information among offenders, which is commonly used in determining sentencing dispositions and placement in treatment, and in assessing risk for relapse and recidivism. Many of these studies are dated, involve small samples, and in general do not consider the likelihood of chance agreement related to dichotomous data or factors influencing rates of agreement. This study explored the rates of agreement between self-reported drug use and drug testing results among arrestees in metropolitan jails, and several factors that may influence the rates of agreement. These factors included age, race/ethnicity, frequency of prior arrests, type of index arrest, and prior history of mental health and/or drug treatment. Data were derived from the ADAM study, conducted by the Office of National Drug Control Policy between 2007 and 2010. The sample consisted of 15,528 arrestees from 10 jail sites across the United States who completed an interview and submitted a drug test.
Findings suggest moderate levels of agreement between self-reported drug use and drug test results, as measured by Cohen’s kappa, the percentage of the sample classified correctly according to self-reports of drug use, and the positive concordance rate, representing those who had a positive drug test and who self-reported drug use. Rates of agreement are consistent with those obtained from previous studies involving offender populations, though it is possible that rates of agreement may have been higher if drug testing occurred prior to the collection of self-report data (Hamid et al., 1999). As discussed in the following section, most of the study hypotheses were confirmed. However, the study indicates that accuracy of self-reported drug use in criminal justice settings is also influenced by key variables (age and race/ethnicity) across the types of drugs used.
There were significant differences obtained for many of the hypothesized predictive variables in the rates of concordance (i.e., agreement) between self-report and drug test results, and the overall classification accuracy of self-reported drug use. Across the study sample, the highest rates of self-report accuracy were obtained for methamphetamine, followed by marijuana. Arrestees who used methamphetamine and marijuana demonstrated substantial rates of agreement between self-report and drug test results, whereas arrestees who used cocaine or opiates demonstrated only moderate concordance. For marijuana, these findings may be explained in part by the longer window of detection, which could be expected to augment rates of agreement between self-report and drug testing. Findings support the hypothesis that arrestees who use marijuana have higher rates of self-report accuracy than other types of drug users, and that users of cocaine have lower rates of self-report accuracy. These findings may be useful in developing specialized assessment and drug testing protocols according to the types of drug use that are most prevalent among a particular offender population.
Contrary to the study hypothesis that lower rates of agreement would be found among younger offenders, there were no significant overall age-related differences in the accuracy of self-reporting when examining general drug use. However, the predicted age-related effects were detected when examining rates of agreement for specific types of drugs, including cocaine, opiates, and methamphetamine. The relationship between age and accuracy of self-reporting for all drugs except for marijuana is striking, and supports the salience of age as a predictor of self-report accuracy. It is unclear why this same age-related pattern did not prevail among marijuana-using arrestees. One possibility is that there is less overall stigma in reporting marijuana use, and that stigma may be a particularly salient factor among younger arrestees.
Study findings support the hypothesis that accuracy of self-reported drug use differs by race/ethnicity, and specifically that African Americans have lower accuracy in self-reporting than other racial/ethnic groups, when considering all types of drug use. However, accuracy of self-reporting among ethnicity/racial groups varied considerably according to the type of drug used. These group differences may yield insights to help craft specialized assessment and drug testing approaches within clinical settings. It is unclear why certain ethnic/racial groups had comparatively lower rates of agreement than other groups, across the different types of drugs examined. Further research is needed to examine these trends, to understand why these differences occur, and how to maximize accuracy in self-reported drug use among the various ethnic/racial groups (e.g., via modifications to assessment and drug testing strategies).
Findings indicate that persons with no prior arrests are more accurate in self-reporting drug use than those with more than 10 prior arrests. However, there did not appear to be an association between prior arrests and accuracy of self-reported drug use. When examining general patterns of accuracy in self-reported drug use among arrestees, no major differences were found across different types of index offenses. Thus, similar assessment and drug testing approaches may be warranted for persons who have varying levels of criminal justice involvement and different types of criminal charges/offenses.
A unique focus of the study was on the effects of prior mental health and substance abuse treatment history on accuracy of self-reported substance use. Of interest, arrestees with a history of substance abuse treatment or of both substance abuse and mental health treatment had significantly higher rates of agreement compared with those reporting no prior treatment. Conversely, arrestees with a history of mental health treatment had lower rates of agreement than those who had not received any prior treatment for one or both the disorders. However, the effects of treatment history on the accuracy of self-reported drug use varied widely across the types of drugs used. One potential explanation for these findings is that persons receiving previous substance abuse treatment may be more attuned to the need to candidly self-report substance use and may be more experienced in sharing this information with staff.
Study Limitations
Although the study involved a very large sample, it was conducted within 10 U.S. metropolitan jails. The 10 jail sites were selected based on geographic diversity, and these sites may not provide a representative sample of the entire country. For example, findings may not represent trends in non-urban areas. The study is also limited to male arrestees, and findings may not be generalizable to women. In fact, several studies indicate that women are more likely than men to accurately self-report substance use (Schuler et al., 2009; Shillington & Clapp, 2000; Weatherby et al., 1994).
Although widely used, urinalysis does not detect small amounts of drugs, or drugs that are rapidly excreted (e.g., heroin, cocaine). As a result, some drug use may not have been detected within the 72-hr time frame examined in the study. It should be noted that most psychosocial assessments conducted in criminal justice settings examine historical substance use that extends well beyond the previous 72 hr. Rates and patterns of self-report accuracy for substance use may be quite different for longer periods of time (e.g., past week, month, or year), although due to limitations in urinalysis technology, for most drugs it is not possible to precisely gauge the accuracy of self-report information beyond 72 hr. Use of drug test procedures (e.g., hair testing) that provide a longer window of detection would allow for a more comprehensive analysis of self-report accuracy among offenders.
Four of the most common drugs of abuse (cocaine, marijuana, methamphetamine, opiates) were examined in the study, although low response rates from both drug testing and interviews prevented evaluation of barbiturates, amphetamine, benzodiazepines, phencyclidine (PCP), and methadone. In addition, the ADAM protocol did not review use of alcohol, club drugs (e.g., ecstasy), or synthetic cannabinoids (e.g., K2, Spice). As one of the three most commonly abused substances among offenders, it would have been useful to provide testing for alcohol and to examine accuracy of self-reported alcohol use. However, as alcohol remains in the body for only several hours, it was impractical to provide this testing in the ADAM study.
The ADAM database provided only limited information about the presence or severity of mental health and/or substance abuse problems. The substance use information provided (self-reported substance use during the past 72 hr, week, month, and year, and drug test results) addressed only the presence of substance use and not the severity of use, or the extent of associated impairment. The only remaining “proxy” variables from the ADAM database included the self-reported history of involvement in mental health treatment and in substance abuse treatment. It would have been useful to have additional diagnostic indicators of mental and substance use disorders and markers of impairment to better ascertain the effects of problem severity on the accuracy of self-reported substance use.
A final concern is that no information was available to describe arrestees in the ADAM study for whom only facesheet information was available. Comparisons between study participants and those who provided only an interview yielded similar demographic profiles, but there were some differences in the frequency of prior arrests and the types of self-reported drug use between the two groups. It is unclear whether these differences may have predisposed the study participants to more accurate reporting of substance use.
Implications for Practice and Research
The study provides support for both the usefulness of self-reported substance use information among offenders and the need to gather collateral information, such as from drug testing. Although offender self-report information is more accurate than chance, it is not entirely reliable, as there is significant underreporting of drug use. Clearly, self-report information should not be used as the sole source of information in determining dispositions (e.g., at the time of pretrial release, sentencing), in developing treatment plans, or in assigning offenders to different levels of treatment or supervision within the criminal justice system. Strategies to increase the accuracy of substance use information (e.g., drug testing, use of collateral sources, enhancing the reliability of self-reporting) are vitally important in developing diagnoses and treatment plan goals, and in matching offenders to appropriate levels of services (Marlowe, 2012; Peters, Rojas, & Bartoi, 2014). These enhancements to screening and assessment practices are likely to result in more effective treatment outcomes in the criminal justice system, including reductions in recidivism (Hiller, Belenko, Welsh, Zajac, & Peters, 2011; Shaffer, 2011).
There is still much to be learned about methods to increase the accuracy of offender self-reported substance use and factors that affect accuracy of this information. For example, identification of predictive factors such as type of drug use, treatment history, and race/ethnicity may trigger the use of specialized approaches in gathering substance use information. This includes more frequent drug testing, compiling collateral information from family members or significant others, use of computerized interview protocols, more comprehensive assessment interviews, and repeated assessment over time. It may also prove useful to develop “risk profiles” to identify persons who are less likely to provide accurate self-report information and who may be candidates for specialized assessment approaches.
Contextual and environmental factors in the criminal justice system may influence the accuracy of self-reported drug use, including interviewer characteristics (e.g., using demographic variables to match interviewers with participants), perceived contingencies for accurate self-reporting of substance abuse, and the timing and format of interviews. For example, informing offenders that drug testing will be provided following an assessment interview, and providing assurances of anonymity and protection of information from use in prosecution may increase the likelihood of accurate reporting of substance use. It would be useful to determine how these factors influence the accuracy of self-report information and whether they have differential impact across various criminal justice settings, including pretrial jail booking, court-based settings (e.g., drug courts), probation and parole, and prison. Research is also needed to explore the relative effects of various predictors of self-report accuracy, and the potential cumulative effects of these predictors and of different approaches to mitigate their effects.
Conclusion
Findings from this study generally support hypotheses regarding the salience of age, race/ethnicity, and type of drug use as factors influencing the accuracy of offenders’ self-reported drug use, and are consistent with previous research involving offender and non-offender samples. However, the accuracy of self-report appears to be a product of the combination between these and other factors (e.g., criminal history, behavioral health treatment history); thus, there is a nuanced relationship between variables of interest and the accuracy of self-reported drug use. Additional research is needed to examine these interactive effects in more detail, and to describe the mediator and moderator effects of these variables in predicting the accuracy of offenders’ self-reported substance use. As noted previously, the study identifies several factors that should be considered by criminal justice and treatment professionals in evaluating self-report accuracy, including offenders’ age, race/ethnicity, type of recent drug use, behavioral health treatment history, and the absence of any prior arrests. Several of these factors (i.e., age, race/ethnicity) are also associated with underreporting of other stigmatizing self-reported information, such as HIV/AIDS status, as well as history of unprotected sex, arrests, and mental health treatment (Emlet, 2006; Farrell et al., 2006; Jeffe et al., 2000; Maxfield, Weiler, & Widom, 2000; Sheeran, Abraham, & Orbell, 1999).
In summary, there is a substantial but not overwhelming level of agreement between drug testing and self-reported drug use among arrestees in metropolitan jails. Accuracy of self-reported drug use is affected by age, race/ethnicity, drug type, number of prior arrests, and prior treatment history, and these factors should be considered in interpreting self-report information compiled in criminal justice settings. The study highlights the need to use diverse strategies in compiling information with substance-involved offenders, to enhance methods of obtaining self-report data, and for research to examine the efficacy of these new practices.
Footnotes
Acknowledgements
The authors would like to extend their thanks to the journal reviewers and editor for comments and recommendations provided on an earlier draft of this manuscript. The data utilized in this study were made available by the National Archive of Criminal Justice Data, operated by the Interuniversity Consortium for Political and Social Research (ICPSR) at the University of Michigan.