Abstract
Past studies have found that underreporting of marijuana use is particularly high. The present study extends previous research that examined the temporal validity of self-reported marijuana use among juvenile arrestees. Furthermore, the present study explores whether the passage of medical marijuana laws in some states have affected the validity of self-reported marijuana use among juvenile arrestees. Using existing juvenile offender interview and urinalysis data from the Arrestee Drug Abuse Monitoring Program (ADAM) for the years 1998 to 2002, we find that the validity of self-reported marijuana use was low, but quite stable, over time even after the threshold for a positive test was changed. However, study sites in states that had passed medical marijuana laws had significantly higher validity levels than states that had not, suggesting that the passage of medical marijuana laws may affect validity of self-reported marijuana use.
Introduction
For more than 30 years, marijuana has been the most widely used illicit drug among adolescents (Johnston, O’Malley, Bachman, & Schulenberg, 2009). Specifically, according to the 2007 Monitoring the Future, 19% of 12th graders reported having used marijuana during their lifetime (Johnston et al., 2009). Past research studies have also found that the extent of underreporting illicit drug use, including marijuana, is particularly high within the criminal justice system (Gray & Wish, 1999; Johnson, Baumler, Yacoubian, Peters, & Ross, 2001; Lu, Taylor, & Riley, 2001). For example, Johnson et al. (2001) conducted a longitudinal analysis of drug user reporting among Houston adult arrestees surveyed through the Arrestee Drug Abuse Monitoring Program (ADAM) between 1990 and 1997 and found that some participants were consistently unwilling to report use over time. Several factors have predicted underreporting including race (Johnson et al., 2001; Kim, Fendrich, & Wislar, 2000; Rosay, Najaka, & Hertz, 2007), gender (Kim et al., 2000), interviewer characteristics (Fendrich, Johnson, Shaligram, & Wislar, 1999; Lord, Friday, & Brennan, 2005), type of substance (Mieczkowski, 1990; Wish, Hoffman, & Nemes, 1997), time frame use (Kim et al., 2000), prior treatment experience (Gray & Wish, 1999), and type of offense (Gray & Wish, 1999). However, no studies have explored the validity of self-reported marijuana use over time and across locations with different marijuana policies.
The purpose of the present study is to further explore the validity of self-reported marijuana use over time and examine whether the passage of medical marijuana laws in some states has affected the validity of self-reported marijuana use among juvenile arrestees. We expand on Yacoubian’s (2001) study, which examined the validity of self-reported marijuana use among juvenile arrestees for the years 1991 to 1997. We use existing juvenile offender interview and urinalysis data from ADAM for the years 1998 to 2002 in five of the original 23 ADAM sites to conduct our analyses. Specifically, the study tests the following research questions:
Research Question 1: Does the temporal validity of self-reported 30-day marijuana use among juvenile arrestees, based on self-report and urinalysis data, change over time?
Research Question 2: Is the average rate of validity of self-reported 30-day marijuana use among juvenile arrestees similar in study sites located in states that have legalized medical marijuana compared to those that have not?
Findings from this research will help provide a further understanding of whether the validity of self-reported marijuana use among juveniles changes over time and across locations, and perhaps offer some explanations as to why such changes (if apparent) do occur.
Justification for the Current Study
Yacoubian’s (2001) study improved our understanding about the limitations of juvenile arrestee self-reported drug use. However, further investigation is merited for several reasons. First, Yacoubian’s (2001) study focused on data collected between 1991 and 1997. Juvenile arrestee data were collected through ADAM up until 2002; therefore, 5 more years of ADAM juvenile arrestee data are available. Importantly, the threshold for testing for marijuana in urine changed in 1996 in the ADAM study. Prior to 1996, the threshold level for marijuana was 100 ng/ml. From 1996 until 2002, the threshold level was lowered to 50 ng/ml. This change was based on recommendations made by the federal workplace testing program and new guidelines that were issued by the Substance Abuse and Mental Health Services Administration (National Institute of Justice, 1997). Following this change, it was expected that more marijuana users would be identified, particularly among those who were occasional to moderate users. It is therefore possible that the change in testing threshold could have an impact on validity estimates.
A final reason for supplementing Yacoubian’s (2001) study is because major changes in the law regarding medical marijuana began to occur in 1996. On November 5, 1996, California voters passed a ballot that legalized the use of medical marijuana. Between 1996 and 2002, several other states with ADAM sites (e.g., Alaska, 1998; Washington, 1998; Nevada, 2000; and Colorado, 2000) enacted similar medical marijuana laws (ProCon.org, 2010). With the implementation of these laws, it is important to examine whether these changes have had any effect on the validity of self-reported marijuana use. The examination is further justified because several researchers have studied marijuana use and found that states that have decriminalized marijuana have slightly higher use rates than other states (Chaloupka, Grossman, & Tauras, 1999; Model, 1993; Saffer & Chaloupka, 1999). Thus a change in drug policy will likely have some initial or delayed effect on societal use and acceptance of drugs, specifically marijuana.
Literature Review
Legalization of Marijuana
The debate over the legal status of marijuana has been ongoing for more than 40 years now. The debate began in the mid-1960s when enforcement of marijuana laws by police became lax and public perceptions of the risks of regular marijuana use declined (Joffe & Yancy, 2004). Proponents of legalization argued that marijuana is a safe drug and that criminal sanctions against possession and personal use represent excessively harsh and unnecessary punishment. They also noted that morbidity, mortality, and economic costs to society associated with tobacco and alcohol use in the United States far outweigh those associated with marijuana use (Joffe & Yancy, 2004; Katel, 2009; Marshall, 2005).
Opponents, on the other hand, argued that marijuana is not a benign substance, pointing to psychopharmacological information demonstrating that marijuana has many of the same features as other illicit drugs. They also called attention to the significant neuropharmacological, cognitive, behavioral, and somatic consequences that have been documented among long-term acute marijuana users. Furthermore, opponents asserted that legalization or decriminalization would likely generate a substantial increase in use, particularly among adolescents and youth, with probable increases in social, economic, and health costs similar to tobacco and alcohol (Joffe & Yancy, 2004; Marshall, 2005). So far, opponents have been successful in maintaining the illegal status of marijuana, but not without persistent challenges.
In 1977, attempting to take advantage of the promarijuana movement, President Jimmy Carter endorsed legislation to remove criminal penalties for possession of an ounce or less of marijuana. However, plagued by scandal, the proposal for marijuana decriminalization was ultimately terminated. This largely put an end to the legalization argument for the foreseeable future. President Regan paved the way for a decade of zero tolerance, including mandatory minimums for marijuana possession. These tough sentencing laws renewed the decriminalization movement. Opposition over the zero tolerance sentencing schemes caused many state legislatures to reduce penalties for marijuana possession, particularly for first time offenders. Into the 1990s, the medical-marijuana movement added further fuel to the fire (Joffe & Yancy, 2004; Katel, 2009).
Medical Marijuana Movement
Although the therapeutic qualities of cannabis were discovered nearly 5,000 years ago, it was not until 1842 that doctors in the United States began prescribing marijuana for various ailments. Marijuana as a medicine remained popular from 1840 to 1900. As the 20th century approached, the use of medical marijuana declined in part because of the variation in the potency of marijuana and the unpredictability of the effects of orally ingested marijuana. Furthermore, more chemically stable drugs had been developed for pain relief (Koch, 1999; Marshall, 2005).
During the 1930s, marijuana was still prescribed by some doctors. However, after the Marihuana Tax Act, doctors found the tax and registration system to be too complex, stressful, and time consuming and many eventually quit prescribing marijuana altogether. Throughout the 1960s, marijuana use dramatically increased and users once again began to recognize the possible medicinal benefits. However, the excitement over the medicinal benefits of marijuana was short-lived due to the Uniform Controlled Substance Act of 1970, which placed marijuana in the Schedule 1 category thereby making it unavailable for doctors to prescribe (Koch, 1999; Marshall, 2005). In 1972, the National Organization for the Reform of Marijuana Laws (NORML), the first Washington-based prolegalization lobbying group, filed a petition to have marijuana downgraded to a Schedule II drug, which would make it available by prescription. The result was years of legal maneuvering with ultimately no victory for advocates of medicalization (Katel, 2009; Marshall, 2005).
During the 1980s, scientists were interested in conducting research on the effectiveness of marijuana as a form of treatment. Six states (New Mexico, California, New York, Tennessee, Michigan, and Georgia) were granted federal permission to conduct research. Thousands of critically ill patients received marijuana for medicinal purposes from the federal government. The results from these studies mostly suggested that marijuana had some medicinal value including relief from pain and chemotherapy-induced nausea. Despite the positive results, the program was nevertheless discontinued in 1992 (Katel, 2009; Koch, 1999).
Despite federal prohibitions, in 1996, California became the first state to legalize marijuana for medical purposes. Voters passed a referendum authorizing anyone to grow or possess marijuana if a physician recommended it for medical reasons. By 2010, there were 13 other states (and the District of Columbia) that had enacted medical marijuana laws: Alaska, Oregon, Washington, Maine, Colorado, Hawaii, Nevada, Montana, Vermont, Rhode Island, New Mexico, Michigan, and New Jersey (Cooper, 2000; Katel, 2009; Koch, 1999; Marshall, 2005; Pro-Con.org, 2010). Furthermore, in October 2009, the Department of Justice announced that it would no longer prosecute individuals who use or distribute marijuana for medical purposes. This announcement represented a small victory for decriminalization and legalization proponents (Stout & Moore, 2009).
Trends in Marijuana Use and Drug Arrests in America
As marijuana policies changed over time, trends in use and marijuana-related arrests have also fluctuated. According to the Monitoring the Future (MTF) project, reported lifetime marijuana use among 12th graders peaked in 1979 at 60.4%. Since then, reported lifetime use among 12th graders gradually declined until 1992, but from 1993 to 1997, use rates doubled. Reported use then continued to decline from 1997 to 2007, reaching 19% in 2007 (Johnston et al., 2009). These trends tend to be consistent across prevalence rates (annual, 30-day, daily) and age groups (8th and 10th graders). Research also suggests that the MTF trends are comparable for older populations as well. For example, the National Household Survey on Drug Abuse found that past month reported use for those aged 18 to 25 peaked in 1979 at 35.6%; for those aged 26 to 34 reported use also peaked in 1979 at 19.7%.
Trends in juvenile drug abuse arrests tend to follow a similar pattern as juvenile self-reported drug use. According to data collected and compiled from the FBI’s Uniform Crime Report, in 1983 juvenile drug arrests reached an all-time low with less than 300 arrests per 100,000 juveniles between the ages of 10 and 17. Between 1983 and 1990, drug arrests among juveniles fluctuated from more than 400 arrests per 100,000 juveniles in 1989 to a low of 300 juvenile drug arrests per 100,000 in 1990. Juvenile drug arrests increased between 1990 and 1996 before peaking in 1997 with approximately 725 arrests per 100,000 juveniles (Puzzanchera, 2009). Since 1997, the juvenile arrest rate for drug abuse violations has trended downward similar to self-reported juvenile drug use (Johnston et al., 2009; Puzzanchera, 2009). One could reasonably conclude that juvenile drug use rates affect juvenile drug arrests.
Societal Approval of Marijuana Use
Several studies have suggested that trends in reported drug use also coincide with society’s approval of the use of certain drugs (Johnson et al., 2001; Kim et al., 2000; Wish et al., 1997). For example, Johnson et al. (2001) found that Houston arrestees were more willing to acknowledge heroin and marijuana use than cocaine use. One explanation for this finding was that marijuana and heroin had become so immersed in the local society that disapproval of use had dropped.
The MTF also provides an opportunity to examine trends in use and disapproval. An increase in personal disapproval of regular marijuana use among 8th, 10th, and 12th graders coincided with a decrease in last 12-month reported marijuana use and vice versa (Johnston et al., 2009). Based on this information, it appears as if personal disapproval of marijuana use and self-reported marijuana use are inversely associated. These results are typically based on self-reported drug use data.
Validity of Self-Reported Drug Use Data
Researchers most often rely on self-reported measures of drug use because of the advantages over indirect measures such as treatment admissions, arrests, emergency room visits, and drug seizures (Harrison, 1997; Johnston et al., 2009; Substance Abuse and Mental Health Services Administration [SAMHSA], 2008). One such study is the National Survey on Drug Use and Health, the major source of information on the patterns and prevalence of illegal drug use and abuse within the noninstitutionalized population in the United States. The MTF study also collects self-reported drug use data annually from a nationally representative sample of public and private school students in 8th, 10th, and 12th grades (Johnston et al., 2009).
Some scholars have suggested that the validity of self-report studies is threatened because drug use estimates are dependent on the accuracy of self-reported drug use (Golub, Liberty, & Johnson, 2005; Harrell, 1985; Harrell, 1997; Rosay et al., 2007). There are several reasons why self-reported drug use data may be inaccurate. First, the accuracy of the data may be affected by respondent’s memory or comprehension (Harrison, Martin, Enev, & Harrington, 2007; Johnson et al., 2001). Harrison et al. (2007) examined this question by asking respondents if they thought they, and others, would have difficulty understanding or remembering drug-related information. Most respondents stated that they would have “no difficulty” understanding (73%) and remembering (58%) drug-related questions. However, respondents were not as confident in the ability of others to understand and remember drug-related information. Specifically, only 42% of respondents believed that others would have “no difficulty” in understanding drug-related questions while only 25% stated that others would have “no difficulty” in remembering drug-related questions. In an attempt to avoid the possibility of individuals misunderstanding questions, some studies, including MTF, have hired trained staff members to administer questionnaires to students so that they can provide clarity if respondents do not understand a question (Johnston et al., 2009).
Two other potential threats to the validity of self-reported drug use are fear of embarrassment and threat of potential sanctions (Guardiola, 1985; Harrell, 1997; Rosay et al., 2007; SAMHSA, 2009). Researchers try to counter these threats by promising confidentiality during data collection. For example, administrators of the National Survey on Drug Use and Health try to reduce inaccurate self-reports by conducting face-to-face interviews with respondents in their home, so that the respondents can be assured that their information will remain confidential and their identity will not be revealed (SAMHSA, 2009).
Yet another factor that affects the validity of self-reported drug use is social desirability (Heeb & Gmel, 2001; Lord et al., 2005). An individual may tailor his or her response based on the perceived acceptability of the use of particular illicit drugs. Again, Johnson et al. (2001) found that Houston arrestees were more willing to admit heroin and marijuana use than crack or powder cocaine use, arguing that this may be based on the notion that marijuana carries less of a social stigma (in Houston) than crack or powder cocaine and heroin has become more ingrained in the local arrestee population.
Lord et al. (2005) examined the applicability of the Social Distance Theory and the Social Attribution Theory in predicting arrestee participation in a drug use survey. Social Distance Theory suggests that respondents will be more willing to participate when they share common characteristics with the interviewer. Social Attribution Theory suggests that respondents choose to participate based on the respondent’s perception about the norms and expectations of an interviewer. To examine the applicability of these theories, data from the Charlotte, North Carolina Arrestee Drug Abuse Monitoring (ADAM) site were used. The study found that the Social Attribution Theory was statistically supported with significant main effects of race, gender, and age on a respondents’ willingness to consent to a drug use survey. Although not statistically supported, Lord et al. (2005) argued that the results also supported the applicability of the Social Distance. For instance, while African Americans were more likely to gain arrestee consent, two thirds of the arrestees were African Americans thus lending support for the Social Distance Theory.
Because of these various validity threats, studies that rely solely on self-reported drug use data have consistently been criticized (Guardiola, 1985). In response to these concerns, some researchers have been engaged in validation studies, with urinalysis being the most often used technique for validating self-reported drug use.
Urinalysis Testing
Drug testing programs have been widely used within the criminal justice system over the past few decades. The purpose of the programs vary, but the most common reasons include screening for recent drug use, identifying chronic users, monitoring and deterring use, and tracking national and local trends. Validating self-reported drug use is another reason for drug testing. The most common technique used for drug use screening within the criminal justice system is urinalysis because of the many advantages it has over other techniques like blood tests and hair analyses (Wish & Gropper, 1990).
The advantages and disadvantages of urine tests for testing criminal justice populations for drug use vary according to the particular type of test used. There are four categories of urine tests available: screening by thin-layer chromatography, screening by immunoassays, confirmation by gas chromatography, and confirmation by gas chromatography/mass spectrometry (GC/MS). One’s desired combination of sensitivity, specificity, and cross-reactivity helps determine which of these tests are most appropriate.
ADAM utilized the Enzyme Multiplied Immune Test (EMIT) to conduct urinalysis tests for arrestees. One advantage of the EMIT test is that it is still relatively inexpensive. A second advantage of the EMIT test is that it is highly sensitive; thus a small concentration of a drug can be consistently detected in a urine sample. Furthermore, minimal training is required to facilitate and evaluate the test results. One last major advantage is that facilities may be able to establish on-site testing programs because of the simplicity of the EMIT process.
Immunoassay tests also have some major disadvantages. First, immunoassay tests have reduced specificity. Thus the test has a limited ability to differentiate between crack cocaine and powder cocaine. Second, there is the possibility of cross-reactivity. In other words, a substance other than the drug in question could produce a positive effect and thereby generate a false positive result. Thus it may be necessary to use a different test to verify the results. Furthermore, while cross-reactivity can occur and create a false positive outcome, adulteration of an individual’s specimen can also lead to false negative results (Golub et al., 2005; Wish & Gropper, 1990).
All urinalysis tests have one common disadvantage—a limited time frame for drug detection. Many of the major drugs of abuse, such as cocaine, heroin, and amphetamines, are quickly metabolized and excreted. The time frame for detection for most drugs, with the exception of marijuana, is 2 to 3 days although this is dependent on purity of the drug taken, the rate of individual use, the test cutoff, and retention of the drug in the body (Mieczkowski, 2002; National Institute of Justice, 2003; Wish & Gropper, 1990) This general disadvantage has led some to speculate on whether a different type of analysis should be used. For now, urinalysis continues to be the primary technique used to validate self-reported drug use (Mieczkowski, 2002; Wish et al., 1997).
The Validity of Self-Reported Drug Use
Prior to the 1980s, self-report data were generally perceived to be reasonably valid. Studies that examined validity unanimously reported validity as “good” or “very good.” However, in 1986, Eric Wish challenged the validity of self-report drug use. More recent validity studies, particularly within treatment facilities and the criminal justice system, have verified these concerns by concluding that underreporting is a common occurrence (Fendrich et al., 1999; Hser, Maglione, & Boyle, 1999; Johnson et al., 2001; Lu et al., 2001; Wish et al., 1997). Table 1 presents a compilation of studies that have explored characteristics and variables that are related to underreporting. Some of these include gender, race, type of drug, offense seriousness, education, family structure, gang membership, interviewer characteristics, marital status, amount of drug use, intensity, frequency of use, location, employment status and age. A number of studies have examined the validity of self-reported drug use among juveniles, including juvenile offenders.
ADAM Studies on the Validity of Self-Report Marijuana Use Among Juvenile and Adult Arrestees
A couple of these studies are particularly relevant here. First, Mensch and Kandel (1988) examined the extent of underreporting among youth with data from the National Longitudinal Survey of Youth (NLSY), a longitudinal survey of young adults conducted since 1979. To validate the results from the NLSY, the self-report data were compared with MTF data and the General Household Survey (GHS) data. Mensch and Kandel (1988) found that underreporting of lifetime prevalence use in the NLSY study was considerable for illicit drugs other than marijuana.
Meanwhile, Feucht, Stephens, and Walker (1994) examined the validity of self-reported drug use data from Cleveland juvenile arrestees surveyed through the Drug Use Forecasting program (which was an earlier iteration of the ADAM program). Over a 2-month period, 169 male juvenile arrestees were asked to participate in the study and provide hair and urine samples. Eighty-eight juveniles provided both a hair and urine sample. Feucht et al. determined that self-reports of drug use appear to severely underestimate the prevalence of cocaine in juvenile arrestee populations. Only 6 of 88 juvenile arrestees admitted to ever using cocaine, while hair assay tests revealed that 50 had used cocaine. Meanwhile, urinalysis tests only identified 7 cocaine users, which is not particularly surprising due to the limited time frame in which cocaine can be detected in urine.
Longitudinal Validity of Self-Reported Marijuana Use among Juvenile Arrestees
Only a small number of studies have examined longitudinal trends in the validity of self-reported drug use, particularly among arrestees. Those that have examined validity trends have mainly focused on adult populations (Golub et al., 2005; Johnson et al., 2001). For example, Johnson et al. (2001) conducted a longitudinal analysis of drug user reporting among Houston adult arrestees surveyed through ADAM between 1990 and 1997. Utilizing kappa statistics, 1 Johnson et al. (2001) found that some participants were consistently unwilling to report use over time. Despite the low agreement, it was found that the proportion of drug-positive adult arrestees who self-reported drug use remained relatively stable over time.
Yacoubian (2001) explored the temporal validity of self-reported drug use among juvenile arrestees using the ADAM data by comparing urinalysis results and self-reported 30-day marijuana use from 33,313 juvenile arrestees who were interviewed between 1991 and 1997 in 12 of the original 23 sites. 2 Using Cohen’s kappa to compare marijuana urinalysis results to self-reported 30-day marijuana use, Yacoubian found only slight agreement between the two raters (urinalysis and self-report); however, the relationship between the two measures of drug use was fairly consistent intrajurisdictionally over the 7-year time period.
Method
History of Arrestee Drug Abuse Monitoring Program
The present study uses data from five different sites (Birmingham, AL, Cleveland, OH, Phoenix, AZ, San Antonio, TX, and San Diego, CA) that collected juvenile information as part of the ADAM program during the years 1998-2002. The ADAM Program—formerly the Drug Use Forecasting (DUF) Program—was established in 1987 by the National Institute of Justice to conduct research on drug use among urban adult and juvenile arrestees in the jail setting (National Institute of Justice, 1999; National Institute of Justice, 2000). The primary goals of the program were to track changes in drug use patterns, determine what drugs were being used in various jurisdictions, alert local officials to trends in use and availability of new drugs, supply data to help study the drug-crime connection, and serve as a research base for policymakers (Johnson et al., 2001; National Institute of Justice, 2000; Yacoubian, 2000). ADAM operations across the country were postponed in 2004 because of federal spending constraints. Before the cessation of ADAM, however, data were collected on hundreds of thousands of recently booked arrestees in 35 sites across the country. The information continues to be used to help communities make educated decisions on the distribution of funds to law and enforcement and drug treatment programs (National Institute of Justice, 2003; Webb, Katz, & Decker, 2006).
Data Collection Process
As mentioned above, the data used in this study are from the years 1998-2002. 3 During these years, for no more than 2 weeks in a single facility every calendar quarter, data were collected on convenience samples of juvenile arrestees. Juveniles who had been arrested within the past 48 hours were approached and asked to participate in the study. Potential participants were read an informed consent form, which stated that no identifying information would be collected and that a urine specimen would be requested at the end of the interview. All juveniles were free to refuse to participate. However, in most sites, more than 80% of individuals approached agreed to be interviewed, and of those participants, more than 80% also agreed to give a urine specimen (National Institute of Justice [NIJ], 1999, 2000, 2003). If the participant agreed to be interviewed, a trained, nonuniformed interviewer would ask the individual a series of questions regarding demographics, past and present drug use, perception of one’s drug problem, and involvement in past and present drug treatment.
Once the interview was completed, participants were asked to provide a urine sample. Drug testing is of particular importance to the ADAM program because it helps analysts to obtain a measure of recent drug use and to assess the validity of self-report data (Johnson et al., 2001; NIJ, 1999). The ADAM program uses the Enzyme Multiplied Immunoassay Testing (EMIT) system to screen for the presence of 10 different drugs (amphetamines, barbiturates, benzodiazepines, cocaine, marijuana, methadone, methaqualone, opiates, PCP, and propoxyphene) in urine. A laboratory conducts all testing and the specificity and accuracy for most drugs is higher than 95% (NIJ, 1999). If a particular drug is present in the urine sample at a level above or equal to a specified cutoff point then the result is considered to be positive for that particular drug. Again, from 1996 to 2002, the cutoff level for marijuana was 50 ng/ml. Prior to 1996, a higher cutoff level (100 ng/ml) was used (NIJ, 1997).
One methodological limitation should be noted in regard to the ADAM data collection process between 1998 and 2002. In all years, there was a relatively high level of missing data for self-reported marijuana use and 2 years, 2001 and 2002, had substantial missing data for urine specimens. Across all 5 years, missing data constituted 32.7% of the sample for self-reported marijuana use and 8.4% of the sample for urinalysis results. Smaller percentages of the urinalysis data were missing because individuals had either refused to participate, were unable to use the bathroom, or did not provide a sufficient quantity to be tested. However, a vast majority of the missing data were simply not obtained during data collection. The proportion of missing data for marijuana urinalysis tests was similar for other drugs. Conversely, percentages of missing data for self-reported use of other drugs varied. Self-reported marijuana use actually had the least amount of missing data, relative to other drugs, in terms of past 30-day use. We discuss the implications of this missing data later.
Results
Data analysis was accomplished in two phases. First, descriptive statistics were calculated to summarize the study sample. Second, kappa statistics were computed to examine the relationship between marijuana urinalysis and self-reported 30-day marijuana use at each of the five sites from 1998 to 2002. Cohen’s kappa is used to measure the agreement between the evaluation of two raters (in this case, urinalysis and self-report) when both are rating the same object (i.e., recent drug use). This measure of agreement is considered to be appropriate when the time periods covered by the self-report and the criterion measure are similar and when both variables have the same number of categories and use the same category values. Perfect agreement is indicated by a value of one; a value of zero indicates that agreement is no better than chance (Johnson et al., 2001; Magura & Kang, 1996; Yacoubian, 2001). Furthermore, to determine whether there were any changes in the level of agreement between each of the years, as well as across the 5-year time period, z tests were performed. 4 A 30-day self-report measure was used, even though the ADAM program also collects data on 3-day self-report use, because marijuana can be detected in urine for up to 30 days depending on the frequency of use and the potency of the marijuana (NIJ, 1999).
Description of the Study Sample
A total of 7,487 juvenile arrestees were originally surveyed in the five selected sites (Birmingham, AL, Cleveland, OH, Phoenix, AZ, San Antonio, TX, and San Diego, CA) between 1998 and 2002. However, only 7,484 were used in these analyses because three arrestees who were interviewed were older than 18 thus disqualifying them as juveniles. The majority of the sample was male (83%). Hispanics (39%) had the largest racial representation followed by African Americans (31%) and Whites (26%). Almost half of the juvenile arrestees were aged 13 to 15 (49%) followed by those aged 16 to 18 (49%) and 9 to 12 (4%). Felonies (51%) made up the majority of the offenses for which the participants were arrested. Furthermore, 75% of the juvenile arrestees were still in school. As shown in Table 2, the demographic characteristics within each individual year are similar to those for the overall sample across all 5 years.
Sample Demographic Characteristics by Year (N = 7,484)
Temporal Comparison of Urinalysis Results to Self-Reported 30-day Marijuana Use by Site
Across the 5-year time period within each site, some variations in level of agreement were found. 5 As shown in Table 3, on average, Birmingham had the strongest level of agreement (.42), whereas San Antonio had the lowest level of agreement (.21). Despite the rather low strength of agreement, the kappa statistics remained relatively stable across the 5-year period for each site with the exception of Birmingham. The range of agreement extended from a low of .06 in San Diego to a high of .40 in Birmingham. Only one site, San Antonio, experienced any significant changes over the 5-year time period at the standard level of p < .05. After a slight increase from 1998 to 1999, San Antonio experienced a significant decrease (p < .01) in level of agreement between 1999 and 2002. Birmingham, Cleveland, and Phoenix also experienced changes, but these changes were not significant (p < .10). Furthermore, when examining yearly changes in level of agreement San Antonio was the only site to experience a significant change. From 2001 to 2002, San Antonio experienced a significant decrease (p < .05) in level of agreement.
Kappa Statistics for Marijuana Urinalysis (50 ng) Results and Self-Reported 30-day Marijuana Use by Site (standard errors in parentheses)
When examining the entire sample over the 5-year time period, the kappa statistics were all relatively low, but remained fairly consistent with a range of .08. There was no significant change in agreement from year to year within sites or across the 5-year time period when considering the overall sample. Thus, based on these results, it does not appear that validity of self-reported 30-day marijuana use among juvenile arrestees changed over time across these five locations. Furthermore, the validity is rather low overall among the entire sample.
Temporal Comparison of Urinalysis Results to Self-Reported 30-day Marijuana Use by Sites with Medical Marijuana Laws
As shown in Table 4, the strength of agreement remained low, but consistent, for the two sites that were located in states with medical marijuana laws (Phoenix 6 and San Diego) and the three sites without such laws (Birmingham, Cleveland, and San Antonio). The average kappa was significantly higher in sites located in states that passed medical marijuana laws compared to those sites located in states that had not passed such a law. Furthermore, over the 5-year time period, the two sites with medical marijuana laws consistently had significantly higher agreement than the three sites without such laws, with the exception of 2001 when the level of agreement was essentially the same. Therefore, it appears that the passage of medical marijuana laws in some states may affect the validity of self-reported marijuana use among juvenile arrestees.
Kappa Statistics for Marijuana Urinalysis (50 ng) Results and Self-Reported 30-day Marijuana Use by Medical Marijuana Law Sites
p < .001.
Results from the present study are comparable to those of Yacoubian’s (2001) study. While the range of agreement from Yacoubian’s (2001) study was lower, the levels of agreement remained fairly low and consistent. It is important to recall that the threshold for a positive marijuana test was changed in 1996 from 100 ng/ml to 50 ng/ml. Because of this change in threshold, it is possible that the kappa statistics would be affected. However, no significant difference was found between the mean kappa from 1991 to 1996 and the mean kappa from 1997 to 2002.
Finally, when Yacoubian’s kappa statistics were compiled with those in the present study (see Table 5), over the 12-year time period (1991-2002), the levels of agreement remained low, but stable, with the exception of the Birmingham site. The range of agreement extended from a high of .41 in Birmingham to a low of .15 in San Diego. Over the 12-year time period, on average, San Antonio had the lowest level of agreement (.21) while the other sites had roughly the same level of agreement (.34 to .39).
Twelve-Year Kappa Statistics for Marijuana Urinalysis Results and Self-Reported 30-Day Marijuana Use by Site
As a final step, medical marijuana sites were compared with nonmedical marijuana sites over the entire 12-year time period (see Table 6). Based on a difference of means test, there were no significant differences in level of agreement during the years 1991-1997 among those states that had passed medical marijuana laws compared with those that had not. However, from 1998 to 2002, with the exception of 2001, sites located in states with medical marijuana laws had significantly higher levels of agreement then sites in states without such laws. Again, it appears that the passage of medical marijuana laws may affect the validity of self-reported marijuana use among juvenile arrestees.
Twelve-Year Kappa Statistics for Marijuana Urinalysis (50 ng) Results and Self-Reported 30-Day Marijuana Use, by Medical Marijuana Law Sites (Standard Errors in Parentheses)
p < .001.
Discussion
In the current study, self-reported marijuana use was validated using urinalysis test results within a sample of 7,484 juvenile arrestees who were interviewed between 1998 and 2002 in five locations. Based on the premise that a kappa statistic below .7 indicates poor agreement (Johnson et al., 2001; Magura & Kang, 1996), the temporal analyses illustrate a consistent unwillingness of juvenile arrestees to accurately self-report marijuana use over the 5-year time period within the full sample. Site-specific temporal analyses also revealed a consistent unwillingness to accurately report marijuana use within all five sites examined here.
The results were generally similar when comparing two sites located in states that enacted medical marijuana laws (Phoenix and San Diego) with three sites located in states that did not adopt medical marijuana laws (Birmingham, Cleveland, and San Antonio). Although juveniles interviewed in sites with medical marijuana laws and those without generally reported low agreement, the medical marijuana law sites had significantly higher levels of agreement (as measured via kappa statistics) than those without such laws.
General Policy Implications
It is rather alarming that two measures of drug use (urinalysis and self-report) used by ADAM have such a low level of agreement among juveniles over a 12-year time period. Golub et al. (2005) found similar inconsistencies between self-reported drug use and urinalysis results among adult arrestees. These findings call into question the reliance on self-report (and urinalysis data) among arrestee populations to formulate drug policy decisions. These findings also raise concerns about the validity of other self-report studies that examine trends in drug use among juveniles and adults. In the absence of biological verification, it remains uncertain whether changes in drug use trends are because of actual changes in use levels or to changes in respondents’ willingness (or lack thereof) to report use. It may also be the case that validity of self- reported drug use would be higher within other nonincarcerated populations (e.g., school samples). This seems to be a particularly important question that warrants further review given the costs associated with annual data collection processes.
It may be impractical, however, to conduct biological tests due to costs and other concerns (privacy, for example). Based on the present findings, as well as Yacoubian’s (2001), collecting urinalysis specimens each year is unnecessary given that the level of agreement between urinalysis and self-reported use, while low, does remain consistent over time. A more cost effective solution would be to simply collect fewer urine specimens or collect specimens perhaps every 2 or 3 years.
Furthermore, it is important to note that this study examined data from the medical marijuana sites fairly recently after new laws had been enacted. At most, there were 6 years of data available to determine whether validity rates had changed since these laws were enacted. Based on this relatively short time frame, it would be useful to replicate this study using a longer time frame to better determine whether the introduction of medical marijuana laws resulted in the higher level of agreement or whether other factors can account for the change. In addition, as of 2010, 14 states had passed medical marijuana laws. Therefore, this study could be improved by including a larger number of sites located in states with medical marijuana laws although these sites would have to focus on adult arrestees if the ADAM data were used.
Implications for ADAM/ADAM II
Again, ADAM stopped collecting data on juvenile arrestees in 2003 and had to fully postpone its operations across the country in 2004 because of federal spending constraints (NIJ, 2003). However, in 2007, ADAM was revitalized under the Office of National Drug Control Policy under the name of ADAM II. ADAM II continues to utilize the same methodology as the original ADAM, but data collection is limited to Atlanta, GA, Charlotte, NC, Denver, CO, Indianapolis, IN, Minneapolis, MN, New York, NY, Portland, OR, Sacramento, CA, and Washington, D.C., and is restricted to male arrestees only (Office of National Drug Control Policy [ONDCP], 2009). Based on the findings from this study, two recommendations might be useful for ADAM II. First, it would be beneficial to begin collecting data specifically on juveniles again. ADAM has a major advantage over traditional self report surveys due to the fact that it collects self-report and urinalysis data, which allows for validating self-reported use and more accurately monitoring trends in arrestee populations (ONDCP, 2009). Second, several ADAM II sites are located in states where medical marijuana laws have been passed (e.g., Denver, Portland, and Sacramento—ProCon.org, 2010). A longitudinal study would better determine whether the changing marijuana laws have truly affected self reporting processes. Since ADAM II is one of the only studies that uses a biological test to validate self-reported use, more states with medical marijuana laws need to be examined through ADAM II.
The Impact of Medical Marijuana Laws on Marijuana Use and Self Reporting
Past research suggests that the use of substances rises and falls, to some degree, with societal acceptance of those substances, particularly since societal acceptance, to some degree, influences laws and law enforcement practices. The free-spirited decade of the 1960s included extensive substance use and greater societal acceptance and tolerance. In the 1980s societal disapproval of drug use increased with the “just say no” message. However, in the past four decades, and particularly since 1990, societal approval and acceptance of both the medical use and legalization of marijuana has steadily increased (Pew Research Center, 2010). In fact, Californians will have an opportunity to vote on the legalization of Marijuana in November 2010 (California Proposition 19, 2010). As marijuana laws move from prohibition to decriminalization (many law enforcement agencies give citations, in lieu of arrests, for simple possession) to restricted legalization (e.g., medical) to legalization, the illicit stigma associated with marijuana will erode and users, including juveniles, will become more comfortable with acknowledging use. As a result, the use and subsequent validity of self reported marijuana use will likely rise in the future. This study provides an early indication that that may be already occurring in two sites that are located in states that adopted medical marijuana laws.
Limitations of the Current Study
The present study was affected by certain limitations that need to be noted. First, the study was limited to juvenile arrestees in five sites across the United States who were selected based on a convenience sampling procedure. It is highly unlikely that each site used the same standards for processing juvenile arrestees, and therefore, the samples are probably not representative of all juvenile arrestees within each city. Furthermore, because only five sites were examined the results are not representative of juvenile arrestees in other geographic locations.
Second, along the same lines, ADAM focuses only on arrestees. Therefore, it cannot be assumed that the results from this study extend to other criminal/deviant populations or noncriminal/nondeviant populations. Future research would benefit from studies examining the temporal validity of self-reported drug use among juveniles, within both deviant and nondeviant populations, based on a probability sample in various locations to assess the broader implications of drug-reporting trends.
Third, as we discussed above, missing data were a particular concern with the ADAM project. This concern may be a function of the convenience sampling approach used or of variations in arrestee processing, data collection protocols, or site management overall. Regardless, ADAM II should seek to reduce missing data to the extent possible. As a reminder, Social Distance Theory suggests that respondents might be more willing to participate when they share common characteristics with the interviewer. Social Attribution Theory suggests that respondents choose to participate based on the respondent’s perception about the norms and expectations of an interviewer. ADAM II should consider these two perspectives when selecting interviewers and within the context of their data collection protocols and questionnaire language. It might even be useful to test variations in protocols across locations.
Fourth, in this study only the validity of self-reported marijuana use was explored. The rationale for choosing marijuana was that the laws governing this particular drug have been rapidly changing in the past few decades. Specifically, since 1996, the use of medical marijuana has been legalized in at least 14 states with many others considering legalization (ProCon.org, 2010). Therefore, since these changes in law have been highly publicized it was assumed that such changes could potentially affect societal approval to marijuana use and subsequently affect validity rates. However, it cannot be assumed that changes in laws regarding other drugs of abuse will have similar effects (or any effect for that matter). For that reason, future research should further examine how changes in drug laws, beyond marijuana, affect the validity of self-reported drug use within and outside of correctional settings.
Footnotes
Appendix
Direction of Disagreement Among Juvenile Arrestees
| Year | % Reported use, tested positive | % Reported use, tested negative | % Denied use, tested positive | % Denied use, tested negative |
|---|---|---|---|---|
| 1998 | 70.6 | 29.4 | 20.4 | 79.6 |
| 1999 | 73.5 | 26.5 | 20.7 | 79.3 |
| 2000 | 71.8 | 28.2 | 13.0 | 87.0 |
| 2001 | 69.2 | 30.8 | 18.9 | 81.1 |
| 2002 | 74.2 | 25.8 | 18.4 | 81.6 |
| Mean | 71.8 | 28.2 | 18.3 | 81.7 |
The author(s) declared no potential conflicts of interests with respect to the authorship and/or publication of this article.
The author(s) received no financial support for the research and/or authorship of this article.
