On the Potential Implications of Reports of Fictitious Drug Use for Survey Research on Juvenile Delinquency

Abstract

A variety of methodological issues have been raised over self-reports of delinquency and its correlates. In this study, we call attention to the provision of untruthful information and provide an investigation of this issue using a survey item that assesses a respondent’s use of a fictitious drug in relation to reports of delinquency and traditional criminological correlates. Bivariate and multivariate analyses were conducted based on data drawn from a probability sample of middle and high school students in Florida. Results show (a) there are important differences on key criminological variables between respondents who report use of a fictitious drug and those who do not; (b) the internal consistency of a variety index of delinquency is particularly sensitive to the inclusion of respondents reporting the use of a fictitious drug; and (c) the effect size of some criminological variables on delinquency may be sensitive to controlling for reports of fictitious drug use. Overall, the inclusion of fictitious drug use items within etiological models may serve as a useful approach to further establishing the reliability and validity of information provided by survey respondents.

Keywords

reliability validity false reporting untruthfulness delinquency FYSAS

Introduction

The use of survey methodologies to examine factors related to the onset, continuation, and desistence of delinquency and crime dominates the field of criminology (Piquero, Farrington, & Blumstein, 2003; Thornberry & Krohn, 2003). Research emerging from such sources as the National Longitudinal Study of Adolescent Health (Add Health), the Monitoring the Future (MTF), the Gang Resistance Education and Training (GREAT) studies, the National Youth Survey (NYS), and the National Longitudinal Survey of Youth (NLSY) has significantly shaped understanding of a variety of topics germane to the field. Likewise, these studies have proved crucial for enabling researchers to test core propositions of many of the leading contemporary theories of delinquency and crime. Suffice it to say, it would be difficult to imagine what the field would look like without the use of survey data.

Given the centrality of survey methodologies for criminological research and theory testing, establishing the reliability and validity of survey data is of critical importance. As it pertains to self-reports of delinquency and crime in particular, Krohn, Thornberry, Gibson, and Baldwin (2010) recently noted that “Throughout the history of the self-report methodology, the issue of central concern to both adherents and critics of the technique has been the accuracy of the data” (p. 512). Importantly, this concern extends to a wide range of items that commonly appear on survey questionnaires, not just self-reports of delinquency. To address these concerns, researchers have developed a number of methods to assess the reliability and validity of survey data. These methods include test–retest reliability assessments (e.g., Huizinga & Elliott, 1986), internal consistency checks (e.g., Hindelang, Hirschi, & Weis, 1981), external validation checks (e.g., Blumstein, Cohen, Piquero, & Visher, 2010; Brame, Fagan, Piquero, Schubert, & Steinberg, 2004; Maxfield, Weiler, & Widom, 2000), developing social desirability bias scales (e.g., MacDonald, Morral, & Piquero, 2011), and item scaling (Osgood, McMorris, & Potenza, 2002; Piquero, Macintosh, & Hickman, 2002).

Yet, there are known limitations associated with many of these methods. For example, although it is possible to conduct external validation checks to determine the accuracy of self-reports relative to official reports of crime among adults, the ability to execute similar checks among adolescents is far more limited (Krohn, Lizotte, Phillips, Thornberry, & Bell, 2013). In addition, test–retest assessments and external validation checks require the collection of information from multiple sources or at multiple periods of time, and usually require stringent institutional review board (IRB) approval given that information cannot be collected anonymously. Furthermore, although internal reliability checks and test–retest assessments can be used to determine the extent to which individuals are responding to items in a consistent manner, they cannot necessarily determine whether a respondent who provides a consistent response to several items in a row does so because that is the attribute that best describes them, or rather because they are being untruthful when responding to each of the items.

For example, a recent study found that among a sample of more than 1,500 high school students, 20% of the respondents reported that they were untruthful when answering at least some items on a survey questionnaire focused on delinquency and its causes (Meldrum, Piquero, & Clark, 2013). In addition, the study found that respondents who self-reported they had been dishonest were lower in self-control and more likely to be non-White. To the extent that survey respondents provide untruthful responses on antisocial behavior and its related factors, the reliability and validity of survey research may be questioned. As the percentage of respondents who provide untruthful information in survey research increases relative to the percentage of respondents who provide truthful information, it might reach a high enough point such that it could raise concerns over the precision of estimates and the nature and size of their effect drawn from subsequent statistical analyses.

Given that limitations exist with current methodologies, coupled with the realization that dishonesty is a legitimate concern when studying antisocial behavior using self-reported data, there is a clear need for innovative strategies to further address the overarching concern of the reliability and validity of survey data within criminology. Self-reports of truthfulness are one such method. In addition to the study by Meldrum et al. (2013) noted above, the British Crime Survey (BCS) has used such an approach. Likewise, in the 2006 Offending, Crime and Justice Survey, Roe and Ashe (2008), speaking to respondent reports on truthfulness stated “98 percent said they answered all offending questions honestly” (p. 9). The National Household Survey on Drug Abuse (NHSDA), conducted by the Substance Abuse and Mental Health Services Administration (SAMHSA), also includes the collection of data on respondent truthfulness.

In a validity study using the NHSDA data, Harrison, Martin, Enev, and Harrington (2007) analyzed self-reports of the truthfulness of participants’ own responses, as well as their perception of the truthfulness of responses given by other people. When commenting on the truthfulness of their own responses, 90% of respondents indicated that they were “completely” truthful when responding to the drug-related questions, whereas 7.9% indicated that they were “mostly” truthful (p. 22). However, these same respondents had a very different view of “most people,” as only 15.9% believed that most people were “completely” truthful and 46.7% believed that most people were “mostly” truthful (p. 22). Subsequent analysis showed that the measure of truthfulness distinguished between over-reporters of drug use and true users: Among 18- to 25-year-olds, it was found that “the likelihood of overreporting was greater for those who admitted to being less than completely truthful in their answers to some drug questions” (Harrison et al., 2007, p. 49).

Although questions tapping into respondent truthfulness are certainly useful, they cannot completely guard against lying or social desirability bias. Thus, other strategies have been devised, one of which entails the insertion of self-report items pertaining to a respondent’s use of a fictitious drug. This approach has been used in the BCS. Specifically, the BCS included a self-reported drug use item for the drug semeron, along with real drug items. In a comparison of BCS data from 1992 and 1998, Maxfield and Babbie (2008) reported that although there were ample reports of the use of real drugs, less than 1% reported using semeron in 1992 and no respondent reported using semeron in 1998.

Inserting fictitious drug items into surveys has been a method utilized by substance use researchers (e.g., Petzel, Johnson, & McKillip, 1973; Sobell & Sobell, 1978) and epidemiologists (e.g., Werch et al., 2008) as well. Informatively, 3.8% of high school students studied by Petzel et al. (1973) reported they had used a fictitious drug. Furthermore, their results indicated that those reporting they had used the fictitious drug were significantly different from non-reporters on almost every question on the survey concerning drug use—their responses were consistently in the direction of increased drug use. Most recently, Meldrum and colleagues (2013) tested Hirschi and Gottfredson’s (1993) claim that individuals who are low in self-control should be less likely to provide reliable information in survey research (see also Piquero, Macintosh, & Hickman, 2000). To test this claim, the authors inserted an item measuring the use of a fake drug called clorovisen into the survey and examined responses to this item in relation to a modified version of the Grasmick, Tittle, Bursik, and Arneklev (1993) Self-Control Scale. The authors found that 2.5% of respondents reported having used the fictitious drug, and that respondents who were lower in self-control were more likely to report having used the fictitious drug, net of age, race, and sex.

Given research findings such as these, we believe it would be fruitful to further evaluate the use and implications of reports of fictitious drug use as a means to identifying survey participants who may systematically falsify their responses. Two points merit discussion. First, many studies that have included fictitious drug use items have simply removed individuals reporting the use of such drugs from analyses rather than making systematic comparisons between individuals who reported using the drugs and those who did not (e.g., Hayes, Hudson, & Matthews, 2004; Hemphill et al., 2012). We view this as a lost opportunity. Just as research has investigated how the correlation between self- and official reports of delinquency and crime systematically varies according to other measured variables (Hindelang et al., 1981; Maxfield et al., 2000; Piquero & Brame, 2008), so too can comparisons be made between individuals who report and do not report the use of a fictitious drug (Meldrum et al., 2013). Second, fictitious drug use items can easily be incorporated into delinquency studies that use survey methodologies. Most delinquency studies have self-report items pertaining to both delinquent behavior as well as substance use. As such, it would require little additional effort or page space to add an additional item pertaining to a drug known by the study researchers, but not revealed to the study participants, to be fake.

Thus, by inserting fictitious drug items into surveys, researchers can accomplish at least three things. First, they can estimate the percentage of a sample that is providing potentially inaccurate information on a survey questionnaire. Second, they can examine whether there are systematic differences between individuals who do and do not report the use of a fictitious drug. Third, they can examine the extent to which the removal or inclusion of individuals reporting fictitious drug use affects the precision of estimates generated from multivariate models.

The Present Study

Although the use of fictitious drug use items is not new, there has been an underutilization of such items within analyses focused on adolescent delinquency. Herein, we investigate three issues by utilizing a self-report item tapping into fictitious drug use inserted into a statewide school-based self-report survey. First, we examine differences between individuals who report having used a fictitious drug and those who do not. Based on the limited body of research investigating this issue (e.g., Meldrum et al., 2013), we hypothesize that there will be significant differences between respondents who report using a fictitious drug and those who do not. Specifically, we anticipate that respondents who report fictitious drug use should evince higher values for delinquency and its correlates. Second, we assess whether the internal consistency of a variety index of delinquency varies according to whether respondents report the use of a fictitious drug. To the extent that a respondent reports using a fictitious drug, there might be reason to suspect such respondents may over-report their involvement in various forms of delinquency, which would artificially strengthen the internal consistency of a variety index. Given this potential, we hypothesize that the internal consistency of a variety index of delinquency will be substantively stronger among those respondents reporting the use of a fictitious drug relative to those who do not.

Third, we investigate whether controlling for self-reported fictitious drug use alters key relationships between delinquency and its correlates. This is an important issue to consider—if controlling for fictitious drug use does alter the strength of these associations, then there may be good reason to have a skeptical eye toward self-reports of delinquency and its correlates provided through survey research because some influential respondents may be over-reporting both delinquency and the severity of other risk factors for delinquency. Accordingly, we hypothesize that the strength of the association between delinquency and its correlates will weaken when controlling for reports of fictitious drug use. To test these hypotheses, we make use of data from a large sample of middle and high school students from Florida. The data are particularly appropriate for assessing the above issues in that the survey includes information on the use of a fictitious drug (derbisol), as well as information on basic demographics, traditional correlates of delinquency from several criminological theories, and a multiple item measure of serious delinquency.

Data and Method

Participants and Procedures

Data for this study came from the 2012 Florida Youth Substance Abuse Survey (FYSAS). The FYSAS is a repeated cross-sectional survey of public middle and high school students in the state of Florida conducted annually since 2000. The survey is based on the Communities that Care Youth Survey (Hawkins, Catalano, & Miller, 1992) and uses a two-stage cluster-sampling methodology. In the first selection stage, separate groups of middle and high schools were randomly selected from 66 of the 67 Florida counties that participated in the 2012 survey (one county did not participate). All public middle and high schools in the state were included in the sampling frame for each county, with the exception of adult education, correctional, or special education schools. The probability of selection for each school was proportional to the size of the school’s enrollment. Thus, larger schools had a higher chance of being selected than smaller schools. In the second sampling stage, survey coordinators were instructed on how to randomly select classrooms from the selected schools to fulfill the survey quota for each school. Because special education and ESOL (English for speakers of other languages) classes were not to be included in the survey, they were not included in the classroom selection list for each school. Recent studies based on the FYSAS data have been reported on elsewhere (Eitle, 2005; Peleg-Oren, Saint-Jean, Cardenas, Tammara, & Pierre, 2009).

For the 2012 administration of the FYSAS, a total of 74,621 students participated in the survey. A passive consent procedure was used for the survey administration in the majority of school districts. Specifically, students were given the consent notification and were asked to give it to their parents. It was then up to the parents to notify the school if they did not want their child to participate in the survey. The overall participation rate among the sampled middle school students was 76.1%, whereas the overall participation rate among the sampled high school students was 72.4%. Administration of the surveys took place during February and March of 2012. Both the classroom teacher and the written instructions on the front of the survey form assured students that participation in the survey was voluntary and that the answers students gave would be anonymous and kept confidential. Students had 50 min to complete the survey.

Of particular note for the current study, when the FYSAS is published by the state of Florida each year, certain exclusionary criteria are used before information on alcohol, tobacco, and other drug (ATOD) use is presented in the State Report. The chief consideration that we focus on is the exclusionary criteria whereby anyone reporting having used the fictitious drug derbisol is removed from the calculation of prevalence rates and other considerations in the State Report. At our request, the raw data file used to generate the State Report before the above exclusionary criteria were implemented was provided to us. As such, any systematic comparisons between the information presented in this study and the information presented in the 2012 FYSAS State Report should not be made.

The FYSAS middle and high school survey instrument is quite lengthy (more than 10 pages), and survey fatigue appeared to play a role in generating item non-response, particularly in later sections of the survey. Although 0.35% of participating students did not respond to the first item on the survey, 12.22% of students did not respond to the last item on the survey.¹ After listwise deletion of cases with missing data on at least one of the variables we describe shortly, a total of 60,363 respondents were included in the analyses presented below (80.9% of the original sample).²

Measures

Fictitious drug use

The FYSAS survey instrument contains items asking about the use of several different substances. Imbedded within the section of the survey asking respondents about their use of these substances was an item asking about their lifetime use of the drug derbisol. Specifically, the item read, “On how many occasions (if any) have you used derbisol in your lifetime?” The response categories were “0 occasions,” “1 to 2 occasions,” “3 to 5 occasions,” “6 to 9 occasions,” “10 to 19 occasions,” “20 to 39 occasions,” and “40 or more occasions.” For the analyses, the measure was dichotomized such that any response other than “0 occasions” was coded 1, whereas the response of “0 occasions” was coded 0. A little less than 3% of respondents (actual figure = 2.8%) included in the analysis reported having ever used the fictitious drug derbisol.

Variety index of delinquency

Although the primary goal of the FYSAS is to provide annual estimates of the prevalence of ATOD use, because it is modeled on the Communities that Care Youth Survey, it includes extensive information on risk and vulnerability factors for antisocial behavior, as well as items measuring antisocial behavior itself. In particular, six items measure the commission of rather serious acts of delinquency over the prior 12 months, including carrying a handgun, selling illegal drugs, stolen or attempted to steal a car or motorcycle, attacking someone with the intent of hurting them, being drunk or high at school, and taking a handgun to school. As with the item measuring derbisol use, the response categories ranged from “0 occasions” to “40 or more occasions.”³ For the analyses, each of the six items was dichotomized (0 = 0 occasions; 1 = at least once) and a variety index of delinquency was constructed by summing together the six dichotomized items (α = .82). Although we report this alpha reliability value, we will later discuss how this value may vary according to whether a respondent reported having used derbisol.

Correlates of delinquency

In addition to the inclusion of delinquency and substance use measures, the FYSAS measures a number of factors and contexts that past research has linked to delinquent behavior, including things related to the neighborhood environment, the home environment, school, peer associations, and demographic variables. Thus, the data provide a rich set of correlates to be examined in relation to reports of fictitious drug use. Furthermore, with the variables to be described shortly, we can assess whether the association between each of them and delinquency is altered when accounting for the measure of reported fictitious drug use.

Neighborhood problems

The FYSAS includes a large number of items dealing with neighborhood conditions and the acceptability of certain behaviors. For the current study, a nine-item indicator of neighborhood problems was constructed from two separate sets of items. The first set asked respondents to report how easy it would be to get alcohol, cigarettes, marijuana, other illicit drugs, and a handgun in their neighborhood. Each of the five items had responses ranging from “very hard” (=0) to “very easy” (=3). A second set asked respondents how much each of the following things describes their neighborhood: crime and/or drug selling, fights, abandoned buildings, and lots of graffiti. These items also had four-category response options of “NO!” (=0), “no” (=1), “yes” (=2), and “YES!” (=3). The nine items were averaged together to create a single measure of neighborhoods problems. The internal reliability of the nine items together was .85 and was higher than subsets of the items.

Delinquent attitudes

To measure delinquent attitudes, a seven-item variable was constructed based on respondent perceptions of how wrong they felt it was for someone their age to steal anything worth more than US$5, pick a fight with someone, attack someone with the intent of hurting them, drink alcohol regularly, smoke cigarettes, smoke marijuana, and use other illicit drugs. For each of the items, responses ranged from “very wrong” (=0) to “not wrong at all” (=3). The seven items were averaged together to create a single variable measuring delinquent attitudes. The internal reliability of the seven items was strong (α = .86).

Peer substance use

The FYSAS includes items asking respondents to report on the substance use of their four closest friends over the past 12 months. A three-item measure of peer substance use was constructed by averaging together responses pertaining to the use of cigarettes, alcohol, and marijuana by the respondent’s four closest friends, where each of the items had potential responses ranging from “0 friends” to “all 4 friends.” Together, the three items demonstrated strong internal consistency (α = .84).

Poor parenting

A 10-item measure of parenting quality was created based on items tapping into things such as attachment to parents (Do you enjoy spending time with your mother?), parental monitoring (Would your parents know if you did not come home on time?), and parental discipline (The rules in my family are clear). Each of the items was based on a four-category response, with higher values indicative of higher quality parenting. Given that all other variables in the analysis were coded such that higher values reflect an increased risk of antisocial tendencies, each of the 10 items was reverse-coded prior to taking the average of each of the items together. The 10-item measure demonstrated strong internal consistency (α = .87).

Poor school grades

The FYSAS includes a single item that asks students to self-report their school grades, which is designed to reflect commitment to school (e.g., Hirschi, 1969). Specifically, the question asks, “Putting them all together, what were your grades like last year?” Responses ranged from “mostly F’s” (=0) to “mostly A’s” (=4). The responses were reverse-coded so that higher values reflected poorer school grades during the prior school year.

Demographic variables

Standard demographic variables are provided in the FYSAS, including an ordinal measure for age (ranging from 10 to 19), a dichotomous measure for gender (male = 1), and a series of binary variables to measure race/ethnicity. Given the size of the sample, we created four binary variables: White, Black, Latino/Hispanic, and Multiracial/Other. The FYSAS allows respondents to select multiple categories (e.g., Black and Pacific Islander) but does not ask respondents to indicate which race they primarily identify with. As such, respondents were included within the dichotomous variables for White, Black, and Latino/Hispanic only if they selected a single racial category. Individuals selecting multiple categories were coded multiracial and included in the “Multiracial/Other” dichotomous variable. For multivariate models, White was treated as the omitted reference category.

In addition to these demographic variables, two additional variables were included in the current study. The first is a dichotomized measure of family structure. Respondents indicated whom they lived with by selecting pre-established options (e.g., mother, stepfather, grandparents, sister, stepbrother). For the analysis, we coded a respondent who was not living in a house with both their biological parents as living in a non-biological family structure (=1), whereas respondents who reported living with both their biological parents were assigned a value of 0. We also included a dichotomized measure for whether the respondent reported living in a household where the primary language spoken was something other than English (=1). We felt that including this measure as a control variable was particularly important, as someone who has poor English skills might confuse the drug derbisol for a drug that they felt actually existed and that they have used. Table 1 provides the descriptive statistics for the analysis sample, while the appendix provides the correlation matrix.

Table 1.

Descriptive Statistics for Full Sample and Split by Reported Fictitious Drug Use.

	Full sample (N = 60,363)				No reported use (n = 58,683)		Reported use (n = 1,680)		t value
Variable	M	SD	Minimum	Maximum	M	SD	M	SD	t value
Age	14.37	2.02	10	19	14.35	2.00	15.17	2.32	−16.49***
Male	0.48	0.50	0	1	0.48	0.50	0.63	0.48	−12.14***
White	0.51	0.50	0	1	0.51	0.50	0.34	0.47	13.76***
Black	0.14	0.35	0	1	0.14	0.34	0.24	0.42	−11.80***
Latino/Hispanic	0.15	0.36	0	1	0.15	0.36	0.13	0.34	2.25*
Multiracial/Other	0.20	0.40	0	1	0.20	0.40	0.30	0.46	−10.06***
Non-Biological Family Structure	0.48	0.50	0	1	0.48	0.50	0.66	0.47	−14.57***
Poor Grades	0.92	0.91	0	4	0.89	0.88	1.76	1.37	−39.19***
Poor Parenting	0.88	0.68	0	3	0.86	0.66	1.64	0.81	−47.43***
Foreign Language Home	0.12	0.32	0	1	0.11	0.32	0.29	0.45	−22.43***
Peer Substance Use	0.97	1.25	0	4	0.93	1.22	2.49	1.44	−51.40***
Delinquent Attitudes	0.64	0.66	0	3	0.60	0.61	1.85	0.97	−81.11***
Neighborhood Problems	0.77	0.69	0	3	0.74	0.65	1.88	0.88	−70.07***
Variety Index of Delinquency	0.44	1.11	0	6	0.33	0.85	4.26	2.19	−170.00***
Fictitious Drug Use	0.028	0.16	0	1	—	—	—	—	—

p < .05. ***p < .001 (two-tailed).

Results

We began by drawing mean comparisons between respondents who reported using derbisol versus those who did not. As shown in Table 1, 2.8% of respondents in the analysis sample reported using derbisol, whereas 97.2% did not report using derbisol. Thus, most individuals did not report having used a fictitious drug. However, an examination of mean differences for nearly each of the variables included in the analysis between those who reported using derbisol versus those who did not report using derbisol does reveal marked differences. Most notably, the mean value for the variety index of delinquency among respondents who did not report using derbisol was low (M = 0.33 with a maximum potential value of 6.00), whereas among those who did report using derbisol, it was significantly and substantively higher (M = 4.26, t value = −170.00).⁴ Likewise, whereas the mean value for peer substance use among non-reporters was 0.93, among those who reported using derbisol, the mean value was 2.49 (t value = −51.40). Mean differences were also present for many of the demographic variables as well. For example, 14% of non-reporters were Black, but 24% of respondents who reported derbisol use were Black (t value = −11.80). Likewise, whereas 48% of non-reporters do not live in a household with both biological parents, that percent was 66% among respondents who reported using derbisol (t value = −14.37). Although only bivariate associations, these differences between non-reporters and those who reported using a fictitious drug indicate there is something distinct about those who do report having used derbisol.

To consider this further, we examined whether each of the variables included in Table 1 predict derbisol use in a multivariate model. To accommodate the left-hand censoring of reported fictitious drug use, we used logistic regression and report robust standard errors adjusted for clustering within the 66 Florida counties. We estimated two separate models, the results of which are displayed in Table 2. Model 1 presents the results without controlling for the variety index of delinquency. Of the 13 predictor variables, only Latino/Hispanic fails to reach statistical significance. Of the significant effects revealed in Model 1, there are rather large, positive effects for Black, poor parenting, foreign language home, delinquent attitudes, and neighborhood problems.

Table 2.

Logistic Regression of Reporting Fictitious Drug Use (N = 60,363).

	Model 1				Model 2
Variable	b	RSE	Z	OR	b	RSE	Z	OR
Age	−.04*	.02	−2.23	0.96	−0.01	.02	−0.75	0.99
Male	.31***	.07	4.62	1.37	−0.10	.08	−1.34	0.90
Black	.97***	.10	9.35	2.65	0.64***	.12	5.25	1.90
Latino/Hispanic	−.10	.12	−0.84	0.90	0.06	.14	0.42	1.06
Multiracial/Other	.37***	.09	4.03	1.44	0.08	.11	0.73	1.08
Non-Biological Family Structure	.23***	.06	3.73	1.26	0.19**	.07	2.66	1.21
Poor Grades	.15***	.03	5.10	1.16	0.08**	.03	2.69	1.09
Poor Parenting	.59***	.06	10.64	1.80	0.50***	.06	8.26	1.64
Foreign Language Home	.81***	.10	7.95	2.24	0.48***	.11	4.51	1.62
Peer Substance Use	.08**	.03	2.59	1.08	−0.07	.04	−1.90	0.93
Delinquent Attitudes	.97***	.06	17.55	2.64	0.36***	.07	4.95	1.43
Neighborhood Problems	1.00***	.05	19.81	2.71	0.29***	.06	4.72	1.33
Variety Index of Delinquency	—	—	—	—	0.87***	.03	31.71	2.39
Constant	−7.08***	.26	—	—	−6.52***	.26	—	—
Log Pseudo-Likelihood	−4,933				−3,674
Wald χ²(13)	3,201***				4,980***
Pseudo R²	.36				.52

Note. Standard errors adjusted for clustering within 66 counties. OR = odds ratio; RSE = robust standard error of beta (SEs not reported for OR).

p < .05. **p < .01. ***p < .001.

Informatively, there is a marked contrast between Model 1 and Model 2, which adds the measure of delinquency. Substantively large changes are observed for the variables male, multiracial/Other, foreign language home, delinquent attitudes, and neighborhood problems. A visual illustration of the change in the magnitude of the odds ratios (ORs) from Models 1 and 2 is shown in Figure 1. Of particular interest, the size of the OR for delinquent attitudes and neighborhood problems decreases by approximately 50%, whereas the ORs for Black, multiracial/Other, and foreign language home also change substantively across the two models. Yet, many of the coefficients in Model 2 remain significant. Thus, support for Hypothesis 1 is found in these data—there are substantive differences between individuals reporting the use of a fictitious drug relative to those who do not report use, and many of these differences remain even when accounting for delinquent behavior.

Figure 1.

Predictors of fictitious drug use.

We next examined to what extent the reliability (i.e., internal consistency) of the six items that comprise the variety index of delinquency differed between all respondents in the analysis sample (N = 60,363), respondents who did not report using derbisol (n = 58,683), and respondents who did report using derbisol (n = 1,680). Figure 2 presents the alpha reliability estimates for each of these three groups and reveals that the reliability of the variety index appears to be materially influenced by respondents who report using derbisol. Specifically, when the reliability of all respondents is estimated, the alpha value of .82 presented in the “Measures” section above is produced. Yet, when the reliability of respondents who did not use derbisol is examined, alpha drops to .69. And, note that alpha jumps to .89 among the group of individuals who reported derbisol use. This indicates that respondents who report using derbisol are far more likely to be reporting that they have committed every one of the six delinquent acts included in the variety index—precisely what was revealed in Table 1. This provides support for Hypothesis 2 and provides additional insight into the substantive differences between respondents who report the use of a fake drug relative to those who do not. It also raises the possibility that these differences may influence measurement issues and structural analyses.

Figure 2.

Internal consistency values for delinquency by sample group.

Thus far, the results provide evidence that individuals who report the use of a fictitious drug are markedly different from individuals who do not on a number of key criminological variables and delinquent behavior itself. Differences in group means for nearly all of the analysis variables, the strong effect of the variety index of delinquency on fictitious drug use, and differences in the reliability of the items included in the variety index between reporters and non-reporters suggest a small yet potentially influential portion of adolescents who participate in survey research might deliberately provide false information. An important consideration, then, is whether such individuals, once identified, should be included in models that provide tests of criminological theories and studies that seek to generate estimates of substance use, or rather removed from etiological models altogether.

Because respondents reporting the use of a fictitious drug report they are more delinquent, experience lower quality parenting, and hold stronger attitudes conductive to delinquency, the possibility exists that these individuals are more likely to use illegal substances and actually be more delinquent than non-reporters. As such, the choice to remove these respondents from analyses could result in a systematic underrepresentation of the prevalence and incidence of substance use among adolescents and limit the variability of delinquent behavior in a sample being used to model the correlates of delinquency, substance use, and crime. Given this possibility, instead of removing such individuals from statistical analyses altogether, researchers could assess whether the inclusion of these cases substantively alters results. In particular, items measuring the use of a fictitious drug could be used as control variables to guard against the possibility that the covariation between reports of delinquency and its correlates could, in part, be the result of systematic bias due to some respondents providing false responses to a variety of survey items.

We considered this possibility in the current data by examining the effects of each of the analysis variables on delinquency before and after controlling for reports of fictitious drug use. Given that 79% of respondents reported not engaging in any delinquent behavior, and that the variance of the distribution for the variety index is greater than its mean, we used negative binomial regression. Model 1 of Table 3 presents the results from the regression model when excluding the control for fictitious drug use. With the exception of the variables Latino/Hispanic and foreign language home, all other variables are statistically significant and predict the count of delinquency in ways that are consistent with prior literature on the correlates of delinquency.

Table 3.

Negative Binomial Regression of Variety Index of Delinquency (N = 60,363).

	Model 1				Model 2
Variable	b	RSE	Z	IRR	b	RSE	Z	IRR
Age	−0.04***	.01	−5.19	0.96	−.03***	.01	−5.24	0.97
Male	0.37***	.02	15.66	1.44	0.35***	.02	16.39	1.42
Black	0.40***	.04	10.09	1.48	0.30***	.04	7.94	1.35
Latino/Hispanic	0.05	.03	1.55	1.05	0.09**	.03	2.96	1.09
Multiracial/Other	0.21***	.02	8.53	1.24	0.18***	.02	7.46	1.19
Non-Biological Family Structure	0.13**	.02	6.14	1.13	0.11***	.02	5.60	1.12
Poor Grades	0.10***	.01	9.99	1.11	0.09***	.01	8.90	1.10
Poor Parenting	0.37***	.02	15.17	1.45	0.34***	.02	15.89	1.40
Foreign Language Home	0.02	.04	0.56	1.02	−0.12**	.04	−3.36	0.89
Peer Substance Use	0.26***	.01	27.06	1.29	0.27***	.01	28.79	1.31
Delinquent Attitudes	0.66***	.02	39.98	1.94	0.58***	.02	33.20	1.79
Neighborhood Problems	0.64***	.02	38.46	1.89	0.54***	.02	33.46	1.72
Fictitious Drug Use	—	—	—	—	1.13***	.05	21.72	3.08
Constant	−3.04***	.11	—	—	−2.93***	.09	—	—
Log Pseudo-Likelihood				−38,773				−38,018
Wald χ²				16,429***				15,451***

Note. Standard errors adjusted for clustering within 66 counties. IRR = incidence rate ratio; RSE= robust standard error of beta (SEs not reported for IRR).

p < .05. **p < .01. ***p < .001.

Model 2 adds the control for fictitious (derbisol) drug use. Informatively, there are a few notable differences between the coefficients presented in the first model and those presented in the second model. Specifically, there appears to have been a suppression effect taking place, as accounting for derbisol use results in the coefficient for Latino/Hispanic to increase in size and become statistically significant. Likewise, the coefficient for foreign language home changes signs from positive to negative and becomes statistically significant. And, although there are not dramatic changes in the statistical significance for other variables in the model, perhaps because of the large sample size, there is attenuation in the size of the coefficients among some of them, providing qualified support for Hypothesis 3. For example, the size of the coefficient for Black is reduced by 25%, from .40 in Model 1 to .30 in Model 2. Similar attenuations are observed for the variables delinquent attitudes and neighborhood problems: The coefficient for delinquent attitudes is reduced from .66 in Model 1 to .58 in Model 2, while the coefficient for neighborhood problems is reduced from .64 in Model 1 to .54 in Model 2. Not surprisingly, fictitious drug use exerts the strongest effect on the variety index of delinquency.

Discussion

Motivated by concerns regarding the reliability and validity of survey data, the current study assessed a unique and underexamined issue—the reported use of fictitious drugs. Although recognized and examined in other areas of research, delinquency researchers have not devoted much time and attention to examining whether key criminological relationships and the reliability of items used to assess such relationships may vary as a result of fictitious reporting. To investigate these issues, we used data from a large sample of adolescents from the State of Florida. Four main findings emerged from the analyses. First, there were important mean differences on key criminological variables between adolescents who reported use of a fictitious drug and those who did not, particularly with regard to self-reported delinquency. Second, a number of key criminological variables predicted reports of fictitious drug use in multivariate models, and these effects in many instances remained after controlling for delinquency. Third, the reliability of a variety index of delinquency was particularly sensitive to the inclusion of respondents in the sample who reported using a fictitious drug. Fourth, controlling for reports of fictitious drug use in regression models predicting delinquency influences the magnitude of the effect sizes of some predictor variables, suggesting the possibility that some adolescents may systematically provide false responses to a number of items contained within surveys of antisocial behavior.

With these findings in mind, there are several points worth considering. First, few studies on juvenile delinquency have samples sizes as large as that found within the FYSAS. As a result, although we observed rather modest changes in effect sizes and statistical significance after controlling for fictitious drug use in the latter regression models presented, the influence of false reporting could be greater within studies that have smaller sample sizes, a feature that characterizes many data sources in criminology/criminal justice. Second, we cannot establish with certainty what a report of having used a fictitious drug represents. On one hand, some respondents may use so many unlabeled substances that they might think they have used derbisol. On the other hand, the concern exists that some respondents may not be paying attention to what they are doing when completing a survey or engaging in outright deception when responding to certain items. To the extent that the latter is taking place more often than the former, this poses concerns for the accuracy of self-reports of other substances, delinquent behavior, and its correlates.

We should emphasize that the findings of this study should not prompt the dismissal of past research findings based on survey methodologies. However, we do feel that untruthfulness may affect some study relationships and, ultimately, study conclusions. Our recommendation is that, going forward, delinquency researchers include questions within their surveys designed to assess potential untruthfulness. Items measuring fictitious drug use is one such strategy; including scales measuring self-reports of truthfulness is another. Such strategies can allow for an array of analyses, much like those conducted in the present study, that will permit researchers to better understand the extent to which potentially deceptive respondents may influence study results. In our minds, the cost is minimal and the gains greater.

The results of this study and our recommendations are not meant to cast a negative light on the self-report method; instead, they serve merely as a caution, one that may be more magnified in self-report studies that assess particularly serious forms of delinquency where few respondents report engaging in the behaviors.⁵ If even half of these individuals are falsely reporting they engaged in the behavior (as they might be falsely reporting having used a drug that does not exist), this has implications for predictive models. Controlling for fictitious drug use may be one method of parceling out those who have actually engaged in delinquency from those who are not paying attention and/or intentionally being deceptive while participating in survey research. Likewise, it is possible that some chronic, high-rate offenders could be exaggerating the frequency of serious acts of delinquency. If there is a strong correlation between high-frequency reporters and fictitious drug use reports, as was true in the current data, this may indicate some difficulty in studying high-rate offenders using survey data.

More generally, our investigation has only considered the salience of the false reporting issue in a preliminary manner. Future studies should examine whether there is more (or less) fictitious reporting at different stages of the life course, that is, childhood, adolescence, and adulthood. Likewise, as the focus of our analyses was on delinquent acts in particular, it would be fruitful to examine fictitious reporting in relation to reports of other drugs, an issue that has been considered by past researchers (e.g., Petzel et al., 1973). In addition, future research that includes self-reports of arrest, external official records of arrest, and self-reported fictitious drug use could enable researchers to consider whether discrepancies between self-reports of arrest and official arrest records are predicted by reports of fictitious drug use. In the end, we hope the approach taken in the current study to investigate a specific reporting issue will encourage researchers to continue to think critically about the accuracy of the information provided by survey respondents.

Footnotes

Appendix

Acknowledgements

The authors would like to thank Hal Johnson and the Florida Department of Children and Families for providing access to the 2012 Florida Youth Substance Abuse Survey (FYSAS) data required to conduct the analyses presented.

Authors’ Note

The views and opinions expressed in this article are solely those of the authors and do not reflect the views of the Florida Department of Children and Families or their employees.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Blumstein

Cohen

Piquero

A. R.

Visher

C. A.

(2010). Linking the crime and arrest processes to measure variations in individual arrest risk per crime (Q). Journal of Quantitative Criminology, 26, 533-548.

Brame

Fagan

Piquero

A. R.

Schubert

C. A.

Steinberg

(2004). Criminal careers of serious delinquents in two cities. Youth Violence and Juvenile Justice, 2, 256-272.

Eitle

(2005). The moderating effects of peer substance use on the family structure–adolescent substance use association: Quantity versus quality of parenting. Addictive Behaviors, 30, 963-980.

Grasmick

H. G.

Tittle

C. R.

Bursik

R. J.

Arneklev

B. J.

(1993). Testing the core empirical implications of Gottfredson and Hirschi’s general theory of crime. Journal of Research in Crime & Delinquency, 30, 5-29.

Harrison

L. D.

Martin

S. S.

Enev

Harrington

(2007). Comparing drug testing and self-report of drug use among youths and young adults in the general population. (DHHS Publication No. SMA 07-4249, Methodology Series M-7). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

Hawkins

J. D.

Catalano

R. F.

Miller

J. Y.

(1992). Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin, 112, 64-105.

Hayes

Hudson

Matthews

(2004). Parental monitoring behaviors: A model of rules, supervision, and conflict. Behavior Therapy, 35, 587-604.

Hemphill

S. A.

Herrenkohl

T. I.

Plenty

S. M.

Toumbourou

J. W.

Catalano

R. F.

McMorris

B. J.

(2012). Pathways from school suspension to adolescent nonviolent antisocial behavior in students in Victoria, Australia and Washington State, United States. Journal of Community Psychology, 40, 301-318.

Hindelang

M. J.

Hirschi

Weis

J. G.

(1981). Measuring delinquency. Beverly Hills, CA: SAGE.

10.

Hirschi

(1969). Causes of delinquency. Berkeley: University of California Press.

11.

Hirschi

Gottfredson

(1993). Commentary: Testing the general theory of crime. Journal of Research in Crime & Delinquency, 30, 47-54.

12.

Huizinga

Elliott

D. S.

(1986). Reassessing the reliability and validity of self-report delinquency measures. Journal of Quantitative Criminology, 2, 293-327.

13.

Krohn

M. D.

Lizotte

A. J.

Phillips

M. D.

Thornberry

T. P.

Bell

K. A.

(2013). Explaining systematic bias in self-reported measures: Factors that affect the under-and over-reporting of self-reported arrests. Justice Quarterly, 30, 501-528.

14.

Krohn

M. D.

Thornberry

T. P.

Gibson

C. L.

Baldwin

J. M.

(2010). The development and impact of self-report measures of crime and delinquency. Journal of Quantitative Criminology, 26, 509-525.

15.

MacDonald

J. M.

Morral

Piquero

A. R.

(2011). Socially desirable response bias in criminology: An example of its effect in testing the effects of self-control. In MacDonald

J. M

(Ed.), Measuring crime & criminality: Advances in criminological theory (Vol. 17, pp. 21-36). New Brunswick, NJ: Transaction Publishers.

16.

Maxfield

M. G.

Babbie

E. R.

(2008). Research methods for criminal justice and criminology (5th ed.). Belmont, CA: Thomson Higher Education.

17.

Maxfield

M. G.

Weiler

B. L.

Widom

C. S.

(2000). Comparing self-reports and official records of arrests. Journal of Quantitative Criminology, 16, 87-110.

18.

Meldrum

R. C.

Piquero

A. R.

Clark

(2013). Does low self-control predict fictitious drug use and untruthfulness? Deviant Behavior, 34, 242-254.

19.

Osgood

D. W.

McMorris

B. J.

Potenza

M. T.

(2002). Analyzing multiple-item measures of crime and deviance I: Item response theory scaling. Journal of Quantitative Criminology, 18, 267-296.

20.

Peleg-Oren

Saint-Jean

Cardenas

G. A.

Tammara

Pierre

(2009). Drinking alcohol before age 13 and negative outcomes in late adolescence. Alcoholism: Clinical & Experimental Research, 33, 1966-1972.

21.

Petzel

T. P.

Johnson

J. E.

McKillip

(1973). Response bias in drug surveys. Journal of Consulting and Clinical Psychology, 40, 437-439.

22.

Piquero

A. R.

Brame

(2008). Assessing the race-/ethnicity-crime relationship in a sample of serious adolescent delinquents. Crime and Delinquency, 54, 390-422.

23.

Piquero

A. R.

Farrington

D. P.

Blumstein

(2003). The criminal career paradigm: Background and recent developments. In Tonry

(Ed.), Crime and justice: A review of research (Vol. 30, pp. 359-506). Chicago, IL: University of Chicago Press.

24.

Piquero

A. R.

Macintosh

Hickman

(2000). Does self-control affect survey response? Applying exploratory, confirmatory, and item response theory analysis to Grasmick et al.’s self-control scale. Criminology, 38, 897-930.

25.

Piquero

A. R.

Macintosh

Hickman

(2002). The validity of a self-reported delinquency scale: Comparisons across gender, age, race, and place of residence. Sociological Methods & Research, 30, 492-529.

26.

Roe

Ashe

(2008, July). Young people and crime: Findings from the 2006 Offending, Crime and Justice Survey (Home Office Statistical Bulletin). London, UK: Home Office.

27.

Sobell

L. C.

Sobell

M. B.

(1978). Validity of self-reports in three populations of alcoholics. Journal of Consulting and Clinical Psychology, 46, 901-907.

28.

Thornberry

T. P.

Krohn

M. D.

(2003). Comparison of self-report and official data for measuring crime. In Pepper

J. V.

Petrie

C. V.

(Eds.), Measurement problems in criminal justice research: Workshop summary (pp. 43-94). Washington, DC: National Academy Press.

29.

Werch

C. E.

Moore

M. J.

Bian

DiClemente

C. C.

Ames

S. C.

Weiler

R. M.

. . .Pokorny

S. B.

(2008). Efficacy of a brief image-based multiple-behavior intervention for college students. Annals of Behavioral Medicine, 36, 149-157.