Some evidences and applications of the randomized response crossed model in a survey of sensitive population behaviors

Abstract

Surveying human behaviors, especially in demographic, social, medical and public health research, often involves sensitive issues. Posing direct inquiries about stigmatizing or threatening topics may lead survey participants to refuse to answer or to give untruthful responses. Nonresponse and misreporting denote measurement errors that are difficult to treat and are likely to yield unreliable analyses of the surveyed topics.

This problem can be mitigated by adopting survey methods that enhance anonymity and respondent cooperation. One possibility is to create a trustful and confidential relationship between the interviewer and the survey participants. Alternatively, it is possible to fully protect privacy by adopting indirect questioning procedures that elicit information without posing sensitive questions directly.

We consider both above-mentioned possibilities showing the results of a real study which explores the effectiveness of the randomized response crossed model proposed by Lee et al. (2013) to produce prevalence estimates for two sensitive traits, cannabis use and its legalization.

Keywords

Cannabis use and legalization mixed-mode research prevalence estimates trustworthy interviewer

1. Introduction

Nowadays, large-scale surveys of human populations delve increasingly into sensitive topics such as abortion, sexual behavior, rape victimization, drug taking, tax evasion, xenophobia, income, and so on. Let us try to imagine the reaction of people engaged in socially undesirable or illegal behaviors when they are directly posed questions like “Have you ever earned illegal income?”, “Are you homosexual?”, or even the sensation of those who are posed confidential questions that pertain to their private sphere, albeit legal and do not trigger social desirability concerns, like “Which party will you vote for in the next election?”, “What is the level of your household income?” It is easy to imagine the embarrassment and the concerns of the interviewees, even when the inquirers do their best to guarantee confidentiality. Probably, most respondents would feel uncomfortable with such direct inquires, may give evasive or misleading replies, or even refuse to answer. Such reactions generate a well-known source of bias in surveys, called social desirability bias, which refers to the tendency to present oneself in a positive light. Survey participants exhibit this bias when they overreport socially acceptable attitudes which conform to social norms (e.g., giving alms, healthy eating, doing voluntary work) or underreport socially disapproved behaviors which deviate from social rules (e.g., gambling, consumption of alcohol, induced abortion, drug taking).

Sensitive research (Liamputtong, 2007; Tourangeau & Yan, 2007; Dickson-Swift et al., 2008) on stigmatizing, highly personal, embarrassing, threatening or even incriminating issues – especially when carried out by traditional techniques based on direct questioning (DQ) survey modes – is likely to produce non-sampling errors, such as nonresponse and untruthful answers, which can lead to inconsistent analyses and erroneous conclusions, including wrong inference about unknown characteristics of the population under study. Although these errors cannot be totally avoided, they can, to some extent, be reduced by enhancing respondent cooperation. There are various aspects to this: how the survey is administered; the presence of the interviewer and whether he/she poses the questions directly; the questionnaire format, the wording and the placing of the sensitive items; the setting for data collection – does the respondents answer in isolation or are others present? All the above factors play a part in encouraging trust and confidentiality (on this, see, e.g., Tourangeau & Smith, 1996; Groves et al., 2004).

In this paper, we discuss the results of a real study which furnishes empirical evidence of possible ways for enhancing respondent cooperation and reducing misreporting in sensitive research. Two aspects, in particular, are investigated: a non-standard data-collection survey method based on indirect questioning (IQ) and the role of the interviewer. Specifically, the study is conceived as a face-to-face mixed-mode interview which allows us to collect information on two topics of different sensitivity, cannabis use and its legalization, and to estimate the prevalence of cannabis users and of people in favour of cannabis legalization. Both the DQ and IQ survey modes are used to collect data in the presence of the interviewer, a person who is a member of the surveyed community and inclined to establish a confidential and trustful rapport with the survey participants. The outcomes of the empirical research well emphasize the positive effects that can be produced by adopting survey strategies based on the IQ and trustworthy interviewers. As expected, the benefits increase when the respondents perceive the investigated topic as being more sensitive.

The remainder of the paper is organized as follows. In Section 2 we introduce the problem of enhancing respondents cooperation in sensitive research. Section 3 is devoted to the description of a particular IQ method: the randomized response crossed model proposed by Lee et al. (2013). In Section 4 we present the main features of a real study on cannabis use and cannabis legalization conceived to evaluate the usefulness of the IQ approach when stigmatizing topics are surveyed. The results of our study are discussed in Section 5. Section 6 concludes the paper with some final considerations.

2. Increasing respondent cooperation in sensitive surveys

In surveys which cover sensitive issues, it is a well-established fact that the decision of the participants to honestly cooperate greatly depends on to what extent they perceive their privacy is being protected. Consequently, survey modes which ensure respondent anonymity or, at least, a high degree of confidentiality, may go some way to improving cooperation and, hence, gathering more reliable information on sensitive topics than is the case with DQ. It is worth remarking that, when comparing alternative data-collection methods, it is crucial to establish the criterion for choosing the most “accurate” method. In the absence of reliable external sources that provide true population benchmarks of the sensitive behaviors under investigation, the most common approach used in comparative validation studies is based on the “more-is-better” [“less-is-better”] assumption according to which a method that successfully induces more [less] reports of a sensitive undesirable [desirable] behavior is expected to yield higher [lower] estimates of the sensitive traits and, therefore, should be judged the more valid.

In order to increase respondent cooperation, survey statisticians have developed many different strategies. One possibility for improving reporting on sensitive topics is to limit the influence of the interviewer from the question-and-answer process on the grounds that the presence of the interviewer tends to increase socially desirable effects. This goal is traditionally pursued by means of self-administered questionnaires (SAQs) with paper and pencil, the computer-assisted telephone interviewing (CATI), the computer-assisted self interviewing (CASI), the audio computer-assisted self interviewing (ACASI) or by the computer-assisted Web interviewing (CAWI). On the contrary, many social researchers stress the collaborative, trustful and non-hierarchical relationship that should be build between the interviewer and the survey participants. In this case, the interviewer self-disclosure or the reciprocal sharing of personal ideas, attitudes and/or experiences concerning matters that might be related to the sensitive topic become important tactics to create a forthcoming and encouraging atmosphere. It may also happen that the interviewer knows most of the members of the stigmatizing group or is himself/herself a member of that group (see, e.g., Liamputtong, 2007; Dickson-Swift et al., 2008). In these situations, the respondents would be less inhibited by the presence of the interviewer and might be more willing to release personal information since they have nothing to hide, show indifference to interviewer opinion, and do not fear that their personal information may be released to third parties for purposes other than the survey ones.

A second possibility to enhance confidentiality consists in the use of alternative questioning methods – different from the conventional survey modes – that have been devised since the sixties to ensure respondent anonymity and that cut down on evasive answers and misreporting of stigmatizing acts. These methods are generally known as indirect questioning techniques (IQTs) and they obey the principle that no direct questions are posed to the survey participants and, thus, there is no need for respondents to openly reveal their true status. In this way, privacy is protected since the information gathered remains confidential and, consequently, the true status of the respondents remains undisclosed to both the interviewer and the researcher.

Since the IQ approach plays a crucial part in this paper, it is worth providing a more detailed discussion in the next Section.

3. The randomized response crossed model

The year 1965 marks the kick-off for the beginning of the IQTs, an ingenious approach conceived to reduce nonresponse and gather franker and more reliable responses to sensitive questions than DQ survey modes. The IQ approach represents a category of strategies for eliciting sensitive information which encompasses various alternatives. For the amount of literature produced since Warner’s (1965) pioneering work, the randomized response (RR) technique (RRT) holds certainly a prominent position among the IQTs. Roughly speaking, the RRT adopts a randomization device (e.g. decks of cards, colored numbered balls, dice, coins, etc.) which determines whether respondents answer the sensitive question or whether they are asked another neutral question, or even provide a pre-specified response (e.g. “Yes”) irrespective of their true status concerning the stigmatizing behavior. The randomization device, therefore, generates a probabilistic relation between the sensitive question and a given answer which is used to make inference about unknown parameters of interest, such as the prevalence of a sensitive attribute in the target population.

A great deal of the work spanning the literature since Warner (1965) on the IQTs is condensed in the monographs of Fox and Tracy (1986), Chaudhuri and Mukerjee (1988), Chaudhuri (2011), Chaudhuri and Christofides (2013), Tian and Tang (2014), Chaudhuri et al. (2016) and Fox (2016).

From the different IQTs, in our study we have employed the RR crossed model (CM) proposed by Lee et al. (2013) to simultaneously estimate, from a single sample, the prevalence of two sensitive attributes, say $A$ and $B$ , and other measures of association. To the best of our knowledge, the model was first implemented in a real study by Perri et al. (2016) to investigate induced abortion and the immigrant status of foreign women. No other empirical studies based on the CM are present in the literature, although different methods can be used to simultaneously investigate two sensitive attributes (see, e.g., Ewemooje et al., 2017). Without loss of generality, the implementation of the CM requires that sampled units be provided with two decks of cards, say Deck-I and Deck-II. Specifically, Deck-I contains two types of cards marked with the statements “I possess the sensitive attribute $A$ ” and “I do not possess the sensitive attribute $B$ ” in a proportion of $p$ and $1-p$ , respectively. Analogously, Deck-II contains cards marked with the statements “I possess the sensitive attribute $B$ ” and “I do not possess the sensitive attribute $A$ ” in a proportion of $q$ and $1-q$ . Each sampled unit is requested to draw one card from each deck, read the statements on the two selected cards without revealing them to the interviewer and then report an ordered pair of responses – in terms of “Yes” or “No” answers – according to whether his/her status does or does not match the question on the card. Hence, each unit provides just one of the possible pairs of responses (Yes, Yes), (Yes, No), (No, Yes) or (No, No). The collected pairs of responses are then used to primarily obtain an estimate of the prevalence $\pi_{A}$ of the attribute $A$ in the population, the prevalence $\pi_{B}$ of the attribute $B$ , the prevalence of units bearing both $A$ and $B$ , say $\pi_{A\cap B}$ , and the prevalence $\pi_{A\cup B}$ of units bearing $A$ or $B$ .

Let $\widehat{\theta}_{11}$ , $\widehat{\theta}_{10}$ , $\widehat{\theta}_{01}$ and $\widehat{\theta}_{00}$ denote, respectively, the proportion of observed responses (Yes, Yes), (Yes, No), (No, Yes) and (No, No) in the sample. Hence the estimation of the unknown population prevalence $\pi_{S}$ , with $S=A,B,A\cap B,A\cup B$ , is achieved by minimizing a squared distance between observed and true unknown population proportions which leads to the following estimators:

$\displaystyle\widehat{\pi}_{A,\text{CM}}=\frac{1}{2}+\frac{(q-p+1)(\widehat{% \theta}_{11}-\widehat{\theta}_{00})+(p+q-1)(\widehat{\theta}_{10}-\widehat{% \theta}_{01})}{2(p+q-1)},$ (1) $\displaystyle\widehat{\pi}_{B,\text{CM}}=\frac{1}{2}+\frac{(p-q+1)(\widehat{% \theta}_{11}-\widehat{\theta}_{00})+(p+q-1)(\widehat{\theta}_{01}-\widehat{% \theta}_{10})}{2(p+q-1)},$ (2) $\displaystyle\widehat{\pi}_{A\cap B,\text{CM}}=\frac{pq\widehat{\theta}_{11}-(% 1-p)(1-q)\widehat{\theta}_{00}}{(p+q-1)[pq+(1-p)(1-q)]}$ (3)

and

$\displaystyle\widehat{\pi}_{A\cup B,\text{CM}}=\widehat{\pi}_{A,\text{CM}}+% \widehat{\pi}_{B,\text{CM}}-\widehat{\pi}_{A\cap B,\text{CM}}.$ (4)

We note that, in real studies, the estimation procedure may lead to estimates of $\pi_{S}$ lying outside the parametric space [0, 1]. In such a case, the truncated estimator, $\widehat{\pi}^{*}_{S,\text{CM}}=\min(\max(0,\widehat{\pi}_{S,\text{CM}}),1)$ , is (customarily) assumed as estimator of $\pi_{S}$ .

4. Application of the crossed model to a cannabis study

In order to evaluate the effectiveness of the CM, we have conducted a real study by investigating two sensitive topics, say cannabis use and cannabis legalization. This Section is, therefore, devoted to describing the salient aspects of our study.

4.1 Why using the crossed model

Measuring levels and patterns of illicit drug use, their determinants, related behaviors and attitudes requires the use of self-reported methods of investigation. However, the validity of self-reported data has long been questioned (see, e.g., Harrison & Hughes, 1997) and assessed by using urinalysis, blood and hair analyses. Although less intrusive survey methods, such as CATI, ACASI and CAWI, are currently used to increase confidentiality, results still suffer from errors mostly ascribable to misreporting. For instance, some studies show that individuals under criminal justice supervision are loath to report drug use on confidential and anonymous surveys, others emphasize that a non-negligible percentage of individuals who test positive for drugs by urinalyses deny having used drugs. Underreporting of drug consumption is therefore well-evident. Similarly, there are reasons to believe that a lesser misreporting may also affect polling findings concerning the legalization of soft drugs, mostly where smoking cannabis is morally wrong.

The nature of the discussed topics well suggests the use of alternative survey modes which may limit the shortcomings of traditional approaches. This idea inspired us to conduct a sensitive research about illegal cannabis use (hereafter identified as the attribute $A$ ) and cannabis legalization (hereafter identified as the attribute $B$ ) using the CM in a face-to-face interview, and compare prevalence estimates with those obtained through direct questioning. It is worth mentioning that many authors have investigated the use of drugs, athletic and cognitive performance-enhancing substances in real life studies using different IQTs. On this see, among others, Cobo et al. (2017) and Perri et al. (2018), and the references within quoted. Instead, to the best our knowledge, no studies using the IQ approach have been produced in the literature on cannabis legalization.

It is reasonable to assume that cannabis use is highly stigmatizing because of the social desirability effect or out of fear of punishment by law. On the other hand, cannabis legalization is reasonably assumed to be a less sensitive issue. Consequently, we expect to get higher estimates for the prevalence of promoters of cannabis legalization than for the prevalence of cannabis use.

4.2 Some data on cannabis use and cannabis legalization in Italy

Cannabis is the most widely consumed illicit drug in the world today, especially by teens, young people and pregnant women. The latest available data collected in Italy for the year 2017 indicate that about one-third (33.1%) of people aged 15–64 years has used cannabis at least once. Cannabis use is more frequent among men (39.1%) than women (26.4%). The prevalence of people who consumed cannabis during the 2017 is 10.2% (12.6% for men and 7.8% for women). The highest prevalence (20.7%) of last-year cannabis use is concentrated among young adults. More detailed information may be found in the 2018 Annual Report to the Parliament on the Use of Drugs in Italy produced by Anti-Drug Policies Department (Dipartimento Politiche Antidroga, 2018, p. 62) set up at the Presidency of the Council of Ministers.1

Although cannabis is considered a soft drug and less damaging to health than hard drugs (e.g. ecstasy and cocaine), in many countries it is forbidden to use, possess, grow or sell cannabis. Nevertheless, in some parts of the world the possession of a “small amount” of cannabis has been decriminalized and use is allowed for medical/religious purposes, or is even legal. Support for making cannabis legal is nowadays increasing around the world. Many people strongly advocate cannabis legalization in the belief that a decriminalization policy would reduce the consumption, particularly among younger age groups, eliminate the illegal trade and associated crime, yield a valuable tax-source to reinvest in welfare and reduce policing costs. In Italy, the percentage of people aged 18 or older who support cannabis legalization has been estimated at 73% by Ipsos Public Affair (2015). According to the opinion poll conduct by the agency SWG in 2014, nearly 46% of interviewees support cannabis legalization or decriminalization, with the peak at 69% observed for the voters of the Democratic Party. Similar findings can be found in a study conducted by Skuola.net2 among about 1500 students aged 11–25, where it is reported that the percentage of those who support legalization is nearly 41% while 40% is not favorable; the remainders do not have a precise opinion in this regard. More recent data released by Eurispes (2019) show that the percentage of Italians who claim to support the legalization of soft drugs is 43.9% (51.1% for men and 36.8% for women), while among young people aged 18–24 the percentage increases up to 63% (61.1% for 25–34 years old).

4.3 The survey plan

A mixed-mode research was conducted in a municipality of about 5,000 inhabitants located in southern Italy with a twofold goal. Firstly, we aimed at simultaneously estimating the prevalence of individuals who have used cannabis at least once in their life and of those who were in favour of its legalization. Secondly, we were interested in evaluating the impact of the interviewer on the DQ survey mode. To serve our purposes, data have been collected on the same people using both the CM and the DQ survey format through face-to-face interviews. The fieldwork was realized by a single interviewer, a qualified and well-instructed final-year master student in Statistics who was well-known by the residents of the municipality. The interviewer recruited a convenience sample of respondents via personal contacts in the two main squares and streets of the municipalities. The interviews were collected during alternating weekends between September 2018–December 2018. On average, each interview took about 20 minutes to be completed. In order to enhance cooperation, contacted people were strongly assured that the collected data would be maintained strictly confidential and would be anonymously used only for scientific purposes by the research group from the University of Calabria which planned the survey. Hence, in advance of data collection, the interviewer explained to the survey participants all the phases of the survey and obtained their informed consent.

Participants were first submitted to a face-to-face interview using a short paper-and-pencil standardized questionnaire containing some generic sociodemographic information about gender, age, education, employment status, marital status and number of children if any. Then, after a collaborative and confidential atmosphere had been established, participants were provided with the two decks of cards needed to run the CM. Finally, DQ was performed.

As described in Section 3, we prepared two decks of cards: Deck-I contained a proportion $p=$ 0.6 of cards marked with the statement: “I have used cannabis at least once in my life” and a proportion $1-p=$ 0.4 of cards with the statement: “I am not in favour of cannabis legalization”; Deck-II contained a proportion $q=$ 0.6 of cards reporting the statement: “I am in favour of cannabis legalization” and a proportion $1-q=$ 0.4 of cards with the statement: “I have never used cannabis in my life”. The interviewer asked the survey participant to perform the CM and report his/her answers in terms of “Yes” or “No”. The interviewer was instructed to check by providing an example – before embarking on the CM – on whether the survey participants fully understood the rules of the randomization device. In the event of doubts or questions raised about the correct execution of the RR experiment, the interviewer was required to explain the rules again and invite the participants to desist if any doubts remained after the third explanation, or if they were unable to implement the technique properly.

After the CM was carried out and the responses collected, the interviewer passed to the third part of the survey by posing the two sensitive questions to the respondents directly:

D1: “Have you ever used cannabis at least once in your life?” D2: “Are you in favour of cannabis legalization?”

In this phase (hereafter DQ1), it is expected that the presence of the interviewer who has established a trusting rapport with the respondents, will encourage them to cooperate and reduce to some extent the embarrassment to answer truthfully. To verify this working hypothesis and evaluate the impact of interviewer on the estimates, DQ was repeated a second time. In so doing, in the fourth stage of the survey (hereafter DQ2), the interviewer collected new responses after posing the following request:

“What would have been your answer to my two previous questions if I had not been a fellow-citizen of yours and you had not known me in advance and/or you had not a trusting rapport with me? Now, please, imagine you never have known me and/or you do not trust in me, and answer to my questions D1 and D2 again”

Under the two stages DQ1 and DQ2, we expect that the results may be significantly different for question D1.

4.3.1 Testing the differences between the CM and DQ survey modes

In order to test the significance of the difference in the prevalence estimates obtained across CM, DQ1 and DQ2, and hence to assess the effect on the respondents of the three survey modes, we have conceived and implemented an “adjusted version” of the McNemar (1947) test for paired data to evaluate CM vs DQ1 and CM vs DQ2. It is worth observing that here the McNemar test is used in its “standard version” to ascertain whether a significant difference exists in the responses of the survey participants under both DQ1 and DQ2 stages. On the contrary, when the CM is used, since survey participants provide misclassified responses, it is not possible to disentangle responses related to attribute $A$ or $B$ and, consequently, in principle it would not be possible to obtain the $2\times 2$ contingency table that summarizes the prevalence of a sensitive attribute and allows us to test the differences of the survey modes. Nonetheless, since responses under DQ1 and DQ2 are available on the same units, we have used this information to obtain a $2\times 2$ contingency table which tabulates respondents who possess the sensitive attribute under the CM and DQ ${}^{*}$ modes, with DQ ${}^{*}=$ DQ1, DQ2.

Our procedure can be summarized in the following steps:

1.
Based on a sample of size $n$ , and starting from the estimated prevalence $\widehat{\pi}_{S,\text{CM}}$ , with $S=A,B,A\cap B,A\cup B$ , compute the marginal totals according to the corresponding survey mode. The resulting $2\times 2$ (empty) contingency table is:

CM

DQ ${}^{}$ $S$ $\bar{S}$ Total

$S$ $n\times\widehat{\pi}_{S,\text{DQ}^{}}$

$\bar{S}$ $n\times(1-\widehat{\pi}_{S,\text{DQ}^{}})$

Total $n\times\widehat{\pi}_{S,\text{CM}}$ $n\times(1-\widehat{\pi}_{S,\text{CM}})$ $n$

2.
Fill the above table with the joint frequencies as follows:

(a)
Derive the contingency table for the CM responses (Yes, Yes), (Yes, No), (No, Yes) and (No, No) from Deck 1 and Deck 2 conditioned to the subsample of respondents bearing the attribute(s) $S$ under DQ ${}^{}$ (hereafter denote this subsample by DQ ${}^{}=$ Yes). Hence, according to the corresponding expression in (1)–(4), compute again the (conditioned) prevalence estimate on the obtained table, say $\widehat{\pi}_{S,\text{CM}|\text{DQ}^{}=\text{Yes}}$ ;
(b)
Compute the number of respondents bearing $S$ under both CM and DQ ${}^{}$ , say $n_{11}$ , as:

$\displaystyle n_{11}=n\times\widehat{\pi}_{S,\text{DQ}^{}}\times\widehat{\pi}% _{S,\text{CM}|\text{DQ}^{}=\text{Yes}}.$
(c)
Complete the contingency table for the McNemar test as follows:

CM

DQ ${}^{}$ $S$ $\bar{S}$ Total

$S$ $n_{11}$ $n\times\widehat{\pi}_{S,\text{DQ}^{}}-n_{11}$ $n\times\widehat{\pi}_{S,\text{DQ}^{}}$

$\bar{S}$ $n\times\widehat{\pi}_{S,\text{CM}}-n_{11}$ $n\times(1-\widehat{\pi}_{S,\text{DQ}^{}}-\widehat{\pi}_{S,\text{CM}})+n_{11}$ $n\times(1-\widehat{\pi}_{S,\text{DQ}^{}})$

Total $n\times\widehat{\pi}_{S,\text{CM}}$ $n\times(1-\widehat{\pi}_{S,\text{CM}})$ $n$

The value of the $\chi^{2}$ McNemar statistic test and the corresponding $p$ -value are reported in the Appendix for the entire sample and for subgroups of it.
5. Results and comments

	CM
DQ ${}^{*}$	$S$	$\bar{S}$	Total
$S$			$n\times\widehat{\pi}_{S,\text{DQ}^{*}}$
$\bar{S}$			$n\times(1-\widehat{\pi}_{S,\text{DQ}^{*}})$
Total	$n\times\widehat{\pi}_{S,\text{CM}}$	$n\times(1-\widehat{\pi}_{S,\text{CM}})$	$n$

	CM
DQ ${}^{*}$	$S$	$\bar{S}$	Total
$S$	$n_{11}$	$n\times\widehat{\pi}_{S,\text{DQ}^{*}}-n_{11}$	$n\times\widehat{\pi}_{S,\text{DQ}^{*}}$
$\bar{S}$	$n\times\widehat{\pi}_{S,\text{CM}}-n_{11}$	$n\times(1-\widehat{\pi}_{S,\text{DQ}^{*}}-\widehat{\pi}_{S,\text{CM}})+n_{11}$	$n\times(1-\widehat{\pi}_{S,\text{DQ}^{*}})$
Total	$n\times\widehat{\pi}_{S,\text{CM}}$	$n\times(1-\widehat{\pi}_{S,\text{CM}})$	$n$

At the end of the data-collection phase, the analyzable cases included 289 participants, aged 16–60 years who correctly gave all the information requested during the face-to-face interview under the three different survey modes, CM, DQ1 and DQ2. With regard to the characteristics of the respondents, gender is almost equally represented (53.3% men against 46.7% women) and about the 40% of respondents are younger than 30 years. Looking at the employment status, about 50% of interviewees are working, 33% do not have a job and 17% are students. Most of the survey participants claimed to have a medium/high level of education (66.4%), while the remainder have a low level (compulsory education). Just over half of the respondents are married/cohabiting and without children (52.3% and 52.6%, respectively).

Table 1 shows point estimates for $\pi_{S}$ , $S=A,B,A\cap B,A\cup B$ , obtained through CM, DQ1 and DQ2 survey modes. Results are given for the entire sample and for certain subgroups of it by sociodemographic characteristics: age, gender, education, employment status, marital status, children. CM estimates are obtained according to expressions (1)–(4) on the basis of the observed frequencies $\widehat{\theta}_{ij}$ reported in Table 2. Some general findings can be drawn from the analysis across CM, DQ1 and DQ1. First, regarding the estimates from the entire sample, the following aspects are worth noting:

Table 1
Point estimates through the CM and DQ survey modes. Symbol $a_{1}$ denotes that the difference $\widehat{\pi}_{S,\text{CM}}-\widehat{\pi}_{S,\text{DQ1}}$ is significant at the 5% level ( $p$ -value $<$ 0.05); $b_{1}$ denotes significance at the 1% level. Similarly, $a_{2}$ and $b_{2}$ refer to the significance of the difference $\widehat{\pi}_{S,\text{CM}}-\widehat{\pi}_{S,\text{DQ2}}$ . Finally $c$ and $d$ denote significance at 5% and 1% levels for the difference $\widehat{\pi}_{S,\text{DQ1}}-\widehat{\pi}_{S,\text{DQ2}}$

		$\widehat{\pi}_{A}$	$\widehat{\pi}_{B}$	$\widehat{\pi}_{A\cap B}$	$\widehat{\pi}_{A\cup B}$
Sample	CM	0.471 ${}^{b_{2}}$	0.685 ${}^{a_{1},a_{2}}$	0.381 ${}^{b_{2}}$	0.775 ${}^{b_{1},b_{2}}$
( $n=$ 289, 100%)	DQ1	0.464 ${}^{d}$	0.664	0.391 ${}^{d}$	0.737 ${}^{d}$
	DQ2	0.280	0.654	0.253	0.682
Age: 16–30	CM	0.544 ${}^{b_{2}}$	0.763 ${}^{b_{1},b_{2}}$	0.516 ${}^{a_{1},b_{2}}$	0.791 ${}^{b_{2}}$
( $n=$ 114, 39.5%)	DQ1	0.544 ${}^{d}$	0.684	0.456 ${}^{d}$	0.772 ${}^{d}$
	DQ2	0.263	0.667	0.237	0.693
Age: 31–60	CM	0.423 ${}^{b_{2}}$	0.634	0.292 ${}^{b_{1}}$	0.765 ${}^{b_{1},b_{2}}$
( $n=$ 175, 60.5%)	DQ1	0.411 ${}^{d}$	0.651	0.349 ${}^{d}$	0.714 ${}^{c}$
	DQ2	0.291	0.646	0.263	0.674
Gender: Male	CM	0.507 ${}^{b_{1},b_{2}}$	0.591 ${}^{b_{1},b_{2}}$	0.367 ${}^{b_{1}}$	0.730 ${}^{b_{1}}$
( $n=$ 154, 53.3%)	DQ1	0.584 ${}^{d}$	0.721	0.487 ${}^{d}$	0.818 ${}^{d}$
	DQ2	0.377	0.714	0.331	0.760
Gender: Female	CM	0.430 ${}^{b_{1},b_{2}}$	0.793 ${}^{b_{1},b_{2}}$	0.396 ${}^{b_{1},b_{2}}$	0.826 ${}^{b_{1},b_{2}}$
( $n=$ 135, 46.7%)	DQ1	0.326 ${}^{d}$	0.600	0.282 ${}^{d}$	0.644 ${}^{c}$
	DQ2	0.170	0.585	0.163	0.593
Education: Low	CM	0.526 ${}^{b_{1},b_{2}}$	0.629 ${}^{b_{1},b_{2}}$	0.444 ${}^{b_{1},b_{2}}$	0.711 ${}^{b_{1},b_{2}}$
( $n=$ 97, 33.6%)	DQ1	0.392 ${}^{d}$	0.516	0.320 ${}^{d}$	0.588 ${}^{d}$
	DQ2	0.124	0.495	0.124	0.495
Education: Medium/High	CM	0.443 ${}^{a_{1},b_{2}}$	0.714	0.349 ${}^{b_{1}}$	0.808
( $n=$ 192, 66.4%)	DQ1	0.500 ${}^{d}$	0.740	0.427 ${}^{d}$	0.813 ${}^{c}$
	DQ2	0.359	0.734	0.318	0.776
Employment status: Working	CM	0.611 ${}^{b_{1},b_{2}}$	0.806 ${}^{b_{1},b_{2}}$	0.502 ${}^{b_{1},b_{2}}$	0.915 ${}^{b_{1},b_{2}}$
( $n=$ 144, 49.8%)	DQ1	0.535 ${}^{d}$	0.708	0.438 ${}^{d}$	0.806 ${}^{c}$
	DQ2	0.375	0.708	0.333	0.750
Employment status: Other	CM	0.331 ${}^{a_{1},b_{2}}$	0.566 ${}^{a_{1}}$	0.260 ${}^{b_{1},a_{2}}$	0.637
( $n=$ 145, 50.2%)	DQ1	0.399 ${}^{d}$	0.629	0.350 ${}^{d}$	0.678 ${}^{c}$
	DQ2	0.189	0.608	0.175	0.622
Marital status: Married/Cohabiting	CM	0.351 ${}^{a_{2}}$	0.682	0.278	0.756 ${}^{b_{1},b_{2}}$
( $n=$ 151, 52.2%)	DQ1	0.371 ${}^{d}$	0.649	0.325 ${}^{d}$	0.695 ${}^{c}$
	DQ2	0.252	0.642	0.245	0.649
Marital status: Other	CM	0.601 ${}^{b_{2}}$	0.688	0.493 ${}^{b_{2}}$	0.797 ${}^{b_{2}}$
( $n=$ 138, 47.8%)	DQ1	0.565 ${}^{d}$	0.681	0.464 ${}^{d}$	0.783 ${}^{d}$
	DQ2	0.312	0.667	0.261	0.717
Children: Yes	CM	0.358 ${}^{b_{2}}$	0.679 ${}^{a_{1},a_{2}}$	0.292 ${}^{a_{2}}$	0.745 ${}^{b_{1},b_{2}}$
( $n=$ 137, 47.4%)	DQ1	0.329 ${}^{d}$	0.628	0.292 ${}^{d}$	0.664
	DQ2	0.234	0.620	0.226	0.628
Children: No	CM	0.572 ${}^{b_{2}}$	0.691	0.461 ${}^{b_{2}}$	0.803 ${}^{b_{2}}$
( $n=$ 152, 52.6%)	DQ1	0.586 ${}^{d}$	0.697	0.480 ${}^{d}$	0.803 ${}^{d}$
	DQ2	0.322	0.684	0.276	0.730

Table 2

Proportion of observed responses for the CM

		$\widehat{\theta}_{00}$	$\widehat{\theta}_{01}$	$\widehat{\theta}_{10}$	$\widehat{\theta}_{11}$
Sample		0.142	0.235	0.450	0.173
Age	16–30	0.158	0.202	0.421	0.219
	31–60	0.131	0.257	0.469	0.143
Sex	Male	0.156	0.292	0.377	0.175
	Female	0.126	0.170	0.533	0.170
Education	Low	0.175	0.258	0.361	0.206
	Medium/High	0.125	0.224	0.495	0.156
Employment status	Working	0.111	0.250	0.444	0.194
	Other	0.172	0.221	0.455	0.152
Marital status	Married/Cohabiting	0.132	0.199	0.530	0.139
	Other	0.152	0.275	0.362	0.210
Children	Yes	0.139	0.197	0.518	0.146
	No	0.145	0.270	0.388	0.197

the ascertainment that $\widehat{\pi}_{A}<\widehat{\pi}_{B}$ and that $\widehat{\pi}_{B}$ does not significantly change across the three survey modes, confirms our expectation mentioned in Section 4.1, i.e. the fact that question D1 about cannabis use is certainly more sensitive than question D2 about cannabis legalization. Cannabis use and cannabis legalization exhibit two different levels of sensitivity that lead the survey participants to follow different response patterns;

although cannabis use denotes a stigmatizing and illegal behavior, estimates for $\pi_{A}$ seem not to be affected by the presence of the interviewer having a trusting rapport with the respondents. In fact, no significant difference stands out when comparing $\widehat{\pi}_{A}$ under CM and DQ1 (47.1% vs 46.4%). On the contrary, the scenario is completely different under DQ2. In this case, the estimated percentage of people who used cannabis is around 28%, a value which is significantly much more lower than the estimates under DQ1 and CM. Note, however, that the estimated percentage is not so different from the last available estimate in Italy at 33.1% (see Section 4.2), a value that can be assumed as a benchmark for validation purposes. Due to the highly sensitive nature of the investigated topic, the large gap between the estimate of $\pi_{A}$ under CM and DQ2 is not surprising and well meets the “more-is-better” assumption. Quite astonishing, instead, is the large difference in $\widehat{\pi}_{A}$ under DQ1 and DQ2 which confirms that trust in the interviewer may considerably encourage respondents to cooperate and answer truthfully. Finally, note that the ascertained underestimation of $\pi_{A}$ under DQ2 exerts an effect on the estimates $\widehat{\pi}_{A\cap B}$ and $\widehat{\pi}_{A\cup B}$ which, although to a lesser extent, are still lower than the estimates obtained under DQ1 and CM;

estimates for $\widehat{\pi}_{B}$ are in line with the results released by IPSOS (73%), the Italian public opinion poll agency mentioned in Section 4.2. This evidence, on a first assessment, may be roughly used to validate the CM. Hence, if we assume that the three methods produce reliable results for the prevalence of people who support cannabis, we can reasonably expect that reliable analyses can also be carried out for cannabis use. We are, therefore, confident that valid estimates have also been obtained for $\widehat{\pi}_{A}$ under CM, DQ1 and DQ2.

Similar findings may be recognized for subgroups of the population although, due to the reduced sample size and the extra variability induced by the CM, some results may appear, in a certain sense, unexpected. For instance, in the group of men, the estimated percentage of cannabis users is lower under CM (50.7%) than under DQ1 (58.4%). Similarly, for the prevalence of those in favour of cannabis legalization under CM (59.1%), DQ1 (72.1%) and DQ2 (71.4%). The latter outcome is perhaps not so surprising since favoring cannabis legalization may denote positive and desirable personal traits, such as an open mind and modern social views. Consequently, some subgroups of the population, contrary to underreporting of cannabis use, may overreport this attitude when it is surveyed by DQ and produce misleading higher estimates. In this case, the CM estimates may provide a more reliable picture of the reality and the “less-is-better” assumption could be applied to validate the CM.

Looking at the estimates obtained using the CM, we observe that cannabis use is more frequently reported by men (50.7%) than women (43%), and that cannabis consumers are more concentrated among people with a job (61.1%). In the same category, not surprisingly, we also find the highest estimated percentage of people who declare to support cannabis legalization (80.6%). A high percentage of cannabis users (54.4%) is found in the age group 16–30 years confirming the tendency that emerges from official statistics according to which the highest prevalence of those who have used cannabis in the previous 12 months is concentrated in young people (see Section 4.2). People with children and with a more stable relation (married/cohabiting) seem to be less prone to cannabis use.

6. Conclusions

In this paper, we discussed the differences in the estimates of the prevalence of two sensitive attributes when different questioning survey modes are employed. The survey design was conceived as a mixed-mode research which allowed us to compare both the DQ and IQ approaches in the presence of the interviewer for the simultaneous estimation of the prevalence of cannabis users and the prevalence of people supporting its legalization.

The results of the study highlight some important issues. First, significant improvements in the prevalence estimates can be achieved when the IQ mode is used rather than questions asked by the interviewer directly. In particular, in almost all considered cases, the randomized response CM led to higher estimates than DQ1 and DQ2 and, according to the “more is better” assumption, we can conclude that IQ worked better than direct mode. Large differences were found especially when comparing CM and DQ2.

Moreover, we intentionally considered two correlated sensitive issues with different levels of sensitivity that, as expected, induced survey participants to behave differently when responding. Differences in the prevalence estimates due to sensitivity can be recognized across the three survey modes employed and, therefore, the idea that respondents were more incline to truthfully answer to questions that they perceived as less stigmatizing is confirmed.

Finally, the results demonstrated that the presence of a trustworthy interviewer plays an important role in enhancing respondent cooperation. Comparison of prevalence estimates through DQ1 and DQ2 suggested that, when the DQ mode is performed, the estimation process can be significantly improved if a trusting rapport is established between interviewer and respondent.

To conclude, we would like to remind the reader to be cautious about making too-strong conclusions from the results here discussed since the study involved a convenience sample and tested different working hypotheses on the same set of data. Future research could certainly give a deeper insight on the investigated matters.

Footnotes

See also http://www.emcdda.europa.eu/publications/country-drug-reports/2018/italy.

See http://www.skuola.net/news/scuola/droghe-leggere-legalizzazione.html.

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Appendix

References

Chaudhuri

(2011). Randomized Response and Indirect Questioning Techniques in Surveys. Chapman & Hall/CRC, Boca Raton, FL.

Chaudhuri

, & Christofides

(2013). Indirect Questioning in Sample Surveys. Springer, Berlin.

Chaudhuri

, & Mukerjee

(1988). Randomized Response: Theory and Techniques. Marcel Dekker, Inc., New York.

Chaudhuri

Christofides

T.C.

, & Rao

C.R.

(2016). Handbook of Statistics 34 – Data Gathering, Analysis and Protection of Privacy through Randomized Response Techniques: Qualitative and Quantitative Human Traits. Vol. 34. Elsevier, Amsterdam.

Cobo

Rueda

M.M.

, & López-Torrecillas

(2017). Application of randomized response techniques for investigating cannabis use by Spanish university students. International Journal of Methods and Psychiatric Research, 26, Article Number: e1517.

Dickson-Swift

James

E.L.

, & Liamputtong

(2008). Undertaking Sensitive Research in the Health and Social Sciences. Managing Boundaries, Emotions and Risks. Cambridge University Press, New York.

Dipartimento Politiche Antidroga (2018). Relazione Annuale al Parlamento sul Fenomeno delle Tossicodipendenze in Italia anno 2018 (dati 2017). Retrieved from http://www.politicheantidroga.gov.it/media/2445/339911.pdf.

Ewemooje

O.S.

Amahia

G.N.

, & Adebola

F.B.

(2017). Estimating prevalence of induced abortion and multiple sexual partners using improved randomized response technique for two sensitive attributes. Communications in Statistics: Case Studies, Data Analysis and Applications, 3, 21-28.

Eurispes (2019). Rapporto Italia 2019. Retrieved from https://eurispes.eu/ricerca-rapporto/rapporto-italia-2019/.

10.

Fox

J.A.

(2016). Randomized Response and Related Methods. Surveying Sensitive Data. Sage Publication, Inc., Thousand Oaks.

11.

Fox

J.A.

, & Tracy

P.E.

(1986). Randomized Response: A Method for Sensitive Survey. Sage Publication, Inc., Newbury Park.

12.

Groves

R.M.

Fowler

F.J.

Couper

M.O.

Lepkowski

J.M.

Singer

, & Tourangeau

(2004). Survey Methodology. John Wiley & Sons, Inc., Hoboken, New Jersey.

13.

Harrison

, & Hughes

(1997). The validity of self-reported drug use: improving the accuracy of survey estimates (NIH Publication No. 97-4147, NIDA Research Monograph 167). National Institute on Drug Abuse, Rockville, MD.

14.

Ipsos Public Affair (2015). Le opinioni degli italiani sulle leggi in materia di droghe leggere (in Italian). Retrieved from http://www.cannabislegale.org/wp-content/uploads/2015/07/opinioni-italiani-leggi-droghe-leggere.pdf.

15.

Lee

C.-S.

Sedory

S.A.

, & Singh

(2013). Estimating at least seven measures of qualitative variables from a single sample using randomized response technique. Statistics & Probability Letters, 83, 399-409.

16.

Liamputtong

(2007). Researching the Vulnerable. A Guide to Sensitive Research Methods. Sage Publications, London.

17.

McNemar

(1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12, 153-157.

18.

Perri

P.F.

Pelle

, & Stranges

(2016). Estimating induced abortion and foreign irregular presence using the randomized response crossed model. Social Indicators Research, 129, 601-618.

19.

Perri

P.F.

Cobo

, & Rueda

M.M.

(2018). A mixed-mode sensitive research on cannabis use and sexual addiction: Improving self-reporting by means of indirect questioning techniques. Quality & Quantity, 52, 1593-1611.

20.

Tian

G.-L.

, & Tang

M.-L.

(2014). Incomplete Categorical Data Design: Non-randomized Response Techniques for Sensitive Questions in Surveys. Chapman & Hall/CRC., Boca Raton, FL.

21.

Tourangeau

, & Yan

(2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859-883.

22.

Tourangeau

, & Smith

T.W.

(1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275-304.

23.

Warner

S.L.

(1965). Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60, 63-69.