A trial of questionnaire administration modalities for measures of sexual risk behaviour in the uniformed services of Peru

Abstract

Summary

Modalities of questionnaire administration may affect data quality, particularly when conducting Biological and Behavioral Surveillance Surveys (BBSS) of uniformed personnel. We aimed to examine differences in administration, reporting and data quality across several common modalities of administration for BBSS endeavors. Prior to a large-scale BBSS endeavor with the uniformed services of Peru, we pilot tested three modes of questionnaire administration among personnel engaged in internal combat: face-to-face interview (FFI), self-administered paper-based interview (SAPI), and audio computer-assisted self-interview (ACASI). Individuals who took the survey using ACASI were less likely to have missing data on measures of sexual risk and alcohol abuse and were more likely to report sexual risk behaviours and symptoms of alcohol abuse; however, more individuals took the survey using SAPI given inadequate time to devote to sitting through an entire FFI or ACASI. Sexually transmitted infections did not vary significantly across modes of questionnaire administration. While more logistically complicated for BBSS efforts in resource-constrained settings, we recommend the use of ACASI in collecting BBSS data from uniformed personnel if conditions are permissible.

Keywords

HIV AIDS sexually transmitted infections prevention,sexual risk alcohol abuse interview questionnaire audio computer-assisted self-interview ACASI uniformed services military personnel surveys Peru surveillance

INTRODUCTION

Biological and Behavioral Surveillance Surveys (BBSS) concerning HIV are often used to inform and evaluate HIV prevention programmes. However, the mode of questionnaire administration for the behavioural component of the BBSS endeavor is rarely considered, and may impact study findings.^1,2 Questionnaires are naturally subject to error, and different questionnaire administration modalities exhibit various strengths and weaknesses. Prior to conducting a large-scale BBSS of the uniformed services of Peru, we pilot-tested three modes of questionnaire administration in uniformed personnel stationed in the Ayacucho region. The modalities included face-to-face interview (FFI), self-administered paper-based interview (SAPI) and audio computer-assisted self-interview (ACASI).

Social desirability bias is a proclivity of the respondent to answer a question based on his/her perception of how the response will be received. Previous studies of questionnaire administration modalities have documented that social desirability bias is a factor that may be somewhat mitigated by the use of ACASI when screening patients for health risk behaviours.^{3
–5} However, studies have generally focused on sexually transmitted infection (STI) clinic patients and other at-risk populations such as drug users.^{4

–9} In population-based studies of military personnel, which may include subsets of high-risk individuals, it is important to better understand which modes of questionnaire administration best reduce social desirability bias. It is also important to consider findings in light of the feasibility and resource constraints, particularly for BBSS endeavors in developing nations.

The uniformed services are recognized as an at-risk population for HIV/AIDS and other STIs.^2,10 This is in part due to the ways in which security institutions and contextual insecurity shapes patterns of HIV risk, and that HIV prevention efforts have been poorly integrated with components of security institutions, such as security sector reforms, humanitarian assistance, and peace-keeping and peace-building efforts.¹¹

Here, we capitalized on a unique opportunity to randomize active duty uniformed personnel to one of three questionnaire administration modalities in the Ayacucho region of Peru during an internal armed conflict. We examine the survey administration process, the prevalence of self-reported risk behaviours and data quality across FFI, ACASI and SAPI questionnaire administration modalities, and we discuss the advantages and disadvantages of their utilization in the uniformed forces of Peru and for BBSS endeavors in uniformed personnel worldwide.

METHODS

From September to October 2010, uniformed personnel (army and police) in the Ayacucho region of Peru were asked to participate in a behavioural questionnaire and serological tests for HIV and other STIs. The study received ethics review board approval from the IRB of NAMRU-6 in Lima, Peru.

A convenience sample was collected from six sites (2 army bases and 4 police bases/stations). At each base, the study team provided potential respondents with a group talk on HIV/AIDS and sexual risk reduction. Potential participants were informed that their responses would be private and that data would be analysed in a de-identified manner. Following the talk, respondents who wished to participate in the study were randomized to one of three modes of behavioural questionnaire administration and a blood draw. All study participants provided free and informed consent to participate in the study following the group talk.

The study design called for subjects to be randomized to one of three modalities of questionnaire administration: FFI, SAPI or ACASI. Subjects were assigned a questionnaire mode of administration based upon the order in which they consented to participate. While this process does not insure complete randomization, it was determined to be the most feasible method given the limited resources, limited time of respondents and site-specific sampling criteria. For FFI, five trained interviewers verbally administered a questionnaire to respondents. The interviewers had experience in HIV/STI prevention counselling, and were trained to administer surveys prior to the study. For SAPI, the questionnaire included seven pages printed on 5 × 10” white paper. Respondents were given a location in which to take the questionnaire, and in cases where respondents had to be called into duty during the questionnaire, they were permitted the ability to return to the testing site to complete the SAPI. Individuals who did not have adequate time to complete a survey but who were willing to participate were automatically given the SAPI. For ACASI, Questionnaire Development System (QDS) software was installed onto tablet netbook computers and was used to administer questions to respondents using a pre-recorded female voice to read questions aloud to respondents through headphones. ACASI respondents provided answers to each question by using a stylus pen to touch responses on the netbook screens, and individual questions were separated by 500-millisecond pauses between screens following a response.

Behavioural measures

The survey instrument included a brief section for demographic measurements, as well as measures of sexual risk behaviour and alcohol abuse.

Measurement for sexual risk behaviours included items derived from Demographic and Health Surveys (DHS) and questions adapted from the Risk Behavior Assessment (RBA).¹² Respondents were asked if in the last 30 days they had sex while drunk or high (0 = no sex drunk or high, 1 = sex while drunk or high), whether they used a condom during sex while drunk or high (0 = used condom, 1 = did not use condom), and the number of sexual partners in a 30-day period (0 = 1 or less, 1 = more than 1 sexual partner). In addition to questions with a 30-day recall period, respondents were asked whether they currently had a casual sexual partner, whether they have had sex with a commercial sex worker (CSW) during their lifetime, and whether condoms were used during the last sexual contact with a casual sexual partner or commercial sex worker.

We measured alcohol abuse using the Rapid Alcohol Problems Screen 4-Quantity Frequency (RAPS4-QF), a short screening instrument which has shown high sensitivity to detect alcohol abuse.¹³ We classified respondents as having probable alcohol abuse for a score of 1 or greater on the 6-item RAPS4-QF, and probable alcohol dependency for a score of 1 or greater on the four-item RAPS4.¹³

Serological measures

To cross-validate any observed differences in reported behaviour across modes, we also compared serological data across modes. Respondents were asked to provide approximately 5 mL of blood prior to taking an interview. Respondent samples were tested for HIV, syphilis, chlamydia, gonorrhoea, herpes simplex virus 2, Human T-lymphotropic virus Type 1 and 2 (HTLV 1/2) and hepatitis B. Respondents screening positive to any one STI were classified as having an STI in comparison to those who did not screen positive for an STI.

Data analysis

All data were uploaded into Microsoft Excel^® and analysed using STATA 10^® (StataCorp LP, College Station, TX, USA) statistical software.¹⁴ We reviewed simple univariate frequencies by mode. To test for differences in missing data by mode, we created dichotomous variables to represent whether data were missing or not missing. We conducted Pearson chi-squared tests to determine whether behaviours and missing data (both scored dichotomously) varied across the three modalities examined, and we used analysis of variance to determine whether age (a continuous variable) varied significantly across modalities. We used logistic regression to estimate the odds ratio (OR) for outcomes among ACASI respondents in comparison to respondents taking the survey by FFI or SAPI (the reference group), and to control for potential confounders in order to estimate the adjusted odds ratio (aOR) for outcomes.

RESULTS

In total, 382 consented and completed a questionnaire and blood test. Fifty respondents completed the questionnaire by FFI, 256 by SAPI and 76 by ACASI. An initial review of the data suggests that individual demographics varied significantly by mode. Respondents completing the interview by FFI were more often male (94%), single (76%) and reported lower education (less than high school completion) (44%), while individuals taking the SAPI were of greater age (mean = 31), and individuals taking the ACASI were more often in the police (43%) (Table 1).

Table 1

Demographic and military characteristics of study participants by mode of questionnaire administration

	Respondent characteristics					Missing data
		FFI	SAPI	ACASI		FFI	SAPI	ACASI
	N	(n = 50)	(n = 256)	(n = 76)	P value*	(n = 50)	(n = 256)	(n = 76)	P value*
Male	337	94.0%	76.3%	84.2%	0.012	0.0%	17.6%	0.0%	0.012
Age (SD)	237	24.5 (9.5)	31.3 (10.2)	29.1 (9.9)	<0.001	16.0%	53.5%	0.0%	<0.001
Single	331	75.5%	40.8%	51.3%	<0.001	2.0%	19.5%	0.0%	<0.001
Officer	331	4.3%	13.9%	15.8%	0.145	6.0%	18.8%	0.0%	<0.001
Police	336	24.0%	42.4%	43.4%	0.045	0.0%	18.0%	0.0%	<0.001
Low education	333	43.8%	6.7%	10.5%	<0.001	4.0%	18.4%	0.0%	<0.001

ANOVA = analysis of variance; FFI = face-to-face interview; SAPI = self-administered paper-based interview; ACASI = audio computer-assisted self-interview;

*P values were derived from chi-squared tests and ANOVA for age (a continuous variable)

There was significant variation in missing data between both demographic and behavioural variables by mode of questionnaire administration. There was a notably larger amount of missing demographic data among SAPI respondents, and in comparison to both FFI and SAPI, individuals who were administered the ACASI exhibited a significantly lower level of missing data (Table 1).

Administration

During study implementation, normal circumstances typical to BBSS endeavors with uniformed personnel arose which impeded a perfect scientific randomization process. This included several instances where electricity was not readily available on base (making ACASI infeasible), and thus subjects were assigned to either FFI or SAPI modalities. The majority of subjects noted that they could not take the interview in one sitting due to occupational duties, and were thus accordingly assigned to the SAPI mode. For this reason, a larger number of subjects completed the interview by SAPI (n = 256) in comparison to ACASI (n = 76) and FFI (n = 50).

Missing data

Missing data on high-risk behavioural items were most prevalent among FFI and SAPI respondents. The likelihood of missing data was generally lowest among ACASI respondents in contrast to the other two modes (Table 2). A particularly high frequency of missing data was observed in SAPI and FFI respondents for having sex while drunk or high (>20% missing), and alcohol abuse (>35% missing in both FFI and SAPI). Among ACASI respondents, missing data for a given item never exceeded 3% for the items tested.

Table 2

Self-reported behaviours and missing data comparing FFI, SAPI and ACASI

Outcomes	FFI (%)	SAPI (%)	ACASI (%)					Adjusted model*
Outcomes	(n = 50)	(n = 256)	(n = 76)	P ^†	OR^‡	95% CI	P	N	aOR	95% CI	P
Tested positive for an STI^§	24.0	27.7	18.4	0.254	0.61	(0.3–1.1)	0.121	221	0.46	(0.2–0.9)	0.030
Sex with a casual partner	26.7	35.6	62.2	<0.001	3.20	(1.9–5.5)	<0.001	191	3.57	(1.9–6.7)	<0.001
Missing	10.0	26.6	2.6	<0.001	0.09	(0.02–0.4)	0.001
Condom use-last sex with casual partner	63.6	56.3	56.5	0.897	0.97	(0.5–2.0)	0.930	81	0.83	(0.3–2.3)	0.718
Missing	8.0	4.5	0.0	<0.001	–	–	–
Ever had sex with a CSW	48.8	45.5	50.0	0.773	1.17	(0.7–2.0)	0.553	190	1.04	(0.6–1.9)	0.890
Missing	14.0	22.7	2.6	<0.001	0.10	(0.02–0.4)	0.002
Condom use during last sex with a CSW	85.0	90.7	18.9	0.000	0.03	(0.01–0.08)	<0.001	84	0.01	(0.001–0.1)	<0.001
Missing	4.8	16.7	0.0	0.014	–	–	–
>1 sexual partner in last 30 days	16.3	12.7	16.2	0.681	1.26	(0.6–2.6)	0.530	198	1.60	(0.7–3.8)	0.284
Missing	14.0	17.2	2.6	0.006	0.14	(0.03–0.6)	0.006
Sex while drunk/high	31.6	34.5	37.8	0.787	1.18	(0.7–2.0)	0.549	192	1.27	(0.7–2.4)	0.448
Missing	24.0	21.9	2.6	<0.001	0.09	(0.02–0.4)	0.001
Used condom during sex while drunk/high	40.9	35.3	32.4	0.757	0.86	(0.5–1.5)	0.595	165	0.81	(0.4–1.6)	0.550
Missing	0.0	2.9	0.0	<0.001	0.96	(0.01–0.2)	0.000
Alcohol abuse	86.2	87.4	98.6	0.000	10.7	(1.4–80.5)	0.022	165	17.76	(1.8–171.8)	0.013
Missing	42.0	37.9	2.6	<0.001	0.04	(0.01–0.2)	0.000

FFI = face-to-face interview, SAPI = self-administered paper based interview, ACASI = audio computer-assisted self interview; CSW = commercial sex worker

*Logistic regression models have been adjusted for the effects of gender, age, relationship status, education, and police occupation

^†Probability values are derived from a Pearson chi-squared test across three modes of questionnaire administration

^‡Odds ratios are derived from a logistic regression for ACASI respondents in comparison to FFI and SAPI respondents (combined as the reference group)

^§STI tests included HIV, syphilis, gonorrhoea, chlamydia, syphilis, herpes, HTLV and hepatitis B

High risk behaviours

ACASI respondents typically reported higher levels of engagement in behaviour with higher risk sexual partners, and a lower level of condom use with specific partners (Table 2). In particular, ACASI respondents were significantly more likely to report sex with a casual sexual partner (OR = 3.2, 95% confidence interval [CI] 1.9–5.5, P < 0.001), were less likely to report condom use during last sexual contact with a CSW (OR = 0.03, 95% CI 0.01–0.08, P < 0.001), and were more likely to report symptoms indicative of probable alcohol abuse (OR = 10.7, 95% CI 1.4–80.5, P = 0.02). After adjusting for potential confounders (which affected the size of the analytic sample), the direction and statistical significance of effects did not notably change.

Serological markers

In total, 25.4% tested positive for an STI. The prevalence of STIs did not vary significantly across modalities (P = 0.25); however, the lowest prevalence of STIs was observed for ACASI respondents (18%) in comparison with FFI respondents (24%) and SAPI respondents (28%). After adjusting for potential confounders, ACASI responders were less likely to test positive for an STI (aOR = 0.46, 95% CI 0.2–0.9, P = 0.03).

DISCUSSION

In this study, we piloted three modes of behavioural questionnaire administration within a sample of uniformed personnel in Peru. We found that high-risk behaviours were more likely to be reported among ACASI respondents, and that missing data were lowest among ACASI respondents. These behavioural differences were likely a result of reporting bias as evidenced by discordant rates of STIs across modes. SAPI was the most feasible mode by which to administer the questionnaire given resource constraints and time available to respondents. These findings have several implications for upcoming BBSS endeavors with uniformed personnel in Peru and Latin America, that are subject to the same ethics review guidelines and requirement that subjects participate voluntarily.

Respondents taking the survey under the ACASI mode showed the most favorable results for a BBSS endeavor. While ACASI respondents were not more likely to engage in sexual risk behaviours, we documented evidence that ACASI responders were more likely to report high-risk behaviours. This was supported given that although more sexual risk behaviours were reported under the ACASI mode, the serological prevalence of STIs was the lowest among ACASI respondents. In addition, the prevalence of missing data was substantially lower across items for ACASI respondents. Multivariable findings adjusted for potential confounders confirmed these associations; however, aORs should be interpreted with caution given the serious reduction in the floating sample sizes that resulted from the missing data patterns, which varied by covariates and mode (Table 2).

While the FFI mode may be a favourable mode as it allows for interviewers to probe, to explain misunderstood concepts to respondents, and to readily interview participants with limited literacy, there may be unobserved interviewer biases that occur during the FFI interview process, particularly with uniformed personnel.¹ Anecdotally, the study team received comments about discomfort from FFI interviewers, who intentionally censored some of the sensitive questions when interviewing respondents of rank. Further, the team reported that respondents were intentionally skipping (or refusing to answer) several questions. This phenomenon may be amplified in the uniformed services setting, where respondents may take additional measures to make their survey void of demographic identifiers and responses to sensitive questions, despite respondents having undergone a thorough consent process where measures of privacy and data security are explicitly described to respondents. Although the SAPI mode was the most feasible to administer in the field, particularly given that it readily allows for interruption of uniformed personnel who are taking a survey (a very realistic condition during BBSS endeavors with active duty uniformed personnel), the preponderance of missing data arguably invalidates the reported prevalence of particular behaviours. Furthermore, not completing a questionnaire in one sitting and returning to a questionnaire site after a work-related event may have unobserved effects on responses provided in a questionnaire.

Our findings concur with other tests of behavioural questionnaire administration modalities in non-uniformed populations. We found that ACASI respondent were slightly more likely to report transactional sex and lack of condom use during transactional sex, which is similar to findings of ACASI responders as seen in the clinical setting.¹⁵ However, we also found that missing data were least prevalent on the ACASI mode, in comparison to a study where FFI was shown to elicit less missing data.³ However, our use of lay interviewers during a BBSS endeavor versus using health-care personnel as interviewers as was used in the clinical setting must likely accounts for the difference. Furthermore, despite training the data collection team for this pilot, it is possible that interview subjects were less likely to want to provide demographic data to an interviewer for fear of having one's results easily identified. In our study, the scope of skipping questions was limited in the ACASI modality, thereby reducing missing data for this mode.

In circumstances where time and electricity are available and where literacy is not an issue, ACASI can be used to effectively complete data collection with uniformed personnel in developing countries. A similar and more extensive ACASI was administered to the Belize Defense Force for their 2009–2010 BBSS endeavor.² In the present study, rates of high-risk behaviour reported among ACASI respondents were similar to those observed in the Belize Defense Force.² The availability of more time and resources to complete BBSS endeavors may allow for effective ACASI implementation within the uniformed armed forces and police.

Limitations

This study has several limitations. First, subjects were assigned to only one mode of questionnaire administration and thus repeated measures are not available to determine whether one mode would elicit more or less information from a single respondent. In addition, a key finding was that missing data and demographic covariates varied by mode, and this may confound the estimated effects of mode on reporting high-risk behaviours. Our ability to control for systematic differences in multivariable analyses was thus compromised by the degree of missing data by mode (resulting in floating sample sizes), although the multivariable analyses did concur with the findings in the full sample. A more rigorous design was not feasible in this population given frank limitations on personnel time, particularly in armed conflict/high-alert settings. Second, the randomization procedure was compromised by the limited amount of time available for certain personnel and/or the availability of electricity on a base. These limitations suggest that researchers should plan for such environmental limitations to the maximum extent possible by carrying power generators for the ACASI and/or surveying personnel during times at which larger spans of time are available (e.g. lunch breaks). Third, the study occurred after a group talk, and may have increased social desirability bias in risk behaviour reporting among participants. Finally, respondents were obtained from a convenience sample, and results are thus not representative. However, the internal validity of the associations detected may be similar for the Peruvian Armed Forces as a whole, or for other uniformed populations where active conflict is an issue.

CONCLUSION

In this study, ACASI responders had a lower prevalence of missing data and reported higher levels of risk behaviour, while not showing elevated STI serologies. Our findings suggest that ACASI is a favourable questionnaire administration mode for reducing social desirability bias and improving data quality in BBSS endeavors with uniformed personnel. If conditions are permissible, we recommend the use of ACASI for collecting questionnaire data in future BBSS studies of uniformed personnel.

Footnotes

ACKNOWLEDGMENTS

Authors express their recognition and gratitude to COPRECOS-PERU Team for their hard work, experience and support during the field work. Special thanks are extended to the participants for making this study possible. Support for this work was provided by the Department of Defense HIV/AIDS Prevention Program. The opinions or assertions contained herein are the private views of the authors, and are not to be construed as official, or as reflecting the views of the Department of Defense HIV/AIDS Prevention Program, Department of the Navy, nor the US. Government. The authors have no commercial or other association that might pose a conflict of interest. HC and SM are employees of the US Government. This work was prepared as part of their official duties. Title 17 U.S.C. §105 provides that ‘Copyright protection under this title is not available for any work of the United States Government.’ Title 17 U.S.C. §101 defines a US Government work as a work prepared by a military service member or employee of the US Government as part of that person's official duties.

References

Chun

Tavarez

Dann

Anastario

. Interviewer gender and self-reported sexual behavior and mental health among male military personnel. Int J Public Health 2011;56:225–9

Anastario

Manzanero

Blanco

HIV infection, sexual risk behaviour and condom use in the Belize defense force. Int J STD AIDS 2011;22:73–9

Ghanem

Hutton

Zenilman

Zimba

Erbelding

. Audio computer assisted self interview and face to face interview modes in assessing response bias among STD clinic patients. Sex Transm Infect 2005;81:421–5

Turner

Des Jarlais

Newman

Paone

Cooley

Gribble

. The differential effects of face-to-face and computer interview modes. Am J Public Health (1971) 2002;92:294–7

Celentano

Macalino

Vlahov

Strathdee

Latkin

. Risk behaviors by audio computer-assisted self-interviews among HIV-seropositive and HIV-seronegative injection drug users. AIDS Educ Prev 2002;14:367–78

Des Jarlais

Paone

Milliken

Audio-computer interviewing to measure risk behaviour for HIV among injecting drug users: a quasi-randomised trial. Lancet 1999;353:1657–61

Lynch

Simoes

Bastos

Metzger

Moreira

. A randomized trial of audio computer and in-person interview to assess HIV risk among drug and alcohol users in Rio De Janeiro, Brazil. J Subst Abuse Treat 2006;30:237–43

Signes

Williams

Gordon

A comparison of the reliability of self-reported drug use and sexual behaviors using computer-assisted versus face-to-face interviewing. AIDS Educ Prev 2000;12:199–213

Simoes

Bastos

Moreira

Lynch

Metzger

. Acceptability of audio computer-assisted self-interview (ACASI) among substance abusers seeking treatment in Rio de Janeiro, Brazil. Drug Alcohol Depend 2006;82(Suppl. 1):S103–107

10.

Anastario

Tavarez

Chun

. Sexual risk behavior among military personnel stationed at border-crossing zones in the Dominican Republic. Rev Panam Salud Publica 2010;28:361–7

11.

de Waal

Jennifer

Manjari

. HIV/AIDS, Security and Conflict: New Realities, New Responses. New York, USA: Social Science Research Council, 2009

12.

Community Research Branch. Risk Behavior Assessment Questionnaire. Rockville, MD: National Institute on Drug Abuse: Community Research Branch. October 1993. p 16–44

13.

Cherpitel

. Screening for alcohol problems in the U.S. general population: comparison of the CAGE, RAPS4, and RAPS4-QF by gender, ethnicity, and service utilization. Rapid Alcohol Problems Screen. Alcohol Clin Exp Res 2002;26:1686–91

14.

Stata 10 [computer program]. College Station, TX: Stata Corporation, 2007

15.

Islam

Topp

Conigrave

The reliability of sensitive information provided by injecting drug users in a clinical setting: clinician-administered versus audio computer-assisted self-interviewing (ACASI). AIDS Care 2012;24:1496–503