Abstract
Social network analysis depends on how social ties to others are elicited during interviews, a process easily affected by respondent and interviewer behaviors. We investigate how the number of self-reported important social contacts varied within a single data collection round. Our data come from Health and Aging in Africa: a Longitudinal Study of an INDEPTH community (HAALSI), a comprehensive population-based survey of individuals aged 40 years and older conducted over 13 months at the Agincourt health and demographic surveillance site in rural South Africa. As part of HAALSI, interviewers elicited detailed egocentric network data. The average number of contacts reported by the 5,059 respondents both varied significantly across interviewers and fell over time as the data collection progressed, even after adjusting for respondent, interviewer, and respondent–interviewer dyad characteristics. Contact numbers rose substantially after a targeted interviewer intervention. We conclude that checking (and adjusting) for interviewer effects, even within one data collection round, is critical to valid and reliable social network analysis.
Measurements of social networks depend on the number and type of social ties to others (Berkman et al. 2000; Smith and Christakis 2008). These ties are typically elicited through interviews, a process easily affected by respondent or interviewer characteristics and behaviors. Understanding social network structure and composition requires substantial amounts of information from respondents (“egos”) about the people (“alters”) they have relationships with (Marsden 1990). Notably, the survey burden associated with network data collection depends heavily on the number of alters elicited through “name generator” questions: Each alter named leads to the repetition of all follow-up questions characterizing the ego–alter relationship (“name interpreters”; Burt 1984).
Interviewers have been identified as a key source of variation in survey responses, particularly for questions that are attitudinal, ambiguous, or have complex skip patterns (West and Blom 2016). Several studies have previously identified interviewer effects on network size (Brüderl et al. 2013; Josten and Trappmann 2016; Marsden 2003; Paik and Sanchagrin 2013; van Tilburg 1998). These interviewer effects may arise from differential understanding of survey questions, and therefore how questions are presented to respondents. Interviewers can also affect which alters are elicited due to their own characteristics (e.g., sex, race, age, or experience), or the nature of the interviewer–respondent dyad (e.g., gender, race, or age homophily), leading to different lines of enquiry, levels of probing, or expectations of social acceptability (Collins 1980; Hox 1994; Marsden 2003; Phung et al. 2015).
Furthermore, if respondents or interviewers are aware that naming more alters substantially increases survey length, then either group may consciously or unconsciously seek to minimize the number of alters named (Eagle and Proeschold-Bell 2015; van der Zouwen and van Tilburg 2001). In cross-sectional surveys, the opportunities for respondents to learn are limited, but those for interviewers will increase as the survey period progresses. Interviewers may try to reduce survey burden, either for themselves or for respondents, by favoring language or probes that decrease the number of alters elicited. Indeed, past studies in Europe have found evidence of interviewers intentionally filtering out questions by entering fewer responses that would trigger more questions. Such filtering behavior has been seen in Europe for interviewers who are being compensated by the interview rather than by the hour (Josten and Trappmann 2016; Kosyakova et al. 2014), for interviewers with prior experience using the relevant screening tool (Matschinger et al. 2005), and where interviewers are under substantial pressure to complete more interviews (Schnell and Kreuter 2000).
We aim to extend this literature by assessing how the number of alters elicited systematically changed over the course of a cross-sectional social network survey of older adults in rural South Africa. We show a substantial drop in alter numbers over time, and a swift reversal following retraining, providing substantial evidence for interviewer effects.
Method
Survey Design
The social network module was one component of the baseline wave of the “Health and Aging in Africa: a Longitudinal Study of an INDEPTH community” (HAALSI) questionnaire conducted in 27 of the 31 villages that comprise the medical research council (MRC)/Wits Rural Public Health and Health Transitions Research Unit in Mpumalanga province, South Africa (hereafter, “Agincourt”; Kahn et al. 2012). The HAALSI study is a population-based longitudinal cohort of men and women aged 40 years and over in rural South Africa and was selected as a random sample of approximately 40% of all age-eligible individuals in the Agincourt demographic surveillance area; 85.9% of eligible individuals approached consented to participate. Interviews progressed from village to village throughout the study period, with interviewers randomly assigned to potential participants within each village.
Data were collected using computer-assisted personal interviews (CAPI). The baseline survey was modeled closely on sister health and aging studies including the U.S. Health and Retirement Studies (HRSs), longitudinal aging study in India (LASI) in India, and China health and retirement longitudinal survey (CHARLS) in China (Arokiasamy et al. 2012; Sonnega et al. 2014; Zhao et al. 2014). It comprised an approximately three-hour household visit including structured quantitative interviews, anthropometric and physiological measurements, and blood draws. Inclusion of a social network module, however, was unique to HAALSI among the HRS sister studies. The structure of the HAALSI social network module was modeled on the network module in the U.S.-based National Social Life, Health, and Aging Project (NSHAP; Cornwell et al. 2008). The HAALSI module, which was started around 30 minutes into the household visit, included one name generator question: “Please tell me the names of six adults with whom you have been in communication either in person or by phone or by Internet in the past six months, starting with the person who is most important to you for any reason.” This question aimed to capture the respondents’ most meaningful recent relationships—those most likely to impact their health and well-being. If the respondent was married and the spouse was not named by the respondent, then the spouse’s name was added as a seventh response. Neither the interviewers, nor the CAPI program, forced respondents to name six alters, despite the name generator wording.
Respondents were then asked a series of “name interpreter” questions about each named alter, including: (1) the alter’s sociodemographics (age, sex, residential location, and relationship with respondent); (2) frequency of contact (in person, by phone/text/e-mail); and (3) frequency of receiving four types of social support (emotional, informational, physical, and financial) from the alter.
HAALSI was granted ethics approval by the University of the Witwatersrand Human Research Ethics Committee, the Harvard T.H. Chan School of Public Health Office of Human Research Administration, and the Mpumalanga Provincial Research and Ethics Committee.
Interviewer Recruitment and Training
HAALSI interviewers were recruited from within the local resident community, among those with high school graduation (“matric pass”) and fluent in both English and xiTsonga (the local language). Twenty of the 29 applicants selected for training were retained for the survey. Four supervisors with previous experience supervising the Agincourt demographic census oversaw the work of the fieldworkers. Both interviewers and supervisors received training specific to the social network module. Interviews began in November 2014. Three interviewers did not continue after January 2015 as performance requirements were not met. In May 2015, an additional seven interviewers were employed to accelerate survey completion. In September 2015, after the bulk of interviews were completed, the three best-performing fieldworkers (based on quantitative data and supervisor reports) were retained to revisit previously unavailable respondents, with other fieldworkers providing occasional assistance. Survey enrollment closed in December 2015. All but four of the 27 interviewers who worked on HAALSI had previous interviewer experience in Agincourt.
Data Monitoring
Beginning in January 2015, HAALSI researchers produced monthly data quality monitoring reports until the end of the survey. Key results were shared with on-site and off-site project managers, who then informed field supervisors at the weekly study management meetings and periodic retraining conducted with field staff (supervisors and interviewers). Social network module retraining in February focused on questions relating to alter ages (which had been missing in 25% of cases) and conflict (5–10% don’t know/refused). Although declining rates of alter elicitation were noted early on, the extent of decline was not clear until later in the year. As a result, the issue was not presented to the field team until June. Despite subsequent discussions with field staff, alter numbers continued to decline to a nadir in September 2015. In October and November 2015, supervisors and project managers held intensive weekly meetings with the remaining interviewers during which they discussed the issue of low alter numbers.
Statistical Analyses
We first described how the number of alters reported varied by: (1) respondent characteristics, using Kruskal–Wallis χ2 tests; (2) interviewer and their characteristics; and (3) date of interview, including what types of alters were named differentially over time. We then conducted multilevel Poisson regression (having tested for and rejected overdispersion), in which respondents were nested within interviewers with random intercepts. Poisson models allowed us to model the count of alters elicited; multilevel models allowed us to decompose the variation in number of alters elicited into parts within and between interviewers. These regression models began with a null model with no covariates, then added in turn: month of interview, village of respondent, other respondent characteristics, interviewer characteristics, and respondent–interviewer dyad characteristics.
Finally, we examined the extent of variation in the rate of decline across interviewers by running a multilevel model on only the first 11 months of data, that is, the period prior to intensive interviewer supervision, including both random intercepts and slopes for each interviewer. The final model was of the form:
where μ ij is the count of alters named by respondent i with interviewer j. The model contained three sets of fixed effects: Xij is a vector of respondent-level covariates, including Mij an indicator for month of interview; Zj a vector of interviewer-level covariates; and Tij a vector of respondent–interviewer dyadic covariates. The model contained three random effects: eij at the individual level, uij a random intercept for each interviewer, and vjMij a random slope for interviewers across months. This last term further decomposed the variation in alter numbers seen between interviewers by allowing for the rate at which interviewers decrease the average number of alters they elicit to vary, so we can determine whether rates declined over time for all, or only some, interviewers.
Results
A total of 5,059 individuals responded to the social network module of HAALSI, describing 15,549 alters in the social network module (Online Supplementary Table S1). Respondents named a median of three alters, with 252 (5.0%) individuals reporting no alters and 532 (10.5%) reporting six alters (227 of whom also had an unnamed spouse who was added as a seventh alter). Over three-quarters (77.8%) of named alters were relatives, mainly living in the same household (34.1%) or village (35.1%) as the respondent. Most nonrelative alters (76%) also lived in the same village as the respondent.
The 27 interviewers conducted between 25 and 351 interviews each, with a median of 211 (Table 1). The mean number of alters elicited per interview varied widely across interviewers—ranging from 1.4 to 6.1. Interviewers who were female, younger, and who conducted more interviews elicited more alters, on average. There was also considerable variation across time, with the mean number of alters elicited in each study month falling from 4.8 in November 2014 to a low of 1.7 in September 2015, before rebounding to 3.1 in the last two months of data collection following retraining (Online Supplementary Figure S1). This downward trend and rebound occurred within interviewers (Figure 1) and appears to correspond to a fall in alters who lived outside the respondents’ household, particularly within the Agincourt study area (Online Supplementary Figure S2).

Mean number of alters reported to each interviewer in each month by the Health and Aging in Africa: a Longitudinal Study of an INDEPTH community respondents. Each circle represents the number of interviews conducted by one of the 27 interviewers in one month; the volume of each circle is proportional to the number of interviews conducted. Each interviewer is represented by the same color over time. (The color version of this figure is available in the online version.)
Descriptive Statistics for HAALSI Interviewers.
Note: χ2 Test is a Kruskal–Wallis test of equality of ranks with one degree of freedom comparing mean number of alters reported by respondents interviewed by interviewers from each category. CI = confidence interval; HAALSI = Health and Aging in Africa: a Longitudinal Study of an INDEPTH community.
The multilevel regression models show that approximately half of the variance seen at the interviewer level in the null model can be explained by the month of interview (Table 2). In addition to month of interview, respondents’ age, gender, education level, marital status, and household wealth are significant predictors of numbers of alters named. No other factors, however, are able to explain the remaining 50% of interviewer-level variance. Neither interviewer gender and age, nor dyadic homophily on these characteristics, predicted number of alters. Finally, when considering only the period up to September 2015, there was significant variability in how rapidly alter elicitation rates fell over time, although all 27 interviewers had a significantly negative slope coefficient (Online Supplementary Figure S3). This rate of decline was positively associated with level of alter elicitation, such that interviewers with higher elicitation rates saw slower falloff in these rates. The predicted incidence rate ratio of elicited alters from model 7 continues to show a clear decline until September 2015 followed by a sharp rise (Online Supplementary Figure S4).
Summary of Mixed-effect Poisson Regressions for Number of Alters Elicited by HAALSI Interviewers.
Note: All statistics for covariates are p values from Wald tests for a linear hypothesis that all k categories of each variable (as shown in Online Supplementary Table S1) are jointly equal to 0, with an associated k − 1 degrees of freedom.
aMonth of interview is categorical for models 1–7 to allow for nonlinearities over time and continuous for models 8 and 9 to allow for meaningful random slope coefficients. Point estimates and confidence intervals for all regressions are provided as Online Supplementary Table S2.
Discussion
In this study of an egocentric social network data collection process within a large, cross-sectional survey in rural South Africa, we show that alter elicitation rates fell systematically as the survey progressed, even after adjusting for respondent, interviewer, and respondent–interviewer dyad characteristics. Even more compellingly, we show that after fieldworkers and supervisors began meeting weekly to discuss fieldwork progress (including reviewing alter elicitation numbers), alter numbers rose sharply for the remainder of the study.
There are several possible explanations for the fall in alter numbers over time. First, later respondents may have learned from friends and family that the interview process was lengthy and could be shortened by reporting fewer alters. Given that the social network module was only one of several within the questionnaire, this seems unlikely. Second, later respondents may have had fewer true. Since this study was rolled out across consecutive villages in an overlapping fashion, the decline over time could represent a geographic pattern. Furthermore, later respondents included those who could not easily be found by the field team; such hard-to-find individuals might have had fewer alters. However, neither of these explanations explains the sharp uptick observed in the last two months of data collection. Third, increased experience may have improved interviewers’ ability to elicit the truly important people in respondents’ lives. Much of the drop in alter nominations was of kin not living in the same household, who might have been less vital to respondents. Yet there was also a drop in the number of alters in daily contact with respondents (Online Supplementary Figure S5), which suggests that at least some truly pertinent alters were lost.
Finally, interviewers may have learned to reduce survey length by eliciting fewer alters. Although interviewers were salaried, rather than paid per interview, there were substantial pressures on fieldworkers to complete interviews more rapidly—since the completion target of two interviews per day was regularly missed throughout the survey period. If interviewers learned that certain modules could be shortened, they may have guided respondents to report fewer alter to speed up the process. This explanation is supported by the sharp increase in alter numbers seen once interviewers were made aware that their elicitation rates were being observed and that higher numbers of alters were to be expected (i.e., a Hawthorne effect).
Variation in alter elicitation rates across interviewers—even with randomized assignment to respondents—is not surprising and has been seen previously for consent rates, reporting on sensitive topics, and indeed naming of alters (Brüderl et al. 2013; Josten and Trappmann 2016; Marsden 2003; Paik and Sanchagrin 2013; van Tilburg 1998). In contrast, variation over time in elicitation rates within interviewers across a survey period has been reported less often. This study is the first to empirically examine possible interviewer learning effects outside of Germany and the second to examine social network data specifically. Our results corroborate findings that interviewers may lead respondents away from longer interviews after learning about the interview process and when under time pressures (Matschinger et al. 2005; Schnell and Kreuter 2000). Our results also suggest that such behavior occurs even when interviewers are salaried (Josten and Trappmann 2016; Kosyakova et al. 2014) and may be placing future employment opportunities at risk—even when stable jobs are very scarce. Interestingly, our findings suggest that not just a subgroup (Matschinger et al. 2005) but all interviewers elicited fewer alters over time, possibly because they worked as a single cohesive field team.
Future social network studies should develop ways to minimize interviewer-associated variation over time in labor-intensive surveys. Possible approaches include: (1) improved training to increase standardization of name generator question delivery and probing; (2) more feedback of alter elicitation rates and more careful education about the importance of comprehensive alter elicitation to the fieldwork team; (3) computer programing strongly encouraging collection of a fixed or minimum number of alters from all respondents (although this requires careful training and allowing interviewers to relax the constraint in the case of truly isolated individuals); and (4) use of self-interview methods, so interviewers are not involved in alter elicitation.
We also note that while roster-style modules, with multiple questions asked about every alter named, may be particularly susceptible to interviewer effects, given the impact of roster length on interview burden, other parts of an interview with loops and skip patterns may also open to control by interviewers (Brüderl et al. 2013). Future work could usefully examine such potential effects.
Strengths and Limitations
A major strength of this study is its use of standard social network data collection methods modeled on the NSHAP study. In addition, this study was based on an existing longitudinal surveillance platform, ensuring well-trained fieldworkers and a strong fieldwork infrastructure. Given these standardized data collection approaches, the effects we report on are likely to generalize to many other settings. Our key weakness is the inability to entirely rule out non-interviewer–led explanations for the observed temporal trends. However, the striking pattern of rebound following intensive interviewer retraining strongly suggests interviewers’ importance in generating the patterns of alter numbers seen. Furthermore, in settings where social networks can be collected via respondent-driven methods (e.g., self- or online interviews) or via analysis of e-mail or social media patterns, the concerns raised here may be less pressing. Nevertheless, interviewers can help increase the validity and reliability of alter elicitation, if carefully trained and supervised. Interviewers are thus likely to be used-and interviewer effects remain a concern-even in more literate and computer-connected populations.
Conclusions
The time and effort required from both interviewers and respondents to measure networks is considerable, thus social network data are vulnerable to measurement error as our findings suggest. We therefore recommend that researchers design network data collection processes to minimize opportunities for interviewer effects, continuously monitor data collection processes, and consider adjustment for both interview date and interviewer identity in any analyses they conduct—even for cross-sectional data.
Supplementary Material
Supplemental Material, harling_FM_8-25-16_supplemental - Interviewer-driven Variability in Social Network Reporting: Results from Health and Aging in Africa: A Longitudinal Study of an INDEPTH Community (HAALSI) in South Africa
Supplemental Material, harling_FM_8-25-16_supplemental for Interviewer-driven Variability in Social Network Reporting: Results from Health and Aging in Africa: A Longitudinal Study of an INDEPTH Community (HAALSI) in South Africa by Guy Harling, Jessica M. Perkins, Francesc Xavier Gómez-Olivé, Katherine Morris, Ryan G. Wagner, Livia Montana, Chodziwadziwa W. Kabudula, Till Bärnighausen, Kathleen Kahn, and Lisa Berkman in Field Methods
Footnotes
Authors’ Note
GH conceptualized the study, conducted the analyses, and wrote the first draft of this article in collaboration with JP and with input from FXGO, KM, and LM. LB, JP, and KM developed the social network measures. LM, FXGO, RW, CK, KK, and LB were involved in all phases of data collection. All authors contributed to the study design, data interpretation, and final revisions to the text. Jessica Perkins is also affiliated to Department of Human and Organizational Development, Peabody College, Vanderbilt University, Nashville, Tennessee and Vanderbilt Institute of Global Health, Vanderbilt University Medical Center, Nashville, Tennessee.
Acknowledgments
We would like to thank the study team, participants, and community of Bushbuckridge, without whom this study would have not been possible. We acknowledge useful comments from participants at Sunbelt XXXVI.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The HAALSI study is funded by the National Institute on Aging from the National Institutes of health (P01 AG041710), and is nested within the MRC/Wits Rural Public Health and Health Transitions Research Unit (Agincourt), supported by Wellcome Trust (grants 058893/Z/99/A, 069683/Z/02/Z, 085477/Z/08/Z, and 085477/B/08/Z), the University of the Witwatersrand, and the South African Medical Research Council. Till Bärnighausen was supported by the Wellcome Trust and by National Institutes of Health (grants R01-AI124389 and R01-HD084233).
Supplementary Material
Supplementary material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
