Abstract
To achieve high return among secondary respondents, primary respondents’ consent rates must be high. In the German Family Panel (pairfam), a large, randomly sampled panel study, primary respondents’ consent rates to surveying their parents were found to be low. Since we suspected that the underlying reason could be interviewer behavior, we tested in an experiment if placing the consent questions in the self-interview (CASI) part of the interview would increase consent rates. Results show that bypassing interviewers by asking respondents directly in the CASI section did not increase consent rates.
Introduction
In recent years, more and more surveys have employed a multi-actor design, in which, additionally to the sampled primary respondents, individuals in their social networks are surveyed as secondary respondents. In the Netherlands Kinship Panel Study (NKPS), for instance, the primary respondents’ partners, children, parents, and siblings are included as secondary respondents (Dykstra et al., 2012). In the QUALICOPC (Quality and Costs of Primary Care in Europe) study, general practitioners (GPs) are primary respondents while their patients are secondary respondents (Schafer et al., 2011). The Workplace Employee Relations Survey (WERS) in UK, which was one of the first surveys of this type, samples workplaces and interviews the most senior manager in each workplace as primary respondent, whereas two employee representatives and a group of randomly sampled employees are surveyed as secondary respondents (Chaplin et al., 2005). The German Family Panel (pairfam), of which we report results in the following, includes the primary respondents’ partners, parents, stepparents, and children in the survey (Brüderl et al., 2015a).
For the success of such singular multi-actor surveys (see classification of types of multi-actor surveys by Pasteels, 2015) both primary and secondary respondents’ response rates are crucial. However, for secondary respondent’s response often a double hurdle to participation exists. Firstly, primary respondents are asked during the interview to give their consent to surveying secondary respondents. Then, only if consent is given, secondary respondents are then invited to the survey and can decide whether to participate or not. Therefore, low response rates as well as bias due to selective participation of secondary respondents are a problem of many multi-actor surveys (Kalmijn and Liefbroer, 2011; Schröder et al., 2013).
In pairfam, low consent rates for surveying parents were achieved in the first waves of the parents’ survey. Prior to the experimental approach taken in the six wave of the panel, the project team investigated the influence of quality of the relationship to the secondary respondents on the probability of consent (Schröder et al., 2013): their analyses detected strong effects, yet relationship quality and individual characteristics were found to account only for a small part of the variance in consent. Particularly, it remained unclear why consent was provided far less often to surveying parents than partners and children. One consideration regarded in particular the role of the interviewers in eliciting consent.
In order to examine if interviewer behavior causes the low consent rates to the parents’ survey observed in pairfam, an experiment was conducted in the sixth wave of the panel that tested whether asking respondents for consent in the CASI part of the interview would increase consent rates.
Concerning participation of primary respondents, the role played by interviewers has long been acknowledged (Groves et al., 2011; Groves and Couper, 2012). In general, involving interviewers to convince individuals to participate in a survey is regarded as a more effective means to lower unit nonresponse than applying self-administered modes (Hox and de Leeuw, 1994). On the other hand, poorly motivated or less capable interviewers might have little success in convincing reluctant respondents to participate (for a review, see Blom et al., 2011: 301). Similar mechanisms may cause nonresponse among secondary respondents. In fact, interviewers may play a vital role in convincing primary respondents that surveying the secondary respondent is important for research and consent is uncritical e.g., regarding privacy protection. On the other hand, it is known that interviewer behavior can also negatively affect data quality (Van der Zouwen and van Tilburg, 2001) as well as that, under specific circumstances, interviewers may deviate from prescribed procedures. For instance, research has found that interviewers tend to artificially cut short respondents’ network size (Brüderl et al., 2013; Paik and Sanchagrin, 2013), influence respondents’ answers to screening questions (Matschinger et al., 2005; Tourangeau et al., 2012), and induce item nonresponse (De Leeuw, 2001). Therefore, it might also be that some of the interviewers deviate from the procedures defined to elicit consent to surveying secondary respondents.
Multi-actor consent questions may be prone to deviating interviewer behavior as interviewers might regard the request as intrusive or because they expect primary respondents to react negatively. In the case of pairfam, these reasons were considered as particularly relevant for the consent to approaching primary respondents’ parents in contrast to partners and children with whom they have usually closer contact. Interviewers might anticipate primary respondents to have more qualms about involving their parents in the survey than partner and children. In addition, questions asking for parents’ participation are the last set of multi-actor consent questions in the pairfam CAPI, so that interviewers may feel overtaxing the respondent’s patience. Furthermore, only in the case of parents the interviewer needs to record the addresses. This represents a more intrusive procedure than just handing over the questionnaire (as it is the case with the partner questionnaire). Moreover, entering addresses is burdensome and time-consuming for interviewers, so that the perceived costs of reading aloud the long question texts and convincing the respondents about the safety of their decision probably exceed the interviewers’ remuneration for this effort. All in all, it wouldn’t be too surprising if interviewers ended up abridging or altogether bypassing this element of the interview.
Design of the German Family Panel
The German Family Panel (pairfam 1 ) is a multi-disciplinary, longitudinal study collecting data from a nationwide random sample of initially more than 12,000 individuals of the three birth cohorts 1971-73, 1981-83, 1991-93 as well as from their partners, parents and children (Huinink et al., 2011). Main topics of the study are intimate relationships and family dynamics, but the survey covers a wide range of additional topics such as, for instance, occupation, well-being, and health. Starting with the first wave in 2008, anchor respondents are interviewed annually via Computer Assisted Personal Interview (CAPI). This interview takes about 60 minutes on average and contains a self-interview section (CASI). For this CASI section - placed in the middle of the interview - the interviewer hands over the laptop to the respondent. This procedure guarantees that the respondent can fill out the modules on sensitive topics such as sexuality and addictive behavior without interviewer involvement.
The field work is conducted by a large professional survey institute. Interviewers are experienced in conducting social surveys, but are not affiliated to scientific research. Interviewer allocation is based on proximity to a given sampling point and the field agency tries to ensure interviewer continuity whenever this is possible: interviewer changes happen mostly in connection with respondents’ residential moves or with an interviewer leaving the project.
The multiactor design of the German Family panel was implemented in two stages: the partner survey was already introduced in the first wave, whereas the other actors have been only surveyed from Wave 2 onwards. The anchors’ partners and parents are surveyed using a mail questionnaire whereas their children (aged 8 to 15 years and cohabiting with the anchor) are interviewed via CAPI as the anchor.
Anchor respondents are asked during the interview to give their consent to surveying the secondary respondents, i.e. their partners, children, and parents, provided the primary and secondary respondents are still in contact. Consent questions are asked after the module asking about the anchor’s relationship to the person, respectively. As the module on intergenerational relations is placed rather late in the interview, the parents’ survey is the last multi-actor element mentioned in the interview. While the child interview is conducted in the anchor’s household and the partner questionnaire can be left with the anchor to pass it on to the partner, the anchor has necessarily to provide the parents’ addresses to the interviewer because the parents’ questionnaires are sent to each parent separately by mail. Provision is made in the procedure that, if respondents are undecided, they can ask their parents before they provide their addresses to the interviewer. The answer category for this option is “yes, but I want to ask my parent first.” In this case, interviewers call them some time after the interview to note the addresses. This option (referred to as “conditional consent” in the following) is chosen rarely (8 percent in Wave 2, but only 1 percent in Wave 5).
The Parents’ Survey in Waves 2-4
In the five waves preceding the experiment in the sixth wave, consent rates varied considerably between types of secondary respondents: the highest consent rates are achieved for surveying children, with 77 percent in Wave 5, and the lowest for surveying the stepmother, with consent for 15 percent of all eligible step-mothers in Wave 5 (Brüderl et al., 2015b). Consent rates for parents altogether are 41.6 percent in Wave 5. Across waves, consent rates decreased from 51.2 percent in Wave 2 when the parents’ survey was conducted the first time to about 40 percent in the following waves. Response rates, i.e. the share of eligible parents who actually responded, have decreased over time from 29.8 percent in the second wave to 26 percent in the fifth wave (see Table 1). Table 1 gives a detailed overview of the development of consent and response rates for the different types of parents over the waves. The first part shows total consent rates, combining unconditional (“yes”) and conditional (“yes, but want to ask parent first”) consent; the second part of Table 1 shows conditional and unconditional consent rates separately; the third part shows response rates based on sent questionnaires and the fourth part response rates based on all eligible parents irrespective of whether a questionnaire was sent to them or not.
Outcomes of the parents’ survey across waves (%)
Note: Consent rates are calculated on the number of eligible parents.
No parent survey run in Wave 1. The East-Germany booster DemoDiff is not included.
As to be expected, the use of the conditional consent option decreased across the waves. In Wave 2, the first one with a parent survey, it was used by roughly 8 percent of the respondents, whereas in all following waves this option was used only in about 1-2 percent of the cases. As the respondents became aware of this element of the multi-actor design, they either asked their parents in advance about their willingness to participate or relied on previous year’s feedback by the parents to make up their mind. In both cases uncertainty about the parents’ decision declined.
Regarding the unconditional consent, the figures declined in the first three waves and increased slightly again in Wave 5. This might be owing to two changes that were introduced in Wave 5: the first one regarded the wording for asking the consent to the parents’ survey.
Waves 2-4: “Just as last time we would very much like to win the support of your mother and your father for this study. For this purpose, we would like to send them a questionnaire by mail. Your mother and your father will get an ARD TV lottery ticket if their completed questionnaire reaches us. We would like to note names and addresses so we can send them a questionnaire. We can only survey parents who live in Germany.” (German Family Panel, 2015a: 206)
Wave 5: “Just as last time we would very much like to win the support of your mother and your father for this study. For this purpose, we would like to send them a questionnaire by mail. The questionnaire is much shorter and is limited to essential information. Information about you will not be shown to your parents. Your mother and your father will get an ARD TV lottery ticket if their completed questionnaire reaches us. We would like to note names and addresses so we can send them a questionnaire. We can only survey parents who live in Germany.” (German Family Panel, 2015b: 208).
But despite these adjustments in Wave 5, consent rates remained below the expectations. Nevertheless, the pairfam team was confident that further improvements would be able to fix the problem. An analysis of the individual patterns of consent showed that even after four waves of the panel, parents’ participation was still fluctuating from wave to wave. For instance, among eligible parents both in wave 4 and 5 and considering only those who were living in Germany at both time points, we found that in 10-15 percent of the cases the anchors’ decisions as to whether to consent or not to the parents’ survey differed (see: Table 2). Changes from consenting to not consenting are easily explainable, since the parents could have informed the main respondents that they were no longer willing to participate. In addition to these cases, however, 10-11 percent of previous year’s refusals converted into an unconditional consent: this was taken as hint that there was room for improvement in consent rates and the experiment was designed to test whether the interviewer were playing a negative or a positive role in the process.
Relationship between consent to parents’ survey in Waves 4 and 5 for biological mothers and biological fathers (counts and column percentage)
The Experiment on Obtaining Consent in the CASI Section in Wave 6
The pairfam team decided to investigate interviewer influence in Wave 6 and moved the consent questions to the CASI part of the interview. Our hypothesis was that interviewers did not follow the procedures prescribed in the CAPI to elicit primary respondents’ consent to contacting their parents. Investigations on interviewer effects in parifam have already shown that some interviewers were taking shortcuts when collecting data on social networks, probably since this module was perceived as particularly long and tedious (Brüderl et al., 2013). Furthermore, we observed a light increase in percentage of consent in Wave 5, after introducing interviewer payment per address collected instead of per returned questionnaire. We hence suspected that interviewers downplayed the importance of parents’ participation or even skipped the questions they regarded as unpleasant (for themselves or for the respondents) and of little economic relevance for the interviewers. Accordingly, bypassing the interviewers by asking the consent questions in the CASI section was expected to increase consent among primary respondents.
To allow for an evaluation of this measure on sound methodological grounds, it was decided to implement a randomized experiment, with 50 percent of the anchor respondents allocated to the usual CAPI protocol and the other 50 percent being asked for consent in the CASI section.
Interviewers were not blind regarding the experiment. The first reason was that it would have been difficult to give them an alternative explanation why some, but not all respondents with contact to their parents were asked for consent (in contrast to past waves). The second reason was that respondents approach interviewers also during the CASI part if they have difficulties with the interview, so that interviewers would have become aware of the experiment in those cases. Moreover, in case of conditional consent (“yes, but want to ask my parent first”), interviewers had to make an appointment for calling the respondent and asking about the parent’s decision.
The experiment implied one major change in the CAPI for all anchor respondents: Since parents were eligible for the parents’ survey only if they were in touch with the anchors, the questions asking for consent had to be placed after the module on intergenerational relationships where the respective information is collected. Until wave 5, this module was placed relatively late in the interview, after the CASI section. Therefore, the succession of the modules had to be changed and the module on intergenerational relations was placed at the beginning of the interview, in order for the information required to assess the eligibility of the parents to be available prior to the beginning of the CASI section. In order to avoid order effects biasing differences between the experimental group and the control group, we changed the sequence of modules in the interview for respondents in both groups.
Results of the experiment are shown in Table 3. Total consent rates, i.e. the sum of conditional and unconditional consent rate, do not differ substantially between the experimental group and the control group. The difference in consent for contacting mothers and fathers is about one percentage point, with consent rates even being lower in the experimental group. Only for stepparents we find that consent is higher in the experimental group (e.g., 30.2 percent vs. 26.3 percent in the case of the stepfather and 13.5 percent vs. 13.3 percent in case of the stepmother) but differences are not significant (t-test, p = .11 for stepfathers and p = .48 for stepmothers). Consent rates across all types of parents are nearly identical (41.12 percent in the experimental group vs. 41.69 percent in the control group).
Consent rates and interviewer effects in experimental and control group (Wave 6)
Note: without DemoDiff sample. Percentages are referred to all eligible parents.
A notable difference is that the share of anchor respondents indicating that they want to ask the parent before giving consent is higher in the experimental group than in the control group. This result holds for all four parent types. The reason for this difference might be that respondents in the CASI group read the three answer categories so the possibility of conditional consent was more prominent. In contrast, in the CAPI group this possibility was not supposed to be explicitly pointed out by the interviewer. Moreover, all in all, interviewers have rationally an incentive in pushing a clear-cut answer regarding consent, e.g. by pointing out that the parents can still refuse to participate, and avoid the conditional consent. In fact, conditional consent increases the workload for the interviewer who has to call the primary respondent to ask for the parent’s address, whereas only a few of those parents for whom the primary respondents gave conditional consent finally provided their address later (22 percent in Wave 6).
Therefore – although results are rather similar regarding primary respondents’ total consent in the interview – a striking difference arises regarding the number of questionnaires sent out by the survey institute. While in the control group 40.6 percent of all parents received a questionnaire, in the experimental group only to 35.0 percent did so. This goes down to the fact that most cases of conditional consent did not translate into a usable address. Consequently, also response rates are lower in the CASI group than in the CAPI group (see bottom lines of Table 3).
Conclusion
A central criterion for the evaluation of singular multi-actor surveys is sufficient secondary respondent’s response: ideally, one wants a high percentage of the eligible others to take part and nonresponse to be as little systematic as possible. This is not the case with the parents’ survey in the German Family Panel (pairfam) where coverage rates among primary respondents’ parents was constantly low in the first waves. The weakest link appeared to be the request for consent to contacting the parents for the survey as the biggest amount of losses happened at that stage. Improving the procedures of eliciting consent from the primary respondents seems to be the first step in order to increase secondary respondents’ participation rates.
One of the hypotheses for the low consent rates was related to interviewer behavior, as we suspected part of the interviewers to shorten or even skip the consent questions. Therefore, an experiment was conducted to test whether consent rates could be increased in that we bypassed the interviewers for this purpose and placed the consent questions in the self-administered section of the interview. The experimental design requested that the sample of primary respondents was randomly divided in an experimental group and a control group of equal size. Whereas the control group was asked for the consent to survey the parents by the interviewer as in the previous waves, the experimental group was asked in the CASI section of the interview, so that the interviewer was not involved in the procedure.
Contrary to our expectations, consent rates were not affected substantially by this measure. Total consent rates were similar in the experimental and the control group. In the experimental group (consent questions self-administered), the share of respondents that gave only conditional consent was, however, higher than in the control group (where this answer category was supposed to be not explicitly mentioned by the interviewer), which lead to an overall lower number of addresses provided than in the control group. In sight of these results, it was decided to return to the CAPI procedure for all respondents in wave 7.
How can this failure of the CASI procedure to increase consent rates be explained? On the one hand, even if deviating behaviors (e.g., cutting short the interview) might have been eliminated in the CASI group, positive effects of convincing interviewers are missing in the CASI as well. These two effects might have evened out. On the other hand, the experiment was conducted in the sixth wave of the panel, after consent to contacting the parents was asked by the interviewer in the four preceding waves. It could be that the resistance against the parents’ survey - possibly caused by the interviewer - in the previous waves continued to have an effect even when the interviewer was not directly involved in the consent questions any more.
We see a number of limitations of our experiment. First, as mentioned before, the sixth panel wave is fairly late for this experiment because to a certain extent routines have built up already in the past waves. Second, interviewers were not blind to the experiment, so they could still influence respondents in order save time. Finally, the response sets in the two groups were not perfectly identical, since the conditional consent alternative was visible at once for the experimental group, whereas interviewers were instructed not to volunteer it.
Nonetheless, the results from the experiment are unambiguous. The self-administered version did not reduce refusal among the primary respondents: on the contrary, in the control group more questionnaires could be sent out and response rates turned out to be higher. In the case of the pairfam study, we can hence conclude that the interviewers’ contribution on request of consent is not a negative one. Considering the evidence collected within the pairfam parents’ survey, it seems advisable to rely on interviewer to obtain consent for a multi-actor design. Furthermore, so far it appears a payment scheme for interviewers that incentivize higher consent rather than higher response seems to have a beneficial effect.
Footnotes
Acknowledgements
This article uses data from the German Family Panel (pairfam), coordinated by Josef Brüderl, Karsten Hank, Johannes Huinink, Bernhard Nauck, Franz Neyer, and Sabine Walper. pairfam is funded as long-term project by the German Research Foundation (DFG).
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
