Maximizing participation from online survey panel members

Abstract

Low survey participation from online panel members is a key challenge for market and social researchers. We identify 10 key drivers of panel members’ online survey participation from a qualitative study and then determine empirically using a stated choice experiment the relative importance of each of those drivers at aggregate and segment levels. We contribute to knowledge on survey participation by (a) eliciting key drivers of survey participation by online panel members, (b) determining the relative importance of each driver, and (c) accounting for heterogeneity across panel members in the importance assigned to drivers. Findings offer immediate practical guidance to market and social researchers on how to increase participation in surveys using online panels.

Keywords

non-response online survey panels stated choice experiment survey participation survey response behavior

Introduction

In 2013, more than half of global research revenues were generated by online surveys (ESOMAR, 2013), and “while the government and academic sectors of the public opinion industry have steadfastly insisted on retaining probability methods for virtually all surveys, the commercial sector has adopted non-probability methods for most survey data collection” (Boyle, Iachan, Freedner-Maguire, & Fakhouri, 2017, p. 7). The use of online panels is expected to continue as a main methodology for market and social research professionals (GreenBook Research Industry Trends Report, 2019). But online panels are problematic because—being non-probability samples of people who have voluntarily signed up—they can suffer from lack of representatively of the population under study and, as a consequence, lack of accuracy of findings (Brüggen, van den Brakel, & Krosnick, 2016). This problem is further exacerbated by the fact that the likelihood of an online survey panel member accepting the invitation by the panel company to complete any given survey is—in contrary to common belief—lower than is the case in probability samples (Pennay, Neiger, Lavrakas, & Borg, 2018). It is critically important, therefore, to find ways how to increase survey participation among online panel members. This is the aim of the present study.

Our work is informed by a number of previous studies which proposed comprehensive models of drivers of survey participation (Brüggen, Wetzels, De Ruyter, & Schillewaert, 2011; Fan & Yan, 2010). Our study extends prior work by developing a model of online panel survey participation which includes both respondent-specific and survey-specific drivers. We elicit potential drivers through a qualitative study, and then test the relative importance of each of those drivers for online panel members at aggregate level, as well as for segments of panel members. To minimize the dependence of our results on specifics of a particular panel provider (Brüggen et al., 2016), we draw a sample for the stated choice model from one major international commercial online panel in two countries and one smaller country-specific panel. Our findings offer actionable recommendations on how to increase survey participation for market and social researchers using online panel samples to collect survey data, for market and social research companies collecting survey data on behalf of clients using online panels, and for online panels themselves.

Models of mail survey response behavior

Technological advances in Internet access and survey software have enabled the development of online surveys using online panel samples since the 1990s. Prior to online surveys and online panels, the dominant self-administration methodology for surveys was by mail. Three models which explain survey participation in the context of mail surveys have been proposed.

The total design method (Dillman, 1978, 2000) assumes that people’s likelihood to participate in a survey increases if the expected benefits of participation are higher than the cost of completing the survey. The strength of the total design method lies in offering practical recommendations for market researchers. Three leverage points help increase survey response rates: reducing perceived cost, increasing perceived benefits, and increasing trust between survey respondents and the entity conducting the study (Singer, 2011). Perceived cost can be reduced by improving the functionality of the survey (Dillman, 1991), for example, by including interesting survey questions early in the questionnaire, making questions easy to understand and answer, and printing the survey as a booklet with a neutral cover design aligned with the survey topic. Perceived benefits can be increased by explaining to respondents how their responses help other people, the importance of their responses to contribute to change, and the exclusivity of participant selection. Trust can be built by accompanying the questionnaire with a personal, thoughtfully written letter explaining the value of the study, and the importance of their participation.

Dillman (1991) notes some shortcomings of the total design method: a lack of empirical evidence of the effectiveness of recommendations to increase survey participation, the exclusion of incentive payments, the reliance of social exchange theory (which assumes direct personal contact between social actors), and the implicit assumption that the method can be applied to any survey, any survey context, and any population under study. The tailored design method (Dillman, Smyth, & Christian, 2014) extends the total design method by taking into consideration the specificities of every survey, including the socio-demographic background of potential respondents, the topic, and the survey situation. It also extends to online surveys and multimodal surveys to tailor survey designs. Multimodal surveys allow respondents to use their preferred device, reducing coverage and non-response error. The online survey–specific extensions include ensuring functionality across various devices, platforms, and browsers; optimizing cognitive processing through a visually appealing layout of questions on the screen; using short and personalized email invitations; timing email contact and number of reminders; and distributing incentives electronically.

The second model puts the construct of cooperation at the center (Childers & Skinner, 1996). Cooperation depends on the expectation of a balanced exchange between researcher and respondent. Both actors incur cost by investing time, effort, and money. And both receive a benefit. If the respondent perceives an inequity, they can take action to restore equity: they can refuse to participate, put minimal effort into survey completion, deliberately give inaccurate responses (thus contributing low-quality survey data), or share with other potential survey participants the inequity of a specific survey. Developing a collaborative relationship with respondents is challenging because “there is no direct personal contact nor is there much recurrent contact” (Childers & Skinner, 1996, p. 92). Childers and Skinner introduce two additional drivers of survey participation in their model: sponsor identification (reinforcing the construct of trust in the total design method) and self-identification (reinforcing the construct of personalization in the tailored design method). If the respondent trusts that the researcher will provide the promised benefits upon survey completion, the respondent develops a specific behavioral intention: commitment to complete the survey. Despite the key role of cooperation, commitment, and trust in this model, its practical value depends on the correct identification of the types of costs and the types of benefits that potential survey respondents take into consideration.

The third model of survey participation is leverage-salience theory (Groves, Singer, & Corning, 2000). It suggests that the power of specific costs and benefits of survey participation differs across respondents. Each respondent has different leverage points. Leverage points also interact in different ways; some can compensate for the lack of others. To decide whether or not to participate in a survey, respondents assess costs and benefits and compare this bundle with an internal threshold.

Underlying all three frameworks are a wide range of costs and benefits. Many have been discussed in relation to mail surveys, including preliminary notifications about the upcoming survey invitation, personalization, follow-ups, return envelopes, sponsorship, appeals, financial incentives, non-financial incentives, the look and feel of the questionnaire, anonymity, survey length, and deadlines (Yammarino, Skinner, & Childers, 1991). A meta-analysis concludes that preliminary notifications about an upcoming survey invitation, follow-ups, appeals to complete the survey in a cover letter, the inclusion of a return envelope, and monetary incentives increase survey participation. Additional drivers include the level of interest in the survey topic and questionnaire color. Different studies draw different conclusions about the effect of specific drivers on survey participation (Dillman, 1991). Critically, drivers of survey participation depend heavily on the mode used to deliver the survey (Dillman, 1991), as does the response rate itself. On average, the response rate to online surveys is about 11% lower than for other modes (Manfreda, Bosnjak, Berzelak, Haas, & Vehovar, 2008). Many of the drivers identified in the context of mail surveys are likely to be obsolete in the era of online surveying, thus warranting a reinvestigation into drivers of survey participation in this particular context.

Drivers of online survey response behavior

Many studies have tested the effect of one or a small number of drivers on online survey participation. We review drivers of online survey participation more broadly here, rather than limiting ourselves to studies of online participation of members of non-probability online survey panels only. We do this to ensure that we are capturing all factors that may have the potential to increase the likelihood of people agreeing to complete a survey.

Key drivers identified in this body of literature include survey length (e.g., Haunberger, 2011; Sax, Gilmartin, & Bryant, 2003), interest in the topic (Brüggen et al., 2011; Huang, Hubbard, & Mulvey, 2003; Keusch, 2013; Zillmann, Schmitz, Skopek, & Blossfeld, 2014), desire to voice one’s opinion, curiosity, enjoyment, the desire to help, recognition and a feeling of obligation to complete the survey (Brüggen et al., 2011), personalized invitations to complete a survey (Joinson & Reips, 2007; Sánchez-Fernández, Muñoz-Leiva, & Montoro-Ríos, 2012; Sauermann & Roach, 2013), the number of reminders (Sánchez-Fernández et al., 2012), other design features of the survey invitation (de Bruijne & Wijnant, 2014; Sauermann & Roach, 2013), trust or the relationship with the organization conducting the survey (Fang, Shao, & Lan, 2009), and questionnaire design (Tangmanee & Niruttinanon, 2015).

Empirical results often contradict each other. In terms of incentives, for example, some studies conclude that prize draws are more effective (Pedersen & Nielsen, 2016; Sánchez-Fernández et al., 2012), some conclude cash payments have a more positive effect on participation rates (LaRose & Tsai, 2014), and others find that incentives do not have much effect on participation rates (Brüggen et al., 2011). The effects of different incentives on survey participation represent the largest sub-area of studies into survey participation, with many studies drawing conclusions about very specific incentive combinations in very specific survey contexts (e.g., Cobanoglu & Cobanoglu, 2003; Gritz, 2004; Huang et al., 2003; Laguilles, Williams, & Saunders, 2011; Sauermann & Roach, 2013; Young et al., 2015). Some studies conclude that drivers of survey participation are only effective under particular circumstances. For example, Mavletova and Couper (2016) find that incentive payments are more effective in increasing survey participation for mobile phone surveys than for PC surveys.

Most of the studies investigating drivers of online survey participation test them in an isolated manner, ignoring potential interaction effects. To the best of the authors’ knowledge, only two studies developed more comprehensive models of drivers:

Fan and Yan (2010) suggest a conceptual framework to improve web survey participation covering (a) survey development (relating to the content, liking the sponsor, survey duration, the presentation of web surveys, language, order in which questions appear, and appearance of the survey in the browser), (b) survey delivery (relating to the method of selecting the sample, how potential respondents are invited to participate, the style of the invitation, the existence and nature of reminders), (c) survey completion (including factors relating to design, the respondent themselves and society more broadly), and (d) survey return (including software quality and data security). This conceptual model has not been empirically tested.

Brüggen et al. (2011) developed a comprehensive model of response motivators—the Survey Participation Inventory. The model postulates a number of different motives that lead to people accepting an invitation to complete a survey. The empirical test of the Survey Participation Inventory reveals the existence of different segments of online respondents who are motivated by different combinations of drivers. The Survey Participation Inventory offers immediate practical recommendations for increasing online survey participation in general and among specific segments, as well as for correcting the representativity of the sample through weighting. Although some of the questions included in the Survey Participation Inventory are reflective of structural survey features, the inventory does not include structural survey features directly.

In summary, empirical results about many of the drivers of online survey participation are inconclusive, mainly because most studies investigate individual drivers in isolation. Our study builds on the work by Fan and Yan (2010) and Brüggen et al. (2011) by developing a comprehensive model including both respondent-based and survey-based factors specifically for the context of online survey panels. Such a model is particularly important in view of leverage-salience theory (Groves et al., 2000), suggesting an interaction effect between drivers. Interaction effects can only be understood if drivers are investigated in combination, not individually. There is also a lack of qualitative research into drivers of online survey participation (e.g., Fan & Yan, 2010; Groves & Couper, 2012), which may offer deeper insights into motivations of online panel members (Brüggen et al., 2011). Most studies of online survey participation use as starting point factors that emerged from the mail survey literature. It cannot be automatically assumed, however, that these same factors drive online survey participation. Some factors may not be relevant in the online context, and entirely new factors may exist. Another gap is the lack of investigation of heterogeneity among online survey participants. This is despite compelling empirical evidence of the existence of substantial heterogeneity (Brüggen et al., 2011). Leverage-salience theory (Groves et al., 2000) postulates the existence of an individual threshold, representing maximum heterogeneity among potential survey participants. But there is still a lack of empirical evidence of both individual thresholds and segment-level differences in the importance of drivers of survey participation. Our study investigates segment-level differences.

We conduct qualitative research to identify drivers of online survey participation. Based on the drivers identified in our qualitative research, we conduct a stated choice experiment to gain insights into the relative importance of each of those drivers for the survey population as a whole and market segments.

Qualitative study

Method

Two researchers conducted face-to-face interviews with Australian study participants recruited by an offline, qualitative recruitment company. We deliberately recruited offline to ensure that we would initially capture the full range of possible drivers of survey participation, some of which may or may not be relevant to online survey panel members. Interviews took about 60 min, and participants received a small payment.

During data collection, we monitored additional insights gained. After the first 11 interviews, no more new insights emerged, suggesting we had reached data saturation (Glaser & Strauss, 2017). We conducted an additional eight interviews to make sure that this was indeed the case and that we were not missing any new potential drivers. Of the final 19 study participants, 10 were female and nine male; 14 have participated in web surveys in the past 12 months and five have not; and 12 were members of an online research panel and seven were not.

To avoid social desirability bias, we used projective techniques at the beginning of the interview. Specifically, we asked participants to describe people who participate in online surveys, people who do not, people who are members of online survey research panels, and people who are not. We then initiated a conversation about enablers and barriers to survey participation by the study participants themselves.

To ensure validity of participants’ behavioral self-reports, we checked evidence of their survey participation behavior. Twelve participants who were members of online survey panels showed us their panel history to confirm that their actual online survey response behavior matched their stated behavior. This part of the interview also offered an additional opportunity for participants to comment on why they had accepted some survey invitations, but not others.

The two researchers analyzed the data resulting from all 19 interviews separately, identifying drivers and barriers, grouping them into themes, and grouping themes into domains. After the independent analysis, the researchers compared results to ensure reliability. This qualitative data informed the development of the conceptual model of online survey participation.

Findings

Ten drivers emerge: incentive payments, speed of completion, ease of completion, topic interest, software functionality, benefit to others, topic knowledge, impact, relationship with brand/organization, and respondent’s opinions being valued (Figure 1). These drivers are not independent; they can reinforce or compensate for one another. The following quotes from study participants illustrate these interaction effects: “I weigh it all up before I decide,” “It really depends on a number of things not just one thing,” “If it has a good incentive I will persevere with a boring topic,” “If I know a lot about it I feel like I should do it even if it is not that interesting.”

Figure 1.

Quantitative survey stated preference conditions.

Knowledge about and interest in the topic

Participants state that knowledge of the topic, product, service, or category is a major driver of their survey participation. Respondents want to feel like an individual, not just part of a crowd. They feel that their knowledge of the topic and their willingness to participate make them special. Knowledge makes the respondent able to complete the survey competently. The following quotes illustrate study participants desire to be treated like subject matter experts: “I want to know that I am chosen because of what I know,” “They ask me because I know about this stuff and I am the right person to do the survey.” Study participants understand knowledge as their own, subjective perception of their ability, authority, and expertise: “Knowing about these products makes it easy for me to answer,” “They need people like me to do the survey.” Feelings of anxiety, inadequacy, and higher cognitive effort result from participants feeling they lack knowledge: “I have to think too hard when I don’t know the topic.” Gaining knowledge also served as a driver: “Learning about topics that I haven’t thought about before or that are interesting to me,” “surveys where you gain knowledge.” Many study participants mentioned that being interested in the topic of the survey motivated them to participate in surveys: “I do it for interesting topics.”

Relationship

Study participants indicated paying more attention to survey invitations if they felt they had a relationship with the organization conducting the survey or the brand under study. They preferred invitations containing a clear, concise explanation, and an indication of the difficulty level of the survey. Respondents highlighted the importance of the reputation of the organization. Reputation, to them, implied a number of aspects, including their personal affiliation with the organization and the confidence that the organization will uphold confidentiality and privacy principles. Participants also cared about the organization being known as listening to the voices of consumers. When the survey invitation is unsolicited, or respondents lack a relationship with the organization conducting the survey, they are more inclined to ignore the request: “If I know who it is from I will open it,” “You get to know the good ones and the bad ones in the survey game.”

Ease of completion

Study participants are more likely to participate in easy surveys than in difficult ones. They assess the level of difficulty by the language and tone of the invitation to participate. Invitations that match respondents’ cognitive ability and communication style increase participation. Complicated invitations prevent participation. Respondents expressed the following preferences and frustrations: “Tell me upfront what the survey is actually about,” “Well worded and clear,” “No poorly worded questions which don’t reflect what they said the survey was about in the introduction,” “I only want to read one or two sentences to decide, not a page,” “When asked in roundabout ways or in several different ways to get an answer rather than just asking directly,” “Thoughtful questions, not marketing nonsense jargon.” Respondents also dislike hypothetical questions and perceive them as much harder, often leading to survey termination: “When I have to imagine an inanimate object to have feelings.”

Software functionality

Design elements emerge as a source of frustration and barrier to survey participation. Respondents prefer basic software functionality, allowing them to complete the survey quickly. Online design features specifically highlighted by study participants included the functionality of the survey software. Functionality drives both online panel membership and survey participation. Experienced respondents are quite aware of which survey software offers good experiences and report a range of frustrations related to function: “When the software doesn’t work, especially if it breaks right at the end,” “When they crash in the middle of a survey,” “You can tell if they are using decent software or the cheap ones that are hard to navigate.”

Speed of completion

Respondents guess the perceived time required to complete an online survey by assessing the perceived length of the task, question format, question order, and the type of response formats. Respondents rarely talk about time in terms of minutes: “quick,” “short,” “don’t take long.” The perception of time is also relative to their survey experience: “I would rather do a few little surveys than a massively long one.” Respondents appear to have lost trust in the accuracy of the time indicated in the survey invitation: “Being invited to a 10-min survey, then kicked out after 14 min as the quota is full or disqualified because you’re not what they are looking for,” “Invited to a survey for 5 min, with a just about acceptable incentive, that then takes 12 min.”

Impact

Believing that completing a survey leads to improvement of products or services increases the likelihood of survey participation. Perceived impact is a key driver of survey participation: “Giving my opinion and influencing future products and research,” “Helping research,” “Helping improve products,” “Feeling you’ve made a difference stated an opinion about something and it changes things,” “a positive outcome for the community.”

Own opinions being valued

Respondents want to express their views and want the opportunity to express through surveys because it makes them feel valued: “Thinking I am valued for helping,” “When it will make a difference and help other people, I feel like they value me.”

Incentive payment

Reinforcing survey completion behavior requires reciprocity. This can be through tangible, extrinsic rewards. Although not all rewards are monetary (“Appreciation when completed,” “The feeling of accomplishment”), incentive payments clearly play a key role (“Getting money/vouchers,” “Being paid for my opinion”). Respondents describe the motivation associated with incentives in one of two ways: “saving up” and “instant gratification.” Respondents who are regular panel members like to save up and use their rewards for Christmas presents or annual treats (“Earning money for extra treats”), recognizing that many small payments accumulate to something desired. Instant gratifiers want free products or cash soon after survey completion and are more motivated by a prize draw.

Benefits to others

Non-panel members or panel members who do not complete surveys are less interested in monetary rewards and more interested in the other intrinsic or social rewards, such as feeling good knowing the survey will benefit others. Those respondents who could discuss how they saw the results of the research or the final product/advertisement were more inclined to have positive sentiment to participate again in the future.

Quantitative study

Method

The aim of the quantitative study was to assess whether the drivers which emerged from the qualitative study phase were indeed effective in changing the participation rate of online survey panel members, and what their effectiveness relative to one another was. We hypothesize that the relative importance of each of the 10 drivers will be different, and that groups of online survey panel members differ with respect to the drivers that affect their survey participation.

We conducted a survey among a randomly drawn sample of 956 panel members in Australia (n = 363), the United Kingdom (n = 299), and New Zealand (n = 294) from three major international online panels. We have deliberately chosen to recruit from major international commercial panels because the way the panel is managed affects the behavior of panel members. By using a major international panel, we ensure that we are capturing mainstream online survey panel respondents. Panel members who completed our survey received the equivalent of eight Australian Dollars in their country’s currency to complete the survey, in line with the online survey panel’s guidelines. The survey was in field for 10 days to avoid any potential bias from early responders. When invitations were sent out, we indicated that the topic of the survey was completing surveys. Given that this is a topic of interest to all panel members, we do not expect a specific response bias from occurring.

The survey included a stated choice experiment (Ben-Akiva, McFadden, & Train, 2019; Hensher, Rose, & Greene, 2015; Quaife, Terris-Prestholt, Di Tanna, & Vickerman, 2018) using combinations of the 10 drivers identified in the qualitative study. Each driver had a positive and a negative level (see Figure 1). Using a fractional factorial design, we created 32 hypothetical survey options (see Figure 2), which were randomly assigned to choice sets. Each respondent received eight choice sets (two respondents completed the full design). For each choice set, respondents indicated whether they would participate in survey A, survey B, or neither of them. The attribute levels used in the choice sets were deliberately formulated using the language used by participants in the qualitative study to ensure that they were meaningful to participating members of the online survey panel. Respondents also answered a few socio-demographic questions, personality items (Rammstedt & John, 2007), and indicated frequency of survey participation, length of panel membership, number of panel memberships, and what annoys and delights them about completing surveys. The survey took some 8 min to complete.

Figure 2.

Leverage-salience theory–based model of online survey participation.

We estimated a latent class model (Hensher et al., 2015) to identify the relative importance of each driver for each segment of respondents. For the multinomial logit (MNL) model, the utility for respondent i when presented with survey option j at choice occasion t is

U_{i j t} = β^{'} X_{i j t} + ε_{i j t}

where X_ijt expresses all survey drivers. The relative weights of each survey driver (parameters β′) are estimated. Unobserved differences (heterogeneity) in utilities are captured by the error term ɛ_ijt. This MNL model assumes the same beta parameters across all respondents. We assume that—among all respondents—there are groups (latent segments) that differ in their survey driver preferences. We assume that different sets of drivers will impact survey participation rate of different respondent segments differently. Given that a respondent is a member of latent segment s (s = 1, . . ., S), the utility for respondent i for survey option j when they make choice t is

U_{i j t} = {β^{'}}_{s} X_{i j t} + ε_{i j t}

where ${β^{'}}_{s}$ is a parameter vector for each latent respondent segment s. From this utility function, we derive choice probabilities, which result in the latent class model. The probability that respondent i, from latent segment s, chooses option j at choice occasion t is

P (y_{i t} = j | s e g m e n t = s) = \frac{\exp ({β^{'}}_{s} X_{i j t})}{\sum_{j = 1}^{J_{i}} \exp ({β^{'}}_{s} X_{i j t})}

For respondent i, the probability of being a member of latent segment s is

P (s e g m e n t = s) = \frac{\exp ({θ^{'}}_{s} Z_{i})}{\sum_{s = 1}^{s} \exp ({θ^{'}}_{s} Z_{i})}

where Z_i represents potentially available additional respondent characteristics that are not associated with a specific choice situation. If additional respondent characteristics are not included in the model estimation, segment probabilities are fixed constants (summing up to 1). Subsequently, each respondent is classified as belonging to the segment for which they have the highest membership probability. We estimate latent segment parameters via maximum likelihood estimation. McFadden’s rho square (ρ² = 1 − LLB/LL0) provides information about the goodness of fit of the estimated model. The minimum Akaike Information Criterion (AIC = −2(LLB − P)) gives an indication of the optimum number of segments to extract (e.g., Gupta & Chintagunta, 1994). For estimation, the dependent variable was the coded choice for a survey option (eight choices per respondent). We effect-coded (−1, 1) the explanatory variables (the survey drivers).

We describe respondent segments using the segmentation variables first. Then we profile them using descriptor variables collected separately to the choice modeling task (Dolnicar, Grün, Leisch, & Schmidt, 2014).

Findings—the relative importance of drivers

Most drivers identified in the quantitative study significantly affect people’s willingness to participate in a survey study. Table 1 contains the MNL model parameters for the positive levels, and the calculated relative importance for each driver is presented in Figure 3. The positive and significant coefficient of the constant indicates that people tend to accept invitations to complete surveys. Receiving a tangible reward (in form of payment) is the most powerful driver, followed by short completion time. Interest in the survey topic is the next strongest driver, followed by the ease with which questions can be answered and the navigational ease of the survey software. Knowledge of the survey topic, a benefit for other people, and survey responses affecting product design also significantly influence survey study participation. For the entire population of survey respondents, having a relationship with the brand or organization they are being asked about in the survey and their responses being valued do not drive survey study participation.

Table 1.

MNL model and segmentation model results.

Driver	Segment 1Mercenary Responders (15%)	Segment 2Decliners (14%)	Segment 3Regular Responders (71%)	1 segment model
Driver	Coefficient	Coefficient	Coefficient	Coefficient
Participation	0.800***	−0.81377***	3.137***	1.332***
Payment
Receive payment	2.798***	0.512***	0.350***	0.547***
Do not receive payment	−2.798***	−0.512***	−0.350***	−0.547***
Quick
Quick to complete	0.631***	0.404***	0.249***	0.267***
Takes a long time to complete	−0.631***	−0.404***	−0.249***	−0.267***
Easy
Easy to answer	0.410***	0.347***	0.172***	0.184***
Difficult to answer	−0.410***	−0.347***	−0.172***	−0.184***
Interest
Interested	0.391***	0.259***	0.198***	0.201***
Not interested	−0.391***	−0.259***	−0.198***	−0.201***
Functionality
Survey software easy to operate	0.323***	0.331***	0.163***	0.176***
Survey software hard to operate	−0.323***	−0.331***	−0.163***	−0.176***
Knowledge
A lot of knowledge	0.310***	0.199***	0.150***	0.159***
No knowledge	−0.310***	−0.199***	−0.150***	−0.159***
Benefit
Other people benefit	0.222*	0.249***	0.163***	0.158***
None benefit	−0.222*	−0.249***	−0.163***	−0.158***
Relationship
Have a relationship	0.208**	0.071	−0.013	0.028
Do not have a relationship	−0.208**	−0.071	0.013	−0.028
Impact
Will have impact	0.179*	0.183***	0.138***	0.129***
Will not have impact	−0.179*	−0.183***	−0.138***	−0.129***
Views known
Important that views are known	0.046	0.060	0.027	0.016
Not important that views are known	−0.046	−0.060	−0.027	−0.016
Segment probabilities	0.150	0.140	0.710
Based on 7,648 choices from 956 respondents Loglikelihood at convergence LL(B)			−6,092.964	−6,632.689
Loglikelihood evaluated at 0 LL(0)			−8,402.187	−8,402.187
ρ²			0.280	0.210
AIC			5,027.000	13,287.400

MNL: multinomial logit; AIC: Akaike information criterion.

***,**,*

Significance at 1%, 5%, and 10% level.

Figure 3.

Relative importance (%) of drivers.

Because panel respondents from three different countries are included in the data collection, we tested for differences between the findings for the three countries and found no significant systematic differences, except for the overall participation propensity. This is an indication that the driver levels were interpreted in the same way in each country and it supports the external validity of the model.

Figure 3 depicts the model of online survey participation resulting from our study: 10 drivers affect participation to differing extent. Incentive payments contribute 29% to the decision to participate; speed of completion 14%; topic interest 11%; ease of completion 10%; software functionality, topic knowledge, and benefit to others 9% each; impact 7%; the relationship with the brand or organization conducting the survey 1%; and the fact that their opinions are valued 1%.

In line with leverage-salience theory, our model of online panel survey participation assumes that respondents compare the perceived cost–benefit combination of a specific survey with an internal threshold. This internal threshold is illustrated in Figure 3 as an area. The higher the benefits of a survey, the larger the area, making the survey more attractive to potential participants. Each survey has a different combination of costs and benefits. In Figure 3, the hypothetical survey represented by the blue shading has a much larger area, suggesting participation rate will be higher than for the hypothetical survey represented by the green shading; the blue area is higher than the personal threshold of respondents more frequently than this is the case for the green area survey. Developers of a survey can use the model to identify opportunities to increase the benefit and, with it, the likelihood of the surpassing panel members’ online survey participation threshold.

Findings—heterogeneity of drivers among segments of survey respondents

The MNL model shown in the right column of Table 1 assumes that all respondents react in the same way to different drivers. This is not necessarily the case. We assume the existence of segments, groups of survey respondents, who are motivated by different drivers. The latent class analysis results in three market segments, representing 15%, 14%, and 71% of respondents, respectively. Table 1 contains the three-segment-level model in the first three columns.

The estimated constants show that 79% of the respondents would accept a survey invitation based on their stated choices. However, there is substantial heterogeneity among groups of respondents: 96% of members of Segment 3 would participate in one of the two surveys offered to them in the choice experiment, but only 60% of members of Segment 1 and 44% of Segment 2 would accept one of those invitations.

By taking part in this study, we know that these panel members are systematically different. To measure potential bias, we pro-actively informed panel members that our study is about survey participation, a topic that is of some interest. Retrospectively, we checked the extent of potential participation bias in our study in two ways. (a) We analyzed how many survey respondents did not select either of the two survey options in our choice model. At aggregate level, 21% respondents did not select one of the alternatives. At segment level, non-selection ranged from 4% to 56%. (b) We analyzed the participation rates of the panel members who completed our study across survey invitations they have received during their lifetime as pane members. This rate ranges from 0.9% to 98.8% (53.4% on average). These numbers suggest that we did not only capture the subset of very active online panel members.

Figure 4 visualizes the relative importance of each driver for each segment. Segment 1 members tend to accept invitations to complete surveys. For members of this segment, the decision to complete a survey is affected by all drivers, except personal attitudes toward survey responding. That means that telling this segment that their opinion matters is unlikely to increase survey participation. The most powerful driver for Segment 1 is payment for completion. This segment is driven by the incentive payment more than any other segment, which is why we refer to them as Mercenary Respondents. Knowing much about the survey topic and having a relationship to the brand or organization that respondents are asked about also positively influences the likelihood of Mercenary Respondents participating in survey studies. In fact, they are the only respondents motivated by brand relationship. The higher the Mercenary Responders’ knowledge about the topic of the survey and the more connected they feel to the brand or organization in question, the higher the likelihood of them participating in the survey.

Figure 4.

Relative importance (%) of drivers.

Table 2 outlines the demographic profiles of each segment where statistically significant differences were identified. The Mercenary Respondents are the smallest segment (n = 129) and have proportionally more 30- to 59-year-olds than the other two segments. Most Mercenary Respondents are working, either full-time (60.5%) or part-time (17.8%), with a small portion retired (11.6%). Over a third of Mercenary Respondents use their laptop (41.1%), whereas less use their PC (31.8%), iPad (4.7%), or tablet (3.9%). Mercenary Respondents are more likely to have been on a panel for more than 2 years (71.3%), or more than 12 months, but less than 2 years (17.8%), which makes them the segment with the highest longevity on an online panel.

Table 2.

Demographic characteristics by segment.

Variable	Item	Segment 1Mercenary Responders (15%)	Segment 2Decliners (14%)	Segment 3Regular Responders (71%)	Total	p value
Age (years)	18–24	3.9%	3.7%	7.4%	6.4%	.001
	25–29	4.7%	2.2%	7.7%	6.5%
	30–34	14.0%	5.1%	11.4%	10.9%
	35–39	14.7%	9.6%	11.7%	11.8%
	40–44	12.4%	7.4%	9.7%	9.7%
	45–49	13.2%	11.0%	9.1%	9.9%
	50–54	12.4%	13.2%	9.8%	10.7%
	55–59	12.4%	11.0%	8.8%	9.6%
	60–64	1.6%	14.7%	8.4%	8.4%
	65 and over	10.9%	22.1%	15.9%	16.1%
Device used	PC	31.8%	46.3%	28.4%	31.4%	.000
	Laptop	41.1%	25.7%	34.3%	34.0%
	Tablet	3.9%	8.1%	6.1%	6.1%
	iPad	4.7%	9.6%	6.8%	6.9%
	Smartphone	18.6%	10.3%	24.5%	21.7%
	PC	31.8%	46.3%	28.4%	31.4%
Longevity on a panel	Only just joined	1.6%	3.7%	5.1%	4.4%	.016
	More than a month	3.9%	4.4%	8.8%	7.5%
	More than 6 months	5.4%	8.8%	9.3%	8.7%
	More than 12 months	17.8%	15.4%	21.0%	19.8%
	Longer than 2 years	71.3%	67.6%	55.9%	59.6%
Employment	Full-time (30 hr or more a week)	60.5%	49.3%	56.7%	56.2%	.023
	Part-time (less than 30 hr a week)	17.8%	13.2%	17.4%	16.8%
	Not currently working but looking for work	3.9%		3.8%	3.2%
	Not currently working not looking for work	3.9%	4.4%	3.9%	4.0%
	Retired	11.6%	27.9%	15.1%	16.4%
	Not able to work	1.6%	3.7%	2.5%	2.5%
	Prefer not to say	0.8%	1.5%	0.7%	0.8%

Members of Segment 2, or Decliners, are characterized by a low baseline motivation to participate in survey studies (significantly negative constant). They frequently decline offers to complete questionnaires. Of all the market segments, Segment 2 is least reactive to incentive payments, although a monetary reward for participation still significantly increases the likelihood of completing a survey. Survey design plays a major role for Decliners. How easy it is to answer the survey questions, how quickly the survey can be completed, and how convenient the navigation of the survey software are all very important to Decliners’ decision to participate in a study. Decliners do care about other people benefiting from them completing the survey, more so than other market segments. Having a relationship with the brand or organization in question, and the fact that their voice is heard do not affect Decliners’ participation.

Decliners are a small segment (n = 136). Proportionally, they have the most (61%) over 50-year-olds with the highest proportion (22%) of respondents over 65. Considering Decliners’ older age skew, they are also most likely of the three groups to be retired (27.9%) and least likely to be working (49.3%). Decliners’ most used device is their PC (46%), followed by laptop (26%), iPad (10%), or tablet (8%). Two thirds of Decliners have been on a panel for more than 2 years (67.6%), or more than 12 months but less than 2 years (15.4%), making them the segment with the second highest longevity on an online panel. (Table 2)

The largest group is Segment 3 (n = 691), or Regular Responders, which contains more than two thirds of all respondents. Regular Responders have a very high inclination to complete surveys when invited (as indicated by a highly significant and positive constant). The monetary incentive represents a significant motivator for members of this segment but is not as dominant a factor as it is for Mercenary Respondents. Rather than one single driver dominating the decision whether or not to complete the survey, these respondents are affected by a range of drivers, but less strongly. Questions that are easy to answer, quick survey completion times, highly functional survey software, and knowledge and interest in the topic positively affect Regular Responders’ willingness to complete surveys. Regular Responders are also motivated by outcomes they imagine may result from the survey, including benefits to people and redesign of products and services.

With the highest proportion of respondents under 29, the Regular Responders are the most evenly proportioned across all age groups. Most Regular Responders are working, either full-time (56.7%) or part-time (17.4%), and a smaller percent of them are not working (retired 15.1%, not looking for work 3.9%, looking for work 3.8%, or unable to work 2.5%). A third of Regular Responders use their laptop (34%), PC (28%), iPad (6.8%), or tablet (6.1%). Over half of the Regular Responders have been on a panel for more than 2 years (55.9%), and a fifth have been on a panel for more than 12 months, but less than 2 years (21.0%). Regular Responders have the highest proportion of new members (just joined 5.1%, over a month 8.8%, 6–12 months 9.3%; Table 2)

Table 3 outlines the Big Five Inventory (John, Donahue, & Kentle, 1991) personality profiles of each segment, where statistically significant differences were identified. Openness shows significant differences between groups. Regular Responders (Bonferroni post hoc test, p = 0.005) rate themselves as having more artistic interests and a more active imagination (Openness mean score = 6.751) than the Mercenary Respondents (Openness mean score = 6.310).

Table 3.

Big Five Inventory personality traits by segment.

	Extraversion	Agreeableness	Conscientiousness	Neuroticism	Openness
Segment 1: Mercenary Responders
M	6.178	7.085	7.698	5.372	6.310
SD	1.730	1.420	1.477	1.908	1.457
Segment 2: Decliners
Mean	6.412	7.125	7.757	5.478	6.551
SD	1.828	1.503	1.599	1.809	1.304
Segment 3: Regular Responders
Mean	6.221	7.129	7.692	5.441	6.751
SD	1.743	1.569	1.547	1.842	1.478
1 segment model
M	6.243	7.122	7.702	5.437	6.663
SD	1.753	1.539	1.544	1.845	1.459
F	.770	.044	.103	.115	.485
p value	.463	.957	.902	.891	.004

Our proposed model of online panel survey participation can be used to modify online surveys to maximize the desirable features. Figure 3 illustrates this: The hypothetical survey depicted in green has many undesirable features but offers a high incentive payment. As a consequence, the area (which can be interpreted as survey participation likelihood) is small. In contrast, the hypothetical survey depicted in blue has many desirable features. As a consequence, the blue area is high, indicating that such a survey is more likely to lead to higher online survey panel member participation. Developers of a survey can use our model to identify areas in which they may be able to increase the benefit (for all panel members or at least specific segment of panel members) and, with it, the likelihood of increasing survey participation. Most of the drivers in our model can be influenced by the researcher, even fundamental drivers such as the ease of use of the survey can be modified by assessing whether the survey platform used does indeed offer respondents the most seamless survey completion experience.

Conclusion, limitations, and future work

Online panels rely on large numbers of panel members who are willing to complete surveys. Although panel members agree in principle to participating in surveys when they join an online panel, they are still free to choose which survey invitations they wish to accept and which they wish to decline. Although online panels are reluctant to provide response rate information because of the commercial sensitivity of this information in terms of their performance, and because of newly developed sampling techniques such as river sampling or real time sampling (Baker et al., 2010), the market and social research industry does acknowledge being challenged by low survey participation from online panel members (ESOMAR & GRBN, 2015).

Our study proposes a model of online panel survey participation that assists market and social researchers to design online surveys in ways that maximize participation, much like Dillman’s (1978) total design method did for mail surveys or the tailored design method (Dillman et al., 2014) did for multimodal surveys. Our model of online panel survey participation is based on leverage-salience theory (Groves et al., 2000): It assumes that each survey is viewed by respondents as a combination of costs and benefits. Each respondent compares the expected net benefit of any given survey with their internal threshold of how much benefit is enough to justify their participation. In our model, the cost–benefit combination is multi-dimensional, consisting of 10 drivers. The joint benefit derived from those 10 drivers determines the net benefit value the respondents compares with their internal threshold.

Building on the empirical evidence that panel members are heterogeneous in their motivations to participate in surveys (Brüggen et al., 2011) and extending the conceptional framework of Fan and Yan (2010), we test multiple drivers of online survey participation including motivations and perceptions of survey functionality and experience. The 10 drivers identified using qualitative research methodology include incentive payments, speed of completion, ease of completion, topic interest, software functionality, benefit to others, topic knowledge, impact, relationship with brand/organization, and respondent’s opinions being valued. Some of these drivers reflect aspects previously identified as increasing mail survey participation (Dillman, 1991), as well as aspects tested in isolation in the context of online panel survey responding.

A stated choice experiment reveals that these 10 drivers affect survey participation differently. Across all online panel members, the incentive payment is the primary driver, in contrast to previous findings (Brüggen et al., 2011). Other drivers that substantially increase survey participation include speed of completion, topic interest, ease of completion, software functionality, topic knowledge, benefit to others, and impact.

Each of the 10 drivers affects online panel survey participation of specific respondents’ segments differently. A small segment of Mercenary Respondents are most influenced by the monetary incentive payment, but are also discerning of survey characteristics such as their interest in the topic, survey length, software functionality, and ease of completion. Mercenary Respondents invest the least possible effort. Decliners are similarly interested in the monetary incentive, but are not influenced by knowledge or interest in the topic. They have the highest rate of declining invitations. The largest segment is that of Regular Responders. They are the most likely segment to complete a survey. A monetary incentive is the strongest driver of participation, but it is not as important as it is for Mercenary Respondents and Decliners. Regular Responders are not overly concerned about survey characteristics or having a relationship with the brand/organization, but they do care about their responses benefiting others.

Our study contributes to the large body of work on survey participation in a number of ways. (a) It starts with a qualitative investigation to ensure the broadest possible range of drivers is revealed. This approach stands in stark contrast to prior work which typically uses experiments to postulate that the survey context being tested is the only driver of behavior. (b) It empirically investigates the impact of the 10 drivers identified in the qualitative study in combination with one another, thus acknowledging the interaction of drivers in line with leverage-salience theory (Groves et al., 2000). (c) By acknowledging that each survey has its own specific combination of costs and benefits, the resulting model of online panel survey participation overcomes the weakness of the total design method (Dillman, 2000). Finally, (d) the investigation of segment-level differences in the importance of drivers of survey participation makes one step in the direction of accounting for heterogeneity of benefit thresholds across survey respondents postulated by leverage-salience theory (Groves et al., 2000). While a lot is already known about online survey participation, our study provides empirical evidence to support managerial decisions to prioritize specific drivers to maximize response rate for online panels.

Our study has a few limitations. First of all, a choice model is hypothetical in nature. It would have been preferable to conduct this study as a field experiment, but given the large number of drivers and the very specific attribute levels, it was not practically feasible to obtain combinations of real survey studies which vary in the required way. Using stated choice, modeling was the next best option, given that predictive validity for choice models using behavior as dependent variable is known to be high (e.g., Swait, 1994). Also, repeating the qualitative study in other countries and contexts may reveal additional drivers. In terms of our model, this is not problematic as additional drivers can easily be integrated and the study replicated. Our conclusions are based on stated preferences. Despite our best attempts to reduce response bias, it is likely that our study is still subject to some degree of non-response bias, as those less likely to respond to a survey about online surveys may have ignored this invitation to participate.

Additional insights could be gained through (a) experimental methods to identify the link with data quality and (b) observations of participants through eye-tracking to identify the association with attentive completion (Brosnan, Babakhani, & Dolnicar, 2019; Lenzner, Kaczmirek, & Galesic, 2011). The stated choice experiment is hypothetical in nature: Study participants did not actually complete the survey they agreed to complete. Study 2 could be replicated in the actual online survey context by selecting surveys that—naturally—have high or low values across the 10 drivers and include self-report questions to determine the perceptions of these drivers for each respondent. Furthermore, all the respondents in the stated choice experiment are, by definition, people willing in principle to participate in online surveys because they have signed up to be members of an online panel. We are not capturing those who refuse entirely, that is, those whose benefit threshold of participation is particularly high. Our findings can also not be used to recruit new online survey panel members, rather they are limited to increasing participation rates among already existing online survey panel members. The study was conducted in English in the United Kingdom, New Zealand, and Australia and may fail to capture drivers relevant to other cultures.

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Australian Research Council Grants LP130100548 (project funding and salary funding), DP110101347 (salary funding) and DP0878338 (data).

ORCID iD

Sara Dolnicar

References

Baker

Blumberg

S. J.

Brick

J. M.

Couper

M. P.

Courtright

Dennis

J. M.

. . . Groves

R. M.

(2010). Research synthesis: AAPOR report on online panels. Public Opinion Quarterly, 74, 711–781.

Ben-Akiva

McFadden

Train

(2019). Foundations of stated preference elicitation: Consumer behavior and choice-based conjoint analysis. Foundations and Trends in Econometrics, 10, 1–144. doi:10.1561/0800000036

Boyle

J. M.

Iachan

Freedner-Maguire

Fakhouri

T. H.

(2017). Characteristics of the population of Internet panel members. Survey Practice, 10, Article 2771. doi:10.29115/SP-2017-0025

Brosnan

Babakhani

Dolnicar

(2019). “I know what you’re going to ask me” Why respondents don’t read survey questions. International Journal of Market Research, 61, 366–379. doi:10.1177/1470785318821025

Brüggen

van den Brakel

J. A.

Krosnick

(2016). Establishing the accuracy of online panels for survey research (Discussion Paper 4, Statistics Netherlands, The Netherlands). Retrieved from https://www.cbs.nl/en-gb/background/2016/15/establishing-the-accuracy-of-online-panels-for-survey-research

Brüggen

Wetzels

De Ruyter

Schillewaert

(2011). Individual differences in motivation to participate in online panels. International Journal of Market Research, 53, 369–390.

Childers

T. L.

Skinner

S. J.

(1996). Toward a conceptualization of mail survey response behavior. Psychology & Marketing, 13, 185–209.

Cobanoglu

(2003). The effect of incentives in web surveys: Application and ethical considerations. International Journal of Market Research, 45, 475–488.

de Bruijne

Wijnant

. (2014). Mobile response in web panels. Social Science Computer Review, 32, 728–742.

10.

Dillman

D. A.

(1978). Mail and telephone surveys. New York, NY: Wiley Interscience.

11.

Dillman

D. A.

(1991). The design and administration of mail surveys. Annual Review of Sociology, 17, 225–249.

12.

Dillman

D. A.

(2000). Mail and Internet surveys: The tailored design method. New York, NY: John Wiley.

13.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. New York, NY: John Wiley.

14.

Dolnicar

Grün

Leisch

Schmidt

(2014). Required sample sizes for data-driven market segmentation analyses in tourism. Journal of Travel Research, 53, 296–306.

15.

European Society for Opinion and Market Research (ESOMAR). (2013). Global Market Research 2013. Retrieved from https://www.esomar.org/uploads/industry/reports/global-market-research-2013/ESOMAR-GMR2013-Preview.pdf

16.

European Society for Opinion and Market Research (ESOMAR), & the Global Research Business Network (GRBN). (2015). Guideline for online sample quality. Retrieved from https://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/Online-Sample-Quality-Guideline_February-2015.pdf

17.

Fan

Yan

(2010). Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior, 26, 132–139.

18.

Fang

Shao

Lan

(2009). Effects of innovativeness and trust on web survey participation. Computers in Human Behavior, 25, 144–152.

19.

Glaser

B. G.

Strauss

A. L.

(2017). Discovery of grounded theory: Strategies for qualitative research. New York, NY: Routledge.

20.

GreenBook Research Industry Trends Report. (2019). 2019 business & innovation GRIT report (Business and Innovation ed.). https://www.flipsnack.com/GRITarchive/2019-business-innovation-grit-report.html

21.

Gritz

A. S.

(2004). The impact of material incentives on response quantity, response quality, sample composition, survey outcome and cost in online access panels. International Journal of Market Research, 46, 327–345.

22.

Groves

R. M.

Couper

M. P.

(2012). Nonresponse in household interview surveys. Chicago, IL: Interscience.

23.

Groves

R. M.

Singer

Corning

(2000). Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly, 64, 299–308.

24.

Gupta

Chintagunta

P. K.

(1994). On using demographic variables to determine segment membership in logit mixture models. Journal of Marketing Research, 31, 128–136.

25.

Haunberger

(2011). To participate or not to participate: Decision processes related to survey non-response. Bulletin of Sociological Methodology, 109, 39–55.

26.

Hensher

D. A.

Rose

J. M.

Greene

W. H.

(2015). Applied choice analysis: A primer. Cambridge, UK: Cambridge University Press.

27.

Huang

J. Y.

Hubbard

S. M.

Mulvey

K. P.

(2003). Obtaining valid response rates: Considerations beyond the tailored design method. Evaluation and Program Planning, 26, 91–97.

28.

John

O. P.

Donahue

E. M.

Kentle

R. L.

(1991). The big five inventory—Versions 4a and 54. Berkeley: University of California, Berkeley.

29.

Joinson

A. N.

Reips

U. D.

(2007). Personalized salutation, power of sender and response rates to Web-based surveys. Computers in Human Behavior, 23, 1372–1383.

30.

Keusch

(2013). The role of topic interest and topic salience in online panel web surveys. International Journal of Market Research, 55, 59–80.

31.

Laguilles

J. S.

Williams

E. A.

Saunders

D. B.

(2011). Can lottery incentives boost web survey response rates? Findings from four experiments. Research in Higher Education, 52, 537–553.

32.

LaRose

Tsai

H. Y. S.

(2014). Completion rates and non-response error in online surveys: Comparing sweepstakes and pre-paid cash incentives in studies of online behavior. Computers in Human Behavior, 34, 110–119.

33.

Lenzner

Kaczmirek

Galesic

(2011). Seeing through the eyes of the respondent: An eye-tracking study on survey question comprehension. International Journal of Public Opinion Research, 23, 361–373.

34.

Manfreda

K. L.

Bosnjak

Berzelak

Haas

Vehovar

(2008). Web surveys versus other survey modes. International Journal of Market Research, 50, 79–104.

35.

Mavletova

Couper

M. P.

(2016). Grouping of items in mobile web questionnaires. Field Methods, 28, 170–193.

36.

Pedersen

M. J.

Nielsen

C. V.

(2016). Improving survey response rates in online panels: Effects of low-cost incentives and cost-free text appeal interventions. Social Science Computer Review, 34, 229–243.

37.

Pennay

Neiger

Lavrakas

P. J.

Borg

(2018). The Online Panels Benchmarking Study: A total survey error comparison of findings from probability-based surveys and non-probability online panel surveys in Australia. Canberra, Australian Capital Territory: Australian National University.

38.

Quaife

Terris-Prestholt

Di Tanna

G. L.

Vickerman

(2018). How well do discrete choice experiments predict health choices? A systematic review and meta-analysis of external validity. The European Journal of Health Economics, 19, 1053–1066.

39.

Rammstedt

John

O. P.

(2007). Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. Journal of research in Personality, 41, 203–212.

40.

Sánchez-Fernández

Muñoz-Leiva

Montoro-Ríos

F. J.

(2012). Improving retention rate and response quality in Web-based surveys. Computers in Human Behavior, 28, 507–514.

41.

Sauermann

Roach

(2013). Increasing web survey response rates in innovation research. Research Policy, 42, 273–286.

42.

Sax

L. J.

Gilmartin

S. K.

Bryant

A. N.

(2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44, 409–432.

43.

Singer

(2011). Toward a benefit-cost theory of survey participation: Evidence, further tests, and implications. Journal of Official Statistics, 27, 379–392.

44.

Swait

(1994). A structural equation model of latent segmentation and product choice for cross-sectional revealed preference data. Journal of Retailing and Consumer Services, 1, 77–89.

45.

Tangmanee

Niruttinanon

(2015). Effects of forced responses and question display styles on web survey response rates. International Journal of Research in Business and Social Science, 4, 54–62.

46.

Yammarino

F. J.

Skinner

S. J.

Childers

T. L.

(1991). Understanding mail survey response behavior: A meta-analysis. Public Opinion Quarterly, 55, 613–639.

47.

Young

J. M.

O’Halloran

McAulay

Pirotta

Forsdike

Stacey

Currow

(2015). Unconditional and conditional incentives differentially improved general practitioners’ participation in an online survey: Randomized controlled trial. Journal of Clinical Epidemiology, 68, 693–697.

48.

Zillmann

Schmitz

Skopek

Blossfeld

H. P.

(2014). Survey topic and unit nonresponse. Quality & Quantity, 48, 2069–2088.