Web Surveys by Smartphones and Tablets

Abstract

Does completing a web survey on a smartphone or tablet computer reduce the quality of the data obtained compared to completing the survey on a laptop computer? This is an important question, since a growing proportion of web surveys are done on smartphones and tablets. Several earlier studies have attempted to gauge the effects of the switch from personal computers to mobile devices on data quality. We carried out a field experiment in eight counties around the United States that compared responses obtained by smartphones, tablets, and laptop computers. We examined a range of data quality measures including completion times, rates of missing data, straightlining, and the reliability and validity of scale responses. A unique feature of our study design is that it minimized selection effects; we provided the randomly determined device on which respondents completed the survey after they agreed to take part. As a result, respondents may have been using a device (e.g., a smartphone) for the first time. However, like many of the prior studies examining mobile devices, we find few effects of the type of device on data quality.

Keywords

data quality measurement error smartphones mobile devices

Tourangeau, Conrad, and Couper (2013) argue that web surveys have good measurement properties. However, their conclusion is based on experiments and other results from web surveys that were almost exclusively done on personal computers (PCs). A growing proportion of web surveys are now completed on smartphones or tablets rather than laptop or desktop computers (Lugtig & Toepoel, 2015; Mavletova, 2013; Struminskaya, Weyandt, & Bosnjak, 2015). Although much of the work on smartphones for web surveys has been done in Europe, many web surveys are being completed on tablets or smartphones in the United States as well.

Smartphones may increase various forms of nonresponse to web surveys as compared to PCs, including the rates of unit nonresponse, item nonresponse, and breakoffs (see Couper, Antoun, & Mavletova, 2017). However, it is not clear whether web surveys done on smartphones maintain the desirable measurement properties of web surveys done on other types of devices. There are at least three reasons to suspect that they may not.

First, the screens on smartphones are smaller than those on tablets or PCs and smartphone respondents may have to scroll to see the entire question or the full set of response options. This could lead to larger response order effects or to more superficial processing of the questions. Web respondents are more likely to select response options that they can see immediately than options that require an action (such as scrolling) to render them fully visible (Couper, Tourangeau, Conrad, & Crawford, 2004). The need to scroll on a smartphone may make it less likely that respondents will see and consider all the options or process other information they need to give an adequate answer to the question. However, a recent paper by Tourangeau et al. (in press) found that smartphone respondents did not show larger response order effects and were no more prone to ignore definitions than respondents who completed the same survey on a tablet or laptop—in fact, smartphone respondents seemed more likely to attend to definitions than tablet or laptop respondents. It is possible that respondents expect to have to scroll based on their experience with smartphones or that they find scrolling no more effortful than making eye movements.

Another source of concern has been the touch screen interface. Both tablets and smartphone use this interface, but the smaller screens on smartphones could interact with the touch screen interface to make it harder for smartphone respondents to register their answer, leading to item nonresponse or inadvertent choice of the wrong answer.

A third source of concern is that smartphone respondents may complete surveys in settings where other people or other potential distractions are present. This could lead to less candid reporting of sensitive information or sloppier reporting overall. However, Mavletova and Couper (2013), as well as a recent replication by Toninelli and Revilla (2016), found little support for these hypotheses.

The studies that have examined how collecting data on smartphones affects the answers respondents give have generally taken two approaches. Some have collected data only on smartphones but have included experiments within that mode, manipulating the number of items per screen or whether the survey used a paging or scrolling design (de Bruijne & Wijnant, 2014; Mavletova & Couper, 2014, 2015; Peytchev & Hill, 2010). Others compared smartphone responses with those obtained by PC or tablet, sometimes in randomized experiments (de Bruijne & Wijnant, 2013; Keusch & Yan, 2016; Mavletova, 2013; Mavletova & Couper, 2015; Wells, Bailey, & Link, 2014) and sometimes in quasi-experimental designs (Lugtig & Toepoel, 2015; Struminskaya et al., 2015).

These past studies support several generalizations:

Smartphone respondents have higher levels of item nonresponse than those completing the survey on tablets or PCs (Keusch & Yan, 2016; Lugtig & Toepoel, 2015; Struminskaya et al., 2015; but see Andreadis, 2015 ; Buskirk & Andrus, 2014).

It takes more time to complete a survey on a smartphone than on a tablet or PC (Andreadis, 2015; Buskirk & Andrus, 2014; de Bruijne & Wijnant, 2013; Keusch & Yan, 2016; Struminskaya et al., 2015).

Answers to narrative open-ended questions are often shorter on smartphones than on tablets or PCs (de Bruijne & Wijnant, 2013; Mavletova, 2013; Struminskaya et al., 2015), though not always (Lugtig & Toepoel, 2015; Toepoel & Lugtig, 2014).

Tourangeau and his colleagues (in press) provide a more comprehensive summary of the prior work comparing smartphones with tablets or PCs (see also Couper et al., 2017).

Still, the past studies have their limitations. Most of them were done in Europe and, with the ones in Russia, some respondents completed the survey on low-tech feature phones rather than on smartphones. In addition, the experiments are often prey to substantial noncompliance, with respondents randomly assigned to one device instead using another device to do the survey. On the whole, though, the results are reassuring—responses by smartphones appear to be very similar to responses obtained by tablet or PC and the differences that are found (such as differences in item nonresponse) are not very large (Couper et al., 2017).

We carried out a randomized field experiment that attempted to minimize some of the problems with the prior work. First, our study used samples selected via address-based sampling, and respondents were contacted and recruited by field interviewers. Second, we randomly assigned respondents to one of the three devices (a smartphone, tablet, or laptop computer). To reduce any selection effects, we informed participants which device—smartphone, tablet, or laptop—they were assigned to only after they had agreed to complete the questionnaire. Third, the interviewers provided the device, so that all the interviews were done on the same smartphone, tablet, or laptop. Noncompliance was not possible in our study; the web program would only accept data provided on the assigned device.

We examined five main outcomes:

item nonresponse,

straightlining;

completion times,

scale reliability (assessed by coefficient α, scale information functions, and differential item functioning [DIF]), and

scale validity (assessed by construct validity coefficients).

In each case, we tested the hypothesis that the outcomes would differ across the three types of devices.

Method

Sample

We selected address-based samples in each of eight counties around the United States; the samples in each county were designed to represent the adult household population there. The counties themselves were selected purposively based on the availability of experienced field staff. Still, the eight sample counties encompassed a range of different types of areas. Seven were in metropolitan statistical areas and the other was rural; one of the seven metropolitan areas involved a small central city. The eight counties included all four Census regions.

Within each county, we selected a full probability sample of addresses (i.e., every address had a known and nonzero probability of selection.) We used Census block groups as the second-stage sampling units (SSUs). Six SSUs were selected from each county with probabilities proportional to size (in housing units). One SSU was on a military base and was replaced with the next SSU in the county. In total, we selected 1,824 addresses, 228 within each sample county and 38 in each sample SSU. One hundred and forty-two of the addresses were vacant, could not be found, or turned out not to be dwellings, leaving a final sample of 1,682 occupied dwelling units.

When they contacted a household, interviewers tried to complete a short screener with an adult there. In households with just one adult, he or she was selected for the main interview; in households with two adults, one of the two was selected randomly; and in households with three or more adults, the last birthday method was used to select one for the main interview. Once the sample person had been designated, he or she was asked to complete the interview on a device that had been randomly determined ahead of time. Sample persons were offered a $20 incentive to complete the questionnaire.

Data Collection

Experienced field interviewers conducted data collection from January 21, 2015, through May 4, 2015. They completed 512 interviews, for an overall response rate (American Association for Public Opinion Research [AAPOR] response rate 3 (RR3); see AAPOR, 2016) of 30.4%. The response rates (smartphone 32.9%, tablet 29.6%, and laptop 28.8%) did not differ significantly across the three device conditions (Rao–Scott χ² = 4.13 with 2 df). Only two cases could not be completed because of problems with cellular coverage, one each in the smartphone and laptop conditions.

Respondents were asked to complete the questions on a device that the interviewer provided while the interviewer remained present.¹ Interviewers were instructed to complete administrative tasks while the respondents answered the survey questions. The overall topic of the questionnaire was health. The interviewer could not change the assigned device type, and the web instrument was programmed to accept data only from the assigned device type.

We examined the demographic makeup of the three device groups, by age, education, sex, Hispanic origin, and race (White vs. non-White). We compared the three device groups on these variables, using the SAS survey procedures that took the clustering of the sample into account. The smartphone group had significantly more females (64.2%) than the tablet and laptop groups (both 50.0%); Rao–Scott χ² = 8.58 with 2 df (p < .05). In addition, the three device groups differed significantly in their racial composition (Rao–Scott χ² = 17.4 with 2 df, p < .001), with the laptop group having the highest proportion of Whites (85.8%), the smartphone group, the lowest (78.1%), and the tablet group in between (80.1%). The three device groups did not differ on the other demographic variables. In the subsequent analyses, we included race and sex as covariates (because they differed across the device types) as well as age and education (because we thought they were likely to be related to the size of any device effects).

Data Collection Devices

The key experimental variable in the study was the type of device respondents used to complete the survey. The devices differed in screen size and input method. The laptop featured a mouse and keyboard, whereas the smartphone and tablet used touch screen interfaces.

Each interviewer was provided with a smartphone (an iPhone 5S), a tablet (an iPad Air), and a laptop (a Fujitsu Lifebook E753). We chose these specific models because they varied markedly in screen size (see Figure 1, which shows the relative sizes of each device). Most tablet and smartphone respondents completed the survey in “portrait” orientation, and Figure 1 displays the screens in that orientation. (The percentage of respondents in portrait mode ranged from 52% to 56% in the tablet condition, depending on the specific screen, and from 88% to 98% in the smartphone condition.) Although the web program did not display the entire laptop screen (see the dashed area in Figure 1), the area that was displayed was still about 50% larger than the tablet screen. The survey was not optimized for smartphones.

Figure 1.

Relative screen size of the three devices. The dashed outline shows actual web browser size displayed on laptop screen. The numbers indicate the size of each device in inches.

Appendix A shows screen shots from the survey for each device type. Each device accessed the web instrument over a cellular network. For the smartphone, this was the telephone’s network. For the tablet and laptop conditions, access was via a wireless hot spot generated by an MiFi device. (An MiFi device is a small, wireless router that generates a WiFi hot spot.) Because the method of getting online was similar for the three devices, download times should have been similar. As a result, completion times are not confounded with differences across devices in connection speeds.

Outcome Measures

We examined five outcome variables. The first was item nonresponse. We counted the number of missing values across the entire questionnaire (a total of 69 items), excluding only the first section. There were no skip patterns involving these items and each respondent should have answered all 69 of them.

The second outcome variable we examined was straightlining (Krosnick, 1991, 1999). The questionnaire included six multi-item batteries. We classified a respondent as a “straightliner” if he or she selected the same response category for every item in any of the six batteries. Straightlining was relatively rare, ranging from less than 1% to 30% across the six batteries. (The items in three of the batteries were displayed one at a time on successive screens, which may have discouraged straightlining.) The analyses presented below grouped respondents by whether they exhibited straightlining on at least one of the six batteries. No one straightlined on all six.

The third outcome variable was the time needed to complete the questions. The program capture the time elapsed from the second section of the questionnaire to the end. (We excluded the first section because interviewers administered that section for about half of the respondents in each device type condition; thus, at the end of the section, the interviewer turned the device over to the respondents who switched to self-administration for the rest of the questions.)

The fourth outcome was the reliability of answers to four batteries of questions, each thought to reflect a single underlying dimension. The four batteries measured subjective well-being, healthy behaviors, perceived social support, and depression. We used coefficient α to measure reliability and compared the three device groups on the α values for each battery. We also conducted item response theory (IRT; Embretson & Reise, 2000) analyses to examine scale information functions and DIF by device type.

The final outcome variable was a measure of scale validity. Specifically, we examined the correlations between the scales and four demographic variables to which they were likely to be related. We compared how these relationships differed across the three devices, on the assumption that weaker relationships indicate higher levels of random measurement error.

Results

The sample counties do not constitute a probability sample, and we did not weight the results. Most of the analyses do take into account the clustering of the data by county, using the SAS survey procedures for this purpose.² The clustering of the sample affects the variability of the results, and the SAS procedures use standard methods to adjust for this (see, e.g., Rao & Scott, 1987). Table 1 shows the key results for the five outcomes we examined.

Table 1.

Data Quality Measures (and Sample Sizes) by Device Type.

Outcome	Device Group			Inferential Statistic
Outcome	Smartphone	Tablet	Laptop	Inferential Statistic
Mean number of missing items	2.18 (180)	1.78 (158)	2.78 (158)	F(2, 7) = 1.82 (ns)
Percent straightlining at least once	50.8 (187)	48.8 (164)	50.9 (161)	Wald χ²(2 df) = 0.82 (ns)
Mean completion times (in minutes)	25.3 (181)	20.8 (162)	22.4 (159)	F(2, 7) = 26.3, p < .001)
Cronbach’s α (standardized)
Scale 1: 6 attitudinal well-being items	.871 (176)	.858 (147)	.904 (138)	χ²(2 df) = 4.56 (ns)
Scale 2: 7 behavioral well-being items	.871 (173)	.892 (148)	.894 (132)	χ²(2 df) = 1.27 (ns)
Scale 3: 19 social support items	.975 (166)	.967 (143)	.977 (127)	χ²(2 df) = 4.42 (ns)
Scale 4: 9 items on depression	.850 (170)	.890 (154)	.866 (141)	χ²(2 df) = 2.95 (ns)
R² with demographic variables
Scale 3: social support	.112 (181)	.089 (161)	.059 (150)	—
Scale 4: depression	.071 (181)	.073 (161)	.170 (150)	—

Note. Parenthetical entries are sample sizes. Mean completion times were computed after we dropped the five fastest and five slowest overall times; the mean number of missing items is based on respondents who answered at least two thirds of the questions.

Item Nonresponse

A few of the sample cases skipped many of the questions. We dropped the 16 “respondents” who skipped a third or more of the questions from our analysis of item nonresponse; they were spread across all three device types. In an analysis of covariance that included the number of missing answers as the dependent variable and device type as the independent variable, with race, sex, age, and education as covariates, we found no significant differences across the three device types (see Table 1).

Straightlining

Overall, about half of the respondents straightlined at least once (50.2%), picking the same answer option for every item in one or more of the six batteries of similar items. This figure rises to about 70% if we include respondents who gave the same answer to every item or to every item but one in at least one of the six batteries in the questionnaire. We carried out logistic regression analyses on both of these outcome variables, using a model that included device type, as well as race, sex, age, and education as covariates. We found no significant differences in straightlining across the three device types.

Completion Times

Like many prior studies (e.g., Andreadis, 2015; Buskirk & Andrus, 2014; de Bruijne & Wijnant, 2013; Keusch & Yan, 2016; Struminskaya et al., 2015), ours found that respondents took longer to complete the survey on a smartphone (25.3 min on average for the sections of the questionnaire that were self-administered for all respondents; the median was 23.2 min) than on a tablet (20.8 min on average; median of 18.8 min) or a laptop (22.4 min on average; median of 19.4 min). The means exclude the five fastest and the five slowest overall completion times—that is, the top and bottom 1% of the times (cf. Ratcliff, 1993). The pattern is the same if we include these extreme observations. The difference across the three device groups in average completion time was highly significant—F(2, 7) = 26.3, p < .001 (see the fourth row of Table 1). A contrast comparing the smartphone group with the other two groups was also highly significant, F(1, 7) = 51.3, p < .001.

Scale Reliability

We assessed scale reliability in two ways. We first calculated values of Cronbach’s α for four sets of items intended to form unidimensional scales. As can be seen from the middle four rows of Table 1, the αs are uniformly quite high—.858 is the lowest—but they do not differ systematically by device type. The χ² values given in the bottom panel of the last columns are from a test originally devised by Feldt, Woodruff, and Salih (1987) and implemented in an R package (Diedenhofen & Musch, 2016). None of them indicates significant differences in scale reliabilities by device type.

We then performed IRT analysis on all four scales and compared scale information functions by device type. Scale information functions indicate the range of the underlying construct where the scale is most useful for discriminating among people—that is, it is a measure of how precise the scale measures people at different levels of the underlying construct the scale is supposed to measure. A higher scale information function indicates more precision and higher reliability. We plotted the scale information function for all fours scales. We display the curves for Scale 3 in Figure 2; the remaining curves are presented in Appendix B.

Figure 2.

Scale information curves by device type for Scale 3.

As shown in Figure 2, the scale information curves for Scale 3 are largely the same across devices. This is also the case for Scales 1 and 2 (see Appendix B). With Scale 4, for the smartphone and tablet groups, the scale measures people with higher levels of depression with more precision, whereas on the laptop, the scale measures less depressed people with greater precision.

We further tested for DIF across the three device types. DIF occurs when different groups respond to an item in different ways. We identified 7 items (of 39) that exhibited significant DIF across the three device types. As shown in Table 2, four of the items are from Scale 1. This scale consists of six subjective well-being items (e.g., “I feel that life is very rewarding”). It appeared that the laptop respondents were the most reliable on Scale 1, with the laptop showing the highest reliabilities for all 4 items with significant DIF. The other 3 items from two other scales that exhibited significant DIF show no consistent pattern; each device type appeared to provide the most reliable answers for one of the items. None of the items from Scale 4 exhibited DIF.

Table 2.

Items Showing Significant Differential Item Functioning by Device Type.

Scale and Item	Device Showing the Highest Reliability	Device Showing the Lowest Reliability
Scale 1
Item 1	Laptop	Tablet
Item 3	Laptop	Smartphone
Item 4	Laptop	Tablet
Item 6	Laptop	Tablet
Scale 2
Item 1	Smartphone	Laptop
Item 5	Laptop	Smartphone
Scale 4
Item 5	Tablet	Smartphone

Scale Validity

We calculated the multiple correlations of each the four scales with four demographic predictors—age, race, education, and sex. Two of the scales (Scale 3 measuring social support and Scale 4 measuring depression) show relatively strong relationships to the demographic variables—R ² of .057 and .067, respectively. The smartphone respondents showed the strongest overall relationship between the demographic variables and social support (R ² = .112), but the laptop group showed the strongest relationship between those variables and depression (R ² = .170).

In addition, several of the device type by demographic variable interactions were significant when we added the relevant interaction terms to the models. To show the nature of these interactions, Table 3 displays how the device type moderates the relationship between the demographic variable and the dependent variable. As is apparent in the table, the interactions do not follow any clear pattern. For example, the relationship of age both to depression and social support varied significantly by device, F(6, 7) = 4.94 and 7.76, both p < .05; however, the smartphone group showed the sharpest decline in depression by age-group, whereas the tablet group showed the sharpest decline in social support across the four age-groups. These results were like those from the DIF analysis, with some apparent differences across device type but no consistent overall direction to those differences.

Table 3.

Patterns of Interaction Effects.

Interaction Effect	Device Showing Largest Difference	Device Showing Smallest Difference
Model of depression
Age × Device	Laptop	Tablet
Gender × Device	Laptop	Smartphone
Model of social support
Age × Device	Tablet	Smartphone
Gender × Device	Smartphone	Laptop
Education × Device	Smartphone	Laptop
Race × Device	Laptop	Tablet

Discussion

Although our survey was not optimized for smartphones, the data from the smartphone group seem, by most standards, to be just as good as those obtained on tablet or laptop computers. Our earlier study (Tourangeau et al., in press) reached similar conclusions. Contrary to our hypotheses in that study, the small screens on the smartphone did not significantly alter the impact of response order, scale direction, scale length, or presentation of items in a grid. In those experiments, smartphone users had to scroll to view critical information, but tablet and laptop respondents could see all the relevant information from the outset. For the most part, our findings are good news for survey researchers—data quality does not suffer noticeably when the data are collected via smartphone.

We did a field experiment and had greater control over the devices on which the respondents completed the survey (because we provided the device they used) than in most past studies. In addition, our respondents were not members of an online panel. Because many respondents were using devices they were not familiar with, we might have expected to find larger differences by device type than past studies have. However, we found few differences across the three types of devices in our measures of data quality—which included item nonresponse, straightlining, coefficient αs, and relationships between scale responses and demographic variables.

Past research has mostly shown somewhat higher item nonresponse, longer completion times, and shorter responses to open-ended questions when the data are collected on smartphones rather than laptops or tablets. We replicated the earlier findings on completion times. Our smartphone respondents took about 13% longer than laptop respondents and more than 20% longer than tablet respondents to complete the same sections of the questionnaire. Because the respondents in all three groups accessed the Internet by similar means, this difference probably does not reflect differences in download times. Our respondents were not panel members and they were not necessarily familiar with the device to which they were assigned. About 62% of the smartphone respondents reported that they owned smartphones. Across all three device groups, smartphone owners were faster in completing the self-administered sections of the questionnaire than respondents who did not own smartphones, but smartphone ownership did not interact with device type.

We also examined scale reliability and scale validity. Again, we failed to find consistent differences in Cronbach’s αs and construct validity coefficients by device type. IRT analyses detected several items that display DIF across the device types; for the first of the four scales we examined, the laptop respondents seemed to give the most reliable answers for 4 of the 7 items in the scale. Relationships between two of the scales and four demographic variables also sometimes differed across the devices, but sometimes the relationships were strongest for the smartphone respondents and sometimes they were strongest for the laptop respondents.

Several things may have contributed to the near absence of clear or consistent device effects. It is possible that the particular smartphone we used (the iPhone 5S, with its relatively high screen resolution) played a role. Had we used a smartphone with lower screen resolution, our findings might have been different. Or it could be that most respondents are now comfortable dealing with smaller screens and touch screen entry, so there really is little effect on data quality of doing the survey on a smartphone. Finally, in our study, interviewers handed respondents the devices and were present as the respondents completed the questions. Thus, any concerns about lack of privacy or high levels of distraction for smartphone respondents did not apply in our experiment; the settings were similar for all three devices.

Overall, we find that smartphones produce data that were quite similar in overall quality to those produced on tablets or laptops, and the latter two device types did not differ from each other much either. All in all, we think this augurs well for data collection via the web.

Supplemental Material

Supplemental Material, Supplement_material - Web Surveys by Smartphones and Tablets: Effects on Data Quality

Supplemental Material, Supplement_material for Web Surveys by Smartphones and Tablets: Effects on Data Quality by Roger Tourangeau, Hanyu Sun, Ting Yan, Aaron Maitland, Gonzalo Rivero, and Douglas Williams in Social Science Computer Review

Footnotes

Appendix A

Appendix B

Authors’ Note

Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the National Science Foundation or the Bureau of Justice Statistics. We thank the Methodology, Measurement, and Statistics Program at the National Science Foundation, the Bureau of Justice Statistics, and Dr. Cheryl Eavey for their support. We are grateful to Rose McAloon for her research assistance; Tamara Bruce and Naoko Okuma for their oversight of the field work; and Abie Reifer, Bryan Davis, and Andrew Heller for their management of the technical and programming aspects of the study.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work reported here was supported by a grant from the National Science Foundation (SES-1261340 to RT), with additional support from the Bureau of Justice Statistics.

Supplemental Material

The supplemental material is available in the online version of the article.

Notes

References

American Association for Public Opinion Research. (2016). Standard definitions final dispositions of case codes and outcome rates for surveys (9th ed.). Oakbrook Terrace, IL: Author.

Andreadis

(2015). Web surveys optimized for smartphones: Are there differences between computer and smartphone users? Methods, Data, Analyses, 9, 213–228.

Buskirk

T. D.

Andrus

C. H.

(2014). Making mobile browser surveys smarter: Results from a randomized experiment comparing online surveys completed via computer or smartphone. Field Methods, 28, 170–193.

Couper

M. P.

Antoun

Mavletova

(2017). Mobile web surveys: A total survey error perspective. In Biemer

Eckman

Edwards

de Leeuw

Kreuter

Lyberg

Tucker

West

(Eds.), Total survey error in practice (pp. 133–154). New York, NY: John Wiley.

Couper

M. P.

Tourangeau

Conrad

Crawford

(2004). What they see is what we get: Response options for Web surveys. Social Science Computing Review, 22, 111–127.

de Bruijne

Wijnant

(2013). Comparing survey results obtained via mobile devices and computers: An experiment with a mobile web survey on a heterogeneous group of mobile devices versus a computer-assisted web survey. Social Science Computer Review, 31, 482–504.

de Bruijne

Wijnant

(2014). Improving response rates and questionnaire design for mobile web surveys. Public Opinion Quarterly, 78, 951–962.

Diedenhofen

Musch

. (2016). cocron: A web interface and R package for the statistical comparison of Cronbach’s alpha coefficients. International Journal of Internet Science, 11, 51–60.

Embretson

S. E.

Reise

S. P.

(2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.

10.

Feldt

L. S.

Woodruff

Salih

F. A.

(1987). Statistical inference for coefficient alpha. Applied Psychological Measurement, 11, 93–103.

11.

Keusch

Yan

(2016). Web versus mobile web: An experimental study of device effects and self-selection effects. Social Science Computer Review. Advance online publication. Retrieved from https://journals-sagepub-com.web.bisu.edu.cn/doi/pdf/10.1177/0894439316675566

12.

Krosnick

J. A.

(1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213–236.

13.

Krosnick

J. A.

(1999). Survey research. Annual Review of Psychology, 50, 537–567.

14.

Lugtig

Toepoel

(2015). The use of PCs, smartphones, and tablets in a probability-based panel survey effects on survey measurement error. Social Science Computer Review, 34, 78–94.

15.

Mavletova

(2013). Data quality in PC and mobile web surveys. Social Science Computer Review, 31, 725–743.

16.

Mavletova

Couper

M. P.

(2013). Sensitive topics in PC web and mobile web surveys: Is there a difference? Survey Research Methods, 7, 191–205.

17.

Mavletova

Couper

M. P.

(2014). Mobile web survey design: Scrolling versus paging, SMS versus e-mail invitations. Journal of Survey Statistics and Methodology, 2, 498–518.

18.

Mavletova

Couper

M. P.

(2015). Grouping of items in mobile web questionnaires. Field Methods, 28, 170–193.

19.

Peytchev

Hill

C. A.

(2010). Experiments in mobile web survey design: Similarities to other modes and unique considerations. Social Science Computer Review, 28, 319–335.

20.

Rao

J. N. K.

Scott

A. J.

(1987). On simple adjustments to chi-square tests with sample survey data. Annals of Statistics, 15, 385–397.

21.

Ratcliff

(1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510–532.

22.

Struminskaya

Weyandt

Bosnjak

(2015). The effects of questionnaire completion using mobile devices on data quality. Evidence from a probability-based general population panel. Methods, Data, Analyses, 9, 261–292.

23.

Toepoel

Lugtig

(2014). What happens if you offer a mobile option to your web panel? Evidence for a probability-based panel of Internet users. Social Science Computer Review, 32, 544–560.

24.

Toninelli

Revilla

(2016). Smartphones vs PCs: Does the device affect the web survey and experience and the measurement error for sensitive topics? A replication of the Mavletova & Couper’s 2013 experiment. Survey Research Methods, 10, 153–169.

25.

Tourangeau

Conrad

F. G.

Couper

M. P.

(2013). The science of web surveys. Oxford, England: Oxford University Press.

26.

Tourangeau

Maitland

Rivero

Sun

Williams

Yan

. (in press). Web surveys by smartphone and tablets: Effects on survey responses. Public Opinion Quarterly.

27.

Wells

Bailey

J. T.

Link

M. W.

(2014). Comparison of online and computer online computer survey administration. Social Science Computer Review, 32, 238–255.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB