The paper measures a gamification effect in longitudinal web surveys among children and adolescents 7–15 years old. Two waves of the study were conducted using a volunteer online access panel in Russia among 737 children. Respondents were randomly assigned to one of the three conditions in the first wave without changing the treatment in the second wave: (1) a text-only survey, (2) a visual survey and (3) a gamified survey. Though in the first wave of the study respondents found it more enjoyable and easier to complete the gamified survey, no differences in participation rates were found between the conditions in the second wave. Contrary to expectations, a higher breakoff was found in the gamified condition. Moreover, it produced lower test-retest reliability correlations than the text-only and visual conditions in all survey questions. The promising gamification effect found in the first wave of the study faded in the second wave. It seems that implementing gamified elements in longitudinal web surveys might differ from the implementation of gamified elements in cross-sectional surveys.
AdamouB. (2013a) Introduction to gamification. Paper presented at 2013 NewMR Lecture Series. Available online at: http://newmr.org/play-again/5f9aa2dd (accessed 8 August 2014).
AmatoP.R., & OchiltreeG. (1987) Interviewing children about their families: a note on data quality.Journal of Marriage and the Family, 49, 3, pp. 669–675.
6.
BeebeT.J., HarrisonP.A., McraeJ.A.Jr, AndersonR.E., & FulkersonJ.A. (1998) An evaluation of computer-assisted self-interviews in a school setting.Public Opinion Quarterly, 62, pp. 623–632.
7.
BorgersN., & HoxJ. (2001) Item nonresponse in questionnaire research with children.Journal of Official Statistics, 17, 2, pp. 321–335.
8.
BorgersN., De LeeuwE., & HoxJ.P. (2000) Children as respondents in survey research: cognitive development and response quality.Bulletin of Sociological Methodology, 66, pp. 60–75.
9.
BorgersN., HoxJ., & SikkelD. (2003) Response quality in survey research with children and adolescents: the effect of labelled response options and vague quantifiers.International Journal of Public Opinion Research, 15, pp. 83–94.
10.
BorgersN., HoxJ., & SikkelD. (2004) Response effects in surveys on children and adolescents: the effect of number of response options, negative wording, and neutral midpoint.Quality & Quantity, 38, pp. 17–33.
11.
BrewerR., AnthonyL., BrownQ., IrwinG., NiasJ., & TateB. (2013) Using gamification to motivate children to complete empirical studies in lab environments.IDC ‘13 Proceedings of the 12th International Conference on Interaction Design and Children. New York, NY, pp. 388–391.
Downes-Le GuinT., BakerR., MechlingJ., & RuyleE. (2012) Myths and realities of respondent engagement in online surveys.International Journal of Market Research, 54, 5, pp. 613–633.
14.
DroletJ., ButlerA., & DavisS. (2009) The survey ‘burden factor’ – how important is respondents’ perception of survey length? Paper presented at CASRO Panel Conference, 2–3 February, New Orleans, LA.
15.
FuchsM. (2005) Children and adolescents as respondents. Experiments on question order, response order, scale effects and the effect of numeric values associated with response options.Journal of Official Statistics, 21, pp. 701–725.
16.
FuchsM. (2008) The reliability of children's survey responses: the impact of cognitive functioning on respondent behavior.Statistics Canada's International Symposium Series: Proceedings.
17.
HagenJ.W., & HaleG.A. (1973) The Development of Attention in Children.Princeton, NJ: Educational Testing Service. Available online at: http://files.eric.ed.gov/fulltext/ED082813.pdf (accessed 9 January 2015).
KönigS. (2011). Online-Forschung mit Kindern.Farmington Hills: Budrich UniPress.
20.
LinkM.W., LaiJ., & BristolK. (2014) Not so fun? The challenges of applying gamification to smartphone measurement in design, user experience, and usability, in MarcusA. (ed.) User Experience Design Practice, pp. 319–327.
21.
MavletovaA. (2014) Web surveys among children and adolescents: is there a gamification effect?Social Science Computer Review, published online, DOI: 10.1177/089443931454531.
22.
McGonigalJ. (2011) Reality is Broken: Why Games Make Us Better and How They Can Change the World.London: Penguin Press.
23.
PulestonJ. (2011) Improving online surveys.International Journal of Market Research, 53, 4, pp. 557–560.
PulestonJ., & MalinoffB. (2011) How far is too far? Traditional, flash, and gamification interfaces, and implications for the future of market research online survey design. Paper presented at ESOMAR 3D Digital Dimensions (Online + Social Media + Mobile) Research, 26–28 October, Miami.
26.
PulestonJ., & RintoulD. (2012) Can survey gaming techniques cross continents? Examining cross cultural reactions to creative questioning techniques. Paper presented at ESOMAR Asia Pacific Conference, 15-17 April, Shanghai.
27.
PulestonJ., & SleepD. (2008) Measuring the value of respondent engagement. Innovative techniques to improve panel quality. Paper presented at ESOMAR Panel Research Conference, 13–14 October, Dublin.
28.
PulestonJ., & SleepD. (2011) The game experiments: researching how gaming techniques can be used to improve the quality of feedback from online research. Paper presented at ESOMAR Congress, 18-21 September, Amsterdam.
29.
SikkelD., SteenbergenR., & GrasS. (2014) Clicking vs dragging: different uses of the mouse and their implications for online surveys.Public Opinion Quarterly, 78, 1, pp. 177–190.
30.
TurnerG., Van ZoonenL., & AdamouB. (2013) Research through gaming: public perceptions of (the future of) identity management.Sage Research Methods Cases, DOI: 10.4135/978144627305013496519.
31.
VaillancourtP. M. (1977) Stability of children's survey responses.Public Opinion Quarterly, 37, 3, pp. 373–387.
