Abstract
Tablet computers (tablets) are positioned to be powerful, innovative, effective, and motivating research and assessment tools. We addressed two questions critical for evaluating the appropriateness of using tablets to study number-line estimation, a skill associated with math achievement and argued to be central to numerical cognition. First, is performance with paper and pencil comparable with performance on a tablet? Second, is comparability affected by students’ preference for one method of presentation? Thirty-two students in Grade 6 estimated targets on a number line; half estimated with paper and pencil and half with a tablet. For both presentation methods, students’ performance was comparable. Students liked both presentation conditions equally but, when asked to choose, most students preferred the tablet. Preference did not influence comparability of results across presentation methods. Finally, students’ reasons for their preferences were explored, along with implications for using tablet applications in research and educational assessment.
As technology evolves, the impact of new tools on psychoeducational assessment needs to be explored. Key questions are whether a new method is likely to yield fundamentally different data from traditional methods and, if so, what might account for any such difference. Tablet computers (tablets) and smartphones now outsell personal desktop and laptop computers (Columbus, 2013) and present new tools for studying child development. Tablets and smartphones are portable, easy-to-use, multi-media enabled, universally accessible devices designed to use applications (apps) available on the Internet, making it possible to collect data easily. All of these features give tablets and smartphones enormous potential to be effective research and assessment tools (Conner, 2015; Dufau et al., 2011).
Despite interest in smartphones and tablets for research (Miller, 2012) and assessment (Davis, Strain-Seymour, & Gay, 2013), little is known about how using tablets compares with using more traditional research tools such as paper and pencil, especially in children. As tablets become more common, it is important to determine whether studies using tablets for task presentation and data collection are likely to yield data comparable with those obtained with conventional methods and tools. It is possible that the digital interface or the novelty of using these electronic devices might systematically improve or degrade performance on a task. We compared using a tablet with using paper and pencil in an area that has been studied with children extensively in the last decade: number-line estimation.
Number-line tasks have been used to measure how children represent numbers and how those representations change over time (Barth & Paladino, 2011; Booth & Siegler, 2006; Siegler & Opfer, 2003; Siegler, Thompson, & Opfer, 2009; Slusser, Santiago, & Barth, 2013). In the number-to-position number-line estimation task, participants are shown a horizontal line with, for example, 0 at the left endpoint and 100 at the right endpoint. Participants are asked to locate a target such as 57 on the line. Studies of performance on number-line estimation are important not only for understanding the development of number-line estimation but also because estimation is a useful academic and everyday math skill (Booth & Siegler, 2008; LeFevre, Greenham, & Waheed, 1993; Newman & Berger, 1984; Opfer & Thompson, 2008; Siegler & Booth, 2005). Researchers have found that performing well on number-line estimation is associated with doing well on other mathematical tasks (Booth & Siegler, 2006; Laski & Siegler, 2007) and with obtaining higher mathematics achievement scores (e.g., Ashcraft & Moore, 2012; Booth & Siegler, 2006, 2008; Schneider, Grabner, & Paetsch, 2009; Träff, 2013). The number line is a concept thought to be central to organizing numerical knowledge (Booth & Siegler, 2008; Siegler & Lortie-Forgues, 2014; Siegler et al., 2009).
Tablets afford distinct advantages for examining children’s number-line estimation. The ability to record robust data easily and instantaneously has always been a goal of researchers. In the case of number-line estimation, tabulating data collected with paper and pencil is laborious and time-consuming because it entails measuring children’s marks by hand with a ruler. With tablets, data on the task can be collected accurately and instantaneously. In fact, commercially available iPad apps designed for research (e.g., EstimationLine from http://hume.ca/ix/) and for teaching (e.g., MathGlow from www.igeneration.com) have recently become available for number-line estimation. Investigating whether students perform similarly on number-line estimation using a tablet compared with using paper and pencil is necessary because future research on this topic, and others, may be conducted with tablets or similar devices.
Despite the potential advantages of tablets, most research on number lines has been conducted with paper and pencil (Barth & Paladino, 2011; Booth & Siegler, 2006; Laski & Siegler, 2007; Opfer & Thompson, 2008; Petitto, 1990; Siegler & Booth, 2005; Siegler & Opfer, 2003; Siegler & Ramani, 2009; Slusser et al., 2013; Thompson & Opfer, 2008, 2010; Thompson & Siegler, 2010; White & Szűcs, 2012), although computerized versions have been used in a few cases (Ashcraft & Moore, 2012; Booth & Siegler, 2008; LeFevre et al., 2010; Newman & Berger, 1984; Pellicano, Aagten-Murphy, Attucci, Klaric, & Burr, 2011; Schneider et al., 2009; Schneider & Siegler, 2010; Sullivan, Juhasz, Slattery, & Barth, 2011). To the best of our knowledge, touch screen or tablet technology has been used with the number-line task in only two studies (Dubé & McEwen, 2014; Segal, 2011). In both cases, the focus was on the relation between gestures and accuracy on number-line estimation. In neither study was estimating on a tablet compared with estimating with paper and pencil.
Comparability studies between established and newer methods for data collection are necessary and are beginning to appear in the research literature. For example, Dufau et al. (2011) successfully used smartphone technology to collect response latencies on a word/non-word lexical decision task programmed as an app (http://www.sciencexl.org/) from more than 4,000 smartphone users. Latency data from these users were highly correlated and similarly distributed to latencies collected using traditional laboratory methods. Recently, researchers demonstrated the feasibility, reliability, and validity of using a smartphone-based app to assess cognitive functioning in the elderly (Brouillette et al., 2013). These studies confirm the comparability of the smartphone platform to more traditional research methods with adults. One study with children has further illustrated the additional measurement affordances of tablets in collecting precise data by demonstrating how children’s anxiety level can be assessed using pencil pressure on a tablet, an index difficult to capture with pencil and paper (LaRoque & Obrzut, 2006). To successfully use tablets to collect data with children, we need to further understand whether data collected from tablets are similar or different from data obtained with other methods such as paper and pencil.
There are several reasons why results may differ for tablets as compared with paper and pencil on math tasks. On one hand, students may be less familiar with tablets than paper and pencil and as a result, the novelty of the device may be distracting. Similarly, students may not attend to material presented on a tablet in a way that is conducive to optimal performance on a cognitive task, a possibility raised in comparability studies involving reading (for a review, see Jabr, 2013). On the other hand, there is a popular perception that children find tablets to be particularly engaging (Hu, 2011), and therefore, they may perform better than with paper and pencil. To examine the effect of mode of presentation on number-line estimation performance, we compared results from Grade 6 students who used paper and pencil to estimate with those who used a tablet using procedures adapted from previous number-to-position estimation protocols (Barth & Paladino, 2011; Booth & Siegler, 2006). We also determined whether students’ preference for tablet or paper and pencil affected comparability across modes of presentation and why.
Method
To compare performance across the two presentation methods, Grade 6 students were assigned randomly to one of two presentation conditions: paper or tablet. Students estimated the positions of target numbers on a line from 0 to 100. After completing the number-line task in the assigned condition, students were asked to judge how much they liked doing the task. Next, students completed five trials in the alternative condition and, finally, were asked which of the two conditions they preferred and why.
Participants
Thirty-two Grade 6 students (17 girls) were each tested individually in a 15-min session. Their ages (in years,months) ranged from 9,11 to 13,00 with a mean of 11,10. Students were drawn from a school in a suburban Canadian town where 83% of adults had at least a high school diploma, 16% had a bachelor’s degree or a higher degree (Statistics Canada, 2013a), and most (93%) residents self-identified as Caucasian (Statistics Canada, 2013b). Whether students had identified disabilities was not assessed, but all were able to complete the task.
Materials and Procedure
Presentation conditions
The task was adapted from the number-to-position estimation procedures of Booth and Siegler (2006) and Barth and Paladino (2011). Students were assigned to one of two presentation conditions, paper or tablet, and estimated 20 targets on the 0 to 100 number line. Stimuli were presented as two sets of 10 targets, with a short break between sets. Each set contained 1 target from each decade and was ordered so that targets from adjacent decades did not appear on consecutive trials. The order of targets within each set was reversed to create two stimulus orders. Presentation condition, gender, and stimulus order were counterbalanced across students.
After estimating targets in their assigned presentation condition (paper or tablet), students estimated targets in the other presentation condition so that they could indicate which presentation condition they preferred. In the alternative presentation condition, students estimated an additional five targets, which was sufficient to give them experience with the other presentation condition.
Materials
The number-line estimation task was presented either as an app on a tablet or with paper and pencil. On an Apple iPad, number-line stimuli were presented as a straight line on the touch screen, 18 cm in length, with 0 marked at the left end and 100 at the right end. The target number was centered 2.5 cm above the line. To make their estimates on the iPad, participants used a Kuel H10 high sensitive stylus. With paper and pencil, the same configuration of the number line and target described above were used except that the trial was presented on an 21.6 cm × 28 cm (8.5 inch × 11 inch) sheet of paper, with a black border around the number line to mimic the tablet display, and children used a pencil instead of a stylus.
Procedure
Instructions were adapted from Booth and Siegler (2006) and Barth and Paladino (2011) and were identical for both conditions except for references to the screen/paper and the stylus/pencil. Students were told,
I want you to show me, by marking with the [pencil, stylus] on this line from 0 to 100, where you think the number at the top of the [page, screen] goes on the number line. Make your best answer as quickly as you can and tell me “done” when you have your answer. I will tell you to [circle your, press the ok button for your] final answer and go to next one.
In both conditions, attention was drawn to the endpoints of the line although the dynamic display of the tablet allowed us to graphically draw attention to the endpoints. For example, when the number line appeared the first time on the tablet, the endpoints were displayed in red and enlarged 200% and then scaled down to the normal size. These dynamic numbers were meant to orient the student to the current number-line range. In the paper condition, the experimenter oriented students to the number-line range by pointing to each endpoint as she spoke. In both presentation conditions, students then practiced making marks by using the stylus or pencil to show the experimenter where the practice targets, 0 and 100, were on the number line.
Next, students estimated the location of 10 targets, each on a separate 0 to 100 number line. After completing each set, the experimenter cheerfully told students in both conditions “Nice job!” The dynamic display of the tablet also afforded the opportunity for including, as part of the feedback after each set, a screen that appeared with a video of some cheerful music and animated dancing rabbits holding a sign that said “Nice Job!” Students in both conditions then estimated the second set of 10 targets.
At the end of both sets, students in both conditions were shown a scale from 1 to 7, and the experimenter asked,
I’m curious to know, on a scale of 1 to 7, where 1 is really dislike [pointed to picture of frowning face icon], 4 is neither like nor dislike [pointed to picture of neutral face icon], and 7 is really like [pointed to picture of smiling face icon], how much did you like doing number lines [on the iPad, with paper and pencil]?
After recording the student’s rating, the experimenter asked “Why?” and recorded the student’s answer.
Finally, to expose students to the alternative presentation condition, students in the tablet condition estimated five targets with paper and pencil, and students in the paper-and-pencil condition estimated five targets with the tablet. After these trials, the experimenter asked, “Which did you like doing number lines on more: the iPad or with paper and pencil?” After recording the student’s preference, the experimenter asked “Why?” and recorded the student’s answer. At the end of the session, the experimenter thanked the student.
Results and Discussion
Number-Line Estimation
To determine whether accuracy on the number-line estimation task varied as a function of estimating with paper and pencil or on a tablet, each child’s percent absolute error (PAE; in Booth & Siegler, 2006, 2008) was calculated as
For example, if a child was asked to estimate the location of 42 on a 0 to 100 number line and placed the mark at the location that corresponded to 55, the PAE on that trial would be calculated as (|42 − 55| / 100) × 100, equaling 13%.
Each student’s mean PAE across 20 trials was calculated. PAE for individual children ranged from 2% to 7%, with an overall mean of 4%. Performance was comparable with previous studies with children of a similar age (mean PAE of 5% in Grade 5 children in Ashcraft & Moore, 2012, and 3% in Grade 6 children in Piatt, Bisanz, & Volden, 2015). To test for differences in students’ accuracy attributable to gender or mode of presentation, a between-subjects 2 (gender: male, female) × 2 (condition: tablet, paper) ANOVA was conducted. No main effects or interactions were found, Fs(1, 28) ≤ 1.74, ps ≥ .19, mean squared error (MSE) = .0214%,
By averaging across 20 trials, initial differences between the two presentation conditions may have been obscured. To test this possibility, we calculated the mean absolute error for each block of five targets. A 2 (condition: tablet, paper) × 4 (block: 1, 2, 3, 4) ANOVA with repeated measures on the last factor revealed no effects. In Block 1, mean PAEs for the two conditions were identical (3.8%). Moreover, a planned comparison on the mean absolute error between the first and last block showed no difference (Ms of 3.8% and 3.7%). Students performed equally well throughout the task irrespective of whether the task was presented with paper and pencil or a tablet.
In studies of number-line estimation, the performance of children in a group is typically evaluated by plotting estimated magnitude (median estimate for each target across all children in the group) as a function of the target (e.g., Booth & Siegler, 2006). The high level of comparability across the two presentation conditions is illustrated in Figure 1, where students’ estimates for all 20 targets are plotted separately for the two conditions. Accuracy with the tablet was highly correlated with accuracy using paper and pencil, r(18) = .998.

Median number-line estimates for Grade 6 students in each of two presentation conditions, tablet or paper.
Students’ Preferences
When asked how much they liked “doing the number lines,” students rated the task in both presentation conditions as equally likeable (M = 5.63, SD = 0.81 for tablet; M = 5.31, SD = 1.10 for paper and pencil). A between-subjects 2 (gender: male, female) × 2 (condition: tablet, paper) ANOVA revealed no main effects or interactions, Fs(1, 28) ≤ 1.33, ps ≥ .25, MSE = .912,
To better understand students’ preferences, we considered the reasons students gave for preferring one presentation condition to the other. All children responded, and 16 unique reasons for preferring one presentation condition to the other were given (see Tables 1 and 2). On examination, responses clustered into three general themes: ease of use, characteristics of the presentation condition, and engagement. The frequency of these themes is shown in Table 1 for students who preferred the tablet and in Table 2 for students who preferred paper and pencil.
Reasons for Preferring the Tablet.
Note. This information comes from the 23 of 32 students who preferred the tablet to paper and pencil.
Reasons for Preferring Paper and Pencil.
Note. This information comes from the 9 of 32 students who preferred the paper and pencil to the tablet.
Several findings about students’ preferences are notable. First, in both presentation conditions, children listed features associated with the general theme of ease of use. More than half (65%) of the students who preferred the tablet mentioned features associated with usability such as speed and clarity. All nine students who preferred paper and pencil mentioned features associated with usability; in particular, eight students thought accuracy was better with paper and pencil. Second, more than half of the students who preferred the tablet cited characteristics such as the stylus, the touch screen, and technology generally as reasons for preferring the tablet. In contrast, of the nine students who preferred paper and pencil, only two mentioned characteristics of the medium, such as paper being more tactile and familiar, as reasons for preferring paper and pencil.
Finally, a theme unique to the tablet emerged: engagement. Of students who preferred the tablet, more than half mentioned features associated with fun, novelty, and engagement. Finding that tablets are “fun” is consistent with recent research exploring the usability of tablets as an assessment tool (Davis et al., 2013). In our study, no student who preferred paper and pencil made any mention of features associated with engagement. This difference may reflect the fact that the number-line app included a very short animation of dancing rabbits at the end of the estimation task, as well as endpoints that were slightly more dynamic than was possible with paper and pencil. With the exception of the short animation, the app was designed not to be particularly engaging nor was the screen design markedly different from pencil and paper. We did not intend to create an app that was more engaging than pencil and paper, and so, we did not systematically control for engagement in our study: This theme emerged as we explored why students preferred presentation modality. In contrast to paper and pencil, tablets may more easily afford the use and integration of design features that may make the task more engaging. Arguably, in psychoeducational research and especially in research with children, investigators should ultimately strive to make their tasks accessible and interesting to participants. Further studies are needed to explore the relations between task engagement, specific design features, and children’s success on a task.
Conclusion
Asking students to estimate with paper and pencil or with a tablet yielded similar results for Grade 6 students on a 0 to 100 number line. Grade 6 students were highly proficient estimators, and it may be that when faced with more difficult number-line ranges, differences in accuracy may emerge as a function of presentation condition. For a familiar, well-learned range, however, no differences in accuracy were found between the two presentation conditions, leading us to suggest that results collected using a tablet are comparable with those collected from paper and pencil. Although generalizability is limited by the sample (a small convenience sample of Grade 6 students from one school), this result is one of the first empirical demonstrations in children showing comparability on a mathematics task across these two formats—paper and pencil versus tablet.
Moreover, the results of this study support the popular view that tablets are engaging (Bonnington, 2012; Hu, 2011). Children liked the tablet and paper and pencil equally but, when asked which mode they preferred, two thirds of the children chose the tablet. Whether these results will hold for younger children who may have had exposure to touch screens from an earlier age remains to be seen. Children with more exposure may find the tablet easier to use and, therefore, preferable but potentially less novel and less engaging.
Finally, although more students preferred tablets, students performed similarly on number-line estimation irrespective of their preferences for one presentation condition over the other. The comparability of number-line estimation performance across presentation conditions is important for linking results obtained using traditional (e.g., Barth & Paladino, 2011; Booth & Siegler, 2006; Petitto, 1990; Siegler & Booth, 2005; Siegler & Opfer, 2003; Siegler & Ramani, 2009; Slusser et al., 2013; White & Szűcs, 2012) and new measurement methods (e.g., Dubé & McEwen, 2014; Segal, 2011). To develop increasingly effective educational interventions and assessments, it will be important to investigate how engagement, performance on assessments, and learning are influenced by personal preference, familiarity with materials, and the design of technological interfaces.
Measurement Implications
Computer tablets have considerable potential for recording robust data easily and instantaneously and are positioned to be powerful research tools. The success of Dufau and colleagues (2011), using a smartphone app to collect data comparable with traditional methods from more than 4,000 participants, illustrates how powerful portable electronic devices may be as a measurement tool. Our results support the comparability between data collected with a tablet and data collected with the more traditional paper-and-pencil method. Other information about number-line estimation, such as response latencies, whether and where multiple marks are made, and the order in which marks are made may also be collected more easily using an app. Collecting robust data on a portable, engaging, readily available computing device is the kind of methodological advance investigators should be striving to integrate into psychoeducational research.
Apps could be adapted to further investigate dimensions of number-line concepts such as the role of multiple representations or different kinds of representations in children’s understanding and learning of the number line (Ebersbach, Luwel, & Verschaffel, 2013). For example, researchers might systematically explore the effects on children’s understanding of fractions and percentages when those representations are presented in dynamic, interactive ways along the number line. Moreover, tablets allow researchers to capture different ways in which children and adults may demonstrate what they know. Being able to capture touch, auditory, video, and even eye-tracking data all on the same portable device may increase our ability to measure the cognitive processes children use in number-line estimation.
Educational Implications
Tablets also have the potential to be innovative, effective, and motivating educational tools (Benton, 2012; Bonnington, 2012; Lynch & Redpath, 2014; Shepard & Reeves, 2011). For example, tablets are suited for supporting learning based on principles of universal design and inclusiveness (Rich, 2010). Tablets may also be used for efficient, effective assessment in classrooms because they are useful for both asking questions and getting answers, as well as for recording and capturing a process (Rich, 2010). The results of this study lend support to the use of tablets for assessment as we demonstrated that tablets themselves do not improve or degrade performance on number-line estimation.
In mathematical learning and assessment especially, using touch screens and tablets may be important because of the dynamic interfaces they afford (Dubé & McEwen, 2014; Segal, 2011). For example, researchers have found that gestures have the potential to enhance teaching, facilitate learning and problem solving, and act as a medium to express knowledge that children do not readily articulate (Alibali & Goldin-Meadow, 1993; Goldin-Meadow, 2006). Apps can be designed to trace or track children’s touches or marks on the screen to assess what children know and how they learn concepts related to number-line estimation, as well as how different gestures might support learning number-line concepts (Dubé & McEwen, 2014; Segal, 2011). For apps and tablets in general, work is needed to systematically explore the relations among the content and design of computer apps and students’ diverse learning pathways and outcomes (Falloon, 2013).
As Dufau et al. (2011) noted, smartphone technology “heralds a new era in behavioural sciences” with wide multidisciplinary apps. Dufau et al. listed economics, social and affective neuroscience, linguistics, and experimental philosophy. To the list may certainly be added developmental science and educational assessment. The results of this study illustrate that for children, tablets can contribute to measuring at least some psychological phenomena such as number-line estimation as well as traditional methods and potentially with greater ease and in a more engaging way.
Footnotes
Acknowledgements
The authors thank K. Welker for research assistance, and the students and teachers who participated in this study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
