The Impact of Screen-Free Zones in an Undergraduate Psychology Classroom: Assessing Exam Performance and Instructor Evaluations in Two Quasi-Experiments

Abstract

Background:

With in-person instruction, many undergraduates look at screens during lectures. Because screens can distract, attempts to curb this behavior through screen policies are increasingly common but understudied.

Objective and Method:

Two quasi-experimental studies (Study 1: N = 412; Study 2: N = 179) in the same Psychology class explore results of restricting screens to one area of the classroom on exam performance and instructor/course evaluation.

Results:

In both studies, these policies improved exam scores in the class. In Study 2, students who chose to sit in the screen-free zone did better on exams. Limiting screen use to the back of the classroom (Study 1) produced student pushback in qualitative comments and lower evaluations in quantitative instructor and course ratings. In contrast, placing screen and screen-free zones side-by-side (Study 2) was accepted without comment by students.

Conclusion:

Together, these studies suggest that creating screen-free zones in classrooms can be beneficial for students, especially for those students who choose to sit in the screen-free section. Additionally, students accept side-by-side division of a classroom into screen and screen-free zones.

Teaching Implications:

Because these studies are ecologically valid, results have implications for how students take notes and how instructors set restrictions on screens in their classroom.

Keywords

teaching psychology digital distraction student engagement teaching and learning strategies instructor evaluation

Screens, including those on cellphones, tablets, and laptops, are ubiquitous in college classrooms. Students often report that they use their laptops to take notes during lecture (McCreary, 2008). However, students also use their devices to engage in activities unrelated to the course (Aguilar-Roca et al., 2012; Fried, 2008; Kraushaar & Novak, 2019). As a response to these behaviors, some college instructors have limited or prohibited the use of screens during their classroom lectures.

Although there are some benefits of laptop use in college courses (Luo et al., 2018; Morehead et al., 2019; Mueller & Oppenheimer, 2014), a number of studies show that using a laptop or cell phone during a classroom lecture is associated with worse performance on assessments related to lecture material (Carter et al., 2017; Glass & Kang, 2019; Hembrooke & Gay, 2003). Further, screens have been found to distract students in class who may not be using a screened device but can see their neighbor’s screen (Fried, 2008; McCreary, 2008; Sana et al., 2013).

Some professors have begun to ban screens entirely from their classes with the hope that it would improve student attention and class climate. These technology bans have been found to improve student performance (Carter et al., 2017; Yamamoto, 2007). Despite the effectiveness of technology bans, some instructors report they fear a backlash if they ban screens in their classes (Yamamoto, 2007) because most students prefer unfettered access to their screened devices during class (Aguilar-Roca et al., 2012; McCreary, 2008). Students’ thoughts on these bans, as shown on teacher evaluations, have been mixed, with many of the negative comments focusing on students’ frustrations with taking notes without a laptop (Yamamoto, 2007).

Instead of complete bans, some professors create screen-free zones in classrooms where there is space for students who do not want to be distracted by others’ screens, and space for students who want to use laptops and other devices. In these classrooms, students can use screens, but only if they sit in specific areas of the room (e.g., back of the class; Aguilar-Roca et al., 2012; McCreary, 2008). Results of these policies have been mixed. For instance, although Aguilar-Roca et al. (2012) did not find differences in test scores between students in a course where screens were restricted and the same course with no restrictions, they did find that a majority of students in courses with screen-free zones supported these policies.

Many studies examining the impact of screens on undergraduate learning have been completed in controlled settings (e.g., Mueller & Oppenheimer, 2014). Only a few scientific studies have looked at the impact of screen-free zones in undergraduate classrooms (i.e., Aguilar-Roca et al., 2012; Carter et al., 2017; Hutcheon et al., 2019), suggesting additional ecologically valid studies of this kind are needed. Further, to our knowledge, no studies have reported student satisfaction with the arrangement, as measured by formal instructor evaluations. To fill in these gaps in the literature, we conducted a series of two quasi-experiments, with the second building on the first. Our research aimed to investigate whether restricting screen use in an undergraduate Psychology course had an impact on student exam scores and instructor or course evaluations.

Study 1

Method

Participants

Participants for this study were students enrolled in an upper division undergraduate Psychology course at UCLA. There were 159 students enrolled in the class in the quasi-experimental condition (2018, Year 2), and 253 students enrolled in the class in the control condition the previous year (2017, Year 1). We do not know why enrollment dropped. About half the students both years were Psychology majors (quasi-experimental condition: n = 83; 52% psychology majors; control condition: n = 150, 59% psychology majors).

Researcher Roles

The lead author was a teaching assistant in the class all 3 years, as was the second author. The third author was the instructor for all 3 years of the study. Hence teaching personnel were constant throughout the 3 years of the research.

Procedure

In the control condition class, there were no policies around screen use, and students were not told any information about the effect of screen use on their learning and performance. In the quasi-experimental condition, on the first day of class, the professor showed a PowerPoint presentation which included main findings from several studies related to the negative impact of screen use on learning. Following this presentation, students were told that they could use screens in class only if they sat in one of the rows in the back of the lecture hall. In both conditions, students were assessed on a 70-question cumulative final exam worth 35% of their grade. Exam questions were different each year.

Also in both conditions, at the end of the course, students were encouraged to fill out an anonymous online evaluation related to the course. On the evaluation, students were asked to provide an “overall rating of the instructor” and an “overall rating of the course” on a scale of 1 (Very low) to 9 (Very high). Although students were asked additional questions on the evaluation form, we chose these two questions for our analysis because these are the only questions used in the faculty evaluation process in the UCLA Psychology department.

Results

As the quarter progressed, more students moved into the screen section, and although the section remained in the back of the classroom, it began to expand beyond the back few rows to accommodate all the students using screens. We did not formally track the precise number of students in each section. However, by the end of the course, we noted that only about 25% of students used the screen-free section.

Formally, we compared results of the final exams to see if the no-screen policy was related to exam performance of the class as a whole. In the control condition, without a screen policy, the mean final exam score was 86% (SD = 7%). The following year in the quasi-experimental condition (screens permitted only in the back of the classroom), the mean score on the final was 90% (SD = 6%). There was a significant difference in final exam scores in favor of the quasi-experimental group. Students in the class where the screen policy was implemented scored on average 4% higher on their final exam than students the year before who had taken the class before implementation of a screen policy, t(410) = −5.55, p < .01, Cohen’s d = 0.57.

To measure student perception of the course, with and without a screen policy, we looked at student course evaluations. Response rates for the instructor evaluations were 75% of the class (n = 193) in the control condition and 65% of the class (n = 105) in the quasi-experimental condition. In the control condition, the mean rating students gave the professor was 7.22 (SD = 1.83). However, in the class with the screen policy, instructor ratings decreased to a mean of 6.29 (SD = 2.30). This was a significant drop in scores t(296) = 3.82, p < .01, Cohen’s d = 0.45. Students’ ratings of the overall course also dropped in the quasi-experimental condition from a mean of 7.05 (SD = 1.98) to a mean of 5.97 (SD = 2.62), t(296) = 4.00, p < .01, Cohen’s d = 0.47.

We also explored students’ perceptions of the screen policy by looking at their responses to the open-ended questions on the course evaluation form. Although students were not explicitly asked to comment on the screen policy, about 20% of the students who wrote comments commented on the screen policy in their course evaluation. Out of the 13 screen-related responses, only one comment was entirely positive. The student who wrote the positive comment said, “[The policy] gave me the opportunity to learn in a more focused environment, and gave the students accustomed to note-taking with their laptops the opportunity to do so.” Yet, another student wrote, “HATE the no-laptop rule…. Unfair to force everyone to the back if they wish to use the computer.” Another student who disliked the policy wrote, “PLEASE get rid of the no laptop policy! We are all adults.” There were no comments about technology in the control condition.

Discussion

We found that implementing a screen policy in an undergraduate Psychology class led to slightly higher exam scores overall. This was the case even though most students in the classroom still used screens. The four percentage point difference in scores might seem small, but the difference between 86% and 90% was the difference between B+ and A-, which is highly meaningful to many students.

Instructor evaluations indicated that more than a few students were displeased with the screen-free policy, even though they had the option of using their own screens in class. One student wrote in their course evaluation, “I like learning with notes and typing everything down since it is a lot faster.” However, even though students may prefer laptops because it is easier for them to type what the instructor says verbatim, this method of note-taking is not very effective for learning and remembering information (Bjork et al., 2013; Mueller & Oppenheimer, 2014). Additionally, this perception of ease may contribute to lower instructor evaluations because students tend to view ease of learning as an indication of a professor’s effectiveness (DeSlauriers et al., 2019) and therefore may rate professors who restrict screen use as less effective. Although students’ evaluations of their instructors offer them important opportunities to share their perspectives, instructors should be aware that implementing a policy that restricts screens could lower their evaluations.

While the class as a whole benefited from the opportunity to sit in areas where they were not distracted by other students’ screens, the findings here left unanswered questions. Our whole-class research design prevented us from examining the relationship between where students sat and how they performed on the exam. Study 2 was designed to explore this issue.

Study 2

Method

Participants

Study 2 participants were students enrolled the following year (2019, Year 3) in the same upper-division Psychology course. One-hundred seventy-seven students were enrolled; again, roughly half the class were Psychology majors (n = 82; 46%). As will be seen in the slight deviations from expected degrees of freedom from analysis to analysis, a few students did not take the multiple-choice midterm or final and/or did not report their seat location.

Procedure

Similar to the year before, on the first day of class, the benefits of not using screens during class and taking notes by hand were briefly presented. In order to size the screen-free zone in line with student preference, on the first day of class, the professor asked students, “Raise your hand if you would like to be in the screen-free section,” then “Raise your hand if you would like to use your screen.” Next, the professor split the lecture hall in such a way that both sections comprised seats in all rows from front to back. Thus, students who used screens were not required to sit in the back of the classroom as they did in Study 1.

As in Study 1, students were assessed on a midterm exam of 40 questions halfway through the quarter and on a final exam of 70 questions at the end of the course. This time, however, students were asked to report where they sat on both exams. See Table 1 for alternative answers to the seating location question on the exams.

Table 1.

Student Self-Reported Seating Location and Mean Exam Score: Midterm and Final Exams (N = 334).

Seating Location, as Reported on Exam	n	Seating Location up to Midterm	Mean Midterm Score	n	Seating Location Midterm up to Final	Mean Final Score
Seating Location, as Reported on Exam	n	%	Mean Midterm Score	n	%	Mean Final Score
Always in the screen section	96	57	84%	92	55	87%
Mostly in the screen section	17	10	85%	24	14	89%
Equally in the screen and the screen-free section	7	4	85%	14	8	88%
Mostly in the screen-free section	16	10	88%	14	8	89%
Always in the screen-free section	31	19	87%	23	14	90%

Analysis

We used a Pearson correlation to examine the relationship between seating location and test scores, and we used t-tests to determine differences in test scores between subgroups of students who changed their seating section between the midterm and the final. However, we were unable to do an ANOVA to test the interaction between all the seating options and exam scores because some of our subgroups were too small.

Additionally, we used t-tests to look for significant differences in course and instructor evaluations between Year 2 (screen section in back of classroom) and Year 3 (screen and screen-free sections side by side). We also used t-tests to compare test scores and student evaluations between Year 1 (no screen policy) and Year 3 (screen and screen-free sections side by side).

Results

Course Exams and Seat Locations

On the exam question where students were asked to report where they sat in the weeks leading up to the exam, more than half of the class reported always sitting in the screen section up until the midterm (n = 96; 57%), and from the midterm to the final (n = 92; 55%). Thirty-one students (19%) always sat in the screen-free section for the first 5 weeks of class, and 23 students (14%) always sat in the screen-free section for the last 5 weeks of class, from the midterm to the final. The remaining students reported sitting in both sections, to varying degrees (see Table 1).

Pearson r correlation analysis revealed that there were significant positive correlations between the amount of time spent in the screen-free section during lecture and scores on the midterm, r(167) = 0.18, p < .05 and the final, r(167) = 0.16, p < .05. These findings indicate that the students who reported sitting in the screen-free zone during lectures scored higher on both the midterm and the final exams.

We also explored whether switching from using a screen more often to using a screen less often midway through the course would lead to improvements on exam scores. First, we took a subset of our sample: those who switched from using a screen in class more to using a screen less (n = 19) and those who switched from not using a screen or not using it much to using a screen more often (n = 29). We compared the mean performance on the midterm and final exam for each group using paired sample t-tests. We found the group who began to use their screens less midway through the course saw, on average, a 4% increase from their midterm scores to their final exam scores (midterm: M = 84%, SD = 7%; final: M = 88%, SD = 6%), and this shift was significant, t(18) = −2.77, p < .05, Cohen’s d = 0.62. Conversely, students who began to use their screens more midway through the course saw a 1% increase from their midterm scores to their final exam scores (midterm: M = 88%, SD = 8%; final: M = 89%, SD = 5%), but this difference was not statistically significant t(28) = −1.17, p = .25, Cohen’s d = 0.23.

Regarding students in the other subgroups, Figure 1 illustrates that all groups saw positive gains from the midterm to the final, but the students in the no-screen section (i.e., the “never screen” group) performed best on the final exam. Additionally, students in the group that began to use their screens less often after the midterm saw the biggest improvement in their exam score. Although we could not check this post-hoc exploratory analysis for significant interactions between all the different subgroups due to small sample sizes, the visualization in Figure 1 tells a descriptive story of the relationship between seating location and test scores.

Figure 1.

Exam scores by seating location groups. Note. Change in exam scores from midterm to final for each seating location group.

Instructor Evaluations

We again looked at student evaluations of the course and the instructor. Approximately 60% of the class (109 students) filled out the instructor evaluations, yet no student mentioned the screen policy. Moreover, mean ratings on the instructor evaluations returned to similar levels present before the screen policy was enacted. In the current study, where students had more freedom to choose where to sit to use screens, the students’ overall mean rating of their instructor, on a scale from 1 to 9, was 7.48 (SD = 1.69). Students’ overall mean rating of the course was 7.04 (SD = 2.11). Compared to the Year 2 class with the screen section in the back of the classroom, student evaluation scores for the Year 3 class with screen and no-screen sections side by side were significantly higher, both for the instructor, t(212) = 4.32, p < .01, Cohen’s d = 0.59 and the course, t(212) = 3.30, p < .01, Cohen’s d = 0.45.

Comparison of Year 1 and Year 3 Data

Finally, we compared the exam scores in Year 1, with no screen restrictions, to the exam scores in Year 3, where screen and no-screen zones were side by side. We found that students in the Year 3 class had slightly higher final exam scores (M = 88%, SD = 7%), compared to the Year 1 class (M = 86%, SD = 7%). This difference was significant, t(423) = 1.99, p < .05, Cohen’s d = 0.20. For students who sat in the screen-free section in Year 3, this difference was even larger. Year 3 students in the screen-free section scored, on average, 4% higher on the final (M = 90%, SD = 6%) than students in the Year 1 class, t(274) = 2.27, p < .05, Cohen’s d = 0.49.

We also compared the student evaluations in Year 1, the control condition, to student evaluations in Year 3, screen and no-screen zones side by side. Although students’ evaluation of the professor and the course were slightly higher in Year 3, this difference was not significant for the instructor, t(300) = 1.22, p = .22, Cohen’s d = 0.15 or the course, t(300) = 0.04, p = .97, Cohen’s d < 0.01. Taken together, these results suggest that placing screen and screen-free zones side by side can support student learning more than a course with no screen restrictions, and students are generally as satisfied with side-by-side zones as with no screen restrictions.

Discussion

Our results support the idea that screen-free sections in large lecture halls can be helpful for students. We showed furthermore that students who do not use screens in class do better on learning assessments. Given that we did not control for or assess underlying achievement in this study, higher achieving students might have sat in the screen-free section. Nevertheless, by showing that students who moved from the screen section to the screen-free section improved their scores, we showed that screen-free sections can help a variety of students.

Given the behaviors we observed in the screen-sections, we hypothesize that the reason students in the laptop section perform worse is not that they are encoding information differently (Mueller & Oppenheimer, 2014), but that they are distracted by other options on their screened devices, as others have shown (Fried, 2008; Subrahmanyam et al., 2013). Future studies could manipulate access to other applications on students’ devices to test if that is, in fact, impeding their attention.

Summary and Conclusion

This study extends to the real world the findings of lab-based studies showing that students in a classroom setting learn better when they do not use screens for taking notes or for multitasking (e.g., Mueller & Oppenheimer, 2014). Yet, limiting screens in a college classroom can be tricky because college students crave autonomy and rate instructors who limit their autonomy lower (Filak & Sheldon, 2003). Indeed, the results of instructor evaluations here show the imperfect nature of student evaluations of their instructor’s effectiveness (see Kornell & Hausman, 2016), which can fluctuate with relatively small changes to a course. Thus, balancing what instructors know about how students learn best (i.e., in screen-free classrooms), while also valuing student choice and feedback, can be difficult. Dividing up the class into side-by-side screen and screen-free zones, as in Study 2, is one resolution to this quandary when it comes to screens in class. Importantly, this kind of division allows students with disabilities who use screened devices as an accommodation the option to sit any distance away from the front of the room.

Implications for Remote Instruction

These results also have important implications for online or remote courses, which have been universal during the COVID-19 pandemic. Our results suggest that during remote instruction, students should only use their computer to watch and listen to the lecture, without the distraction of visiting other websites, and that their exam performance will be better if they take notes by hand.

Limitations and Conclusion

Despite the ecological validity that our study adds to the current literature, the results here may not generalize to all Psychology classrooms for several reasons. First, we did not consider tablets or eWriters (i.e., devices where students can write on a screen with a stylus) as differentiated from laptops, but some students or instructors might consider these devices to be more like pen and paper. Second, it is unclear if creating screen sections or limiting screens in Psychology classes with drastically different content, like quantitative methods, would have similar effects. Finally, we only informally tracked students’ use of technology during class, and, in our second study, we relied on student reports of where they sat. More rigorous tracking of student behavior during class might show that students were not accurate reporters.

Even with these limitations, these results are useful for Psychology instructors who provide classroom instruction in a variety of courses. Taken together, these results suggest that it would be beneficial to some students if their instructors encourage students to put their screens away during lecture, even some of the time. Sharing results of studies that have addressed this issue, particularly the results of Study 2, is one way to do this. Overall, the advice we would give instructors is: Create screen-free sections in your classroom, but give students the option of where to sit. And, the advice we would give to students is: If you are worried about paying attention, or your grade, sit in the screen-free section.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Salvador R. Vazquez

References

Aguilar-Roca

N. M.

Williams

A. E.

O’Dowd

D. K.

(2012). The impact of laptop-free zones on student performance and attitudes in large lectures. Computers & Education, 59(4), 1300–1308. https://doi.org/10.1016/j.compedu.2012.05.002

Bjork

R. A.

Dunlosky

Kornell

(2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444. https://doi.org/10.1146/annurev-psych-113011-143823

Carter

S. P.

Greenberg

Walker

M. S.

(2017). The impact of computer usage on academic performance: Evidence from a randomized trial at the United States Military Academy. Economics of Education Review, 56, 118–132. https://doi.org/10.1016/j.econedurev.2016.12.005

Deslauriers

McCarty

L. S.

Miller

Callaghan

Kestin

(2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257. https://doi.org/10.1073/pnas.1821936116

Filak

V. F.

Sheldon

K. M.

(2003). Student psychological need satisfaction and college teacher-course evaluations. Educational Psychology, 23(3), 235–247. https://doi.org/10.1080/0144341032000060084

Fried

C. B.

(2008). In-class laptop use and its effects on student learning. Computers & Education, 50(3), 906–914. https://doi.org/10.1016/j.compedu.2006.09.006

Glass

A. L.

Kang

(2019). Dividing attention in the classroom reduces exam performance. Educational Psychology, 39(3), 395–408. https://doi.org/10.1080/01443410.2018.1489046

Hembrooke

Gay

(2003). The laptop and the lecture: The effects of multitasking in learning environments. Journal of Computing in Higher Education, 15(1), 46–64. https://doi.org/10.1007/BF02940852

Hutcheon

T. G.

Lian

Richard

(2019). The impact of a technology ban on students’ perceptions and performance in introduction to psychology. Teaching of Psychology, 46(1), 47–54. https://doi.org/10.1177/0098628318816141

10.

Kornell

Hausman

(2016). Do the best teachers get the best ratings? Frontiers in Psychology, 7, 1–8. https://doi.org/10.3389/fpsyg.2016.00570

11.

Kraushaar

J. M.

Novak

D. C.

(2019). Examining the affects of student multitasking with laptops during the lecture. Journal of Information Systems Education, 21(2), 241. http://jise.org/Volume21/n2/JISEv21n2p241.pdf

12.

Luo

Kiewra

K. A.

Flanigan

A. E.

Peteranetz

M. S.

(2018). Laptop versus longhand note taking: Effects on lecture notes and achievement. Instructional Science, 46, 1–25. https://doi.org/10.1007/s11251-018-9458-0

13.

McCreary

J. R.

(2008). The laptop-free zone. Valparaiso University Law Review, 43, 989–1044. https://scholar.valpo.edu/vulr/vol43/iss3/2

14.

Morehead

Dunlosky

Rawson

K. A.

(2019). How much mightier is the pen than the keyboard for note-taking? A replication and extension of Mueller and Oppenheimer (2014). Educational Psychology Review, 1–28. https://doi.org/10.1007/s10648-019-09468-2

15.

Mueller

P. A.

Oppenheimer

D. M.

(2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159–1168. https://doi.org/10.1177/0956797614524581

16.

Sana

Weston

Cepeda

N. J.

(2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24–31. https://doi.org/10.1016/j.compedu.2012.10.003

17.

Subrahmanyam

Michikyan

Clemmons

Carrillo

Uhls

Y. T.

Greenfield

P. M.

(2013). Learning from paper, learning from screens: Impact of screen reading and multitasking conditions on reading and writing among college students. International Journal of Cyber Behavior, Psychology and Learning, 3(4), 1–27. https://doi.org/10.4018/ijcbpl.2013100101

18.

Yamamoto

(2007). Banning laptops in the classroom: Is it worth the hassles? Journal of Legal Education, 57(4), 477–520. https://ssrn.com/abstract=1078740