Abstract
Over 5 semesters and 10 sections, students in a 300-level research methods course used clickers to respond to knowledge-based and opinion/attitude questions about clicker use, the effectiveness of assignments, self-confidence regarding skills and abilities, and more. Changes during the semester and student answers to questions about pedagogy and the execution of the course were compared to course grades (total points earned). Students perceive clicker use positively, reporting that they attend more, pay better attention, and participate more than they normally would due to clicker use. Other positive indicators do emerge linking the achievement of course goals to student performance. Outcomes are discussed in the context of how scientist–educators are encouraged to study scholarship of teaching and learning-type questions in order to be effective teachers.
The use of audience- or student-response systems (hereafter clickers) is an active teaching approach that leverages the benefits of student engagement with the technologically savvy preferences of today’s digitally native students. This active learning benefit may be enhanced in large classes (Caldwell, 2007; Hoekstra, 2008; Morling, McAuliffe, Cohen, & DiLorenzo, 2008; Trees & Jackson, 2007) where hand raising may not be as efficient and participation can be achieved with anonymity for those “reluctant participators” (Fallon & Forrest, 2011; Graham, Tripp, Seawright, & Joeckel, 2007). In similar studies, Stowell and Nelson (2007) and Elicker and McConnell (2011) reported that when clicker, flashcards, and hand-raising methods of student engagement were directly compared, students were most positive toward clicker use.
Faculty research into the beneficial effects of clickers tends to focus on one of the three areas: student performance, student engagement, and the demonstration of behavioral phenomena in the classroom. For instance, faculty members participating in a focus group reported that the major approaches in their clicker use included taking attendance, stimulating discussion, gauging student retention of concepts, and offering review sessions prior to an exam (Dallaire, 2011). I suggest that a fourth area be added to the potential clicker uses: a tool that allows an instructor to address pedagogy/scholarship of teaching and learning (SoTL)-type research questions as a scientist–educator. Using clickers as a pedagogically based data collection methodology fits nicely with the recent emphasis for psychology teachers to adopt a scientist–educator professional model (American Psychological Association [APA], 2011; Bernstein et al., 2010).
Other training models predate the scientist–educator model. The Boulder model emphasized a scientist–practitioner approach in psychology, and the Vail model emphasized a practitioner–scholar training model (Craig, 1992). The Puget sound model—emerging from the National Conference on Undergraduate Education in Psychology held at the University of Puget Sound in 2008 (see Halpern, 2010)—embraced a scientist–educator model. Faculty members at all levels who strive to be scientist–educators make a commitment similar to scientist–practitioners in that the scientific outcomes of psychological research are applied to the design and execution of teaching psychology. Scientist–educators design, implement, measure, and assess instructional strategies and use a continuous iterative process for pedagogical improvements. APA (2011) further encouraged scientist–educators to “become proficient in their use of commonly used technologies as a means to promote learning, and they encourage their students to develop these proficiencies as well” (p. 3). Faculty investment in understanding clicker technologies has professional development advantages as well as provides another tool used by effective teachers. The originators of the scientist–educator model offer compelling justifications for the model:
A scientist-educator treats professional work as an inquiry into the effectiveness of practice. It is critical to be familiar with evidence-based practice in the teaching of psychology, identifying those methods that are appropriate to one’s own teaching. Central to this enterprise is the systematic collection of evidence regarding the effectiveness of teaching and use of these data to guide the development and refinement of both the conceptual understanding of teaching and its practice in an iterative, recursive fashion. The scientist-educator reflects on the results of instruction, makes that work visible to peers, and redesigns course conception, measures, and activities accordingly. (Bernstein et al., 2010, p. 30)
Many of the early research questions addressed regarding to clicker use involved how clickers may or may not influence student course performance. The effect of clicker use on learning outcomes is mixed, with some researchers reporting a beneficial effect (e.g., Morling et al., 2008; Poirier & Feldman, 2007; Shaffer & Collura, 2009), some indicating no effect (e.g., Elicker & McConnell, 2011; Fallon & Forrest, 2011; Morgan, Eckerd, & Morgan, 2007), and at least one researcher reporting a detrimental effect (Anthis, 2011). Studies in this area include both controlled lab studies experimentally testing for clicker use effects and embedded classroom-based studies.
Another common area of research interest involved how clickers affect student engagement (Hill & Smith, 2011) and the social environment of the classroom. Hoekstra (2008) reported that when students in a clicker section of a course were compared with those in a “control” section of a course, students in the clicker section reported increased activity, cooperation, and conceptual applications. Stowell and Nelson (2007) reported a similar pattern of results, but in addition, slight increases in student enjoyment were reported, which may be linked to improved metacognitive monitoring. Active responding, coupled with instant feedback and the potential for anonymity, provide the potential for classroom advantages beyond hand-raising and the flashcard-raising methods. However, these benefits are related in part to a faculty members’ ability to use clicker pedagogy wisely, or as Trees and Jackson (2007) stated, “if students want to be involved and engaged, they are more likely to perceive clickers positively in terms of both learning and involvement processes” (p. 35). Stowell and Nelson demonstrated that clicker use is positively associated with increased participation and student engagement; however, it is important to remember that not all students will have a positive experience in their use of clickers (Dallaire, 2011). Varied examples exist of the clicker being used as an instructional aide in the classroom (Anthis, 2011), including the demonstration of behavioral research outcomes (e.g., Langley, Cleary, & Kostic, 2007) such as the false memory effect and the levels of processing effect (Cleary, 2008).
An additional utilitarian outlet for clickers may be in support of the scientist–educator systematically addressing SoTL-type research questions. Clicker use can facilitate answering pedagogical questions of interest. For example, Dallaire (2011) studied student perceptions of clicker use in relation to self-reflections about their major and their course performance. In this study, 40% of participants indicated that clickers were an extremely useful learning tool. Electing to use clickers in the classroom is not a pedagogical choice that excludes other pedagogical decisions. Graham, Tripp, Seawright, and Joeckel (2007) emphasized that clickers can be used in conjunction with conceptests, problem-based learning, and other active learning approaches. In a study designed to disentangle the pedagogy from the technology, Elicker and McConnell (2011) determined that “the questions themselves are what mattered in students’ perceptions of their usefulness rather than the response method” (p. 149). Although strong pedagogy in the design of clicker content is warranted, Fallon and Forrest (2011) suggested that part of the beneficial effect could be the novelty or joy of using clickers (and over time, novelty would wane). Christopherson (2011) also emphasized that the benefit of clicker use may be more due to the changes in pedagogy in the classroom rather than the implementation of clickers, or in her words, the clicker “simply helps instructors recognize some useful teaching moments” (p. 290). Scientist–educators would be well served to know if any benefits accrued from clicker use are bona fide and reliable or just due to novelty.
It is typical for new research tools to be scrutinized for some time in order for researchers to establish the utility of new techniques and demonstrate how a new research approach may have added value or simplify previous complex methodological procedures. The type of skepticism that still exists with regard to clicker use in the classroom is reminiscent of the skepticism that was once applied to the use of the Internet as a research methodology (Gosling, Sandy, John, & Potter, 2010; Gosling, Vazire, Srivastava, & John, 2004). For example, Gosling, Vazire, Srivastava, and John (2004) conducted research that compared Internet samples to traditionally recruited samples, and concluded that the findings are consistent and generalize similarly. In fact, when promoting the possible use of the Internet for data collection, potential benefits included the efficiency of data collected, dispensing with the need for manual data entry, and the relative inexpense. Given that students may already be using the same clicker for multiple courses during their college career, some of the same advantages leveraged by Internet use may also be applicable to clicker use; hence, the need for published research about the comparability of existing and new methodologies as well as the need for demonstrations of the enhanced utilitarian function that clickers may provide. Although the utility of clickers is evident in published studies regarding student engagement, student performance (although with mixed results), and as a demonstration tool, my goal was to demonstrate via multiple examples how using clicker technology can facilitate a scientist–educator’s efforts to address SoTL questions of interest.
Method
Participants
Students enrolled in my 300-level research methods course (10 sections over 5 semesters, beginning Fall 2008; N = 308) participated by using clickers throughout their 15-week course. Clicker use was an expectation of the course, with 10% of a student’s final grade based on clicker use during class sessions. Because my clicker use began as an in-class activity and not as a research project, I obtained retrospective institutional review board (IRB) approval for this research; additionally, I obtained additional information about the enrolled students from my registrar’s office. Based on the data from the registrar, 53.9% were women, 17.5% were men, and 28.5% did not report gender.
Materials
Many different SoTL-type questions were addressed during each semester. Near the beginning and end of each semester, a 12-item clicker effectiveness survey was administered via clicker (Magyar-Moe, Becker, Burek, McDougal, & McKeel, 2008)—see Table 1 for these early–late semester comparisons. Throughout the semester, I asked questions about student study habits, such as asking questions in class, where to seek help when needed, using a notecard method for writing a literature review, and so on. These items are presented in the Results and Discussion section presented below.
Percentage Yes Responses to Early Semester/Late Semester Clicker Perception Items.
Note. All chi-square tests df = 1. Given 12 chi-square tests, a Bonferroni correction was calculated, with statistical significance achieved with p < .004; for these outcomes, all items (except for Item 2) remain statistically significant. Effect size is based on 0–1 probabilities, as per Lenth (2009).
Procedure
I implemented a required clicker use component to my research methods class beginning Fall 2008. Students purchased clickers and registered them via a TurningPoint registration tool available on Blackboard. Thus, student performance using their clicker throughout the course was linked to my gradebook (including total points earned and final grade). During the semester, some clicker questions (presented using TurningPoint embedded in PowerPoint slides; www.turningtechnologies.com) were such that the student received a point for any response; other clicker questions were factual in nature, and required the correct response to earn a course point. All of the SoTL-related items presented here were the point-for-any-response type, encouraging students to answer honestly. At the end of the five semesters, I received a retrospective IRB approval to “connect” my accumulated clicker data to course performance data; I also added some student demographic data via a request to our registrar.
Results and Discussion
What Are Student’s General Perceptions About Using Clickers, and Do Those Perceptions Change Over the Course of a Semester?
For each of the 12 clicker perceptions items (from Magyar-Moe et al., 2008) presented in Table 1, there is a significant change in the percentage of yes responses over time (including a Bonferroni correction) with significant decreases for Items 1, 4, 5, 9, 10, and 12; significant increases for items 3, 6, 7, 8, and 11. First, it is interesting to note that even relatively small changes in the percentage of yes responses can lead to a statistically significant chi-square result 1 (effect sizes are presented in Table 1). Second, even though there is significant change over time for these items, the overall changes warrant continued use of clickers. For instance, at the beginning of the semester 89.7% replied yes that they participate more in the class as a result of clicker use, but by the end of the semester 83.4% respond yes. Even though this change is statistically significant, the overall level of yes responses (still over 80%) indicates that students believe clickers do enhance class participation. The drop over time may be indicative of what others have reported (Fallon & Forrest, 2011; Morgan, Eckerd, & Morgan, 2007) as the novelty effect for using clickers, especially early in the semester. Even if this is a novelty effect, by the end of the semester over 80% of students respond yes to a positive effect of clickers on attention and encouraging class attendance. Additionally, at the end of the semester, more than 70% respond yes when asked about clickers helping to understand lecture material better and allowing participation in a class when the student normally would not participate. Positive student reactions and perceptions to clicker use have been reported by other researchers as well (Morling et al., 2008; Poirier & Feldman, 2007; Shaffer & Collura, 2009).
How Can Clicker Methodology Be Used to Address Different Types of Pedagogical Effectiveness Questions? Are These Student Responses Related to Student Course Performance?
Specific responses for each of the items are presented in parentheses immediately following each possible survey response. For the following four clicker questions, students responded either true or false: (a) If I have a question about an assignment, I will ask my question (95.4% true, 4.6% false); (b) I am so shy that I doubt if I will ever ask a question during class (26.3% true, 73.7% false); (c) If I ask a question during class, my classmates will think I am dumb (20.1% true, 79.9% false); and (d) If I ask a question during class, the instructor will think I am dumb (13.8% true, 86.2% false). I conducted independent means t tests for each of the four individual items comparing true–false responders to their points earned in the course; there was only one outcome that marginally approached significance: those answering true (M = 864.56, SD = 59.44) versus those answering false (M = 892.08, SD = 68.67) to the clicker item If I ask a question during class, the instructor will think I am dumb, t(194) = −1.97, p = .051. Students who tend to think that by asking questions the instructor will think they are dumb also tend to earn fewer points in the course.
For the item When I have a question about an assignment, I prefer to ask the question …, students could respond with (a) in class (23.4%), (b) via e-mail (65.0%), or (c) during office hours (11.6%). For the item When I have a question about an assignment, I prefer to ask …, participants answered with either (a) a classmate (19.0%), (b) the instructor (77.4%), or (c) a teaching assistant (3.6%). For the item Generally, my preference for working on an assignment is to work on it …, participants answer with either (a) as soon as I receive it (16.6%), (b) after some time to think about it (47.7%), (c) whenever I get the time (27.1%), or (d) right before it is due (8.5%). One-way analyses of variance were conducted for each of these items, and there were no statistically significant differences between the response groups on total points earned in the course.
These last 4 items were all answered on a Likert-type agreement scale (from 1 = strongly disagree to 5 = strongly agree), and these outcomes were correlated with total points earned. The correlation outcomes were as follows: (a) Using the idea notecards helped me write a better introduction section, r(188) = .11, p = .138; (b) The idea notecards were a waste of my time, r(189) = −.26, p < .001; (c) The idea notecards helped me integrate ideas from multiple studies, r(192) = .26, p < .001; and (d) I think I will use the notecard method in other classes to help with writing assignments, r(189) = .05, p = .530. Students who earned more points in the course tended to disagree more that the notecards were a waste of time, and students earning more points agreed more that the notecards helped them to integrate ideas from multiple assignments.
Conclusion
Researchers continue to accumulate evidence in support of the use of clickers in the classroom. Clicker usage is linked to student performance (e.g., Shaffer & Collura, 2009), improvements in student engagement (e.g., Stowell & Nelson, 2007), and as a method for demonstrating psychological phenomena (e.g., Cleary, 2008). Interestingly, it may not be the use of a clicker per se but the changes in pedagogical approach to use the clicker that leads to beneficial outcomes (Christopherson, 2011; Fallon & Forrest, 2011). Based on the data presented here, I suggest there is a fourth potential benefit: scientist–educators using the clicker as a data collection device to enhance available knowledge about the SoTL.
Scientist–educators continually strive to assess the effectiveness of instructional strategies and varied pedagogical approaches. I used clickers to assess student perceptions of clicker use, both at the beginning of the semester and at the end of the semester for 10 course sections over 5 semesters. Although student opinions do change over the course of the semester, the overall responses are encouraging. By the end of the semester, note the percentage of yes responses to the following items: I participate more in class as a result of clicker use (83.4%); clickers positively affect my level of attention (81.9%); using clickers encourages me to attend class (83.1%); clickers help me understand lecture material better (77.6%); and clickers allow me to participate in class when I would normally not participate in class (76.2%). As others point out (e.g., Christopherson, 2011), these beneficial outcomes may not be due to the use of a clicker device, but the changes in pedagogy and course structure that I made. Nevertheless, as a scientist–educator, I am pleased to have evidence that positively links clicker use to students (a) paying more attention, (b) attending class more, (c) understand lectures better, and (d) participating more often in a course that the student would not normally participate.
Clickers can be utilized to assist in exploring potentially meaningful relationships between perceptions about student learning outcomes and overall course performance; however, it is sometimes difficult to find pedagogical interventions that impact course performance (Tomcho & Foels, 2008). With sufficient numbers of scientist–educators systematically studying similar teaching and learning scenarios, a corpus of best practices could emerge that helps students positively link self-confidence and self-efficacy to course performance. Student self-confidence is an important outcome to consider; if a knowledgeable student is metacognitively unaware of knowing what they know, then that student’s ability to apply and act upon that knowledge seems limited at best.
As students reach more challenging portions of any course, educators strive to teach effective problem-solving skills, and scientist–educators strive to measure their own success at teaching problem-solving skills. For instance, when student performance during the course started to dip but students were not asking more questions during class, I wanted to better understand student question-asking behavior. Results from this sort of inquiry early in the semester and using clicker methods to instantly display class results to everyone can lead to fruitful discussions about learning strategies. Although not statistically significant, it is interesting to note that students who think that by asking a question during class their instructor will think they are dumb tend to earn fewer course points compared to those that do not think that. I also ask students about the effectiveness of a notecard assignment I use to help students integrate different research ideas from multiple studies as students prepare to write their first literature review. In addition to receiving feedback about the assignment, using an SoTL approach I can compare those clicker responses to total points earned. There is a significant negative correlation with total points and agreement scale responses to the item the idea notecards were a waste of my time and a significant positive correlation with the idea notecards helped me integrate ideas from multiple studies. By taking this scientist–educator approach, I receive feedback about student perceptions of the assignment during the course. If some students do think that the notecard approach is a waste of time, then I have feedback for the following semester as to making the connection between the assignment and the ultimate task (writing a literature review) more relevant or perhaps tweaking the assignment in some fashion.
There are certainly other ways to garner feedback from students about their perceptions of assignments, learning objectives, course performance, and so on. However, using clickers, there are the added advantages of anonymity and increased student engagement, which can be particularly useful in a large class environment and with shy students (cf, Stowell, Oldham, & Bennett, 2010) who would not normally participate. Not only can the use of clickers be effective in aiding scientist–educators answer interesting pedagogical questions, but also a systematic exploration of SoTL issues helps educators to be ethical teachers. Pusateri (2012) persuasively argued by linking to the current American Psychological Association ethics code that ethical teachers habitually reflect on their own teaching effectiveness, seek professional development opportunities, and work to enhance their pedagogy skill sets regarding teaching and learning. Using clickers can be one avenue by which scientist–educators can demonstrate effective and ethical teaching.
Footnotes
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.