You Can Lead a Horse to Water

Abstract

An increasing number of publishers offer online companion websites that students pay to access with the expectation that using these will significantly increase their grade. This research assessed the students’ attitudes toward and the efficacy of components of MyPsychLab, a companion site for Introduction to Psychology. Students reported liking MyPsychLab and felt it helped their grades. Composite data across three semesters and three professors suggest that use of MyPsychLab is associated with better grades for individual students. However, making its use a course requirement did not improve the overall class grade or passing rate. Whether it was optional or required, not all students used MyPsychLab, suggesting a need to improve students’ use of available and beneficial resources.

Keywords

online resources introductory psychology practice tests

Introduction to Psychology is among the most widely offered and popular courses among college students for a variety of reasons. Many students take Introduction to Psychology because they intend to major in psychology, which is consistently among the most popular majors in the United States (U.S. Department of Education, 2012). In addition, there are students who take it because it fulfills a general education requirement; others take it because it is relevant to their nonpsychology major, such as nursing or criminal justice; and students who take it simply because it sounds like it might be interesting and relevant to their lives. These diverse reasons result in a classroom of students approaching the class with different aims, needs, levels of interest, and motivation, which presents a number of challenges for the instructor.

Despite the diversity of students’ motivations and the breadth of instructor’s goals, Introduction to Psychology is a critically important course in the curriculum of any psychology program; it is a foundational course introducing many basic concepts and preparing students for more specialized mid- and upper level psychology courses. In addition, successful completion of this course is critical for engaging students within the discipline and retaining them within the major. The attribute of this course as being important yet uniquely challenging makes Introduction to Psychology one of the hardest courses to teach, even for veterans (Dunn & Chew, 2006; Hackney, Korn, & Buskist, 2006) and has resulted in many journal articles and conference sessions focusing on pedagogical interventions designed to improve and enhance both teaching and learning (Hill, 2006).

The challenges and importance of this course, the choices of which textbook to use, which pedagogical aids are valuable or cost effective, and students’ perception of textbooks are matters that require some attention (Gurung & Daniel, 2006; Johnson & Carton, 2006; Marek & Christopher, 2011; Marek, Griggs, & Christopher, 1999). In an effort to foster student achievement and engagement, textbook publishers are frequently and increasingly packaging their texts with a variety of ancillary resources, including student companion websites providing students with an array of resources to help them study (Marek et al., 1999). These websites may include practice tests, resource tools, study guides, flash cards, interactive demonstrations, and real-world application of the textbook content. It seems likely that utilizing these varied resources should result in an improvement in a student’s grades as well as increase his or her engagement with the material and enjoyment of the class. In particular, these online components have the added benefit of embracing technology, which is the focus of much research attention (Ludwig & Purdue, 2006), and there is evidence to suggest that the use of multimedia can improve retention (Mayer, 2001).

Among the most popular components of these platforms are the practice tests, providing the potential advantage of allowing students to take practice quizzes before they complete exams (Peat & Frankin, 2002) and therefore benefiting from the testing effect, which suggests that prior testing enhances later retention (Roediger & Karpicke, 2006). However, empirical evidence of efficacy of these online quizzes is minimal and lacks consensus. Landrum (2007) found that students performed significantly better on unit exams than on online quizzes, suggesting that these quizzes served as a good teaching tool for the students. Similarly, Grimstad and Grabe (2004) found that students who voluntarily utilized the practice quizzes performed better on course assessments and exams than students who did not choose to use the practice quizzes. The success of these resources may depend on how students use the quizzes; for example, Marek and Christopher (2011) reported that students preferred to use practice exams as a way to learn the material presented in an introductory course, and Brothen and Wambach (2001) found that practice quizzes were ineffective study strategies compared to reading the presented course materials. In essence, there is insufficient evidence regarding the efficacy of practice quizzes.

Despite the wide availability and use of ancillary textbook packages, there remains relatively little empirical research examining their impact. In particular, beyond the evidence that students will reap benefits from some of the pedagogical aids, from an instructor’s point of view, an important additional question is whether requiring students to use these resources rather than making them voluntarily available will result in an increase in average class grade. To put it bluntly, is it worthwhile to require students to purchase access to these resources? This article presents an assessment of one such support site—Pearson’s MyPsychLab—and its impact on students’ grades as well as students’ perceptions of the efficacy of the resource.

Method

Participants

Participants were 187 students at Trinity Washington University enrolled in seven sections of Introduction to Psychology. The first author taught three sections: Fall 2010 (n = 28), Spring 2011 (n = 28), and Fall 2011 (n = 17). The second author also taught three sections: Fall 2010 (n = 18), Spring 2011 (n = 35), and Fall 2011 (n = 35). A third instructor in Fall 2011 taught the seventh section (n = 26). We did not collect demographic information from our sample. However, the college is women only, predominantly Black and Hispanic, and traditionally aged college students.

Procedure and Material

All students used the same introductory textbook by Ciccarelli and White (2009, 2012) published by Pearson Education. This text comes with an online platform, MyPsychLab, which includes an electronic version of the text, practice tests, media assignments, study tips, and other resources. In Fall 2010 (n = 46), the online component was either not used, in the case of one section (n = 28), or presented as an optional resource for students in another section (n = 18). In Spring 2011 (n = 63) and Fall 2011 (n = 78), purchasing the online component was required, and scores on the practice tests counted toward students’ grades. In Fall 2011, we offered students extra credit for completing a presemester survey about motivation, interest, and the perceived difficulty of the class and a postsemester survey about study habits and the use of MyPsychLab.

Students’ grades on in-class tests, the MyPsychLab practice tests, and final class grades were included in the analysis. In-class tests were closed book and a mixture of multiple-choice and short answers. The MyPsychLab practice tests were multiple-choice quizzes that students completed in their own time prior to the in-class exam. Students were able to use their notes and the textbook during the completion of the MyPsychLab tests. Instructors took portions of the in-class tests from the textbook’s test bank; therefore, the MyPsychLab practice tests were ideal preparation for the in-class tests. However, the use of both the textbook’s practice tests and test bank may have resulted in some overlap in the questions included on the practice tests and the in-class tests, although given the large number of potential questions in the pool for each, this overlap was likely minimal. Students’ final class grade consisted of the in-class tests and, where relevant, the MyPsychLab scores, other assignments, attendance, and in some cases minimal extra credit. All student scores were percentages. Because percentage data have the potential to violate assumptions of normality of analysis of variance, we applied an arcsine transformation to all dependent variables measured as percentages. The significance test statistics present analyses using the transformed variable as the dependent measure; however, the descriptive statistics present raw percentages to retain interpretability.

Students’ grade point average (GPA) information helped to clarify other findings. Both overall and semester GPA were available; however, because many of the students—in particular those in the fall semesters—were in their first semester of college, for consistency, we used the GPA for the semester during which the student took Introduction to Psychology for all analyses.

Results

There were no significant differences between instructors for the students’ in-class test scores, F(2, 184) = 1.01, p = .37, η² = .01, or final class grades, F(2, 185) = 0.69, p = .50, η² = .02; therefore, the data are combined for all further analysis.

Efficacy for Students’ Grades

Overall, students’ scores on the MyPsychLab practice tests were positively correlated with their in-class test scores, r(139) = .57, p < .001, and their final class grades, r(140) = .65, p < .001. In addition, there was a significant difference, t(182) = 4.76, p < .001, d = 0.70, between the in-class test scores of those students who completed the MyPsychLab tests (M = 71.25, standard deviation [SD] = 15.08) and those who did not (M = 58.01, SD = 20.75). There was also a significant difference, t(183) = 6.15, p < .001, d = 0.89, between the final class grade of those students who completed the MyPsychLab tests (M = 75.01, SD = 14.25) and those who did not (M = 59.68, SD = 22.58).¹

This initial pattern of data suggests the value of incorporating MyPsychLab as a required component of Introduction to Psychology classes. Indeed, analyzing these data at the end of Fall 2010 was the impetus for us adopting this requirement in the Spring and Fall 2011 semesters. However, when comparing the composite class grades across these semesters, there was no significant difference between classes in which MyPsychLab was not used, optional, or required for students’ in-class test grades, F(2, 183) = 1.30, p = .28, η² = .01, or overall class grades, F(2, 184) = 1.03, p = .36, η² = .01.² Indeed, when students were grouped into one of three groups—failed the class (final grade < 60%), low pass (final grade 61–70%), and pass (final grade ≥ 71%)—the frequency of students falling in each of these categories was not related to whether MyPsychLab was not used at all, was optional, or required, χ²(4) = 6.81, p = .15, φ_c = 0.19.

The benefit of MyPsychLab for individual students, but not the class as a whole, may be the result of individual students’ own motivation. However, students’ self-rated motivation, their level of interest, and how difficult they expected the class to be, were not correlated with their scores on any of the assessments and did not differ between those students who subsequently used MyPsychLab and those who did not. However, this may be partially a social desirability or ceiling effect because the average self-rated motivation and interest were both high (M = 3.45, SD = 0.70 and M = 3.40, SD = 0.74, respectively, both on a scale of 1–4).

Alternatively, it might be that the students who chose to utilize MyPsychLab were—for want of a better word—better. To examine this possibility, we compared the GPAs of students who did and did not use MyPsychLab during the semesters that it was required. There was a significant difference, t(106) = 2.70, p = .008, d = 0.52, between the GPA of those students who completed the MyPsychLab tests (M = 2.67, SD = 0.88) and those who did not (M = 2.15, SD = 1.11). However, even when the students’ GPAs were controlled for, students’ scores on the MyPsychLab practice tests were still significantly correlated with their in-class test scores, r(135) = .32, p < .001, and their final class grades, r(135) = .44, p < .001. Furthermore, when GPA was recoded into a categorical variable (≤ 1.00, 1.01–2.00, 2.01–3.00, ≥ 3.00) and entered as a layer in the chi-square test for independence between the passing rate of the class (fail/low pass/pass) and whether MyPsychLab was not used in that class (not used/optional/required), these variables remained unrelated to one another. This suggests that even when controlling for the students’ GPAs, MyPsychLab does not influence the overall pass rate of the course.

Students’ Perceptions

Examining the survey data of those students who used MyPsychLab and completed the end of the semester survey (n = 40), students reported that using MyPsychLab helped them prepare for in-class tests (M = 2.95, SD = 0.88 scale of 1–4) with 30% saying it was very helpful and a further 40% saying somewhat helpful. Likewise, they felt it helped increase their overall class grade (M = 2.80, SD = 0.88 scale of 1–4) with 25% saying it was very helpful and 35% saying somewhat helpful. The students indicated that the practice tests were the most helpful component, with 35% of survey respondents listing them as the most helpful. Students who responded to the survey had marginally higher GPAs (M = 2.46, SD = 1.08) than those who did not (M = 2.00, SD = 1.15), t(75) = 1.83, p = .071, d = 0.41, suggesting that stronger students respond to even minimal extra credit opportunities or that stronger students felt compelled to share their experience of MyPsychLab.

Completion of the chapter exams was the only element of MyPsychLab that was required as part of the students’ grades, and it was not mandatory that they utilize any of the other available components. However, because they paid for access to MyPsychLab, the hope was that they would make full use of the range of resources offered. Most of the survey respondents indicated that they used the required practice tests (95%), but far fewer reported using any of the other available resources: chapter pretests (23%), chapter posttests (18%), study guides (50%), chapter audio (13%), media assignments (13%), psychology in the news (3%), flash cards (25%), and the e-book (28%). Students rated the study guides as the second most useful component on MyPsychLab (18%) and the e-book as the third most useful (15%). The impact of these tools on students’ grades is harder to assess, given the small number of respondents to the survey; however, examining the in-class test scores and final class grades for students who did and did not report using each of these components showed no significant differences.

Discussion

The results of this analysis suggest that using MyPsychLab, specifically the practice exams, is associated with improved performance on in-class tests and consequently an increased class grade, even when controlling for the students’ GPAs. This supports research concerning the value of practice tests for student performance (Landrum, 2007; Peat & Frankin, 2002). However, making MyPsychLab required does not significantly increase the average class grade or the proportion of students who pass the class. This may be in part due to the number of students who choose not to utilize the resource, despite it being a required part of the class. Across all semesters, comparable passing rates regardless of MyPsychLab use suggest that it may serve as just another resource that will help the grades of those students who utilize it. However, whether it is required or not, some students will choose not to use it, in the same way they do not use other available and encouraged resources. The hope was that tying the use of MyPsychLab explicitly to the students’ grades would be sufficient motivation for students to use it and would hold them accountable for the way they use it (Gurung & Daniel, 2006). In essence, we hoped to force all our students into the good habits associated with doing well; unfortunately, the data suggest that this is not always possible. Merely making a pedagogical aid available, or even required, does not necessarily improve student performance (Gurung & Daniel, 2006).

Limitations and Future Directions

The only use of MyPsychLab tracked by instructors was whether students had a grade or not for the practice tests in the MyPsychLab gradebook. Based on a comparison of the in-class and final grades of students who did and did not complete the practice tests, it is evident that MyPsychLab benefited individual students’ grades. However, those students who completed the MyPsychLab practice tests may also be utilizing other tools or resources. One limitation of the current study then is that the current data do not allow us to assess which of these components are influencing students’ grades. The student survey responses suggest that the students are primarily using the required practice tests and that reported use of the other components is not associated with improved performance. However, the number of students completing the survey was too low to draw any strong conclusions from this small sample. Therefore, future research should examine the popularity and impact of all components of MyPsychLab individually. Although students do not have the option of purchasing individual components, this could influence what aspects of the resource instructors require students to use.

Because final grades included MyPsychLab practice tests, it is not surprising that students who completed the practice tests had higher final class grades than did other students. However, a similar pattern for students’ individual in-class test scores suggests that the completion of the MyPsychLab practice tests did boost learning. Preliminary evidence using GPA data indicates that students who did not complete practice tests might be weaker students; however, given the bidirectional nature of the relationship between the use of resources and a student’s GPA, we cannot be sure of this. Future research should explore the characteristics of students who do or do not use available resources, the reasons for students’ choices, and whether resource use yields differential benefits for stronger and weaker students.

In the present research, the practice tests were worth approximately 10% of the students’ final grade. Increasing incentives for practice test completion might lead to increased student compliance with the requirement and potentially increased average class grades. Although future research might explore the effect of differential incentives on students’ performance, questions would remain regarding the instructor’s role in mandating good study habits.

Conclusion

The use of MyPsychLab practice tests significantly improves the grades of individual students, and from this perspective it is an effective resource. However, before requiring students to purchase and interact with MyPsychLab, we caution instructors to consider that we cannot force all students to engage with course resources in the way we would like. Although the incremental improvement seen among our students who chose to use MyPsychLab is encouraging, in all classes there were students who, despite the instructor’s best efforts and all available resources, did not perform at the desired level. You can lead a horse to water, but sometimes it just will not drink. As instructors and researchers, we would do well to consider ways in which we might encourage all students to make use of available resources with demonstrated efficacy, such as MyPsychLab.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Brothen

Wambach

(2001). Effective student use of computerized quizzes. Teaching of Psychology, 28, 292–294. doi:10.1207/S15328023TOP2804_10

Ciccarelli

White

(2009). Psychology (2nd ed.). Upper Saddle River, NJ: Prentice Hall.

Ciccarelli

White

(2012). Psychology (3rd ed.). Upper Saddle River, NJ: Prentice Hall.

Dunn

D. S.

Chew

S. L.

(2006). Grounding the teaching of introductory psychology: Rationale for and overview of best practices. In Dunn

D. S.

Chew

S. L.

(Eds.), Best practices for teaching introduction to psychology (pp. 1–10). Mahwah, NJ: Lawrence Erlbaum Associates Inc.

Grimstad

Grabe

(2004). Are online study questions beneficial? Teaching of Psychology, 31, 143–146. doi:10.1207/s15328023top3102_8

Gurung

R. A. R.

Daniel

(2006). Evidence-based pedagogy: Do text-based pedagogical features enhance student learning? In Dunn

D. S.

Chew

S. L.

(Eds.), Best practices for teaching introduction to psychology (pp. 41–55). Mahwah, NJ: Lawrence Erlbaum.

Hackney

Korn

J. H.

Buskist

(2006). Learning to teach introductory psychology: Philosophy and practice. In Dunn

D. S.

Chew

S. L.

(Eds.), Best practices for teaching introduction to psychology (pp. 57–69). Mahwah, NJ: Lawrence Erlbaum.

Hill

G. W.

(2006). Foreword. In Dunn

D. S.

Chew

S. L.

(Eds.), Best practices for teaching introduction to psychology (pp. xii–xvi). Mahwah, NJ: Lawrence Erlbaum.

Johnson

Carton

(2006). Introductory psychology without the big book. In Dunn

D. S.

Chew

S. L.

(Eds.), Best practices for teaching introduction to psychology (pp. 83–92). Mahwah, NJ: Lawrence Erlbaum.

10.

Landrum

R. E.

(2007). Introductory psychology student performance: Weekly quizzes followed by a cumulative final exam. Teaching of Psychology, 34, 177–180. doi:10.1080/00986280701498566

11.

Ludwig

T. E.

Perdue

(2006). Multimedia and computer-based learning in introductory psychology. In Dunn

D. S.

Chew

S. L.

(Eds.), Best practices for teaching introduction to psychology (pp. 143–158). Mahwah, NJ: Lawrence Erlbaum.

12.

Marek

Christopher

A. N.

(2011). What happened to the first “R”? Students’ perceptions of the role of textbooks in psychology courses. Teaching of Psychology, 38, 237–242. doi:10.1177/0098628311421319

13.

Marek

Griggs

R. A.

Christopher

A. N.

(1999). Pedagogical aids in textbooks: Do college students’ perceptions justify their prevalence? Teaching of Psychology, 26, 11–19. doi:10.1207/s15328023top2601_2

14.

Mayer

R. E.

(2001). Multimedia learning. New York, NY: Cambridge University Press.

15.

Peat

Frankin

(2002). Supporting student learning: The use of computer-based formative assessment nodules. British Journal of Educational Technology, 33, 515–523.

16.

Roediger

H. L.

Karpicke

J. D.

(2006) Test-enhanced learning: Taking memory tests Improves long-term retention. Psychological Science, 17, 249–255. doi:10.1111/j.1467-9280.2006.01693.x

17.

U.S. Department of Education, Institute of Education Sciences. National Center for Education Statistics. (2012). Fast facts: Most popular majors. Retrieved from http://nces.ed.gov/fastfacts/display.asp?id=37