A Multisite Study of Learning in Introductory Psychology Courses

Abstract

Four hundred and fifty-four students enrolled in introductory psychology at different institutions across the nation participated in a study of factors related to learning. Key measures included an instructor rating, ratings of textbook quality and helpfulness, study time, student self-evaluations of study behaviors, approach to learning, self-report of learning, and a measure of quiz performance using biopsychology and learning chapter questions from a College Board Advanced Placement exam. The authors found significant predictors of both self-reported learning (deep approach, less surface approach, instructor ratings, student self-evaluations, and study behaviors) and quiz performance (grade point average, study time, metacognitive activity, and less use of a surface approach to learning). These results are discussed in the context of optimizing student learning and providing a foundation for future research.

Keywords

student learning instructor textbooks study behavior

Any examination of how students learn necessitates a focus on at least three major components: student behaviors (e.g., study techniques), instructor behaviors (how is learning facilitated?), and the means by which the content is delivered (e.g., textbooks). In a multisite study, we assessed how student study behaviors, textbook quality, and instructor quality influence learning.

Many researchers have attempted to classify study techniques and to identify the techniques that are optimal (e.g., for reviews, see Entwistle & McCune, 2004; Gurung & Schwartz, 2009; Hattie, 2009). Study behaviors are broadly defined as behaviors functioning to acquire, organize, synthesize, evaluate, remember, and use information (Crede, & Kuncel, 2008; Gettinger & Seibert, 2002)—specific examples include time management, goal setting, selecting what to study (how and where), taking good notes, reading, and self-testing. Although clearly establishing the link between study behaviors and performance, large meta-analytical studies (Crede & Kuncel, 2008; Hattie, 2009) do not provide specific prescriptions of how exactly students should study.

There are also many ways the instructor can influence learning. As summarized by Buskist (2011), volumes of information are available about excellent teaching, and a number of factors such as instructor characteristics (ideal/master teacher factors) may all influence student learning (Buskist & Benassi, 2012; Fink, 2003; Gurung & Vespia, 2007; Keeley, Smith, & Buskist, 2006). In addition, there are suggestions of effective ways to teach. Bain (2004) culled a set of characteristics identifying the best college teachers. Similarly, Buskist (2004) identified a set of features held by master teachers, such as passion, organization, and inspiration. However, few of these efforts have addressed instructor variables as they impact student performance.

The modality by which information is transmitted is also very important, and the primary sources of content delivery tend to be the instructor and the textbook, both of which are highly variable. The vast research literature on textbooks show that textbooks have little similarity in content and vary in length, writing style, number and usefulness of pedagogical aids used, applied or research focus, tone, and comprehensiveness (Christopher, Griggs, & Hagans, 2000; Griggs, Bujak-Johnson, & Proctor, 2004; Gurung, 2004; Landrum & Hormel, 2002; Nevid & Lampmann, 2003; Sappington, Kinsey, & Munsayac, 2002; Weiten, Deguara, Rehmke, & Sewell, 1999).

We designed this study to improve on three major shortcomings with previous research. First, few studies focus on more than one influence on learning. Learning is a complex process influenced by many factors (Daniel & Poole, 2009). Hattie (2009) used meta-analytic strategies and examined 15,000 studies to tease out variance in learning due to teachers, students, and other factors, but research outcomes that examine student behaviors, teacher characteristics, and textbook quality in one study are not present in the literature. Second, many relevant studies primarily use self-reported learning as the sole outcome measure. Student perceptions of their own learning are not good proxies for learning (Hacker, Dunlosky, & Hacker, 1998; Kennedy, Lawton, & Plumlee, 2002; Wesp & Miele, 2008; Zinn, Magnotti, Marchuk, Schultz, Luther, & Varfolomeeva, 2011; see Dunning, Heath, & Suls, 2004, for a review of issues associated with self-assessment). In this study, we assessed student perceptions of their own learning, but we also utilized a standard measure of objective learning (e.g., quiz) for comparison across sites and classes taken from items vetted for use in the Advance Placement (AP) Psychology exam (College Board, 2004). The latter parallels a common form of assessment of learning in introductory psychology courses and offers a more rigorously developed and standardized assessment than has been used previously. Finally, most studies are conducted with students from one institution (e.g., Lammers & Smith, 2008), or one course (Chase & Houmanfar, 2009), or using one textbook (e.g., Gurung, 2004). We assessed students from different parts of the United States using different textbooks.

Our primary research goal was to contrast the different factors that influence student learning in the introductory psychology class. Introductory psychology is the highest enrollment class in our discipline and students may be particularly likely to overestimate their learning, given that the content of psychology seems to be common sense. Psychology instructors are also particularly well positioned to address students’ study habits and metacognitive abilities, given that areas of the field focus on learning and behavior. We reviewed the literature to identify key variables and included them here to assess and account for correlations between factors and the consequent relationship to learning. For example, we measured the extent to which students take a deep approach to learning. Students adopting a deep approach set out to understand what they have read or heard for themselves, which makes it more likely they will grasp the authors’ meaning and learn the material better (Entwistle, 2009). Researchers from cognitive psychology suggest that study behaviors such as repeated testing and metacognitive strategies (e.g., assessing what is known) are key to learning (Bjork, 1994; Winne & Nesbit, 2010). Consequently, we examined the extent of students’ awareness of their knowledge. We also asked students how they rated their textbook (Gurung & Martin, 2011). Finally, we assessed student perceptions of instructor characteristics. In short, we measured four major contributors to learning: (a) student approach to learning (e.g., deep/surface), (b) study behaviors, (c) perceived textbook quality, and (d) instructor ratings by students. We also tested the extent to which each predicted self-reported learning and scores on a quiz.

Method

Participants

Four hundred and fifty-four students from different universities participated in this study (59% women; 65 students did not indicate their sex). Two schools were large public universities (one in the Midwest, N = 252 from three sections of introductory psychology, one in the South, N = 151 from two sections), and the last school was a small liberal arts college in the South (N = 28). The remainder of the sample did not identify a school. The average age was 19.26 (SD = 2.58). The majority (58%) were first-year students. The remaining students were sophomores (20%), juniors (5%), and seniors (5%). Of those reporting a grade point average (GPA; N = 400), the average GPA was 2.97 (SD = 0.58). Ethnicity data were not collected.

Materials

Participants completed an online questionnaire designed to capture various aspects of student learning behavior. We modified existing scales to measure ratings of the instructor, study strategies, textbook quality, attitude toward learning, and wrote items to capture student behavior. A complete copy of the 95-item questionnaire is available upon request.

Instructor ratings

We used items derived from the Teacher Behaviors Checklist (Keeley, Smith, & Buskist, 2006). We asked students to indicate how each of the 20 different characteristics described their instructor: communicates well, uses relevant examples, humorous, likeable, passionate/enthusiastic, approachable, organized, effectively uses visual aids, conscientiousness, encouraging/caring, entertaining, knowledgeable, easy to understand, engaging, uses a good mix of teaching approaches/styles, professional, challenging, respectful, is a fair grader, and flexible. Students responded on a Likert-type scale from 1 (strongly disagree) to 9 (strongly agree). We summed responses on all 20 items to create a total score (Cronbach’s α = .96).

Textbook assessment

We used a short version of the Textbook Assessment and Usage scale (Gurung & Martin, 2011), utilizing nine questions designed to assess the main components of a textbook (study aids, figures, visual appeal, boxed information, tables, research studies, writing, examples, and photographs). Students first rated their textbook on the quality of each of the items (with regard to placement, relevance, and clarity). Students responded on a Likert-type scale from 1 (poor) to 9 (excellent). Next, students indicated the extent to which each component of the textbook helped them to understand the material. Students responded on a Likert-type scale from 1 (not helpful at all) to 9 (very helpful). We created two total scores, summing items for quality and helpfulness. Both scales showed high reliability (α = .90 for text quality and α = .91 for text helpfulness).

Study time

We used items modified from the Study Behavior Checklist (Gurung, Weidert, & Jeske, 2010) and created a composite score from the 11 items to represent total time studied. We asked students to report the amount of time in minutes, spent on each of the 11 study behaviors in preparation for their most recent exam: read the text, make up examples to understand material, read notes, rewrite notes and/or skim notes, review information highlighted from text, take notes from the book, use concept checks and other textbook pedagogical aids, study with a friend, memorize definitions through repetition, think of mnemonic/memory devices, and review figures and tables in text.

Self-evaluation related study activity (SERSA)

Because student perceptions of their own learning tend to guide their study behavior (Gurung & McCann, in press), we assessed student perceptions of their learning and study-related activity with two sets of questions. One set of three questions (SERSA-metacognition, α = .93) asked how much students remembered the material for the most recent exam, how much students’ thought they understood the material, and how confident students’ were that they knew the material. A second set of four questions assessed quizzing behavior, given that quizzes are a major way students’ can test their knowledge (SERSA-quizzing, α = .84). We asked students general questions about their use of quizzing results (online or in-class) as feedback to study more for the exam, how often they reviewed quiz results to see what they got wrong, how often they reread parts of the book associated with missed quiz questions, and how often they closely read the book before taking the quiz.

Student attitude toward learning

We used the approaches to learning and studying items from the Shortened Experiences of Teaching and Learning Questionnaire (SETLQ; Tait, Entwistle, & McCune, 1997). The subscale of the SETLQ measured three different approaches to studying and learning: deep approach (9 items, α = .76, e.g., I’ve looked at evidence carefully to reach my own conclusions about what I’m studying), surface approach (4 items, α = .76, e.g., I’ve tended to take what we’ve been taught at face value without questioning it much), and organized effort (4 items, α = .87, e.g., I’ve organized my study time carefully to make the best use of it). We created composite scores for each approach.

Student performance

We measured performance or learning in two ways. Students indicated how much they were learning in the class (self-reported learning) by responding to a singular item on a 9-point scale ranging from 1 = nothing to 9 = a lot. Additionally, all students also took a 15-item quiz that covered biology (eight questions) and learning (seven questions). We utilized items from the released 2007 AP exam with permission from the College Board. We created a composite quiz score (α = .86).

Procedure

We created the survey with Qualtrics software. A link to the survey was sent to students in the first and third authors’ classes. In addition, we sent the survey link to three colleagues teaching different sections of the class at different institutions. Some sections already completed quizzes online; however, all participants were asked to complete our quiz questions online using online survey software (Qualtrics). All classes included some evaluative component to gauge student comprehension of the course material; our online quizzes were asked in addition to the normal course and testing formats utilized by the respective instructors. We timed the release of the survey participation request to students in the introductory psychology classes for after they had covered the sections/chapters on biology and learning in class (approximately the sixth week of the semester). We told students to complete the survey in response to their introductory psychology class and the most recent exam. Students completed the quiz out of class (online) and some received research credit for participation. We could not measure how many students the e-mail was sent to and correspondingly do not have a return rate measure. Student participants had a chance to win one of the three music players.

Results

Our primary goal was to compare the relative contributions of factors such as the instructor, textbook, student attitude toward learning, and student study behavior, on learning and performance to provide a detailed picture of student learning behavior. To that end, we first provide descriptive statistics reporting a variety of behaviors and then predict self-reported learning and scores on our quiz. Table 1 provides the mean values and standard deviations for the major variables and the extent each correlates with the two main outcome variables, self-reported learning and quiz performance. All four categories of measures significantly correlated with quiz scores and all categories except study time correlated with self-reported learning. SERSA correlated with both learning measures, whereas online quizzing activity correlated with quiz scores. Five study behaviors correlated with quiz scores, whereas only two correlated with self-reported learning. Quiz scores and self-reported learning were correlated, r(444) = .20, p = .005.

Table 1.

Descriptive Data for Major Variables and Correlations With Learning

Variable	M	SD	Range	Quiz Score	Self-Reported Learning
Age	19.26	2.58	18–47	.14**	−.02
Grade point average (GPA)	2.97	.86	.86–4.00	.29**	.13*
Textbook quality	6.93	1.29	2.33–9.00	.10*	.39**
Textbook usefulness	6.85	1.32	2–9.00	.09	.40**
Teacher behavior	161.82	21.81	20–180	.15**	.48**
Deep processing	5.41	1.32	1.33–8.33	.08	.30**
Surface processing	3.95	1.52	1.00–8.70	−.34**	−.26**
Organizational effectiveness	5.62	1.63	1.00–8.75	.21**	.33**
SERSA-quizzing	5.73	2.39	1.00–9.00	.21**	.26**
SERSA-metacognitive	6.64	1.52	1.00–9.00	.31**	.47**
Total study time	342.93	348.65	0–2650	.19**	.09
Review highlighted material	26.23	42.53	0–420	.14**	.08
Think of mnemonics	18.84	83.77	0–1600	.01	.02
Make up examples	18.05	25.86	0–250	.07	.05
Review figures and tables	18.13	27.38	0–300	.03	.00
Memorize definitions	50.46	68.97	0–500	.15**	.05
Use pedagogical aids to test knowledge	23.43	39.29	0–500	.21**	.16**
Read notes	50.56	58.87	0–500	.08	.02
Study with a friend	29.78	100.19	0–1080	.08	.01
Rewrite notes	33.74	49.81	0–434	.06	.01
Take notes from book	25.54	52.01	0–480	.17**	.07
Read the text	49.12	74.99	0–600	.19**	.13**
Self-reported Learning	7.05	1.67	1–9	.20**	—
Quiz score	6.03	3.97	0–14	—	.20**

Note. SERSA = self-evaluation related study activity. Total study time to read the text measured in minutes.

*p < .05. **p < .01.

Predicting Learning

We next conducted analyses predicting self-reported learning (how much the students said they learned in class) and their score on our quiz. We conducted two multiple regression analyses. In each regression, we entered all four major factors in as a block: student factors that could influence the outcome (GPA, amount of time studied, and approaches to learning, SERSA, instructor ratings, and textbook factors [quality and perceived helpfulness]). Table 2 shows a summary of the two regression analyses.

Table 2.

Summary of Multiple Regression Analyses Predicting Learning

Variable	Quiz Score	Self-Reported Learning
Grade point average (GPA; β)	.24***	.06
Gender (β)	−.05	.00
Text quality (β)	.03	.04
Text usefulness (β)	−.07	.02
Deep processing (β)	−.06	.10*
Surface processing (β)	−.19**	−.11*
Organizational effectiveness (β)	−.02	.03
Teacher behavior (β)	−.02	.33***
Total study time (β)	.11*	−.04
SERSA-metacognitive (β)	.21**	.21***
Online quizzing behavior (β)	.18**	.08
Total R ²	26%***	36%***

Notes. SERSA = self-evaluation related study activity. Two separate regressions were conducted, one for each type of learning measure. β = Standardized β weights for full equation. R ² = total variance accounted for the block.

*p < .05. **p < .01. ***p < .001.

The predictor variables accounted for 26% of the variance in quiz scores. Five variables were significant: students’ GPA, surface processing (negatively related), total study time, SERSA, and online quizzing activity.

The predictor variables accounted for 36% of the variance in self-reported learning. Four variables were significant. The extent students held a deep approach, a surface approach (negatively related), ratings of the instructor, and SERSA all predicted self-reported learning.

Discussion

A variety of factors influence student learning, and the results of our study illustrate the relative importance of categories of key factors and highlight differences between predicting learning using self-report measures and an objective test. Our results imply that factors that may guide student study behavior, such as their approach to learning and SERSA, are significantly related to their learning. This finding reinforces the emphasis recently placed on student responsibility in their own learning (Quality Principle 1, American Psychological Association, 2011) and suggests that efforts to guide students in productive study and monitoring behaviors may positively impact learning.

A finding of particular interest involves the “approach” that students reported to guide their own learning (Entwistle, 2009). The more a student used a surface approach, the lower their quiz scores and their self-reported learning. Likewise, taking more of a deep processing approach was associated with higher self-reported learning (though not quiz scores). Whereas the exact source of these differences is difficult to ascertain in a broad study such as this, this latter difference may indicate a disconnect between what many introductory psychology students think they are learning, and what their quiz performance indicates. In the absence of objective measures of learning (e.g., formative quizzing), students who are studying deeply may think they are learning (self-reported learning) and inaccurately gauge their actual progress (quiz score). This possibility is supported by the finding that the measure of SERSA, which included questions about how students utilized the results of quizzing, significantly correlated with quiz scores. Another potential explanation is that of the testing effect; students exposed to multiple quizzes or tests can achieve beneficial test performance as testing is an effective method of studying (Roediger & Karpicke, 2006). It could be that utilizing the testing effect is a concrete metacognitive strategy that can positively influence student performance—continued research in this area is needed to help disentangle these possible explanations for our data.

We conducted two separate multiple regression analyses (self-reported learning and quiz learning) to answer the question “what predicts learning?” Students at all GPA levels tend to overestimate their performance in general psychology (Prohaska, 1994) and our results show how misleading using only self-reported learning measures can be; thus, the predictors of the two types of learning are not the same. Teacher behavior predicted self-reported learning but not an objective measure. Total time studied and online quizzing behavior predicted an objective measure but not self-reported learning. We believe that the latter discrepancy indicates that students may not fully integrate their study behaviors into their own sense of learning, although said behaviors do indeed influence their actual learning as measured by an objective quiz.

Our results provide new challenges for pedagogical research attempting to identify optimal study skills. Not surprisingly, study behaviors continue to play a critical role in determining student success. These behavioral outcomes also suggest that students’ inaccurate perceptions of their learning may encourage less effective study behaviors. However, there is evidence that interventions to improve student awareness of their learning and develop study skills can be effective (Winne & Nesbitt, 2010). For example, Hattie (2009) found that interventions to improve study skills were significant in improving learning with effect sizes ranging from .59 (for teaching study skills) to .71 (for spaced vs. mass practice) and .69 (metacognitive strategies), significantly above the average effect sizes found (.40). Indeed, our data also indicate a positive correlation between a student’s use of organizationally effective behaviors and quiz scores. We recommend introductory psychology teachers consider designing exercises that get students to improve awareness of learning.

We were surprised by some lack of associations. For example, instructor rating did not predict quiz score, although it was highly related to self-reported learning. As the perception of the quality of instruction increases, so do student self-perceptions of learning or vice versa. The literature is clear that instructor characteristics are significantly related to student motivation, enthusiasm for the subject, and enjoyment of the course (Buskist, 2004), in addition to perceptions of learning. It is noteworthy that once we included in the analysis other factors more closely related to student effort and behavior, the role of the instructor was diminished. Similarly, ratings of textbook quality and helpfulness did not predict quiz scores or self-reported learning. Whereas both textbooks and instructors are clearly important as evidenced by the significant zero-order correlations between these variables, neither were significant predictors of objective learning as compared to student approaches to learning. Our results thereby reinforce the relative importance of the student role in their own learning.

A study such as this is designed to provide a broad overview of the major factors associated with learning in introductory psychology courses. Thus, firm conclusions cannot be developed at the level of practice. Indeed, the data came from different teachers in different universities using different textbooks. We also acknowledge that our main objective measure (AP exam questions) was not embedded into one of the courses’ exams. It is likely that students may have been more motivated to take the tests if their own class grade was dependent on their performance. However, this study was designed to point the way for future research to more closely examine important factors associated with learning. Importantly, our results encourage faculty to more explicitly attend to how their students’ study and offer future researchers a data-based rationale to more closely examine the relative value and complex interactions of student characteristics with regard to instructional materials, strategies, and instructor variables.

Footnotes

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The first and second authors received a research grant from McGraw-Hill Higher Education.

References

American Psychological Association. (2011). Principles for quality undergraduate education in psychology. American Psychologist, 66, 850–856.

Bain

(2004). What the best college teachers do. Cambridge, MA: Harvard University Press.

Bjork

R. A.

(1994). Memory and metamemory considerations in the training of human beings. In Metcalfe

Shimamura

(Eds.), Metacognition: Knowing about knowing (pp. 185–205). Boston: MIT Press.

Buskist

(2004, September). Ways of the master teacher. APS Observer, 17, 23–26.

Buskist

(2011). 7 Steps to becoming an excellent teacher (or at least a better teacher). Presentation at the 33rd Annual National Institute on the Teaching of Psychology, St. Pete Beach, FL.

Buskist

Benassi

V. A.

(Eds.). (2012). Effective college and university teaching: Strategies and tactics for the new professoriate. Thousand Oaks, CA: SAGE.

Chase

Houmanfar

. (2009). The differential effects of elaborate feedback and basic feedback on student performance in a modified, personalized system of instruction course. Journal of Behavioral Education, 18, 245–265. doi:10.1007/s10864–009–9089–2

Christopher

A. N.

Griggs

R. A.

Hagans

C. L.

(2000). Social and abnormal psychology textbooks: An objective analysis. Teaching of Psychology, 27, 180–189.

College Board. (2004). Writing: A ticket to work…or a ticket out. A survey of business leaders. New York: Author.

10.

Crede

Kuncel

N. R

. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3, 425–454. doi:10.1111/j.1745–6924.2008.00089.x

11.

Daniel

D. B.

Poole

. (2009). Learning for life: An ecological approach to pedagogical research. Perspectives on Psychological Science, 4, 91–96. doi:10.1111/j.1745–6924.2009.01095.x

12.

Dunning

Heath

Suls

J. M

. (2004). Flawed self-assessment: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5, 69–106. doi:10.1111/j.1529–1006.2004.00018.x

13.

Entwistle

(2009). Teaching for understanding at university: Deep approaches and distinctive ways of thinking. New York, NY: Palgrave Macmillan.

14.

Entwistle

N. J.

McCune

(2004). The conceptual bases of study strategy inventories. Educational Psychology Review, 16, 325–346.

15.

Fink

L. D.

(2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Jossey-Bass.

16.

Gettinger

Seibert

J. K.

(2002). Contributions of study techniques to academic competence. School Psychology Review, 31, 350–365.

17.

Griggs

R. A.

Bujak-Johnson

Proctor

D. L.

(2004). Using common core vocabulary in text selection and teaching the introductory course. Teaching of Psychology, 31, 265–269.

18.

Gurung

R. A. R.

(2004). Pedagogical aids: Learning enhancers or dangerous detours? Teaching of Psychology, 31, 164–166.

19.

Gurung

R. A. R.

Martin

(2011). Predicting textbook reading: The textbook assessment and usage scale. Teaching of Psychology, 38, 22–28. doi:10.1177/0098628310390913

20.

Gurung

R. A. R.

McCann

(in press). Study techniques and teaching. In Schwartz

Gurung

R. A. R.

(Eds), Evidence-based teaching in higher education, Washington, DC: American Psychological Association.

21.

Gurung

R. A. R.

Schwartz

(2009). Optimizing teaching and learning: Pedagogical research in practice. Malden, MA: Blackwell.

22.

Gurung

R. A. R.

Vespia

K. M

. (2007). Looking good, teaching well? Linking liking, looks, and learning. Teaching of Psychology, 34, 5–10. doi:10.1207/s15328023top3401_2

23.

Gurung

R. A. R.

Weidert

Jeske

A. S.

(2010). A closer look at how students study (and if it matters). Journal of the Scholarship of Teaching and Learning, 10, 28–33.

24.

Hacker

D. J.

Dunlosky

Graesser

A. C.

(1998). Metacognition in educational theory and practice. Mahwah, NJ: Lawrence Erlbaum.

25.

Hattie

(2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London, England: Routledge.

26.

Keeley

Smith

Buskist

. (2006). The Teacher Behaviors Checklist: Factor analysis of its utility for evaluating teaching. Teaching of Psychology, 33, 84–90. doi:10.1207/s15328023top3302_1

27.

Kennedy

E. J.

Lawton

Plumlee

E. L.

(2002). Blissful ignorance: The problem of unrecognized incompetence and academic performance. Journal of Marketing Education, 24, 243–252.

28.

Lammers

Smith

(2008). Learning factors in the university classroom: Faculty and student perspectives. Teaching of Psychology, 35, 61–70. doi:10.1080/00986280802004586.

29.

Landrum

R. E.

Hormel

. (2002). Textbook selection: Balance between the pedagogy, the publisher, and the student. Teaching of Psychology, 29, 245–248. doi:10.1207/S15328023TOP2901_04

30.

Nevid

J. S.

Lampmann

J. L.

(2003). Effects on content acquisition of signaling key concepts in text material. Teaching of Psychology, 30, 227–330.

31.

Prohaska

. (1994). ‘I know I’ll get an A’: Confident overestimation of final course grades. Teaching of Psychology, 21, 141–143. doi:10.1207/s15328023top2103_2.

32.

Roediger

H. L.

Karpicke

J. D.

(2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210.

33.

Sappington

Kinsey

Munsayac

. (2002). Two studies of reading compliance among college students. Teaching of Psychology, 29, 272–274.

34.

Tait

Entwistle

N. J.

McCune

(1997). ASSIST: A reconceptualisation of the Approaches to Studying Inventory. In Rust

(Ed.), Improving student learning: Improving students as learners. Oxford, England: Oxford Brookes University, Oxford Centre for Staff and Learning Development.

35.

Weiten

Deguara

Rehmke

Sewell

(1999). University, community college, and high school students’ evaluations of textbook pedagogical aids. Teaching of Psychology, 26, 19–21.

36.

Wesp

Miele

(2008). Student opinions of the quality of teaching activities poorly predict pedagogical effectiveness. Teaching of Psychology, 35, 360–362. doi:10.1080/00986280802374617.

37.

Winne

P. H.

Nesbit

J. C.

(2010). The psychology of academic achievement. Annual Review of Psychology, 61, 653–678.

38.

Zinn

Magnotti

Marchuk

Schultz

Luther

Varfolomeeva

(2011). Performance vs. effort: More about what makes the grade. Teaching of Psychology, 38, 10–15. doi:10.1177/0098628310390907