ToP ’s Greatest Hits

Age of Articles

Only one of the Top 20 total citations articles is from the 1970s, five are from the 1980s, 12 are from the 1990s, and two are from the 2000s. The mean and median ages are 20.55 and 20.50 years, respectively, with a range of 7 to 35 years. These articles are older than those on the average citations Top 20 list. The mean and median ages of these latter articles are only 9.65 and 8.50, respectively, with a range of 3 to 26 years. Also, in contrast with the articles on the total citations list, none are from the 1970s, only one from the 1980s, five from the 1990s, and 14 from the 2000s. Whereas only 10% of the articles on the total citations list are from the 2000s, 70% of the articles on the average citations list are. The age biases for the two indexes are also demonstrated by the fact that 13 (65%) of the 20 articles on the average citations list have received 100% of their citations in the last 10 years, but this is only true for one (5%) of the 20 articles on the total citations list.

The differing impact of age on the two lists was expected because older articles have the advantage in compiling total citations given their longer lives, but recent articles have the advantage in computing average citations per year given their shorter lives. Even though these age biases work to minimize overlap between the two Top 20 lists, there is surprising commonality between the lists with 40% (8) of the 20 articles on each list also appearing on the other list. Being on both lists is quite a feat for an article. This means that the article is both cited frequently and typically cited frequently over a substantial period of time.

Length of Articles

Because longer articles have been found to receive more citations (Frosch et al., 2010; Seglen, 1997), we expected that of the 32 articles on one or both lists, most would be the longer, more substantial articles that are now termed Topical Articles, and this was the case. Twenty-four of the 32 articles (75%) are Topical Articles, 6 (19%) are Faculty Forum articles, and only 2 (6%) are Methods and Techniques articles. All of the articles that appear on both lists are what we have classified as Topical Articles.

Academic Dishonesty

The most cited article for both indexes, Davis, Grover, Becker, and McGregor (1992), reported the findings for a large survey study on academic dishonesty. Davis et al. has 107 total citations and a citation yearly average of 5.35 citations. Although this article is 20 years old, it continues to be cited frequently. For example, it had 15 citations in 2009 alone, the most for any ToP article in 1 year and greater than the total number of citations ever for 95.93% of the entire set of articles published in ToP (1975–2011).

Why has the Davis et al. (1992) article been cited so often? Whereas some articles on “cheating” had been published in ToP prior to the Davis et al. article (e.g., Bellezza & Bellezza, 1989), these articles only dealt with very specific aspects of cheating, such as using crib notes or copying on multiple choice tests. The Davis et al. article, however, reported the findings of a seminal, large survey study involving more than 6,000 students regarding the prevalence, causes, techniques, faculty and institutional responsibility, deterrent measures, and punishment dimensions of academic dishonesty. To the best of our knowledge, no one had done a survey study of this scale and scope previously, or has done one subsequently. Given that academic dishonesty was and continues to be a perennial problem in higher education, Davis et al.’s thorough study of this problem is likely the major reason for this article’s great impact. Interestingly, Davis and Ludvigson (1995) provided some additional data from 2,153 upper division undergraduate students regarding the frequency of cheating, reasons for cheating, and influence of penalties on cheating and a proposal for remediation of academic dishonesty. This later article has also been cited frequently but not nearly as often as the earlier one. It was tied for 32nd in total citations with 22 and was 51st in average citations per year with an average of 1.29.

Given the impact of Davis et al. (1992), academic dishonesty seems to be a topic of much interest to the teaching research community. However, Davis et al. is the only article on either Top 20 list that deals with academic dishonesty. The remaining seven articles that appear on both lists are concerned with five other important topics (mentor–student relationships in psychology graduate programs, the undergraduate psychology curriculum, introductory psychology textbooks, critical thinking, and active learning), and five of these seven articles report the findings of large surveys like the one reported by Davis et al.

Mentor–Student Relationships

Cronan-Hillix, Gensheimer, Cronan-Hillix, and Davidson (1986) and Clark, Harden, and Johnson (2000) reported findings for surveys examining mentor–student relationships in psychology graduate training, a topic for which little empirical data existed at the time of either study. Cronan-Hillix et al. surveyed 90 graduate students and reported findings on the prevalence and role of mentors and the personality characteristics and functions of mentors from the viewpoint of students. Similarly, Clark et al. reported the survey findings for nearly 800 graduate student respondents in clinical doctoral programs (both PhD and PsyD programs) on the prevalence of mentor relationships, personality characteristics of mentors, the functions of mentoring, and the evaluation and importance of the mentoring relationship. Although neither of these studies is recent, both continue to be frequently cited because of the scarcity of large-scale, survey data on this critical topic of graduate student mentoring. For example, Clark et al. was cited 11 times in 2011.

Undergraduate Psychology Curriculum

The two articles by Perlman and McCann (1999a, 1999b) address another important issue—the undergraduate psychology curriculum; and like the three articles already discussed, both report findings from large survey studies. Perlman and McCann (1999a) reported the results of a survey of 400 college catalogs that identified the most frequently listed undergraduate psychology courses for all four institutional types per the Carnegie Foundation for the Advancement of Teaching taxonomy. Perlman and McCann (1999b) reported the results of a survey of 500 college catalogs that identified the modal undergraduate psychology program for the four institutional types. Demonstrating further importance of this topic, Messer, Griggs, and Jackson (1999), who reported the findings of a survey of 292 college catalogs to identify undergraduate psychology degree options and major requirements, tied for 23rd in average citations with 1.69 per year and for 32nd in total citations with 22.

Analysis of Introductory Textbooks

The remaining survey study that appears on both lists, Marek, Griggs, and Christopher (1999), reported the ratings of 411 psychology students on 15 pedagogical aids in introductory psychology textbooks for familiarity, likelihood of use, and value. In addition, Marek et al. examined 37 introductory textbooks to determine the prevalence of the 15 aids. This study addressed the question of whether these learning aids that add to the length and possibly the cost of the texts are justified by their probability of use and perceived helpfulness. Another article on the total citations Top 20 list, Weiten (1988), had earlier found that the number of pedagogical aids present in a text did not have a significant predictive influence on professors’ impressions of pedagogical quality. Weiten measured 29 objective variables (e.g., number of pedagogical aids present, number of references, and size of glossary) for a sample of 43 introductory textbooks. He also surveyed 857 psychology professors (but there were only 156 usable returns) for their familiarity with the 43 texts and, depending upon their familiarity, to provide ratings on four dimensions (e.g., pedagogical quality and level of discourse). Weiten’s seminal textbook study was highly influential and stimulated numerous other textbook studies, such as the one by Marek et al., in the ensuing decade.

There is another introductory textbook study on the total citations Top 20 list, Gorenflo and McConnell (1991). Gorenflo and McConnell analyzed the reference lists for 24 introductory psychology textbooks to determine the most frequently cited journal articles in introductory texts. This resulted in a massive database with 37,590 bibliographic items. Interestingly, there was not one article that was cited in all 24 textbooks. The highest citation total for an article was 22. Whereas textbook research was a heavily investigated topic in the late 1980s and throughout the 1990s, interest in this topic has lessened recently. The total number of citations for all three of these textbook studies combined was only seven during the past 5 years. Current introductory textbook studies typically tend to be far more limited in scope, assessing the presence and nature of the coverage of specific topics, such as disability (Goldstein, Siegel, & Seman, 2010) or ethics (Zucchero, 2011).

Critical Thinking, Writing, and Active Learning

The remaining two articles that appear on both lists, King (1995) and Yoder and Hochevar (2005), are concerned with using questioning to teach critical thinking and how active learning can improve student performance on exams, respectively. King’s article appeared in the special issue, “Psychologists Teach Critical Thinking” (Halpern & Nummedal, 1995). Two other articles in this special issue on critical thinking, Angelo (1995) and Wade (1995) also appear on the total citations Top 20 list.

King’s (1995) article described a model of inquiry and how to apply it in cooperative-learning contexts in the classroom as well as in individual reading and study settings. The model emphasizes helping students develop a habit of inquiry by learning to ask thoughtful questions. King’s program of research provides strong support for this inquiry-based learning model, contributing to its credibility and likely to the impact of this article. At first, the frequent citation of Angelo’s (1995) critical thinking article seems somewhat of an anomaly given its length, only a little over one page. Upon closer inspection, however, it is easy to discern a possible reason for this article’s impact––its subject matter. It describes classroom assessment techniques for critical thinking, a topic of much interest to teachers, and Angelo is a coauthor of a seminal handbook on classroom assessment techniques (Angelo & Cross, 1993). This article continues to be cited with 13 (50%) of its 26 citations coming in the last 5 years.

Wade’s (1995) article covers an aspect of teaching critical thinking that is also of much interest to teachers, using writing to develop and assess critical thinking. Wade is a well-respected authority on critical thinking and has (with her colleague, Carol Tavris) authored extensive critical thinking materials. As is likely the case for the other two articles from this special issue on critical thinking, Wade’s reputation probably made a significant contribution to her article’s impact. The only other article on either Top 20 list in addition to Wade’s that deals with writing, Hettich (1990), appeared in a special issue of ToP, “Psychologists Teach Writing” (Nodine, 1990). Hettich’s article dealt specifically with journal writing, a technique that allows students to connect specific course concepts with their experiences. This article’s impact is likely due to the fact that student journal writing is a popular teaching/learning technique that has been used for decades in psychology, sociology, English, education, and other fields. Even given its long history, journal writing still seems to be of interest to the teaching community. Connor-Greene (2000), who demonstrated that journal writing increased student learning and exam grades, tied for 22nd on the average citations list with an average of 1.75 citations per year.

Finally, Yoder and Hochevar (2005) is noteworthy because its total citation data are really strong given that it was published so recently. Its large impact is likely due to its general topic, active learning, and more specifically to Yoder and Hochevar’s finding of empirical support for the efficacy of active learning techniques, indicating that active learning can improve students’ performance on examinations versus lecture, autonomous reading, or video without discussion.

Demonstrating the widespread interest in research on active learning, three other articles on this topic are on one or the other of the two Top 20 lists—Benjamin (1991), Poirier and Feldman (2007), and Gier and Kreiner (2009). Benjamin focused on ways to make large classes into small classes to facilitate active learning and on strategies to promote active learning in the large class. Poirer and Feldman also focused on promoting active learning in large introductory psychology classes, specifically using individual response technology (clickers) to do so. Gier and Kreiner demonstrated that it is possible to incorporate effective active learning methods into PowerPoint-based lectures.

Technology in Teaching

Given their use of computer technology, Poirier and Feldman (2007) and Gier and Kreiner (2009) could also have been grouped with the five other articles on one or the other of the Top 20 lists that deal with technology in teaching. Indicating the strong current research interest in this topic, these two articles along with four other articles—Baker, Wentz, and Wood (2009), DeBord, Aruguete, and Muhlig (2004), Morling, McAuliffe, Cohen, and DiLorenzo (2008), and Stowell and Nelson (2007)—appear on the average citations Top 20 list. Another technology article, Kulik, Kulik, and Cohen (1980), appears on the total citations list. Thus, of the 32 articles on the two lists, more concern technology in teaching than any other topic.

Like Poirier and Feldman (2007), Morling et al. (2008) examined the efficacy of clickers in large introductory classes. Morling et al., however, found that using clickers had only a small, positive effect on exam scores and that students using clickers did not report feeling significantly more engaged during class. Stowell and Nelson (2007) also examined the benefits of clickers versus standard lecture and two other response systems (hand-raising and response cards) on student participation and learning in simulated introductory psychology classes. They found that using clickers or response cards led to more class participation but no significant improvement on postlecture quiz scores. Students using clickers, however, did report greater positive emotion during lecture. The influence of these two articles is largely due to the continuing interest for empirical data on the efficacy of using clickers in teaching. Additional recent articles on this topic include Anthis (2011), Dallaire (2011), and Stowell, Oldham, and Bennett (2010), which all cite the Morling et al., Poirier and Feldman, and Stowell and Nelson articles.

Unlike clickers and PowerPoint™, which are frequently used in psychology classrooms, Baker et al. (2009) examined teaching and learning using Second Life^®, the most active virtual world, multiuser environment in higher education. Psychology teachers can use the virtual world as a space to meet with students and to create labs, buildings, and objects that can be used to learn psychology content and skills. The influence of this article is likely due to its cogent discussion of using virtual worlds as educational tools and the current interest in their use in online education (e.g., Stadtlander & Giles, 2010).

DeBord et al. (2004) was concerned with the effectiveness of simpler computer-assisted teaching methods—computer-assisted visuals versus overhead transparencies in lecture and using versus not using an optional website. As with many studies like this one, results showed that whereas students liked the computer-assisted teaching interventions, the interventions had no effect on student performance. The remaining study dealing with instructional technology, Kulik, Kulik, and Cohen (1980), was concerned with the use of much older technology. Kulik et al. reported the findings of a meta-analysis of more than 300 studies on five types of instructional technology, some of which younger teachers may not even know about, such as Keller’s Personalized System of Instruction (Keller, 1968) and Postlethwait’s Audio-Tutorial Approach (Postlethwait, Novak, & Murray, 1972). Given that Kulik et al. reviewed the effects of older instructional technologies, the article is mainly historically significant now as evidenced by the fact it has received only five citations in the past 10 years.

Diversity

Two articles on the average citations list are concerned with diversity—Case (2007) and Kernahan and Davis (2007). The impact of these two articles is likely due to the increasing research interest in the efficacy of diversity courses in psychology. Both of these articles provided quantitative data indicating the effectiveness of such courses. Case’s results included greater awareness of White privilege and racism and more support for affirmative action by the end of the term. Kernahan and Davis also found increases in awareness of racism, in realization of the extent of White privilege, and in feelings of racial guilt and responsibility. Undoubtedly, these reported positive benefits of the diversity courses have also contributed to the research impact of these two articles.

Assessment

Two articles on the average citations Top 20 list, Schaeffer, Epting, Zinn, and Buskist (2003) and Keeley, Smith, and Buskist (2006), are concerned with the Teacher Behavior Checklist (TBC) developed by Buskist, Sikorski, Buckley, and Saville (2002). The TBC identifies student and faculty perspectives on the qualities and corresponding behaviors of effective teachers. Schaeffer et al. surveyed 95 faculty and 231 students at a Midwestern community college using the TBC to assess their perspectives on effective teaching. In general, faculty and students’ rankings of the top 10 qualities or behaviors of effective teachers were very similar, thereby replicating Buskist et al.’s earlier findings for the TBC. Keeley et al. converted the TBC to an evaluative instrument to assess teaching and then evaluated this modified TBC via factor and psychometric analyses, providing evidence that the modified TBC is a psychometrically sound instrument that clearly identifies specific target behaviors that instructors can alter to improve their teaching. The impact of these two articles is likely due to their finding of empirical support for an instrument to not only identify the qualities and behaviors of effective teachers but also possibly to use as a tool for assessing teaching.

Halonen et al. (2003) also dealt with assessment, but outcomes assessment rather than individual teacher assessment. This article discussed a comprehensive multidimensional rubric to describe the progress of students’ acquisition of scientific inquiry skills throughout their academic experience and provides an example of an authentic assessment that demonstrates use of the rubric. Development of this rubric centered on some ideas from Halpern (1988), an older ToP article on assessing student outcomes. Like Halonen et al., the Halpern article has also been influential, just missing making the total citations list. It tied for 21st with 24 citations. The impact of Halonen et al. can be seen in recent articles on the assessment of more specific rubrics for grading APA-style introductions (Stellmack, Konheim-Kalkstein, Manor, Massey, & Schmitz, 2009) and for assessing student learning outcomes using a class assignment (Thaler, Kazemi, & Huscher, 2009).

Two other articles deal with student assessment (testing) issues—Benjamin, Cavell, and Shallenberger (1984) on the total citations list and Leeming (2002) on the average citations list. Benjamin et al. reviewed the findings of 33 studies on changing initial answers on objective tests and found that most students who change their answers improve their test scores and that the majority of answer changes are from incorrect to correct, dispelling the myth that one should not change answers on objective tests. Leeming evaluated the exam-a-day procedure to determine whether it improves performance. He found that giving a short exam at the start of every class led to significantly better grades than in classes where the same material was taught but with only four exams. The exam-a-day students also believed that this procedure led to more studying and better learning and reported that they liked the procedure. The impact of both articles most likely stems from the fact that both deal with important testing issues—changing answers on objective tests and the value of more frequent testing.

Student Misconceptions and Knowledge

The oldest article on either Top 20 list is Vaughan (1977), fourth on the total citations list with 40 citations. Interestingly, this was Vaughan’s only article published in ToP. Her study dealt with introductory psychology students’ misconceptions about psychology, their prevalence, and the difficulty in dispelling them. This was and still is a topic of much interest to psychology teachers, especially introductory psychology teachers. Vaughan developed a true–false Test of Common Beliefs (TCB) and found that introductory psychology students enter the course with a wide variety of misconceptions, but these have little effect on their performance in the course and are resistant to change by text reading and class discussion. Vaughan’s article was very influential for the next decade or so, stimulating much research and more articles on student misconceptions. Three of these articles have also gathered many citations. Gutman (1979) and Lamal (1979) have 24 citations and were among those tied for 21st on the list, and Gardner and Dalsing (1986) tied for 32nd with 22 citations. Some other studies (e.g., Brown, 1984; Gardner & Hund, 1983; Ruble, 1986), however, found fault with several of the TCB items. Because of these latter findings and the results of Griggs and Ransdell (1987) that indicated maybe rather than assessing student misconceptions, the TCB itself may have been misconceived, this cottage industry of research slowed down considerably. Empirical research on misconceptions in introductory students, especially on trying to dispel them (e.g., Kowalski & Taylor, 2009), however, continues, and citations to Vaughan continue at a decent rate (10 in the last 10 years).

Bunnell (1992), another article on the total citations list, examined the historical awareness of psychology majors and found that before taking a course in the history of psychology, students showed a low level of recognition of eminent contributors and their contributions to the history of psychology, markedly inferior to that of faculty members. Bunnell concluded that without explicit instruction, students acquire little knowledge of the history of their discipline. The impact of this article seems to have grown with 16 (59%) of its 27 citations coming in the last 10 years, likely related to continued interest in the structure of the undergraduate psychology curriculum (e.g., Stoloff et al., 2010) and the role that the history of psychology course plays in that structure.

Careers for Psychology Majors

Ware and Meyer (1981) surveyed 275 psychology majors who had graduated between May 1973 and August 1978 with respect to several dimensions such as additional education, income, employment status, and satisfaction with the major. Whereas these survey data were influential in the past, their influence has greatly lessened because of their age (over three decades old). The Ware and Meyer article has had no citations in the last 5 years and only two in the last 10 years.

Using Feature Films in Psychology Courses

The only article remaining to be discussed is Fleming, Piedmont, and Hiam (1990), which is on the total citations Top 20 list. Fleming et al. examined the effectiveness of using feature films in an interdisciplinary course on psychopathology, Psychology and Film: Images of Madness. Two other articles on using feature films in psychology courses were also ranked relatively high in total citations. Anderson (1992) was tied for 26th with 23 citations, and Boyatzis (1994) was tied for 52nd with 19 citations. Anderson was concerned with the use of feature films in a psychology and the law course, and Boyatzis with their use in teaching social development in a Middle Childhood course. All three of these studies found that students responded positively to the films and that using feature films led to valuable educational experiences. It appears that studies on the viability of using feature films in psychology courses have had far more impact than one might think, and this impact continues. For example, Fleming et al. received 17 (65%) of its 26 citations in the last 10 years.