How Effective are Pedagogical Agents for Learning? A Meta-Analytic Review

Abstract

Research on the use of software programs and tools such as pedagogical agents has peaked over the last decade. Pedagogical agents are on-screen characters that facilitate instruction. This meta-analysis examined the effect of using pedagogical agents on learning by reviewing 43 studies involving 3,088 participants. Analysis of the results indicated that pedagogical agents produced a small but significant effect on learning. The overall mean effect size was moderated by the contextual and methodological features of the studies. The findings revealed that the use of pedagogical agents were more beneficial for K-12 students than post-secondary students. Pedagogical agents that communicated with students using on-screen text facilitated learning more effectively than agents that communicated using narration. The findings of this study have implications for advancing theory and practice, as well as highlighting productive future directions for research.

Get full access to this article

View all access options for this article.

References

Adcock

A. B.

Duggan

M. H.

Nelson

E. K.

Nickel

(2006). Teaching effective helping skills at a distance: The development of project CATHIE. The Quarterly Review of Distance Education, 7(4), 349–360.

Adcock

A. B.

Van Eck

R. N.

(2005). Reliability and factor structure of the attitude toward tutoring agent scale (ATTAS). Journal of Interactive Learning Research, 16(2), 195–217.

Adesope

O. O.

Nesbit

J. C.

(2012). Verbal redundancy in multimedia learning environments: Ameta-analysis. Journal of Educational Psychology, 104(1), 250–263.

Atkinson

R. K.

(2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94(2), 416–427.

Atkinson

R. K.

Mayer

R. E.

Merrill

M. M.

(2005). Fostering social agency in multimedia learning: Examining the impact of an animated agent's voice. Contemporary Educational Psychology, 30, 117–139.

Ayers

Sweller

(2005). The split-attention principle in multimedia learning. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 19–30). New York, NY: Cambridge University Press.

Baylor

A. L.

(2002). Expanding preservice teachers' metacognitive awareness of instructional planning through pedagogical agents. Educational Technology Research and Development, 50(2), 5–22.

Baylor

A. L.

Kim

(2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450–457.

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2008). Comprehensive meta-analysis (version 2.2.048). Englewood, NJ: Biostat.

10.

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2009). Introduction to meta-analysis. Chippenham, Wiltshire: Wiley.

11.

Burgoon

J. K.

Bonito

J. A.

Bengtsson

Cederberg

Lundeberg

Allspach

(2000). Interactivity in human-computer interaction: A study of credibility, understanding, and influence. Computers in Human Behavior, 16, 552–574.

12.

Cheng

Y. M.

Chen

L. S.

Huang

H. C.

Weng

S. F.

Chen

Y. G.

Lin

C. H.

(2009). Building a general purpose pedagogical agent in a web-based multimedia clinical simulation system for medical education. IEEE Transactions on Learning Technologies, 2(3), 216–225.

13.

Choi

Clark

R. E.

(2006). Cognitive and affective benefits of an animated pedagogical agent for learning English as a second language. Journal of Educational Computing Research, 24(4), 441–466.

14.

Clark

R. E.

Choi

(2005). Five design principles for experiments on the effects of animated pedagogical agents. Journal of Educational Computing Research, 32(3), 209–225.

15.

Clark

R. E.

Choi

(2007). The questionable benefits of pedagogical agents: Response to Veletsianos. Journal of Educational Computing Research, 36(4), 379–381.

16.

Cooper

Hedges

L. V.

(2009). Research synthesis as a scientific process. In Cooper

Hedges

L. V.

Valentine

J. C.

(Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 3–16). New York, NY: Russell Sage Foundation.

17.

Craig

S. D.

Gholson

Driscoll

D. M.

(2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94(2), 428–434.

18.

DaPra

C. S.

Mayer

R. E.

(2011). Effects of gesture and voice on online learning with animated pedagogical agents. Poster session presented at the 2011 annual meeting of the American Educational Research Association.

19.

Dehn

D. M.

van Mulken

(2000). The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies, 52, 1–22.

20.

Domagk

(2010). Do pedagogical agents facilitate learner motivation and learning outcomes? The role of the appeal of agent's appearance and voice. Journal of Media Psychology, 22(2), 84–97.

21.

Dunsworth

Atkinson

R. K.

(2007). Fostering multimedia learning of science: Exploring the role of an animated agent's image. Computers & Education, 49, 677–690.

22.

Egger

Smith

G. D.

Schneider

Minder

(1997). Bias in meta-analysis detected by a simple graphical test. British Medical Journal, 315, 629–634.

23.

Flather

M. D.

Farkouh

M. E.

Yusuf

(1997). Strengths and limitations of meta-analysis: Larger studies may be more reliable. Controlled Clinical Trials, 18, 568–579.

24.

Frechette

Moreno

(2010). The roles of animated pedagogical agents' presence and nonverbal communication in multimedia learning environments. Journal of Media Psychology, 22(2), 61–72.

25.

Ginns

(2005). Meta-analysis of the modality effect. Learning and Instruction, 15, 313–331.

26.

Graesser

A. C.

Chipman

Haynes

B. C.

Olney

(2005). AutoTutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions on Education, 48(4), 612–618.

27.

Graesser

Jackson

G. T.

Ventura

Mueller

Person

(2003). The impact of conversational navigational guides on the learning, use and perceptions of users of a web site. Proceedings of the AAAI Spring Symposium on Agent-Mediated Knowledge Management. Menlo Park, CA: AAAI Press.

28.

Gulz

(2004). Benefits of virtual characters in computer based learning environments: Claims and evidence. International Journal of Artificial Intelligence in Education, 14, 313–334.

29.

Heidig

Clarebout

(2011). Do pedagogical agents make a difference to student motivation and learning? Educational Research Review, 6, 27–54.

30.

Hershey-Dirkin

Mishra

Altermatt

(2005). All of or nothing: Levels of sociability of a pedagogical software agent and its impact on student perceptions and learning. Journal of Educational Multimedia and Hypermedia, 14(2), 113–127.

31.

Higgins

Thompson

S. G.

Deeks

J. J.

Altman

D. G.

(2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557–560.

32.

Holmes

(2007). Designing agents to support learning by explaining. Computers & Education, 48, 523–547.

33.

Hostetter

A. B.

(2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137(2), 297–315.

34.

Johnson

W. L.

Rickel

J. W.

Lester

(2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments. International Journal of Artificial Intelligence in Education, 11, 47–78.

35.

Kennedy

Archambault

(2012). Offering preservice teachers field experiences in K-12 online learning: A national survey of teacher education programs. Advance online publication. doi: 10.1177/0022487111433651.

36.

Kim

Baylor

A. L.

Shen

(2007). Pedagogical agents as learning companions: The impact of agent emotion and gender. Journal of Computer Assisted Learning, 23(3), 220–234.

37.

Kim

Ryu

(2003). Meta-analysis of the effectiveness of pedagogical agent. In Lassner

McNaught

(Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003 (pp. 479–486). Chesapeake, VA: AACE.

38.

Kirk

(2009). Performance, perception, and choice of animated pedagogical agent. Dissertation Abstracts International: Section A, 70(4), 1247A. (UMI No. 3356553).

39.

Kirschner

P. A.

Sweller

Clark

R. E.

(2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.

40.

Kizilkaya

Askar

(2008). The effect of an embedded pedagogical agent on the students science achievement. Interactive Technology and Smart Education, 5(4), 208–216.

41.

Lester

J. C.

Towns

S. G.

Fitzgerald

P. J.

(1999). Achieving affective impact: Visual emotive communication in lifelike pedagogical agents. International Journal of Artificial Intelligence in Education, 10(3–4), 278–291.

42.

Lipsey

M. W.

Wilson

D. B.

(2001). Practical meta-analysis. Applied Social Research Methods Series (Vol. 49). Thousand Oaks, CA: Sage.

43.

Louwerse

M. M.

Graesser

A. C.

Mitchell

H. H.

(2005). Social cues in animated conversational agents. Applied Cognitive Psychology, 19, 693–704.

44.

Louwerse

M. M.

Graesser

A. C.

McNamara

D. S.

(2008). Embodied conversational agents as conversational partners. Applied Cognitive Psychology. Published online in Wiley InterScience (www.interscience.wiley.com). doi: 10.1002/acp.1527

45.

Low

Sweller

(2005). The modality principle in multimedia learning. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 147–158). New York, NY: Cambridge University Press.

46.

Lusk

M. M.

Atkinson

R. K.

(2007). Animated pedagogical agents: Does their degree of embodiment impact learning from static or animated worked examples? Applied Cognitive Psychology, 21, 747–764.

47.

Mayer

R. E.

(2005a). Cognitive theory of multimedia learning. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 31–48). New York, NY: Cambridge University Press.

48.

Mayer

R. E.

(2005b). Principles of multimedia learning based on social cues: Personalization, voice, and image principles. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 201–212). New York, NY: Cambridge University Press.

49.

Mayer

R. E.

(2005c). Principles for managing essential processing in multimedia learning: Segmenting, pretraining, and modality principles. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 169–182). New York, NY: Cambridge University Press.

50.

Mayer

R. E.

Dow

G. T.

Mayer

(2003a). Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds? Journal of Educational Psychology, 95(4), 806–813.

51.

Mayer

R. E.

Sabko

Mautone

(2003b). Social cues in multimedia learning: Role of speaker's voice. Journal of Educational Psychology, 95(2), 419–425.

52.

Moreno

(2005). Multimedia learning with animated pedagogical agents. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 507–523). New York, NY: Cambridge University Press.

53.

Moreno

Mayer

R. E.

Spires

H. A.

Lester

J. C.

(2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177–213.

54.

Moreno

K. N.

Person

N. K.

Adcock

A. B.

Eck

R. N. V.

Jackson

G. T.

Marineau

J. C.

(2002). Etiquette and efficacy in animated pedagogical agents: The role of stereotypes. Paper presented at the AAAI Symposium on Personalized Agents, Cape Cod, MA.

55.

Moreno

Reislein

Ozogul

(2010). Using virtual peers to guide visual attention during learning, a test of the persona hypothesis. Journal of Media Psychology, 22(2), 52–60.

56.

Moundridou

Virvou

(2001). Evaluating the impact of interface agents in an intelligent tutoring systems authoring tool. Advances in Human-Computer Interaction I: Proceedings of the PC-HCI (pp. 371–376). Patras, Greece: Typorama Publications.

57.

Moundridou

Virvou

(2002). Evaluating the persona effect of an interface agent in a tutoring system. Journal of Computer Assisted Learning, 18, 253–261.

58.

Murray

Tenenbaum

(2010). Computerized pedagogical agents as an educational means for developing physical self-efficacy and encouraging activity in youth. Journal of Educational Computing Research, 42(3), 267–283.

59.

Paas

Renkl

Sweller

(2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4.

60.

Paas

Van Merrënboer

(1994). Variability of worked examples and transfer of geometrical problem solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122–133.

61.

Park

(2006). The effects of seductive augmentation and agent role on learning interest, achievement, and attitude. Dissertation Abstracts International: Section A, 67(4), 1303A.

62.

Plant

E. A.

Baylor

A. L.

Doerr

C. E.

Rosenberg-Kima

R. B.

(2009). Changing middle-school students' attitudes and performance regarding engineering with computer-based social models. Computers & Education, 53, 209–215.

63.

Rickenberg

Reeves

(2000). The effects of animated characters on anxiety, task performance, and evaluations of user interfaces. Proceedings of the CHI (pp. 49–56). New York, NY: ACM Press.

64.

Rosenthal

(1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86(3), 638–641.

65.

Song

Khan

K. S.

Dinnes

Sutton

A. J.

(2002). Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. International Journal of Epidemiology, 31, 88–95.

66.

Sweller

(2005). Implications of cognitive load theory for multimedia learning. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 19–30). New York, NY: Cambridge University Press.

67.

Sweller

(2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22, 123–138.

68.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn and Bacong

69.

The APA Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist, 63(9), 839–851.

70.

Theodoridou

K. D.

(2010). Learning with Laura: Investigating the effects of a pedagogical agent on Spanish lexical acquisition. Dissertation Abstracts International: Section A, 70(9), 3431A.

71.

van Merriënboer

J. J. G.

Kester

(2005). The four-component instructional design model: Multimedia principles in environments for complex learning. In Mayer

(Ed.), The Cambridge handbook of multimedia learning (pp. 71–93). New York, NY: Cambridge University Press.

72.

van Mulken

André

Muller

(1998). The persona effect: How substantial is it? In Johnson

Nigay

Roast

(Eds.), People and computers XIII: Proceedings of HCI'98 (pp. 53–66). Berlin, Germany: Springer.

73.

Veletsianos

(2010). Contextually relevant pedagogical agents: Visual appearance, stereotypes, and first impressions and their impact on learning. Computers & Education, 55(2), 576–585.

74.

Veletsianos

(2012). How do learners respond to pedagogical agents that deliver social-oriented non-task messages? Impact on student learning, perceptions, and experiences. Computers in Human Behavior, 28(1), 275–283.

75.

Veletsianos

Miller

(2008). Conversing with pedagogical agents: A phenomenological exploration of interacting with digital entities. British Journal of Educational Technology, 39(6), 969–986.

76.

Veletsianos

Miller

Doering

(2009). Enali: A research design framework for virtual characters and pedagogical agents. Journal of Educational Computing Research, 41(2), 171–194.

77.

Yung

H. I.

(2009). Effects of an animated pedagogical agent with instructional strategies in multimedia learning. Journal of Educational Multimedia and Hypermedia, 18(4), 453–466.