Effects of Pedagogical Agents on Students’ Mathematics Performance

Abstract

This experimental study investigated the effects of the use of two versions of a pedagogical agent named personal instructing agent (PIA) on the mathematics performance of students. The first version exhibits synthetic facial expressions while the second version does not exhibit facial expression (i.e., neutral facial expression). Two groups of students with the same levels of prior knowledge in mathematics utilized two different versions of PIA. The first group—the facial group—utilized a PIA that provides textual and facial expressions feedback (happy, sad, surprise, and neutral facial expressions). The second group—the nonfacial group—used the same software except that PIA only exhibited neutral facial expression. The study showed that the mathematics scores of the students in the facial group significantly improved as compared with those who are in the nonfacial group. The posttest scores of the facial group were found significantly higher than those of the nonfacial group. The study showed that PIA that exhibited synthetic facial expressions improved students’ mathematics learning. It is concluded that synthetic facial expressions and textual feedback of pedagogical agent can be utilized to help students learn to solve mathematics problems. Limitations and recommendations are also presented.

Keywords

artificial emotion feedback facial expression emotion pedagogical agent personal instructing agent

Introduction

Mathematics is commonly perceived as a difficult subject. It poses cognitive challenges to students, and its difficulty is often linked to negative affects (e.g., boredom and anxiety) and dispositions (e.g., dislike of the course and lack of confidence; Brown, Brown, & Bibby, 2008). Consequently, students tend to discontinue taking the course because of perceived difficulty and perceived lack of relevance (Brown et al., 2008). Students who continue learn mathematics in different phases because they differ in levels of cognitive abilities. This makes teaching mathematics challenging since teachers have to attend to the different learning needs of each student (Arroyo et al., 2014). Intelligent tutoring systems (ITS) may address these pedagogical concerns.

ITS provide learners a nonthreatening environment that is customized to students’ learning needs (Bringula, Basa, Dela Cruz, & Rodrigo, 2015; Green, 2011). The inclusion of pedagogical agents (PA) in ITS further provides customized learning experience for students (Schroeder, Adesope, & Gilbert, 2013) since the PA can provide conversational and human-like feedback. A number of studies show that ITS with PA can improve learning of students (Bringula et al., 2015; Graesser, Chipman, Haynes, & Olney, 2005; Kizilkaya & Askar, 2008). ITS developers acknowledge that the emotional states (or sometimes used interchangeably with affective states; see Malekzadeh, Mustafa, & Lahsasna, 2015; Strain & D’Mello, 2011) of learners play a key role in learning since these states influence human cognitive abilities and learning (Damasio, 2004; Lin, Wu, & Hsueh, 2014; Maxwell, 2014; Megill, 2014; Strain & D’Mello, 2011). Detection of affective states of learners as well as affect responses is embedded in the design of PAs and showed that PAs with embedded emotions (or artificial emotions) can influence students’ learning (Chen et al., 2012; Lin et al., 2014; Malekzadeh et al., 2015; Mao & Li, 2010).

PAs with embedded emotions are developed to assist students’ learning in mathematics (e.g., Arroyo et al., 2014; Rodrigo et al., 2012; Woolf et al., 2009). However, the current designs of the PAs are focused on the detection of affect states of learners which in turn, provide emotion-regulation strategies to support positive affects and to counter negative ones. This study attempted to contribute to the existing thread of studies of PAs in mathematics by designing a PA that exhibits facial expressions while solving simple linear equation. The PA displays synthetic facial expressions of happiness, sadness, surprise, and neutral depending on the correctness of step in solving linear equations regardless of the affect state of learners. In short, this study considered synthetic facial expressions (subsequently referred as facial expressions) as forms of feedback when students are solving mathematics problems. Furthermore, this study attempted to compare the mathematics performances of two groups of students that used two different versions of PA.

Literature Review

Characteristics of a Good Mathematics Teacher

Studies consistently found that the good mathematics teachers are those who provide individual feedback and those who check the progress of their students by going from desk to desk (Kaur, 2008, 2009; Murray, 2011; White, Barnes, Lawson, & Johnson, 2009). This can be explained by the fact that students give value to one-on-one help where explanations are individualized or customized to the learning needs of the students (Kaur, 2009; Murray, 2011). Teachers who provide individualized feedback express implicitly that they understand their students’ effort to learn the subject (Murray, 2011; Raufelder et al., 2016). This leads students to perceive that their teachers know them individually and to feel that they are being encouraged to solve more problems (Murray, 2011; Patrut & Spatariu, 2015; Raufelder et al., 2016; White et al., 2009).

Given the foregoing findings, it can be said that a mathematics teacher is patient and does not rush explaining mathematical concepts (Murray, 2011). Likewise, it is safe to say that being understanding, passionate, pleasant, kind, and caring are the most desirable positive characteristics of a mathematics teacher (Murray, 2011; Patrut & Spatariu, 2015; Raufelder et al., 2016; White et al., 2009).

Emotions and Learning

Emotions, feelings, and affect are used interchangeably (Malekzadeh et al., 2015; Shouse, 2005; D’Mello, Craig, Witherspoon, McDaniel, & Graesse, 2008; Strain & D’Mello, 2011). Nonetheless, different authors offer definitions to distinguish one from the other. Emotions are defined as “psychological states that comprise thoughts and feelings, physiological changes, expressive behaviors, and inclinations to act” (Manstead, 2007, p. 285). In short, emotions (e.g., surprise, happiness, anger, fear, disgust, and sadness; Maxwell, 2014) are subjective projections or displays of feelings (Shouse, 2005; Tone, 2007). Feelings occur when a person checks a sensation with previous experiences and labels that sensation (Shouse, 2005). Meanwhile, affect is the general term that contains emotions, feelings, and moods (D’Mello, Craig, Witherspoon, McDaniel, & Graesse, 2008; Isen, 2007; Tsahuridu, 2009). Also, affect is more focused on the state of feelings (such as joy, calmness, love, anxiety, boredom, confusion, delight, and frustration; see Isen, 2007; Larsen, 2007; Rodrigo & Baker, 2009; Tsahuridu, 2009) upon which the body takes action based on these states (Shouse, 2005).

The study of emotions is important in the context of learning because it can influence human cognitive abilities (Damasio, 2004; Di Martino & Zan, 2011; Maxwell, 2014; Megill, 2014). In particular, emotions affect in the mathematics performance of the students. Estrada, Young, and Isen (1994) and Isen and Shalker (1982) show that students who are experiencing positive emotions can improve problem-solving abilities. This is also confirmed in the study of Connor and Davidson (2003) who found that students who feel happy perform better than students who feel sad, angry, or scared of a mathematics subject.

It is also important for teachers to display positive emotions when teaching (Patrut & Spatariu, 2015). According to Krapp (2002), teachers who show positive emotions can stimulate the interests and intrinsic motivations of their students. In turn, the positive disposition of teachers can facilitate positive emotions of students (Malekzadeh et al., 2015) and ultimately, student–teacher engagement in the learning process (Patrut & Spatariu, 2015; Ruzek et al., 2016; Taylor, 2010). On the other hand, if a teacher always manifests negative emotion like anger during instruction, this may lead to being detached from teaching and from the students (Hascher, 2010). Thus, scaffolding of positive emotions can increase positive emotions and learning (Strain & D’Mello, 2011).

Artificial Emotions, PAs, Feedback, and Learning

Kizilkaya and Askar (2008) revealed that students who used ITS with a PA had a better science achievement level than students who did not use ITS. PA can also provide textual feedback. Wang et al. (2008) incorporated polite feedback (e.g., “It might be useful to read the paragraph.” “You might want to read the paragraph.”) to a PA. It was shown that students who utilized the polite version of software achieved better learning outcomes than students who utilized a version that lacked this feature.

Advancements in technology enable developers to animate PAs that exhibit embedded emotions and detect affective states of its users. These design features, together with the correctness and number of emotion recognition of students, and pedagogical actions are among the factors that affect the satisfaction of learners on the design of affective tutoring systems (Mao & Li, 2010). Chen et al. (2012) showed that students who used an animated PAs (or avatar) were more willing to continue their readings and were more likely to complete their exercises than those who used a nonempathic avatar. The empathic version of the avatar, which served as a friend, provided motivational cues and empathic facial expressions based on the expressed emotions of learners. Emotions of learners were expressed by clicking buttons that correspond to four-pair emotional responses. This strategy regulates and corrects the emotions of learners during the learning process (Mao & Li, 2010; Lin et al., 2014).

Kim, Baylor, and Shen (2007) conducted two experiments to determine the impact of emotion and gender of a PA as a learning companion (PAL). The first experiment was focused on a PAL with positive, negative, and neutral emotions while the second experiment involved an empathic (responsive and nonresponsive) PAL. The researchers reported that the empathic reaction had positive impact on the interest and self-efficacy of learners. Likewise, the gender of PAL had positive impact on recall. In a similar study, Van der Meij (2013) showed students who utilized an animated pedagogical agent had higher motivation and learning than those students in the no-agent group. It was concluded that that the animated pedagogical agent could enhance students motivation and learning in software training. Hernández et al. (2016) concluded that PA is considered useful for learning.

Feedback mechanisms are also incorporated in PAs. Mory (1996) defines feedback as a computer-generated message in response to learners’ activities. The concept of feedback is utilized in the development of ITS for mathematics. Corbalan, Paas, and Cuypers (2010) showed that feedback on all problem-solving steps helped students to learn more effectively and to have higher motivation to solve the problem. A feedback can be given on each line of a solution every time a student solves a linear equation (Bringula et al., 2015). The study of Rodrigo et al. (2012) included feedback mechanisms and affective responses (happy, sad, and anger). In the study, the researchers utilized a tutoring system where the PA-assisted students learning scatter plot diagrams. In the same study, an artificial agent, also referred to as a dog named Scooter, is a PA that assists students to construct a scatter plot diagram. Scooter and to show happy emotions when they are not gaming the system. Scooter also gives occasional positive messages to encourage students as well as exhibits sadness when the students commit mistakes in solving the problem. The increasing levels of displeasure from repeated mistakes are interpreted by Scooter as gaming the system. Scooter gets angry and reprimands the student from gaming the system.

Research Framework, Statement of the Problem, and Hypothesis

The foregoing review of related studies provided the framework of this study. The study adapted the concept of emotions given by Maxwell (2014) but added a neutral state of emotion. In this study, only two positive emotions (surprise and happiness) and one negative (sadness) were chosen due to ethical considerations. These emotions served as basis in the development of facial expressions of the PA. Since the students in the study are in the initial stage of learning mathematics, it would be inappropriate to show anger, fear, or disgust when students committed mistakes during the learning process. This study considered repeated mistakes as a form of perseverance and not as gaming the system. The PA in this study is caring, understanding, and polite. Thus, the PA displays a sad facial expression but encourages the students to continue until the correct answer is achieved.

Furthermore, the PA gives praises when correct solution is given by the student. It also gives encouragements and textual feedback (i.e., hints) when the answer is incorrect. It is developed in such a way that students will have an impression that it is kind, caring, and nonthreatening. The study incorporated line-by-line individualized feedback mechanism on each step of solving a mathematical problem. The detailed discussion of the PA and its facial expressions are given in the next section. The concept of the study is depicted in the research paradigm shown in Figure 1. In this paradigm, there are two sets of students (facial and nonfacial group) who utilized different versions of software.

Figure 1.

Research paradigm of the study.

This study attempted to determine whether facial expressions combined with textual feedback of a PA could serve as formative reactions to a student’s works in progress in solving mathematics problems. The concept of the study is shown in Figure 1. Based on the paradigm, the study sought to find answers to the following research questions (RQs).

RQ1: Do the software usage of the students in the facial and non-facial group differ in terms of number of problems solved, number of problems not solved, time spent using the software, and number of hints used in solving an equation?

The first RQ examined the software utilization of the participants of the study. It attempted to determine the levels of engagement of the students in the experiment. This is based on the reports of White et al. (2009) that students believed that they would participate in the class when their teachers explain the topics very well and when their teachers encourage them to participate. Also, teachers that provided one-on-one feedback were found to increase the number of problems solved in mathematics (Murray, 2011).

Moreover, Patrut and Spatariu (2015) showed that emotions had significance influence on the ability of the students to learn new information or to solve problems. Students who were experiencing positive emotions tend to solve problems in different ways and are motivated to complete the assigned tasks. Consequently, students prefer teachers who can assess the latter correctly and can provide feedback and empathy. Ruzek et al. (2016) also found that the emotional support of teachers had positive impact on the behavioral engagement of the students in the class. Students were more participative in classroom discussion in terms of responding to teacher’s questions, paying attention, and doing the assigned tasks. Appreciating the students’ works, acknowledging the learning progress of the students, considering individual learning phases, identifying learning difficulties, and providing encouragement were also identified as desirable qualities of teachers that could engage the students in the class (Raufelder et al., 2016). In a computerized learning environment, an empathic PA could influence the engagement of the students in learning (Chen et al., 2012).

In this regard, the first hypothesis to be tested in this study is that students in the facial group tend to be more engaged in the experiment than those in the nonfacial group in terms of number of problems solved (H1a), number of problems not solved (H1b), time spent using the software (H1c), and number of hints used in solving an equation (H1d).

RQ2: How Do the Facial and Nonfacial PA Influence Students’ Mathematics Performance?

This RQ justifies the investigation of the mathematics performance of the students before and after using the software. In view of this RQ, it is hypothesized that the use of facial version of PA would lead to better mathematics performance of the students compared with the use of the nonfacial version of PA (H2). This is based on the reports that emotions can influence the problem-solving abilities of the students (Connor & Davidson, 2003; Estrada et al., 1994; Isen & Shalker, 1982; Krapp, 2002; Ruzek et al., 2016; Taylor, 2010). Furthermore, the findings of Kizilkaya and Askar (2008), Wang et al. (2008), Van der Meij (2013), Patrut and Spatariu (2015), Malekzadeh et al. (2015), and Hernández et al. (2016) those students who utilized animated PAs achieved better learning outcomes than those students who utilized nonanimated PAs were consistent.

Methodology

Research Design, the Subjects, Determination of Sample Size, Sampling Design, and Sampling Technique

Grade 7 students of one public high school in Manila participated in the study. Using statistics calculator, a minimum sample size of 26 per group was computed (Soper, 2016). The sample size was computed based on the following parameters: anticipated effect size = 0.80, desired statistical power level = 0.80, and probability level = .05. Two groups with 31 students each participated in the study. However, two participants did not complete the experiment; thus, only 29 students were involved in one group. The sections were preselected by the school to ensure that the participants had the same levels of mathematics skills.

This experimental study utilized the randomized pretest–posttest control group design (Figure 2). Students were categorized based on their previous mathematics performance to ensure that the mathematics skills of students in each group were balanced. Their teacher categorized the mathematics performance of the students into three classifications: low, average, and high. These classifications are reliable because teachers know their students’ capabilities (Cheong, Parajes, & Oberman, 2004; Lambert, 2002; Reeve, 2006). Students were transferred either to the facial (denoted by letter X in Figure 2) or nonfacial group (denoted by letter C in Figure 2) to balance the number of low, average, and high performing students until the two groups had the same number of students and same levels of classification. Afterwards, one group was assigned to a personal instructing agent (PIA) that exhibits facial expressions (called the facial group), and the other one was assigned to a PIA that does not exhibit facial expression (except neutral; called the nonfacial group). The two groups were randomly assigned to a version they would utilize through lottery. The facial group had 17 female and 12 male participants while the nonfacial group had 20 female and 11 male participants. (This whole process of random assignment of participants to each group is denoted by R in Figure 2).

Figure 2.

Randomized pretest-posttest control group design.

A pretest was given to determine if both sets of participants had the same levels of prior knowledge in mathematics. It was administered a day after the topic was introduced to the class. As shown in Table 1, the mean difference, M(facial group) = 36.38; M(nonfacial group) = 34.29; d = 2.09, between the two groups is not significant, t(58) = 1.09, p > .05. Thus, participants had the same levels of prior knowledge in mathematics. It supported the classifications of the teacher.

Table 1.

Prior Knowledge of Students in Mathematics.

Test of difference	Mean (M)	Mean difference (d)	t	df	Significance
Pretest—Facial group	36.4	2.1	1.09	58	.28
Pretest—Nonfacial group	34.3	2.1	1.09	58	.28

Software Utilized

The study utilized the PIA. It is a web-based application ITS that can assist users in solving single variable linear algebraic equations. It runs on any Internet-connected computer operating system platform installed with the latest Internet browser version and Adobe Flash Player. Users can enter the equation they want to solve or they may let PIA generate a question for them. Afterwards, users can enter their solutions on a step-by-step basis. Each time the solution is entered, the PIA detects whether the move is correct or not. If the move is incorrect, the PIA gives hints to correct the solution (Figure 4). Users are allowed to abandon the problem and post a new one. This is the tutoring process. All solved and unsolved equations, hints expressed by PIA, and time spent tutoring are logged in a database of PIA.

There are two versions of PIA. The first version can exhibit facial expressions such as happy (Figures 3 and 6), surprise (Figure 4), sad (Figure 5), and neutral (Figure 6). PIA is depicted as a young female teacher who knows her student’s name (Figure 3.). She shows happy facial expressions when users entered a correct step. She also shows a sad facial expression and provides hints to solve a problem when users entered an incorrect step. She exhibits a surprise facial expression when users committed a mistake after three consecutive correct steps. She also gives a hint at a wrong step of the users. A neutral facial expression is displayed when none of the first three facial expressions is exhibited. Neutral facial expression is the default emotion of PIA. It does not get angry even though the students repeatedly commit mistakes. Instead, it shows a sad facial expression and encourages the student to continue solving the problem (Figure 7).

Figure 3.

One of the happy facial expressions exhibited by PIA if the student submitted a correct step in solving the equation.

Figure 4.

PIA showing a surprise facial expression which at the same time is giving a hint on the solution of her student named Jan.

Figure 5.

A sad facial expression exhibited by PIA if the student submitted a series of incorrect steps.

Figure 6.

Neutral facial expression of PIA. This is also the default facial expression of PIA.

Figure 7.

Facial expressions exhibited by PIA. (a) Neutral, (b) Happy, (c) Surprise and (d) Sad.

Data Gathering Procedure and the Research Instrument

Students actually have a slight idea on linear equation. The experiment was conducted a day after the topic on linear equation was introduced to the students. The experiment lasted for five consecutive days. During the first day, students took a pretest to determine their prior knowledge in mathematics. Afterwards, they utilized a PIA for three consecutive days. This was the intervention period. Each session of the intervention period lasted for 45 minutes. While the experiment lasted for 45 minutes, the average time spent tutoring each group was about 30 minutes. The remaining time was dedicated to preparation, setup, and logging to computers. A posttest was administered on the fifth day. The letter O shown in Figure 2 denotes the pretest and posttest administrations in data gathering.

The participants of the study were observed during the duration of the experiment to clarify the results of the study. There was no checklist utilized in the observation method since the behavior of the students would only be visible during the experimentation. Three members of the research team observed the behaviors of the students during the experiment. The observed behaviors were discussed immediately after the experiment with the fourth member of the team. The fourth member of the research team facilitated the discussion in order to validate the observations of the three observers. Follow-up informal interviews were also conducted to further clarify the results of the study.

The tests were adapted from the study of Bringula et al. (2015) and Matsuda et al. (2012). There were two versions of the tests, α(Test A) = 0.92; α(Test B) = 0.91. The contents were validated by the mathematics high school teacher. The teacher agreed that the contents were drawn from the curriculum. The two versions had different items but their level of difficulties were the same. Students did not take the same version of the test twice. For example, if a student took Test A during the pretest, then that student was obliged to take Test B during the posttest. The students were instructed that the tests were a right-minus wrong type of test. This is a type of test where all items with incorrect answers are deducted from the total of all items with correct answers. Nonetheless, if the students opt not to answer the question by leaving the item blank, the item is considered wrong but no score is deducted from the total correct scores. It aims to deter the students from guessing answers. They were also informed that the test results would be forwarded to their teacher.

The tests contained topics on equation solving (10 items), identifying terms (21 items), identifying equivalent terms (10 items), demonstrating the next step in a solution (5 items), identifying equivalent expressions (10 items), and identifying errors in a solution (5 items). Overall, the tests contained a total of 61 items. Equation solving involved finding the value of a variable. Term identification involved classifying a term as a constant, variable, and like terms in a given expression (e.g., 2 is a coefficient). This part can be answered by either true or false.

In identifying equivalent terms, students responded with either true or false to determine whether two terms were alike (e.g., 7X and 9 are like terms). Next step was a mathematics skill test that involved identification of a subsequent correct step in solving equations. It was a multiple-choice item test. A sample equation was given (e.g., a−6 = 2), and participants were given a chance to choose from four choices. Example, the choices were as follows: (a) subtract six from both sides, (b) subtract two from both sides, (c) add six to both sides, and (d) add a to both sides. Finally, in error identification, students were made to identify and justify the flaw in a solution.

Statistical Treatment of Data

The study utilized descriptive statistics such as sum, means, and standard deviations. Tests of difference between means were employed to determine significant differences in mathematics performance and software usage of the participants in the facial and nonfacial group. A .05 level of significance was adopted to determine the reliability of the findings.

Findings

RQ1: Software Usage of the Participants

Table 2 shows the learners interaction with the two versions of the software. The learners in the facial group solved 995 problems (M = 34.3, SD = 12.6) and posted 1,004 problems that were unsolved (M = 34.6, SD = 23.2). PIA generated 893 hints (M = 30.8, SD = 30.8) in assisting the students. Students in the facial group spent 29.4 minutes in using the software on average. Meanwhile, there was an equal number of solved and unsolved problems (Total = 1,101, M = 35.5) in the nonfacial group. However, the unsolved problems were more dispersed (SD = 28.5) than the solved problems (SD = 11.3) in the nonfacial group. The participants in the nonfacial group utilized more hints (Total = 902, M = 29.1, SD = 20) and utilized the software longer (Total = 1,003, M = 32.4, SD = 21.4) than did the facial group. However, none of the indicators of learners’ interaction with the software was found significant at .05 level of confidence. Hence, the first hypothesis stating that the students in the facial group tend to be more engaged in the experiment than the nonfacial group in terms of number of problems solved (H1a), number of problems not solved (H1b), time spent using the software (H1c), and number of hints used in solving an equation (H1d) is not supported.

Table 2.

Test of Difference Between Means on Learner Interface Usage of Facial (n = 29) and Nonfacial (n = 31) Groups.

Learners’ interaction	Group	Total	Mean (M)	Standard deviation (SD)	Mean difference (d)	t (df = 58)	Significance
Solved problems	Facial	995	34.3	12.6	1.21	0.391	.697
Solved problems	Nonfacial	1,101	35.5	11.3	1.21	0.391	.697
Unsolved problems	Facial	1,004	34.6	23.2	0.90	0.133	.895
Unsolved problems	Nonfacial	1,101	35.5	28.5	0.90	0.133	.895
Hints given by PIA	Facial	893	30.8	24.3	1.70	0.297	.768
Hints given by PIA	Nonfacial	902	29.1	20.0	1.70	0.297	.768
Time spent tutoring (in minutes)	Facial	852	29.4	16.0	2.98	0.606	.547
Time spent tutoring (in minutes)	Nonfacial	1,003	32.4	21.4	2.98	0.606	.547

PIA = personal instructing agent.

RQ2: Influence of Facial and Nonfacial PA on Students’ Mathematics Performance

Table 3 shows the pretest and posttest scores of both sets of respondents. The average pretest scores of the facial group are 36.4 while the nonfacial group is 34.3. The dispersion of the pretest scores is almost the same, SD(facial) = 8.2; SD(nonfacial) = 8.4. Posttest scores of the facial group are higher (M = 38.7) than the scores of the nonfacial group (M = 34.2). However, the posttest scores of the facial group (SD = 8.9) are more dispersed than those of the nonfacial group (SD = 6.5).

Table 3.

Test of Difference Between Means on Mathematics Performance of Facial and Nonfacial Groups.

Test of difference	Mean	SD	Mean difference (d)	t	df	Significance
Pretest—Facial group	36.4	8.2	2.3	2.38	28	.02*
Posttest—Facial group	38.7	8.9	2.3	2.38	28	.02*
Pretest—Nonfacial group	34.3	8.4	0.1	0.06	30	.953
Posttest—Nonfacial group	34.2	6.5	0.1	0.06	30	.953
Posttest—Facial group	38.7	–	4.5	2.02	58	.048*
Posttest—Nonfacial group	34.2	–	4.5	2.02	58	.048*

*p < .05.

Table 3 also shows that the mean difference (d = 2.3) between the pretest and posttest scores of the emotion group is unlikely to have arisen from sampling error, t(28) = 2.38, p < .05. On the other hand, the difference (d = .01) between the pretest and posttest scores of the nonfacial group is insignificant, t(30) = 0.06, p > .05. When the posttest scores of both groups are compared, it reveals that the facial group has a higher mean posttest scores than that of the nonfacial group (d = 4.50, t(58) = 2.02, p < .05). Therefore, the hypothesis stating that the use of facial version of PA would lead to better mathematics performance of the students compared with the use of the nonfacial version of PA is supported by the study.

Observations and informal interviews were conducted to further clarify the results in Table 3. Different behaviors were observed during the experiment (e.g., looking from terminals to terminals of their classmates, visiting classmates to observe how they were doing, and seeking help from classmates). However, it was decided that behaviors relating to emotions or facial expressions were reported. It was observed that there were students who were inclined to be happy when PIA was providing happy facial expressions. Meanwhile, students in the nonfacial group did not show any reaction when they got correct answers in the experiment.

Discussion

It is clearly shown that the results shown in Table 2 indicate that both sets of participants were engaged in the experiment and that the two versions of the software were fully utilized. Statistical tests show that there are no significant differences on the software utilization of the participants. It is therefore concluded that students have the same levels of utilization of the facial and nonfacial versions of the software. It is worth noting that the result of this study is different from the findings of Chen et al. (2012). The engagement of the participants of both groups and the discrepancy in the study of Chen et al. (2012) can be explained by two external factors. First, the experiment was conducted during their mathematics classes. As such, the chances of being involved in the experiment were high. Second, students were fully informed about the experiment. They were informed that the test results after the intervention period would be given to their teacher.

This finding is important since it shows that the design of a PA may not necessarily be the reason for students to engage in learning mathematics. Instead, external factors (such as the two mentioned in this study) could explain the utilization (or non-utilization) of a PA by the participants. Hence, this study informs educational technology researchers that they have to be cautious when concluding that the design of animated PAs could engage the students in the experiment. It can also be noted that this study did not support the studies of White et al. (2009), Murray (2011), Ruzek et al. (2016), and Raufelder et al. (2016). While said studies compared the levels of students’ class participation where their teachers exhibited positive and negative emotions, this study determined the effect of the PA’s exhibited positive and neutral facial expressions.

The result of the study suggests that the students who used the facial version of PIA are more likely to solve problems correctly than their counterparts in terms of pretest and posttest scores. Estrada et al. (1994), Isen and Shalker (1982), Krapp (2002), Connor and Davidson (2003), Ruzek et al. (2016), and Taylor (2010) revealed that teachers who show positive emotions can influence the learning process of their students in an actual classroom environment. The current study extended the findings of these studies by showing that virtual learning environments are also helpful to students in solving mathematics problems provided that synthetic facial expressions and textual feedback are incorporated in these environments. Finally, the results also suggest that facial expressions and textual feedback are necessary components of a PA. This is consistent with the studies of Kizilkaya and Askar (2008), Wang et al. (2008), Van der Meij (2013), Patrut and Spatariu (2015), Malekzadeh et al. (2015), and Hernández et al. (2016).

In the study, it was also observed that students imitated happy facial expressions of PIA every time they got a correct answer. When PIA showed happy facial expressions, students uttered words of happiness (e.g., Yehey!) and showed gestures of emotion (e.g., clapping hands). These are the same body gestures generated by the agent. Follow-up interviews with two students revealed that they felt pleasure when they got a correct solution. This pleasure is reinforced by the expression of the PA. As a result, they were stimulated to express happy emotion with the avatar. In turn, they felt that they were on the right track in learning the topic.

However, students did not exhibit any reaction when PIA was either sad or surprised. It was observed that some students said out loud “Why is my answer wrong?” or “What is wrong with my solution?” Afterwards, students attempted to fix the problem using the hint given by PIA. This is an indication that even though students were confused with their answers, the textual feedback of PIA could assist them. PIA, saying a polite hint (e.g., “The answer is wrong. Have you tried simplifying the right side?”), can push the students to finish the problem. This could explain the findings in Table 3 where there is a significant difference between the pretest and posttest scores of the facial group but not in the nonfacial group.

Students in the nonfacial group did not exhibit happy emotion even though they got correct answers. The nonfacial version of PIA shares the same capabilities of the facial version of PIA except that the former only displays neutral facial expression. It is interesting to note that, although the nonfacial version PIA is able to give the same hints, students are not able to learn from it. The result suggests that textual feedback alone may not be an effective mechanism in learning linear equation. For a PA to be more effective, hints could be combined with facial expressions as feedback.

It should be noted that the learning process in this study happens in a computerized learning environment and that the facial expressions exhibited are only artificially generated. Therefore, the study offers a vivid contribution to the literature in three ways. First, hints together with facial expression feedback serve as a strategy for learning mathematics. While textual feedback confirms correctness of a solution, facial expressions of the PA strengthen and reassure the emotions of the students. Furthermore, facial expression of PA shows that it is delighted in the progress of the students and confirms that the learners are on the right track in learning the course. The facial expressions of the avatar serve as signals to the progress of their solving problem capabilities. Second, previous studies linked emotions and learning in the context of actual classroom environment. This study shows that learning can still be expected with the utilization of a PA that exhibits synthetic facial expressions. Lastly, learners tend to have positive emotional reactions even if the PIA generated synthetic facial expressions.

The results of this study inform instructional designers and educational technology developers that it is necessary to develop PAs that induce facial expressions as forms of learning feedback since textual feedback alone may not be sufficient to teach students effectively. In addition, the study calls for a multidisciplinary approach where educational technology developers may collaborate with teachers or child psychologists to ensure that the PA is giving correct textual and facial expression feedback.

Conclusions, Recommendations, and Limitations

This study attempted to determine the effects of synthetic facial expressions on the mathematics performance of the students. Toward this goal, two versions of software were utilized by two groups of students. There were two hypotheses tested in this study. On the basis of the findings presented, the first hypothesis stating that students in the facial group tend to be more engaged in the experiment than the nonfacial group in terms of number of problems solved, number of problems not solved, time spent using the software, and number of hints used in solving an equation is not supported. Hence, the interactions of the participants with the PA were the same regardless of the version they utilized.

Meanwhile, the second hypothesis stating that the use of facial version of PA would lead to better mathematics performance of the students compared with the use of the nonfacial version of PA is supported. Thus, a PA that provides textual feedback and facial expressions is able to teach the students mathematics solving. This study provided evidence that students can learn from a PA even though it only exhibits synthetic facial expressions. It disclosed that students tend to be happy when the PA expressed happy facial expressions at the time the student got correct solutions in a mathematics problem. The happy emotion of the students for getting a correct solution to a mathematics problem is reinforced by the happy facial expressions and the textual feedback of the agent. Students interpret these gestures of the agent as an indication that they are on the right track of learning mathematics.

This study revealed that happy facial expressions of the PA can influence the emotion of the students. However, the study did not investigate the level at which each facial expression could contribute to the increase (or decrease) in the mathematics performance of participants. In addition, there are certain factors that are not included in the current study that might influence the findings of the study. For example, it is unclear if gender or other demographics influence the findings of the study. The second factor that might influence the results of the study is the size of the samples. The study was limited to two group settings as indicated by the sample size parameters. Hence, it is suggested that the sample size parameters be increased to address the statistical limitations of the study. Lastly, the intervention only lasted for 90 minutes. The results of the study could be different if the intervention period is extended.

Moreover, it is unclear how students may react if they may have already acquired the necessary mathematics skills. Will they consider PIA over reacting when it exhibits positive facial reactions? Future investigation can answer this RQ. Furthermore, the current study did not investigate how facial expressions can influence the retention and how these facial reactions are used with textual feedback. A follow-up study is needed to explain the relationship between emotional and textual feedback. Lastly, attitudes toward the software and attitudes toward mathematics are not determined in the current study. Further studies can shed light on these research gaps.

Footnotes

Acknowledgment

The authors are greatly indebted to the administration, faculty, and students of San Francisco High School.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is partially supported by the University of the East.

Author Biographies

Rex P. Bringula is a faculty member of the College of Computer Studies and Systems, University of the East (UE-CCSS).

Ian Clement O. Fosgate, Jr received his degree in computer science, magna cum laude, from UE-CCSS. He is currently working as a freelance web designer.

Neil Peter R. Garcia received his degree in computer science, cum laude, from UE-CCSS. He is a former research and development unit team leader of the UE-CCSS. He is a Ruby on rails developer.

Josf Luinico M. Yorobe received his degree in computer science from UE-CCSS. He is currently working as a programmer.

References

Arroyo

Woolf

B. P.

Burelson

Muldner

Rai

Tai

(2014) A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. International Journal of Artificial Intelligence in Education 24: 387–426. doi:10.1007/s40593-014-0023-y.

Bringula

R. P.

Basa

R. S.

Dela Cruz

Rodrigo

M. M. T.

(2015) Effects of prior knowledge in mathematics on learner-interface interactions in a learning-by-teaching intelligent tutoring system. Journal of Educational Computing Research 54(4): 462–482. doi:10.1177/0735633115622213.

Brown

Bibby

(2008) “I would rather die”: Reasons given by 16-year-olds for not continuing their study of mathematics. Research in Mathematics Education 10(1): 3–18. doi:10.1080/14794800801915814.

Chen

G.-D.

Lee

J.-H.

Wang

C.-Y.

Chao

P.-Y.

L.-Y.

Lee

T.-Y.

(2012) An empathic avatar in a computer-aided learning program to encourage and persuade learners. Educational Technology & Society 15(2): 62–72.

Cheong

Y. F.

Parajes

Oberman

P. S.

(2004) Motivation and academic help-seeking in high school computer science. Computer Science Education 14(1): 3–19.

Connor

K. M.

Davidson

J. R. T.

(2003) Development of a new resilience scale: The Connor-Davidson Resilience Scale (CD-RISC). Depression and Anxiety 18(2): 76–82. doi:10.1002/da.10113.

Corbalan

Paas

Cuypers

(2010) Computer-based feedback in linear algebra: Effects on transfer performance and motivation. Computers & Education 55: 692–703. doi:10.1016/j.compedu.2010.03.002.

Damasio

(2004) Theory of thinking faster and faster. Discover 25(5): 44–49.

Di Martino

Zan

(2011) Attitude towards mathematics: A bridge between beliefs and emotions. ZDM Mathematics Education 43: 471–482. doi:10.1007/s11858-011-0309-6.

10.

D’Mello

S. K.

Craig

S. D.

Witherspoon

McDaniel

Graesse

(2008) Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction 18(1): 45–80. doi:10.1007/s11257-007-9037-6.

11.

Estrada

Young

Isen

(1994) Positive affect influences creative problem solving and reported source of practice satisfaction in physicians. Motivation and Emotion 18: 285–299.

12.

Graesser

Chipman

Haynes

Olney

(2005) AutoTutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions on Education 48(4): 612–618.

13.

Green, D. T. (2011). Intelligent tutoring systems for skill acquisition (Unpublished manuscript). The University of Arizona, Tucson, Arizona.

14.

Hascher

(2010) Learning and emotion: Perspectives for theory and research. European Educational Research Journal 9(1): 13–28. doi:10.2304/eerj.2010.9.1.13.

15.

Hernández

Pérez-Ramírez

Zatarain-Cabada

Barrón-Estrada

Alor-Hernández

(2016) Designing empathetic animated agents for a B-learning training environment within the electrical domain. Educational Technology & Society 19(2): 116–131.

16.

Isen, A. (2007). Positive affect. In R. F. Baumeister & K. D. Vohs (Eds.), Encyclopedia of social psychology (Vol. 2, pp. 680–682). Thousand Oaks, CA: SAGE Publications.

17.

Isen

Shalker

T. E.

(1982) The effect of feeling state on evaluation of neutral, positive, and negative stimuli: When you accentuate the positive, “do you eliminate the negative? Social Psychology Quarterly 45: 58–63.

18.

Kaur

(2008) Teaching and learning of mathematics: What really matters to teachers and students? ZDM Mathematics Education 40(6): 951–962. doi:10.1007/s11858-008-0128-6.

19.

Kaur

(2009) Characteristics of good mathematics teaching in Singapore grade 8 classrooms: A juxtaposition of teachers’ practice and students’ perception. ZDM Mathematics Education 41(3): 333–347. doi:10.1007/s11858-009-0170-z.

20.

Kim

Baylor

A. L.

Shen

(2007) Pedagogical agents as learning companions: The impact of agent emotion and gender. Journal of Computer Assisted Learning 23(3): 220–234.

21.

Kizilkaya

Askar

(2008) The effect of an embedded pedagogical agent on the students’ science achievement. Interactive Technology and Smart Education 5(4): 208–216. doi:10.1108/17415650810930893.

22.

Krapp

(2002) Structural and dynamic aspects of interest development: Theoretical considerations from an ontogenetic perspective. Learning and Instruction 12(4): 383–409. doi:10.1016/S0959-4752(01)00011-1.

23.

Lambert

(2002) A framework for shared leadership. Educational Leadership 59(8): 37–40.

24.

Larsen

R. J.

(2007) Independence of positive and negative affect. In: Baumeister

R. F.

Vohs

K. D.

(eds) Encyclopedia of social psychology, Thousand Oaks, CA: SAGE Publications, pp. 468–470.

25.

Lin

H.-C. K.

C.-H.

Hsueh

Y.-P.

(2014) The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability. Computers in Human Behavior 41: 514–522. doi:10.1016/j.chb.2014.09.052.

26.

Malekzadeh

Mustafa

M. B.

Lahsasna

(2015) A review of emotion regulation in intelligent tutoring systems. Educational Technology & Society 18(4): 435–445.

27.

Manstead

A. S. R.

(2007) Emotion. In: Baumeister

R. F.

Vohs

K. D.

(eds) Encyclopedia of social psychology, Thousand Oaks, CA: SAGE Publications, pp. 285–291.

28.

Mao

(2010) Agent based affective tutoring systems: A pilot study. Computers & Education 55: 202–208. doi:10.1016/j.compedu.2010.01.005.

29.

Matsuda, N., Cohen, W. W., Koedinger, K. R., Keiser, V., Raizada, R., Yarzebinski, E., … Stylianides, G. (2012). Studying the effect of tutor learning using a teachable agent that asks the student tutor for explanations. In M. Sugimoto, V. Aleven, Y. S. Chee, & B. F. Manjon (Eds.), Proceedings of the international conference on digital game and intelligent toy enhanced learning (DIGITEL 2012) (pp. 25–32). Los Alaminos, CA: IEEE Computer Society Press.

30.

Maxwell

(2014) Emotion. In: Feinstein

(eds) From the brain to the classroom: The encyclopedia of learning, Santa Barbara, CA: Greenwood, pp. 188–192.

31.

Megill

(2014) Emotion, cognition and artificial intelligence. Minds & Machines 24(2): 189–199. doi:10.1007/s11023-013-9320-8.

32.

Mory, E. H. (1996). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology: A project of the association for educational communications and technology (pp. 745–783). New York, NY: Macmillan.

33.

Murray

(2011) Secondary students’ descriptions of “good” mathematics teachers. Australian Mathematics Teacher 67(4): 14–21.

34.

Patrut

Spatariu

R.-P.

(2015) Implementation of artificial emotions and moods in a pedagogical agent. In: Tettegah

S. Y.

Gartmeier

(eds) Emotions, technology, design, and learning, Cambridge, MA: Academic Press, Elsevier Inc, pp. 63–86.

35.

Raufelder

Nitsche

Breitmeyer

Keßler

Herrmann

Regner

(2016) Students’ perception of “good” and “bad” teachers—Results of a qualitative thematic analysis with German adolescents. International Journal of Educational Research 75: 31–44. doi:10.1016/j.ijer.2015.11.004.

36.

Reeve

(2006) Teachers as facilitators: What autonomy-supportive teachers do and why their students benefit. The Elementary Journal 106(3): 225–236.

37.

Rodrigo, M. M. T., & Baker, R. S. J. D. (2009). Coarse-grained detection of student frustration in an introductory programming course. Paper presented at the International Conference on Educational Computing Research, Berkeley, CA, USA.

38.

Rodrigo

M. M. T.

Baker

R. S. J. D.

Agapito

Nabos

Repalam

M. C.

Reyes

S. S.

San Pedro

M. O. C. Z.

(2012) The effects of an interactive software agent on student affective dynamics while using an intelligent tutoring system. IEEE Transactions on Affective Computing 3(2): 224–236. doi:10.1109/T-AFFC.2011.41.

39.

Ruzek

E. A.

Hafen

C. A.

Allen

J. P.

Gregory

Mikami

A. Y.

Pianta

R. C.

(2016) How teacher emotional support motivates students: The mediating roles of perceived peer relatedness, autonomy support, and competence. Learning and Instruction 42: 95–103. doi:10.1016/j.learninstruc.2016.01.004.

40.

Schroeder

N. L.

Adesope

O. O.

Gilbert

R. B.

(2013) How effective are pedagogical agents for learning? A meta-analytic review. Journal of Educational Computing Research 49(1): 1–39. doi:10.2190/EC.49.1.a.

41.

Shouse, E. (2005). Feeling, emotion, affect. M/C Journal, 8(6). Retrieved from http://journal.media-culture.org.au/0512/03-shouse.php.

42.

Soper. (2016). A-priori sample size calculator for student t-test. Retrieved from http://www.danielsoper.com/statcalc/calculator.aspx?id=47.

43.

Strain

A. C.

D’Mello

S. K.

(2011) Emotion regulation during learning. In: Biswas

Bull

Kay

Mitrovic

(eds) Artificial intelligence in education: Lecture notes in computer science Vol. 6738, Berlin, Germany: Springer Berlin Heidelberg, pp. 566–568.

44.

Taylor

J. S.

(2010) Learning with emotion: A powerful and effective pedagogical technique. Academic Medicine 85(7): 1110.

45.

Tone

(2007) Cognition in emotion, Amsterdam, The Netherlands: Rodopi.

46.

Tsahuridu

E. E.

(2009) Emotion. In: Wankel

(ed.) Encyclopedia of business in today’s world, Thousand Oaks, CA: SAGE Publications, pp. 587–589.

47.

Van der Meij

(2013) Motivating agents in software tutorials. Computers in Human Behavior 29(3): 845–857. doi:10.1016/j.chb.2012.10.018.

48.

Wang

Johnson

W. L.

Mayer

R. E.

Rizzo

Shaw

Collins

(2008) The politeness effect: Pedagogical agents and learning outcomes. International Journal of Human-Computer Studies 66: 98–112. doi:10.1016/j.ijhcs.2007.09.00.3.

49.

White, B., Barnes, A., Lawson, M., & Johnson, W. (2009). Student perceptions of what makes good teaching. Paper presented at Annual Conference of the Australian Teacher Education Association, Albury, Australia.

50.

Woolf

Burleson

Arroyo

Dragon

Cooper

Picard

(2009) Affect-aware tutors: Recognising and responding to student affect. International Journal of Learning Technology 4(3/4): 129. doi:10.1504/IJLT.2009.028804.