Abstract
Prediction allows learners to adjust behavior toward the future by exploiting information pertaining to the present and the past. Through a field study, we examined whether poor performing students are truly unaware of not knowing their deficiencies as the illusion of knowing (IoK) phenomenon implies. College students’ ability to predict their final test performance was surveyed as a function of experience (before and after the test), performance level, and self-efficacy. In this study, high performers’ prospective and retrospective predictions were more accurate and confident than those of poor performers. Although poor performers overestimated their grades (as predicted by IoK), they were less confident in their predictions. Furthermore, both their prediction accuracy and confidence benefited from the experience of taking the test. These findings, coupled with the lower self-efficacy of poor performers, suggest that prediction errors involving inflated estimations reflect the wishful thinking exhibited by students who are aware of their lack of competence but may have little confidence in their abilities.
Successful performance on an upcoming task (e.g., a test or assignment) relies on the extent to which the knowledge and skills that a person possesses overlap with the particular demands of the task. Yet, successful performance begins well before the actual task is executed. It stems from the person’s ability to accurately predict performance on the upcoming task (metacognitive awareness) as well as commitment to close the gap (metacognitive control) if the degree of overlap between these two domains is small (Avhustiuk et al., 2018). Estimation of future performance is all but simple, especially in settings, such as college, where a great deal of information is to be processed and retained. In such settings, accuracy can easily become the casualty of unrealistic expectations, misperceptions, and misinterpretations not only of the knowledge and skills one possesses but also of the processing and knowledge demands of the task to be carried out (Dunning et al., 2004). Yet, one’s predictions can affect both current and future behavior, shaping learning and ultimately test performance. Consider, for instance, students who are preparing for an upcoming test. Estimation of their likely performance will affect decisions regarding the amount of time and type of activities they devote to learning and effort expended at test. It follows that prediction accuracy is a key aspect of test performance.
In a very broad sense, “predictive processing refers to any type of processing which incorporates or generates not just information about the past or the present, but also future states of the body or the environment” (Bubic et al., 2010, p. 2). As stored information about the self and current environmental information are used to imagine and predict future events, metacognition can be said to be the essential ingredient of successful learning (Avhustiuk et al., 2018). It involves two interrelated processes: cognitive monitoring, which refers to learners’ awareness of their knowledge and competence in relation to specific task demands, and cognitive control, which represents their willingness and ability to use strategically the information gathered to adjust cognition and action. These processes roughly correspond to the distinction between prediction, which refers to the representation of an expected event, and anticipation, which describes the impact of the prediction on current behavior (Butz et al., 2003). Within this framework, the illusion of knowing (IoK) phenomenon is the belief that knowledge and/or skills have been attained when, in fact, the opposite is true (Dunning et al., 2004). When cognitive monitoring is disrupted by IoK, cognitive control and ultimately performance are impacted.
Poor performers are said to be particularly affected by IoK, which leads them to not only overestimate their knowledge and capabilities (Hacker et al., 2000; Kruger & Dunning, 1999) but also be unaware of their lack of knowledge and skills (i.e., deficient subjective confidence). Recently, a pattern that contradicts the very essence of IoK in the classroom has been reported by Miller and Geraci (2011) and Al-Kuhayli et al. (2019). Poor performers indeed overestimate their test performance, but they are not overconfident in their predictions. On the contrary, they appear to treat their overestimation as a form of wishful thinking that can temporarily protect their self-concept from the imminent adversity brought about by bad grades. As poor performers do not trust much their inflated estimates, they cannot be defined as “unskilled” and “unaware” (Ehrlinger et al., 2008) or “blissfully incompetent” (Williams, 2004) as IoK implies.
Wishful thinking can be deemed the reflection of a pervasive phenomenon known as the optimism bias, the unrealistic confidence that underlies one’s tendency to overestimate the likelihood of positive events and underestimate the likelihood of negative events (Sharot, 2011). Whereas awareness of one’s weaknesses may be expected to lead poor performers to consider corrective actions, thereby spurring a call to action, wishful thinking can considerably reduce the chances that such actions be exercised, thereby promoting inertia. Inflated estimations ignited by wishful thinking are a metacognitive paradox since feedback from past test experiences, instead of being corrective, has left such thinking largely unscathed. As such, inflated estimations may reflect poor performers’ tendency to discount or rationalize the predictive validity of past signals of unsuccessful learning (i.e., mismatches between expected and realized outcomes; O’Brien, 2013). If so, poor performers select not to rely on the informational value of such signals for corrective actions, such as updating self-knowledge and pursuing behavioral adaptations, to be exercised in the present as well as in the future. However, if wishful thinking is at work, acts of discounting or rationalizations are likely to be motivated by the lack of confidence in one’s abilities to address deficiencies, rather than lack of awareness of said deficiencies. As discounting strategies and rationalizations rest on the tenuous compromise between the wish to protect one’s self-concept from aversive information and the plausibility and validity of the information produced by the signal (Ditto & Lopez, 1992; Pyszczynski & Greenberg, 1987), learners may be open to rectifying optimistic predictions at least slightly if relevant information of sufficient clarity becomes available (e.g., after having completed a test).
In recognition of the role that predictive processing plays in self-regulatory activities during learning, the following predictions can be made if wishful thinking rather than illusory thinking is at work in the classroom: First, poor performers will be sensitive to information that may cast doubt on their rosy predictions. Of course, after having taken a test, more information about its content becomes available. If poor performers are aware of their shortcomings concerning particular test demands, they will reluctantly integrate this information, and rectify their inflated estimates but only slightly so that they can enjoy a fleeting reprieve from the harsh reality of a bad grade and its implied message regarding low competence. Conversely, because high performers have a reasonably good idea of their knowledge and competence, their predictions and related confidence will be less likely to benefit from test experience. Thus, if students are asked to predict their performance before and after a test, and indicate their confidence in such predictions, we hypothesize that poor performers’ retrospective predictions will benefit the most from the experience of having taken the test.
Second, if poor performers are aware of their shortcomings, they may lack the confidence necessary to address deficiencies and hopefully find remedies. A “can-do attitude” is the expression of self-efficacy which refers to one’s underlying confidence to perform well across various tasks and situations (Bandura, 1989). General self-efficacy is a motivational trait that is developed over one’s lifespan from the accumulation of successes and failures (Chen et al., 2000). It is negatively correlated with wishful thinking (Romppel et al., 2013). Thus, it is reasonable to expect that if a “can-do attitude” is possessed by high performers, and a “can-do-little attitude” is possessed by poor performers, reliance on wishful thinking will also distinguish the two groups.
Method
Participants
Participants were female undergraduate students (n = 1,260) of a university in Saudi Arabia (SA). They were enrolled in a general education course covering Arabic and Islamic culture which is an adjunct to the Core Curriculum of the university developed by the Texas International Education Consortium (TIEC). Arabic and Islamic culture is taught as a sequence of four required units that are distributed across consecutive semesters to cover different topics. The selected unit was the second and third in the sequence to ensure a sample of students who were already acclimated to the requirements of college life and who would be in another course of the sequence in the following semester for debriefing purposes. Participants were Arabic-English bilingual speakers whose English competency had been verified through standardized tests prior to admission. Their age ranged from 18 to 25. Due to gender-segregation rules, a corresponding male sample was unattainable. Treatment of participants complied with the guidelines of the Office for Human Research Protections of the U.S. Department of Health and Human Services as well as with the code of ethics of the American Psychological Association. The selected sample included students for whom the midterm, final test, and confidence ratings were available (93.84% of the registered students).
Materials and Procedure
Table 1 illustrates the sequence of assessment activities implemented. At the beginning of the semester, the New General Self-Efficacy (NGSE) inventory (Chen et al., 2001) was administered. The inventory required students to report the extent to which they agreed with each of eight statements of general confidence in one’s competence to deal effectively with life challenges. Agreement was reported on a five-point Likert-type scale from “strongly disagree” (1) to “strongly agree” (5). The NGSE’s reliability, as measured by Cronbach’s alpha, was .80.
Sequence of Assessment Activities.
Note. NGSE = New General Self-Efficacy.
The midterm and the final tests involved articles or chapters of socio-cultural issues pertaining to the Middle East. Each test asked students to summarize key ideas, principles, and events from selected passages, as well as to critically analyze their content for the purpose of application, comparison, contrast, and evaluation (Bloom, 1976). Students were told in advance that the final test would be preceded and followed by a brief self-assessment activity. Specifically, the first and last pages of the final test required students to predict their grades on a scale from 0 to 100. They were also asked to report their confidence in the predictions made on a Likert-type scale from 0 (not at all confident) to 4 (extremely confident). Students were instructed to report what they reasonably believed to be their grade at the time of the inquiry. They were explicitly discouraged from reporting what they wished their grade to be. In addition, no questions regarding wishful thinking propensities or other personality measures were asked to ensure a “business-as-usual” atmosphere in the classroom.
Debriefing sessions were incorporated into the lectures of the course during the following semester. In addition, informal focus groups (as part of debriefing sessions) were conducted with students and instructors regarding their views of self-assessment. To ensure the collection of frank comments and respect anonymity, individual comments were recorded without links to the self-assessment data collected in class during the previous semester.
Results
Prediction accuracy was measured as the difference between estimated and actual final test grades. A perfectly accurate self-assessment was represented by a value of 0. Overestimation and conservative estimation were represented by values above and below 0, respectively (see Table 2).
Mean and Standard Error of the Mean of Key Variables by Final Test Performance Level: High (A: 90–100; n = 333), Mid-High (B: 80–89.99; n = 302), Mid-Low (C: 70–79.99; n = 289) and Poor (D and F: 0–69.99; n = 336).
Significant difference between prospective and retrospective estimates (i.e., effect of test experience).
A 2 (test experience: prospective and retrospective estimates) × 4 (performance level: A B, C, and poor) mixed factorial analysis of variance (ANOVA) was conducted on accuracy. There was a main effect of performance level, F(3, 1,256) = 332.44, MSE = 398.42, p <.001, ηp2= .443, illustrating a shift from overestimation to underestimation as performance improved. There was also a main effect of test experience, F(1, 1,256) = 67.32, MSE = 45.97, p <.001, ηp2= .051, indicating that accuracy was greater for retrospective than prospective estimates, as well as a significant interaction, F(3, 1,256) = 14.12, MSE = 45.97, p <.001, ηp2= .033. Tests of simple effects illustrated a pattern for poor performers contrary to IoK. Namely, poor performers, as well as mid-low and mid-high performers, slightly corrected their overestimations after having taken the test, indicating that they were able to benefit from the experience, ts ≥ 2.70 (p < .013 as per Bonferroni correction). High performers made conservative estimates before the test which were not significantly different from those made afterward.
The same ANOVA performed on subjective confidence displayed a main effect of performance level, F(3, 1,256) = 11.29, MSE = 2.01, p <.001, ηp2= .026, which underscored poor performers’ little confidence in their overestimations compared with that of other students. There was also a main effect of test experience F(1, 1,256) = 4.64, MSE = .370, p <.001, ηp2= .004, but not an interaction (F = 1.30, ns). To ensure comparison with accuracy data, tests of simple effects were conducted in each group. Only poor performers exhibited a decline in confidence as a function of test experience, t = 2.55 (p < .013 as per Bonferroni correction).
In sum, poor performers overestimated their test performance but were less confident in their overestimations. Both their accuracy and confidence benefited from the experience of taking the final test. Did these students lack confidence in their abilities? A one-way ANOVA, which was conducted on self-efficacy scores with performance level as the factor, was significant, F(3, 1,256) = 4.66, MSE = .213, p =.003, ηp2= .011. Tests of simple effects indicated that the general self-efficacy of poor performers was significantly below that of mid-high and high performers, ts ≥ 2.71 (p < .008 as per Bonferroni correction). The other groups’ self-efficacy did not differ.
Focus groups, created through accidental sampling, and debriefing sessions incorporated into the lectures of the course provided qualitative information regarding students’ views of their self-assessment and performance. Inquiries yielded qualitative findings that informed the results of the quantitative tests discussed above. For instance, we found that prospective and retrospective estimates pushed to the forefront of students’ mind the difference between desired outcomes and those that could be anticipated following an objective assessment of their knowledge and skills in response to test demands. This difference was often cited by students who reported being displeased with test outcomes but rarely by those who reported being pleased. Displeased students frequently noted that they had not been surprised by the challenging content of the final test. Their explanations for less than desirable performance ranged from insufficient time for reviewing to their failed understanding of the test materials during the semester. Yet, most students who were unhappy about test outcomes spontaneously admitted that they were more hopeful about their ability to answer test questions before opening the test than after the test was completed (i.e., when retrospective estimates were made). Not surprisingly, displeased students often mentioned that they felt compelled to correct their initial estimates downward after completing the test. When probed further about the meaning of “hope” in their initial predictions, their comments exhibited fatalism, generalized optimism, or both. Namely, they remarked that future events and outcomes are the domain of a higher authority to whom one has to acquiesce and/or are to be viewed optimistically because encouragement, even if self-produced, is key to optimal performance.
In sum, poor performers’ attempts to compromise were corroborated by post-facto focus-group interviews. These opinions are informative as they are consistent with the quantitative evidence collected in class during the previous semester. Of course, our reliance on convenience sampling, the absence of a link between individual comments and the data collected in class, and the post-facto timing of the debriefing limit the value of this qualitative evidence if considered in isolation.
Discussion
The results of this study offer a window into the human mind as a predictive rather than merely reactive device (Maldonato & Dell’Orco, 2012). Accordingly, the human mind is viewed as a powerful hypothesis-testing device that relies on mental models (i.e., understandings of the world) derived from experience to estimate current and future states under conditions of uncertainty. When students are asked to predict their upcoming test performance, uncertainty exists since precise and practical knowledge of the extent to which test demands match their current expertise is often incomplete and sometimes inaccurate. If the human mind is a hypothesis-testing device that relies on experience, shouldn’t it be self-correcting and thus reduce the likelihood of inflated, self-serving predictions based on faulty and/or incomplete information as test experience accumulates? The costs of being wrong are reasonably high. Our results suggest that reliance on faulty and/or incomplete information may not adequately explain poor performers’ overestimations. Poor performers face two hurdles. They are aware of their shortcomings but lack the necessary confidence to tackle deficiencies and find effective remedies. Test experience can help poor performers to curb wishful thinking, but only slightly so that they can still find comfort in an unlikely positive outcome. Their estimates can be characterized as “a compromise between the wish to reach a particular conclusion and the plausibility of that conclusion given the available data” (Ditto & Lopez, 1992, p. 569).
The key difference between high and poor performers reflects the distinction between predictions that are confirmed by reality and errors of prediction. Whereas matches carry little informational value to the learner as they signal adequate learning, mismatches have noteworthy informational value as they signal unsuccessful learning. Objectively registering outcomes that deviate from expectations can be costly, as additional cognitive resources need to be devoted to understanding the unpleasant outcome of a bad grade in a manner that does not devalue one’s self-concept markedly. Yet, it can also be beneficial, as it can lead to adaptive reactions which may include updating one’s knowledge and strategically changing study activities. Not surprisingly, discrepancies between expected and realized outcomes have been postulated to promote learning (see Schultz et al., 1997). The conditions under which either self-correcting action or inaction (bury one’s head in the sand) is chosen demand further study. The behavioral data collected in our investigation merely suggest that discrepancies between realized and expected events may require a compromise, the precise nature of which is the realm of future research.
Our findings question the pervasiveness of IoK in the classroom. They are consistent with evidence from the Western world (Miller & Geraci, 2011) that poor performers lack confidence in their inflated predictions of performance, and with evidence illustrating that experience can rectify inflation (Al-Kuhayli et al., 2019) but not entirely solve the general confidence problem that poor performers exhibit. Because our study involves Middle Eastern students, a population often neglected in research on metacognition, these findings highlight a common pattern across the globe for predictions of test performance. Yet, limitations exist, including the issue of the extent to which students’ grade predictions rely on past test performance and whether gender differences exist. Although our sample was limited to female students, it is important to note that, contrary to stereotypes, gender differences in self-confidence are less than ubiquitous (Lundeberg et al., 2000).
Footnotes
Authors’ Note
Authors contributed to the present study equally.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
