Enhancing Students Conceptual Understanding: An Interventional Study on the Knowledge Revision Components Framework Using Online Materials

Theory of Conceptual Change and Knowledge Revision

Undoubtedly, the process of correcting misconceptions should be grounded in the theory of conceptual change. This theory can be viewed from two perspectives: knowledge-as-theory and knowledge-as-elements (Özdemir & Clark, 2007). In the knowledge-as-theory view, based on Piaget’s learning theories, students are unlikely to change their understanding unless they encounter problems that cannot be resolved within their current conceptual framework. For a radical conceptual change to occur, the student must be dissatisfied with their initial understanding, and the new scientific knowledge to be assimilated must be understandable, reasonable, and effective (Posner et al., 1982). In addition, the knowledge-as-elements perspective, proposed by diSessa (1993) focuses on the nature of knowledge elements. It suggests that learners’ knowledge is an unstructured collection of simple elements formed through interaction with the physical world offering an organic view of their development and complexity (diSessa, 1993). Neither this nor the knowledge-as-theory perspective offers a singular explanation for the complexity of conceptual change. Instead, both provide different dimensions for understanding the richness and multidimensionality of this process (Özdemir & Clark, 2007).

On the other hand, Kendeou (2024) characterizes conceptual change as a fundamental transformation of a student’s understanding of a concept and specifically states that it is used in fundamental misconceptions. He offered the theory of knowledge revision, which mainly focused on the learner’s simple, factual inaccuracies or misunderstandings instead of false mental models or deep-rooted misconceptions. This theory explains that learners will modify their existing knowledge and schemas if they are dissatisfied with their current understanding when it fails to resolve a problem scientifically and accurately. However, the existing literature provides limited insights into readers’ cognitive processing during knowledge revision (Kendeou, 2024). Kendeou envisaged the need to investigate situations in which knowledge revision can occur when conceptual change does not occur and that this can be considered a paradigm shift in this field. Kendeou makes sense of the philosophy and theoretical structure in this approach by building on refutation texts (Butterfuss & Kendeou, 2021; Kendeou & O’Brien, 2014). The KReC and KReC-MD frameworks are used to form the knowledge revision theory, which replaces students’ background knowledge with accurate information and reorganizes conflicting knowledge, thereby creating conditions that facilitate knowledge revision during refutation text reading and supporting successful learning in the presence of misconceptions (see Kendeou, 2024 for the details of the theory). The theoretical pattern of the present study is based on the knowledge revision approach developed from Kendeou’s philosophical perspective.

Refutation Text and Knowledge Revision

Refutation texts state previously acquired but incorrect knowledge, then directly refute it, and finally provide an alternative and correct knowledge with a causal explanation (Kendeou et al., 2014, 2016; van den Broek & Kendeou, 2008). These powerful persuasion tools (Hynd, 2001) have three basic features (Weingartner & Masnick, 2019): (i) misconception, (ii) refutation, and (iii) evidence. It usually starts by stating a common misconception among students. Then, this misconception is refuted. Refutation is supported by describing evidence that supports currently accepted scientific views. Thus, students are consecutively confronted by their misconceptions, the correct expression of these misconceptions, and the related scientific evidence.

The effectiveness of these tools in persuading the reader can be explained by the fact that they create cognitive dissonance between existing erroneous ideas and new information and that they are understandable, reasonable, and useful (Hynd, 2001). Tippett (2010) states that readers’ metacognitive awareness about the inaccuracy of their current knowledge, consistent and reasonable presentation of correct information, and the nature of the information highlights the power of these texts.

Prior knowledge has a critical impact on the processes and products of learning experiences (Vosniadou, 2019). It has been known for a long time that if this prior knowledge contains misconceptions, serious problems arise in the process of learning new knowledge (Dellantonio & Pastore, 2021). In knowledge revision terminology, prior knowledge and new information on the relevant subject are taken into account simultaneously (Kendeou et al., 2017). In knowledge revision, the students’ misconception arising from previous knowledge is gradually reduced according to newly encoded correct information (Butterfuss & Kendeou, 2021).

The KReC explains understanding how knowledge revision can occur during reading (Kendeou et al., 2017). The KReC covers five key principles: encoding, passive activation, coactivation, integration, and competing activation (see Kendeou et al., 2014). According to this framework, information in long-term memory (which cannot be erased) can be activated while reading a text, by the use of encoding and passive activation principles. Such activation supports the integration of new information with the student’s prior knowledge. These two pieces of information (prior knowledge and newly acquired knowledge) compete during the activation process. Knowledge revision occurs when new knowledge wins the competition thanks to its activation over prior knowledge (Kendeou et al, 2014; Will et al., 2019). Kendeou et al. (2014) argue that “the KReC framework can contribute to a deeper understanding of many of the existing models and theories of conceptual change and/or knowledge revision” (p. 391).

The examination of the literature on the refutation texts reveals that the researchers generally try to explain the changes in the student’s scientific knowledge base (Kendeou et al., 2017) by using a single term, namely “conceptual change” or “knowledge revision.” It is noteworthy that the term “knowledge revision” has been extensively used in the science learning literature (Flemming et al., 2020; Harsch & Kendeou, 2023; Hunsu et al., 2023) since the introduction of the KReC framework (Kendeou et al., 2014). Kendeou et al. (2017) explain their use of this term to express intentional revision or modification of misconceptions in two aspects: (i) Knowledge revision directly addresses the difficulty of making certain assumptions about the structure and nature of prior knowledge. (ii) It is operationalized (rejects the delete-and-replace view of conceptual change). Therefore, it causes the activation of students’ existing misconceptions to be gradually reduced and/or eliminated, thus intervening in pre-existing misconceptions.

As many researchers similarly state (Kim & Kendeou, 2021; Weingartner & Masnick, 2019; Will et al., 2019), refutation texts contain short-term interventions, while misconceptions are deep-rooted and resistant structures to change (Asberger et al., 2021; Dellantonio & Pastore, 2021). Therefore, this study follows the methods of (i) explaining the change in students’ misconceptions with the term knowledge revision and (ii) conducting a research process based on the KReC framework.

Refutation Text Interventions in the Literature

Evidence from past refutation text intervention studies suggests that when students misconceptions and scientific concepts are simultaneously activated, integrated, and compared, the attention is focused on the conflict between these two beliefs, and refutation texts become more effective tools for resolving misconceptions than texts without refutation (Danielson et al., 2016; Schroeder & Kucera, 2022; Torsney et al., 2021). In addition, previous studies have used some textual, visual, rhetorical, or multimodal tools in conjunction with refutation texts to enhance their effect of them. For example, Danielson et al. (2016) showed that refutation texts combined with analogy or graphics and analogy were more successful in encouraging revision of scientific topics in college students. Butterfuss and Kendeou (2021) extended the understanding of knowledge revision from the process of readers reading a single refutation text to interacting with multiple texts or documents, thus extending KReC to knowledge revision components-multiple documents (KReC-MD). These two frameworks contain the same assumptions and conditions; KReC-MD also proposes that “source reliability and intertextual integration are two key factors that influence the success of knowledge revision in the context of multiple documents” (Kendeou, 2024, p.11). Butterfuss (2020) conducted a series of experiments testing the assumptions of the KReC-MD framework and these key factors. These experiments indicated that undergraduate students demonstrated greater intertextual integration when reading refutation texts from high-credibility sources compared to non-refutation texts from low-credibility sources, and that students who engaged in greater intertextual integration experienced more substantial knowledge revision. In addition to these, Johnson et al. (2022) found that a multimodal refutation intervention using an authentic video from an expert source was effective in reducing misconceptions about a socio-scientific topic such as the COVID-19 pandemic compared to a group that did not receive the intervention. In the present study, considering the different interventions and integrations of refutation text in past studies, we brought the refutation text to bear on a digital content organizer, which we can integrate into multiple and different types of educational content.

The Online Advance Organizer Concept Teaching Material (ONACOM)

With a versatile structure that can be integrated into teaching methods, ONACOM is a digital content organizer that offers the opportunity to upload different digital content (such as video, image, URL, and text) to concepts on a concept map. ONACOM differs from the examples in the related literature with its features such as having its database structure, users having their profile pages, and teachers and students having different user authorizations (see Korur, Toker & Eryılmaz, 2016). This material has been used in teaching science topics such as light, sound, force, and motion to students at different grade levels, and evidence has been provided that it has increased students’ achievement and attitudes (Korur, Toker & Eryılmaz, 2016; Yılmaz & Korur, 2021). In addition, Türksoy (2019), in his experimental study integrating augmented reality applications into ONACOM, provided strong evidence that this material helped fourth-grade primary school students to construct and retain new knowledge in science subjects. ONACOM has been developed to increase the conceptual understanding of students in various studies (Korur, Toker & Eryılmaz, 2016; Türksoy, 2019). There is no study in the literature on the integration of ONACOM with refutation texts to contribute to students’ concept acquisition and knowledge revision processes. The usage of ONACOM in this study is compatible with the KReC framework (Kendeou et al., 2014) since it can support the coactivation, integration, and competing activation processes.

The Present Study

This study was conducted experimentally on three groups. In the first group (Treatment-1), the teaching process of the “matter and change” subject was carried out using the refutation texts integrated into ONACOM in line with the KReC framework (ONACOM-RT). As for the second group (Treatment-2), the teaching process was carried out using the refutation texts in line with the KReC framework (RT). In the third group (Control), the teaching process was carried out by a method included in the related curriculum (TMC), in which no refutation texts or refutational explanations were used. In this context, to test the effects of the applied methods among the groups and within the groups, the null hypotheses of the study are as follows:

In addition to testing the seven null hypotheses, we aimed to identify robust misconceptions across all groups. The sample of the study, method, measurement tools, and ethical considerations were approved by the Provincial Directorate for National Education of Antalya, Türkiye (#22025226). All participants in the study were volunteers.

Design and Participants

The study had a quasi-experimental pretest-posttest, treatment-control group research design. In this design, ONACOM-RT and RT were applied to the treatment groups, whereas TM-C was applied to the control group. The two treatment groups received different interventions than a single control group. While multiple control groups could enhance the robustness of the findings by providing additional comparisons, they also increase the complexity of the experimental design. Kirk (2013) notes that multiple control groups can be beneficial but are not always required. Given the context of this study—specifically the number of variables, available sample size, and the hypothesis being tested—a single control group was sufficient to address the specific research questions and the variables under investigation. Matter and Change Three Tier Test (MCT) was administered to all groups at the beginning of the study and 3 weeks after the treatment to determine the students’ conceptual understanding and misconception scores (see Table 1). The school applications were performed in line with the Matter and Change unit for fourth graders.

OPEN IN VIEWER

Table 1.

The Quasi-Experimental Design of the Study.

Groups	Pre-test/pre-scores	Methods	Treatment period	Post-test/post-scores
Treatment-1	MCT/	ONACOM-RT		MCT/
Treatment-2	Pre-conceptual understanding score (CUS)	RT	3 weeks	Post-conceptual understanding score (CUS)
Control	Pre-misconception score (MS)	TMC		Post-misconception score (MS)

The accessible population includes all fourth graders in Konyaaltı and Muratpaşa, the two biggest central districts of the province of Antalya, Türkiye. Since individual student selection and group assignment are not feasible, a two-stage stratified cluster random sampling was employed for a representative sample. Schools served as the strata, and classes as the clusters. First, the two most crowded state schools in these districts were chosen. Second, two classes were randomly selected from each school.

Data were collected from 188 students from the 2 schools in the mentioned districts of Antalya. There were 63 students (30 girls and 33 boys) in the control group, 62 students (32 girls and 30 boys) in the Treatment-1 group, and 63 students (28 girls and 35 boys) in the Treatment-2 group.

Variables and Score Types

Pretest scores were designated as covariates, posttest scores as dependent variables, and the teaching methods used (ONACOM-RT, RT, and TMC) as independent variables. Two types of scores were calculated based on both students and items. The scores based on students were calculated as pretest and posttest scores to perform inferential statistical analyses. The scores based on items were those that reflected percentages based on tiers and those that examined the validity of the data collection instrument.

Data Collection Instrument

Matter and Change Three Tier Test (MCT)

The test was developed by the first two authors of this study. Initially, an item pool was created to design the MCT, aimed at identifying misconceptions and assessing conceptual understanding. Most items were crafted by the authors to align with the learning outcomes in the Turkish science curriculum (The Turkish Ministry of National Education [TMNE], 2018). The MCT consisted of three tiers: the first tier presented a question, the second tier asked for the reasoning behind the response, and the third tier gauged participants’ confidence in their answers (options: “I am sure,”“I just guess,”“I am not sure”). The face and content validity of the MCT was confirmed by experts (three faculty members, two doctoral students, and two master students). A pilot study was conducted with 156 fifth graders who had studied “Matter and Change” to assess the validity and reliability of the MCT. The pilot results showed a significant correlation between students’ scores on the first two tiers and their confidence in the third tier (r[156] = .718, p < .05).

A classical item analysis was conducted for the first two tiers, as the third tier had only yes–no responses. The difficulty and discrimination indices were calculated for the first two tiers of each item. For item difficulty, one item (item 6) was classified as very difficult (.00–.19), five items (items 1, 3, 4, 5, and 7) were difficult (.20–.39), two items (items 2 and 9) were moderate (.40–.59), six items (items 8, 10, 11, 12, 15, and 16) were easy (.60–.79), and two items (items 13 and 14) were very easy (.80–1.00). According to the threshold values of Crocker and Algina (2008), the test had moderate difficulty.

In misconception tests, it is recommended to detect assessment errors by identifying correct answers with incorrect reasoning (false positives) or incorrect answers with correct reasoning (false negatives) (Hestenes & Halloun, 1995). Minimizing false positive and false negative rates to below 10% is crucial for improving content validity in multiple-choice tests (Hestenes & Halloun, 1995). Based on these analyses, items 2 and 9, which had discrimination values below .19 and false positive/negative rates above 10%, were removed, as other items adequately covered their learning outcomes. The remaining items had discrimination indices between .200 and .417, indicating the discrimination was acceptable (Ebel, 1954). After removing two items, the KR-20 coefficient for test reliability was .77, reflecting acceptable consistency in student responses (Fraenkel et al., 2012). Following the pilot study, expert feedback was re-collected for the MCT, and three items were revised. An expert also reviewed the grammar and vocabulary. The final version of the MCT included 14 items assessing nine misconceptions. Alternative sets representing these misconceptions are shown in Table 2 (see for details Korumaz, 2018).

OPEN IN VIEWER

Table 2.

Alternative Sets Showing Misconceptions According to Three Tiers.

Misconception	Misconceptions alternative sets	Number of items
1. Heat and temperature are the same	1.1 A 1.2 A 1.3 A	1
2. Temperature varies depending on the size of the object	3.1 A 3.2 A 3.3 A	1
	3.1 C 3.2 A 3.3 A
3. Heat can flow in all directions continuously	2.1 B 2.2 A 2.3 A	3
	2.1 C 2.2 A 2.3 A
	4.1 A 4.2 A 4.3 A
	4.1 C 4.2 A 4.3 A
	6.1 A 6.2 A 6.3 A
	6.1 C 6.2 A 6.3 A
4. The material of the object affects the temperature	5.1 A 5.2 A 5.3 A	1
	5.1 A 5.2 C 5.3 A
5. Even if we perceive properties of matter such as color and smell, if the matter is not visible, it does not exist in the environment	7.1 B 7.2 A 7.3 A	2
	13.1 A 13.2 A 13.3 A
	13.1 C 13.2 A 13.3 A
6. Gases are not matter because most gases are invisible, and they do not have mass	8.1 A 8.2 A 8.3 A	1
7. The change of state is a physical and chemical change	9.1 A 9.2 A 9.3 A	3
	12.1 B 12.2 A 12.3 A
	12.1 C 12.2 A 12.3 A
	14.1 A 14.2 A 14.3 A
	14.1 C 14.2 A 14.3 A
8. Materials may only exhibit the properties of one of the states of matter	10.1 B 10.2 A 10.3 A	1
	10.1 C 10.2 A 10.3 A
9. Melting/freezing/boiling/evaporation is mostly considered to be properties of only water	11.1 A 11.2 A 11.3 A	1

Teaching Materials

Refutation Texts

Refutation texts for misconceptions in that specific unit were included in certain slides in presentations. During the treatments, nine refutation texts were distributed to the Treatment-1 and Treatment-2 groups in printed form and shown in PowerPoint presentations. In presentations, the instructional presentation method was used to guide refutation text. Two refutation texts were used in the first week, four in the second week, and three in the third week. The refutation texts were designed similarly to the studies of Hynd et al. (1997) and Tippett (2010). The introduction of the refutation text included a question that expresses the misconception and emphasizes that the student who read the text might also have that misconception, which helped internalization and caught attention. The rest of the text consisted of scientific knowledge involving statements aiming to correct and refute the misconception. An example of an English translation of the refutation text about heat and temperature is presented in Figure 1.

OPEN IN VIEWER

Figure 1.

Sample refutation text and its components.

Treatment and Control Group Instructions

The KReC-based ONACOM-RT method was applied to the Treatment-1 group. Before the treatment started, the teachers in the Treatment-1 group were informed by the researcher about using the ONACOM. One of the researchers, who participated in lessons during the process alongside the classroom teacher, acted as merely a guide and kept details related to the process of application of the method by filling in a method application form. The ONACOM-RT method was applied to one class in each of the two selected schools for 3 weeks by dividing one class hour into four stages. In the first stage of the method, the related page of the ONACOM map was displayed (see Figure 2). The concepts in the related concept map were elaborated. In the second stage, the teacher made explanations related to the content and introduced the main concepts of the subject from the map. The refutation texts prepared to eliminate the misconceptions found in the topics of that week were used in proper steps. At this stage, the teacher presented some of the digital content that was assigned to the concepts and students analyzed them. This stage essentially reflected the coactivation principle of the KReC framework since it was the time that new information came into contact with the misconception.

OPEN IN VIEWER

Figure 2.

Sample ONACOM map prepared for matter and change unit.

In the third stage, using the relevant digital content from ONACOM, the subject matter was explained through examples, and the students were asked to provide further examples. During lecturing, the teacher offered more explanations related to the subjects by using digital content (such as pictures, and videos) on ONACOM. This process corresponds to the integration principle in the KReC framework. In the fourth stage, students were given a little extra time to review the content they wanted. At this stage, the teacher also asked the students to re-read the refutation texts, aiming to ensure that correct scientific knowledge had been established and misconceptions were eliminated. Since the activation of the newly encoded true scientific knowledge has increased considerably and since the objective was to reduce or eliminate the activation of false information, this stage mainly reflected the KReC framework’s competing activation principle. At the end of the stage, students solved the ONACOM assessment questions. Students were told that they could also use ONACOM from their homes by logging on to their profile pages.

In the Treatment-2 group (RT) the whole process was applied in the same manner except for ONACOM. Nine refutation texts, related teaching materials, and digital contents were classified according to lessons in 3 weeks; and in every lesson, the KReC framework principles were taken into account carefully. At the end of the lessons, the students solved questions from a worksheet, the assessment questions were reviewed one by one, and the teacher clarified any.

In the control group (TMC) teachers covered the subjects by using methods and techniques they preferred concerning the learning outcomes of the Matter and Change unit given in the science curriculum. In the TMC group, the application also lasted 3 weeks. The teachers also used educational videos or digital content such as animations from an online learning environment (e.g., www.morpakampus.com) in several lessons. Moreover, the teachers also prepared worksheets for the students about the subject every week and made the students take various notes using the science course book and notebook during the lecture. On the other hand, neither refutation texts nor refutation explanations were used during lecturing in the TMC group. One of the researchers also participated in control group lessons, filling in a method application form to keep details about the process of application of the method, and usage of various materials.

Data Analysis

Data cleaning and missing data analysis were initially conducted. Outliers affecting the normality of the distribution were identified using Mahalanobis distance values. Two participants from the control group (participants 170 and 188) had Mahalanobis values with p < .05, classifying them as extreme outliers, and were removed from further analyses. MANCOVA assumptions, which included those of the paired-sample t-test, were assessed. Skewness and kurtosis values for all variables fell within the acceptable range (−1.5 to +1.5), indicating normal data distribution (Field, 2009). The maximum observed skewness was 1.052, and the maximum kurtosis was −.998, further confirming normality.

Regression homogeneity, variance/covariance homogeneity, linearity, sphericity, and independence of observations in MANCOVA were all tested, with no significant violations observed. The interaction term Method * MS-PRE * CUS-PRE yielded p = .156 for MS-POST and p = .760 for CUS-POST, confirming the homogeneity of the regression assumption. Additionally, the similarity in group sizes (Treatments-1, 2/control < 1.5) supported regression homogeneity. The equality of variance and covariance matrices was verified using Box’s M-test (M = 24.097, F[6, 851702.7] = 3.953, p = .001). Box’s M-test, which assesses the homogeneity of covariance matrices across groups, is highly sensitive, particularly when group sizes exceed 30 participants. As such, a significance threshold of .001 was deemed acceptable (Hair et al., 2006). Levene’s test was used to evaluate the homogeneity of variance, revealing a non-homogeneous distribution of variances for both dependent variables (MS-POST, p = .003; CUS-POST, p = .019). In instances where Levene’s test indicates variance differences, it is appropriate to examine the variance ratio for the dependent variables. For sample groups ranging from 30 to 60 participants, variances are considered homogeneous if the ratio of the largest to smallest variance falls between 2 and 3 (Field, 2009). The variance ratio for MS-POST was 2.74 (2.917/1.063) and for CUS-POST, it was 1.15 (7.323/6.378), both under the critical value of 3.

All analyses were conducted using SPSS (IBM SPSS, 2013) and MS Excel (Microsoft Corporation, 2013). This study, or its analysis plan, was not preregistered. Due to data-sharing restrictions imposed by the Ministry of National Education’s General Directorate of Legal Services, supporting data is unavailable but can be provided upon reasonable request.

Results of the Between-Group Analysis

The MANCOVA analysis was performed to test the first hypothesis of the study. The MANCOVA indicated that when pretest scores were controlled, there was a significant difference between the effects of the treatments in the groups on collective dependent variables (F[4, 364] = 20.404, p < .005, Wilks’Λ = .667, partial η² = .183). In summary, the methods had significant treatment effects on MS-POST (F[2, 183] = 11.854, p < .001, partial η² = .115) and with large effect sizes on the CUS_POST (F[2, 183] = 41.701, p < .001, partial η² = .313) when the MS-PRE and CUS-PRE were controlled. The methods explained 18.3% of the variance of the dependent variables. To evaluate the post hoc tests in terms of adjusted alpha, the alpha constituent needs to be recalculated using the alpha adjustment approach. Since there were two dependent variables in this study, the alpha should be reduced from .050 to .025 to maintain the Type I error rate for the joint null hypothesis at the pre-specified α joint of .050 (Rubin, 2021). There were still statistically significant effects of the methods on the two dependent variables for the adjusted alpha level. One of the reasons for this significant difference might be that students not only realized their misconceptions but also learned the concepts through a certain systematic knowledge revision framework with the help of refutation texts and refutation texts integrated online teaching material (Butterfuss & Kendeou, 2021; Hunsu et al., 2023; Kendeou et al., 2017).

According to the results of MS-POST in Table 3, there were no significant differences between the mean misconception scores of the ONACOM-RT and RT. However, the mean MS-POST of the students in the control group (TMC) was significantly higher than that of the students in both the ONACOM-RT and RT groups separately. It can be said that the TMC method applied to the control group was not as effective as the methods applied to the treatment groups in eliminating misconceptions of students. The ONACOM-RT group students had significantly higher CUS-POST mean scores than the students in the other two groups. The mean CUS-POST of the students in the RT group was significantly higher than that of the students in the TMC group. The advanced graphical analysis in Figure 3a shows that the sharp decrease in the mean misconception scores might indicate that in the ONACOM-RT and RT groups’ misconceptions were mostly eliminated during the process. However, the moderate decrease in the scores of the TMC group might indicate that the elimination of the misconceptions in this group was quite limited. According to the advanced graphical analysis of conceptual understanding scores in Figure 3b, the highest increase in mean scores was in the ONACOM-RT group. Therefore, the ONACOM-RT was the most effective method among the three groups in reducing students’ misconceptions while increasing their conceptual understanding.

OPEN IN VIEWER

Table 3.

Pairwise Comparisons.

Dependent variable	(I)	(J)	Mean variance (I-J)	Std. error	p	95% confidence interval for difference
						Lower	Upper
MS-POST	TMC	ONACOM-RT	1.051*	.228	.000	.501	1.601
		RT	.831*	.228	.000	.283	1.380
	ONACOM-RT	RT	−0.220*	.229	.712	−.772	.333
CUS-POST	TMC	ONACOM-RT	−3.735*	.409	.000	−4.720	−2.749
		RT	−1.833*	.408	.000	−2.816	−.851
	ONACOM-RT	RT	1.901*	.411	.000	−.911	2.891

Note. N = 188.

*

p < .025.

OPEN IN VIEWER

Figure 3.

(a) MS advanced graphical analysis and (b) CUS advanced graphical analysis.

Results of the Within-Group Analysis and the Robust Misconceptions

The conceptual understanding scores and misconception scores were analyzed within the groups to analyze the second to seventh hypotheses of the study. At the beginning, the students’ mean CUS scores (CUS-PRE; X ¯ _{ONACOM_RT} = 2.77, X ¯ _RT = 2.57, X ¯ _TMC = 2.63) and MS scores (PRE-MS; X ¯ _{ONACOM_RT} = 2.18, X ¯ _RT = 1.78, X ¯ _TMC = 2.03) were close to each other for all three groups. The paired sample t-test related to the data obtained from the students in the Treatment-1 group was performed to test the second null hypothesis. A statistically significant difference was found between CUS-PRE ( X ¯  = 2.77; SD = 1.68) and CUS-POST ( X ¯  = 7.24; SD = 2.18) with a large effect size (t[61] = −14.295, p < .001; Cohen’s d = 2.30), then the null hypothesis was rejected. At the end of the lecturing performed with the ONACOM-RT method, the conceptual understanding scores of the students in the Matter and Change unit increased by 32% (4.47 out of 14).

When the misconception scores in the same group were investigated with the paired sample t-test to test the third null hypothesis, a statistically significant difference was detected between MS-PRE ( X ¯  = 2.18; SD = 1.71) and MS-POST ( X ¯  = .95; SD = 1.03) with a large effect size (t[61] = 6.058, p < .05; Cohen’s d = .871). Also, there was a noticeable decrease (approximately 9%; 1.23 decrease out of 14) in the students’ misconception scores. It was determined that the ONACOM-RT method considerably increased students’ conceptual understanding and eliminated their misconceptions about the Matter and Change unit. The misconceptions of the ONACOM-RT were analyzed in terms of item scores for three tiers. There were three different scores before (PRE) and after (POST) lecturing respectively: MBI-1, MBI-2, and MBI-3.

In the ONACOM-RT group, five out of six misconceptions were eliminated, the misconception percentage was reduced, but not eliminated for one misconception, and no misconceptions were detected in three of them (see Figure 4). An example of a misconception that was eliminated is “Heat can flow in all directions continuously” (see Table 1). After the ONACOM-RT treatment, 26.9% of the students answered the items corresponding to this misconception incorrectly in the first tier. This rate dropped to 8.1% after the ONACOM-RT treatment. Since the POST-MBI-3 value is below 10%, it can be said more properly and consistently that the detected misconception has been eliminated. On the other hand, the misconception that “Temperature varies depending on the size of the object” was not eliminated and it could be named “robust.”

OPEN IN VIEWER

Figure 4.

Misconceptions of the ONACOM-RT group for the tiers before and after the treatment.

The data of the students in the RT group was used to test the fourth hypothesis. The paired sample t-test indicated that statistically significant difference with a large effect size between CUS-PRE ( X ¯  = 2.57; SD = 1.65) and CUS-POST ( X ¯  = 5.22; SD = 2.66) scores (t[62] = −7.739, p < .05; Cohen’s d = 1.20). After the treatment through the RT, the conceptual understanding scores of the students increased by 19% (2.65 out of 14). Concerning the fifth hypothesis, the paired sample t-test performed for the misconception scores in this group indicated a statistically significant difference with a medium effect size between MS-PRE ( X ¯  = 1.78; SD = 1.42) and MS-POST ( X ¯  = 1.04; SD = 1.22) scores (t[62] = 3.176, p = .002; Cohen’s d = .56). After the treatment through the RT, the students’ misconception scores decreased by 5% (.74 out of 14). The misconceptions of the RT group were also analyzed in terms of item scores for three tiers to identify robust misconceptions.

It was observed that two of the six misconceptions were eliminated, and the percentage of misconceptions was reduced in four of them (see Figure 5). After the treatment applied in the RT group, the robust misconceptions that could not be eliminated were “Temperature varies depending on the size of the object,”“Heat can flow in all directions continuously,”“The material of the object affects the temperature,” and “Gases are not matter, because most gases are invisible, and they do not have mass.”

OPEN IN VIEWER

Figure 5.

Misconceptions of the RT group according to the tiers before and after the treatment.

In the control group (TMC), a statistically significant difference with a medium effect size was found between CUS-PRE ( X ¯  = 2.63; SD = 1.94) and CUS-POST ( X ¯  = 3.42; SD = 2.52) scores (t[62] = −3.160, p = .002; Cohen’s d = .351), indicating the sixth hypothesis was rejected. At the end of the lecturing through the TMC method, the conceptual understanding scores of the students increased by 6% (.79 out of 14). When the misconception scores were examined, no statistically significant difference was detected between the MS-PRE ( X ¯  = 2.03; SD = 1.69) and MS-POST ( X ¯  = 1.95; SD = 1.71) scores (t[62] = .361, p = .719); so, the seventh hypothesis was accepted. The mean post-misconception score of the students only decreased by 0.6% (.08 out of 14). At the end of the lectures using the TMC method, the misconception percentages of the students did not considerably decrease (see Figure 6).

OPEN IN VIEWER

Figure 6.

Misconceptions of the TMC group according to the tiers before and after the treatment.

One of the five misconceptions was eliminated, the misconception percentage of four misconceptions decreased, and the percentage of one misconception increased, there was a possible misconception, namely “Even if we perceive properties of matter such as color and smell, if the matter is not visible, it does not exist in the environment” in TMC. In this direction, it was revealed that it would be a challenging process to eliminate misconceptions with a teaching method that was not planned with regards to a systematic approach to eliminating misconceptions, and it was also possible to create misconceptions. The robust misconceptions in this group were, “Heat and temperature are the same,”“Temperature varies depending on the size of the object,”“Heat can flow in all directions continuously,” and “The material of the object affects the temperature.”

Limitations and Future Research

The present study has several limitations. First, while results indicate improved conceptual understanding and reduced misconceptions in the ONACOM-RT and RT groups, these findings occurred immediately post-intervention. Future research should assess long-term effects to address concerns about the sustainability of refutation texts (Prinz et al., 2022; Will et al., 2019; Zengilowski et al., 2021). Secondly, all interventions were conducted by student’s own teachers in different classes, which could introduce variability. However, one of the authors of this study observed the process to ensure fidelity to the interventions. Third, the ONACOM online tool allowed students to create profiles and access refutation texts outside class; however their usage was not tracked. While we viewed this as an opportunity for enhanced learning (Yılmaz & Korur, 2021), future studies should measure how often students engage with such tools. Fourth, a total of nine refutation texts were used at different times, which may have influenced recall and targeted knowledge revision due to the varying time intervals between the texts and the posttest. Fifth, students could not indicate confidence in their answers at each tier, potentially leading to inflated scores (Kaltakci Gurel et al., 2015). This limitation existed in all three-tier tests. Lastly, the study’s 3-week duration may not have been sufficient for younger students to achieve deeper conceptual understanding. Longer exposure to these tools could lead to a more comprehensive evaluation of their effects, such as a greater reduction in misconceptions.

Future studies should explore the impact of extended intervention periods to determine students’ understanding and retention of concepts. This would offer valuable insights into the time needed to achieve more robust learning outcomes. Future studies could also investigate how an innovative online tool (such as ONACOM) can be designed to address and eliminate misconceptions in educational settings, particularly for primary school students. Finally, educators should be mindful of outlined principles (such as co-activation, integration, and competing activation) of the KReC when developing teaching materials to better support students in overcoming robust misconceptions.