Reading-Metacognitive Strategy Awareness and Use in Reciprocal Teaching Settings: Implementing a Computerized RMSA System

Abstract

The present experiment was composed of both qualitative and quantitative methods and aimed at investigating Reading Metacognitive Strategy Awareness (RMSA) and use by the aid of three inventory techniques including a questionnaire, a think-aloud protocol, and a Computerized RMSA System. The computerized system was specifically designed and validated to satisfy the needs of this study in measuring and practicing Reading Metacognitive Strategies. It also tried to explore any possible effects of Computerized RMSA System on reading comprehension of the participants. To this end, a sample of 25 university students majoring in English Translation and English Literature from a University in Iran were selected based on purposive and clustered sampling. The research data were selected using a number of instruments including the reading section of Test of English as a Foreign Language as pre/posttest, a researcher-made Reading Metacognitive Strategies questionnaire, think-aloud protocol and a Computerized RMSA System. The research findings show that the questionnaire, think-aloud and Computerized RMSA System results vary significantly. Moreover, Computerized RMSA System has a positive effect on reading comprehension of learners of English as a Foreign Language. Pedagogical implications of the findings are discussed.

Keywords

computer assisted inventory computer assisted instruction reading comprehension reading metacognitive strategies reciprocal teaching

Introduction

Second and foreign language learning has rightly been described as a multifaceted and dynamic process in which success remarkably relies on proficiency in different skills including reading skill. The importance of reading skill has encouraged many researchers to search for the best ways to deal with reading comprehension and explore solutions to put theoretical views in practice (Elleman & Oslund, 2019; Catts, 2018). It is believed that second language learners use specific strategies to compensate for the reading comprehension difficulties and that researchers called some of those compensatory strategies as reading metacognitive strategies (Djudin, 2017; Razak et al., 2018). According to Anderson (2002) readers’ awareness, monitoring and regulating of such strategies while reading a text is called metacognitive awareness. These strategies are so important that can influence all the steps of learners’ reading comprehension and encourage the development of learning activities (Wang et al., 2009).

“Thinking about thinking” pertains to metacognition. This statement was first proposed by an American developmental psychologist John H. Flavell (1979). Metacognition which is further referred to as cognition of cognition (Flavell, 1979; Metcalfe & Shimmura, 1994) has been an area of interest to educational researchers for more than 40 years (Al-Jarreh et al., 2018). Researchers have been interested in distinct aspects of metacognition including the effect of metacognition on different language skills and systems (Coskun, 2010; Lv & Chen, 2010; Marzano et al., 2001; Selamat & Sidhu, 2013; Tsai, 2009) more specifically its effects on reading comprehension (Boulware-Gooden et al., 2007; Curwen et al., 2010; Veenman et al., 2006) and the plausibility of teaching metacognitive strategies (Ellis & Denton, 2010; Haider & Al-Naqabi, 2008; Kistner et al., 2010; Leon-Guerrero, 2008; Leutwyler, 2009; McBride & Dosher, 2002). Some researchers were also eager to design or use distinct methods such as questionnaires, think-aloud protocols, interviews etc separately to investigate metacognitive strategy awareness and use (Liu et al., 2014; Sarac & Karakelle, 2012; Schellings, 2011). Considering the wide range of studies in the realm, this study mainly focused on investigating awareness and use of metacognitive strategies with the aid of three different inventory techniques including a questionnaire, think-aloud protocol and researcher-made Computerized RMSA System. The study also tried to explore the effect of this system on reading comprehension of Iranian learners of English as a Foreign Language.

Literature Review

Reading-Metacognitive Strategy Instruction

Reading skill and reading metacognitive strategies are important features of every foreign language learning that can be implemented either cooperatively or collaboratively using a reciprocal reading procedure (Asikcan & Saban, 2018; Bishara & Kaplan, 2018; Dagoc & Tan, 2018; Fitrisia et al., 2015). In cooperative and collaborative reading sessions, teachers and learners can scaffold other learners’ learning and reading; thus, scaffolders can help learners have better comprehension of the texts they are reading. Scaffolding as a pedagogical approach is widely used in teaching different aspects of language including metacognitive strategy training (Azevedo & Hadwin, 2005; Bannert, 2006). It is rooted in Vygotsky’s (1978) view on the interaction taking place between teachers and learners with the teacher taking the role of scaffolding and supporting learners’ progress.

Reciprocal teaching can best be explained by ideas proposed by Iserbyti et al. (2010) and Ward and Lee (2005), Vygotsky’s (1978) social-historical perspective. The social-historical theory related to social constructivism, and points out that the essential role of social interaction is to develop human cognition. Li and Kam (2011) refer to Vygotsky’s theory and claim that learning is triggered by social interaction with the impact of “more knowledgeable other” and creating “a zone of proximal development”. Zone of proximal development refer to a level of improvement “as determined through problem-solving under adult guidance or in collaboration with more capable peers” (Vygotsky, 1978, p. 86)”. It is believed that the learning environment which is covered with teacher and peer collaboration and cooperation “serve as opportunities for more knowledgeable others for tutoring as well as trigger the zone of proximal development for the students to achieve the learning task” (Li & Kam, 2011, p.29)

Scaffolding is closely linked to what has been referred to as reciprocal teaching by Palincsar and Brown (1984). Reciprocal learning/teaching is an instructional model in which students work in pairs to master lesson content. According to Mosston and Ashworth (2002), they cooperate in well-defined roles of doer and observer (i.e., tutor and tutee) to maximize their own and each other’s learning. While one learner is doing (doer), the other learner (observer) is observing, analyzing the doer’s performance, and giving performance-related feedback (Mosston & Ashworth, 2002).

Several models for teaching learning strategies in both first and second language context have been developed (Chamot & El-Dniary, 1999; Cohen, 1998; O’Malley & Chamot, 1990; Oxford, 1990); all these instructional models reinforce the importance of developing learners’ metacognitive understanding of the value of learning strategies and suggest that teacher demonstration and modeling can facilitate this process (Duffy et al., 1988). However, if the strategy inventory system which is designed, developed and validated by the researchers of this study is used for instructional purposes, there is no need for the direct modeling of the teacher. Explicit teacher modeling takes place when a teacher provides the learners with an easy procedure to follow while doing a task (Lunenberg et al., 2007; Richards & Lockhart, 1996). In fact, in Computerized RMSA System, strategies would be learnt and internalized without the teacher’s direct or indirect instructions.

Another characteristic of the instructional models is that they usually stress the importance of providing opportunities for practicing the strategies to enable autonomous strategy use (Chamot et al., 1999; Harris, 2003; O’Malley & Chamot, 1990; Oxford, 1990). This feature is available in Computerized RMSA System since all reading comprehension texts and steps are accompanied by elements of metacognitive strategies.

One other feature that all models share is the fact that learners should evaluate how a strategy has worked, choose strategies for a task and actively transfer strategies to new tasks (Chamot, 2004). In Computerized RMSA System, learners are passively active in strategy use. In other words, they are not actively aware of the strategies; rather, they subconsciously use and internalize reading metacognitive strategies while reading English texts.

In fact, Computerized RMSA System is a semi-direct method of strategy training which helps overcome the limitations of traditional strategy training models (including Anderson’s model, 1991; Tsai, 2009). The readers process reading comprehension tasks which are assimilated and accompanied with reading metacognitive strategies. In other words, reading strategies are indirectly incorporated into every step of reading comprehension. Metacognitive reading strategies are higher-order performance methods that refer to planning, monitoring and evaluating the success of a learning activity (Pressley & Afflerbach, 1995). Metacognitive reading strategy awareness is of interest not only for what they indicate about the ways learners arrange their interaction with context, but also for how the use of strategies is related to effective reading comprehension (Mokhtari & Reichard, 2002). In Computerized RMSA System, computer took a key role since it was used as a means with which reading comprehension was taking place. This is a semi-direct method of strategy awareness and instruction where learners can use reading metacognitive strategies while comprehending the text.

Reading-Metacognitive Strategy Inventory

The rich literature (Becirovic et al., 2017; Dardjito, 2019; Fazeli, 2012; Gavora et al., 2019; Mokhtari et al., 2018; Sheikh et al., 2019) related to the investigation of Metacognitive Strategy Awareness shows great interest in exploring metacognitive strategy in reading; however, the studies mostly use prefabricate questionnaires to measure metacognitive strategy awareness. One reason is that most researchers find this instrument as a sophisticated technique for data collection. Young (2015) states that questionnaires are among the most popular methods of data collection in social science and he further added that this data collection technique is also among the most misused. Meanwhile, some researchers try to enhance their research instrument by modifying the questionnaire and making it more contextualized. When it comes to the modification of questionnaires, researchers (Jacob & Paris, 1987; McLain et al., 1991) usually increase the number of items, extend psychometric properties, improve reliability and validity or just reconsider the construct of metacognition (Mokhtari & Reichard, 2002). Thus, it causes great misuse of specific questionnaires.

The researchers of this study like many other researchers (Berg, 2009; Rahman, 2016; Ravitch, 2010; Sallee & Flood, 2012) assume that questionnaires may not generate feasibly valid and practical data just because of learners’ and participants’ lack of interest in reading the whole items of the questionnaire, shortage of time for answering the questioners as well as confidentiality and privacy reasons. According to Mokhtari and Reichard (2002), “effects to develop metacognitive awareness inventories have been well intentioned but generally not satisfactory from a measurement perspective” (p. 250). The instruments available are mostly in questionnaire form which encourages learners to only read some sentences and rate them based on their expectations. Generally, most of reading metacognitive strategy measurement tools including Jacobs and Paris’(1987) Index of Reading Awareness, Pereira-Laird and Deane’s (1997) Self-report of Reading Strategy Use, Schmitt’s (1990) Reading Strategy Awareness Questionnaire, Miholic’s (1994) Metacognitive Awareness of Reading Strategies, and Mokhtari and Reichard’s (2002; 2018) Metacognitive Awareness of Reading Strategies aimed at diagnosing learners’ strategy use only by reading few items and selecting the scale that best fits their cognitive processes. This cannot be considered as a valid indicator of how learners use reading metacognitive strategies. Questionnaires are mostly used as an inventory system just because of their conveniences (Harrison & Vallin, 2017).

Another data collection technique which is interchangeably being used by many researchers who are willing to survey mental processes of learners is the think-aloud protocol. Think-aloud protocols can provide rich insights on language learning strategies, and were also used by some researchers (Olson et al., 1984; O’Malley & Chamot, 1990; Vandergrift, 1997, 2003) to gain an understanding of the participants’ underlying processing. As stated by Vandergrift (2003), a think-aloud procedure includes two phases: a training phase and a data collection phase. Whereas the training phase aims at helping participants to know how to think aloud, the data collection phase deals with asking the participants to think aloud and collect data.

According to Ericsson and Simon (1980), the data gathered using the think-aloud technique will always be incomplete so that it might exclude a number of underlying thought processes. The reason relies on the fact that some ideas are only in working memory and it’s hard to verbalize ideas untill they are in long-term memory.

This study is closely linked with MacNamara’s (2011) Creed that metacognition and strategy use are crucial to deep, long-lasting comprehension and learning, but their assessment is challenging. Firstly, MacNamara (2011) states rightly that “students’ judgments of what their abilities and habits are, and measurements of their performance often do not match” (p. 159). Secondly, “it is generally assumed that strategy use (metacognition, metacomprehension) is separable constructs from the underlying skills germane to the target task” (MacNamara, 2011, p. 159). In this study, the researchers try to compensate the RMSA inventory systems of questionnaire type which were developed in many studies (Mokhtari et al., 2018; Mokhtari & Reichard, 2002; Mokhtari & Sheorey, 2002). It also puts an attempt to bypass the problems faced while using the think-aloud protocol.

There are also researchers who assume that assessing underlying mental skills cannot be easily handled using computer (Heinrich & Wang, 2003; Paterson, 2002). In contrast, researchers such as Cox and Clark (1998) and Reid (2002) pointed out that mental skills including metacognitive strategies can be evaluated using computerized mechanisms. According to Sim et al. (2004), the computer- assisted assessment is multifold and encompasses a) adaptive testing, b) analysis of the content of discussion boards, c) automated essay making, d) delivery of exam papers, and e) objective testing. The system devised here pertains to a triangulation of three characteristics (i.e. b, d, & e) proposed by Sim et al. (2004).

Computerized RMSA System

The development of Computerized RMSA System was directed by some steps including (a) reviewing the related literature on metacognitive strategies and reading comprehension and checking the feasibility, (b) setting up language and software experts, (c) designing the program, (d) programming, (e) testing, and (f) evaluating. The steps have already been proposed by Turel and Mckenna (2013, pp. 188–190) to design a language learning software used here to originate a system which is used for both teaching and measurement purposes.

We trusted certain research creeds on reading metacognitive strategy inventory (Gersten et al., 2001; Pressley, 2000) as helpful and fruitful resources for developing the system. Rosenblatt’s (1978) theory of reader response which emphasizes the transition between readers and the available text was implemented. In his theory, Rosenblatt (1986) clarifies the essence of the transaction or exchange between reader and text suggests a reciprocal or mutual relationship between the reader and the text. This theory encompasses not only a top-down (Anderson & Pearson, 1984) but also a bottom-up model of the reading process. There was also an attempt to use all the metacognitive strategies proposed by O’Malley and Chamot (1990) and Oxford (1990). Notably, metacognitive strategies were injected and penetrated within the three steps of reading, i.e. pre-reading, reading, post-reading. This was an important quality of the system since the steps represent the theoretical concepts of what constitutes a metacognitive strategy inventory.

When preparing the blueprint of the system, the researchers were assisted by two linguistic experts who have both knowledge and experience of teaching reading and assessing metacognitive strategies. The experts reviewed the initial steps of the system to help metacognitive strategies merge well with reading comprehension. This primary review resulted in a three-interface system with underlying steps. Some aspects of the system are explained here:

The Inside Interface

The inside interface was designed and developed to encourage learners to apply the metacognitive strategies indirectly. This involves engaging learners in planning the activities during the pre-reading and post-reading stages. To indirectly figure out the awareness and use of this strategy type, the first page of the inside interface as an instance includes “a choice between an automatic and manual arrangement of activities”. For detecting the directed attention as a type of centering one’s learning strategy, learners are free to direct their attention to the text only or check the text-related pictures while reading the text. For the strategy of monitoring, learners can “check the answers to comprehension question as often as they need”. Also, they can “focus on learners’ preferences in correcting the wrong answers”. These are only few instances of indirect elicitation of awareness and use of metacognitive strategies.

The Outside Interface

Unlike its name, the outside interface is interwoven within the inside interface; however, it covers the strategies used in the inside interface and tries to add reciprocal aspect to the system. The outside interface which provides possibilities for collaboration with the teacher emphasizes indirect use of metacognitive strategies such as problem identification, performance evaluation, selective attention, and finding out learning. For planning, “learners can discuss the reason for reading the text”. In the discussion forum, the teacher and learners can jointly use strategies which are implemented in distinct discussion topics.

The Trace Interface

The system also has the capacity to automatically keep a record of the learners’ written notes on key words and main grammatical points in printable notebooks and discussion annotations. The teacher-researcher can monitor learners RMSA through the trace left in the third interface of the system (Supplemental Table 1). She can then estimate the frequency and duration of learners’ attention to each RMSA-related activity. The researcher obtains a report presenting which activities the learners have selected during the pre-reading and post-reading stages. Traces are also used to give extra qualitative information on duration and time management of each participant on every single reading metacognitive strategy. Overall, statistical and qualitative information of the learners’ activities in the system would be presented in the trace interface. It is important to note that the classification of items in Computerized RMSA System was consulted by Sim et al. (2004) who proposed some categories for designing questions in the computer-assisted assessment.

Two major schools of thought for considering the quality in qualitative research are Dixon-Woods et al.’s (2004) view on methodology and Lincoln et al.’s (2011) view regarding the interpretation of results. For the sake of methodology, the researchers can produce a checklist of underlying factors for considering and reviewing the clarity and appropriateness of the research questions. This includes description and appropriateness of sampling, data analysis, levels of support and evidence for claims (Leung, 2015). The checklist used for the validation of the study was mainly designed with a focus on the traces in the trace interface. The checklist represents a file to keep the observed data which exists in the trace interface of the system. Based on the observed data, the system users and assessor can make inferences about the strategies.

Validity issue in Computer Assisted Strategy Assessment had been reviewed and discussed by Chapelle (1996). According to Chapelle (1996), “validity justification consists of an argument relevant to understanding the meaning of observed data for making inferences about strategies” (p. 55). As triangulation is a powerful strategy for improving the validity of research, in this study, we transferred qualitative data into quantitative data to further study the validity of the system. In fact, we used Confirmatory Factor Analysis to test whether measures of Computerized RMSA System were consistent with the model proposed by the researchers. Owing to this fact, the researchers used Confirmatory Factor Analysis to see if the data fit the hypothesized measurement model. This was implemented to overcome the validity problem of Computer Assisted Strategy Assessment use proposed by Chapelle (1996). According to Joreskog and Sorbom (1979), the Modification Indexes value for a parameter gives the expected drop in the model Chi-square values when the parameter is estimated freely. Mokhtari et al. (2018) reported that Modification Indexes greater than 10 implies an indication of misspecification for the respective parameter. In the present study, the candidates of the factor loading parameter for all aspects were statistically significant (P<.05).

Administration of this system needs a computer for every participant and interaction of the researcher (teacher) with the participants. However, this interaction is limited to the phases where discussion forums are selected by the participants. Although there is no limitation regarding the time of completion, an average of 15–20 minutes is fair for administration. The administration procedure is threefold. Firstly, the researcher (in this study, the teacher-researcher) should explain the use of the system and the researcher should enable the participants in the use of computer in case they have any problems. Secondly, all participants have to be directed to consider all the instructions and steps of reading that they find necessary. Finally, the participants should be encouraged to complete the task at their pace.

For scoring and Computerized RMSA System, the researchers need to conduct a content analysis of the trace interface. Although the researchers can carry out the scoring individually, they might also need to ask another colleague or researcher to analyze the results. This is especially achieved for generating reliable and valid results. The researchers of this study organized and prepared a scoring checklist for determining the types of metacognitive strategies used by the participants in a semi-direct manner (see Supplemental Table 2).

The interpretation of the information generated using this instrument is based on a taxonomy of RMSAU by O’Malley and Chamot (1990) and Oxford (1990). Three levels of reading metacognitive strategies which were coded in the scoring checklist would provide a tangible standard that can be used for interpretation of the scores. These scores can be arranged as indicating high, moderate and low levels of strategy awareness and use. Generally, the average score shows the frequency of the use of such strategies by the participants.

This Study

Although various researches on reading metacognitive awareness and regulation have been carried out to date, the very sense of semi-direct inventory and instruction of reading metacognitive strategies evokes further investigation. The studies to date have not rigorously investigated the indirect and semi-direct inventory methods of metacognitive strategy awareness; rather, they directly asked the learners to perform a questionnaire or talk aloud to show their strategy use and awareness. The researchers of this study aimed at indirect investigation of RMSA and its effects on general reading comprehension in reciprocal teaching setting which requires teacher’s collaboration with the learners. Hence, the present study was undertaken to bridge this research gap by formulating the following research questions:

What are the possible reading metacognitive strategies in reciprocal teaching model?

Does computerized RMSA in reciprocal teaching model have any effects on reading comprehension?

Method

Participants

Participants in a classroom discourse are usually the teachers and the learners. Language classrooms are no exceptions and every teacher and learner will hold the responsibility of teaching and learning. The participants of this study who were selected based on purposive and clustered sampling procedures were 25 university students majoring in English Translation and English Literature of Payamnoor University in Iran. The design of the study allowed for the participation of 7 male 18 female university students regardless of age and first language background. It is believed that the participants were suitable for the study for two distinct but related reasons. Firstly, they were adult language learners who could collaborate and cooperate with their teacher. Secondly, the students could reflect on the underlying process required for the task completion in the computerized RMSA inventory system.

Instruments

For the purpose of the study, the researcher used a number of instruments including (1) Preliminary English Test, (2) reading section of Test of English as a Foreign Language as pre and posttest, (3) a Metacognitive Strategy Awareness questionnaire, (4) think-aloud protocol, and (5) a Computerized RMSA System.

Firstly, the researcher used a 42-item standard Preliminary English Test with a ceiling score of 60 to select participants on the basis of their test scores and proficiency levels. Preliminary English Test is a standardized English proficiency test and assesses general English proficiency from beginners to advanced levels. The reliability of this test was calculated in SPSS using KR-21 and reliable test scores were obtained (r = 0.86). The second data collection instrument was extracted from the reading comprehension section of Test of English as a Foreign Language. The test was administered as pre and posttest to estimate any possible improvements in reading comprehension after using Computerized RMSA System.

Moreover, a 30-item RMSA questionnaire which was developed and validated by the researchers was used to measure participants’ RMSA (see Supplemental Table 3). The structure of the questionnaire was theoretically based on Oxford’s (1990) classification of three general types of metacognitive strategies; that is to say, centering your learning, arranging and planning, and evaluating, each with its sub-components. Furthermore, the reading dimension of this questionnaire was underpinned by Lee and VaPatten’s (1995) interactive model of reading. Additionally, its construction was based on Dornyei’s (2003) general and detailed instructions on constructing a questionnaire. Learners were supposed to choose an answer on a five-point Likert Scale ranging from never true to always true. It is important to note that the reliability of the researchers made questionnaire was calculated in SPSS using Cronbach’s Alpha Coefficient. As stated by Dornyei (2007), the best method of estimating the internal consistency of the items is using Cronbach’s Alpha. The Cronbach’s Alpha for the present questionnaire was r = 0.899, which indicates a high level of internal consistency for the items of the questionnaire.

Another data collection technique was the think-aloud protocol. A think-aloud protocol was applied to describe the participants’ thoughts while working on reading comprehension task. Since think-aloud protocols can provide rich insights on language learning strategies, they were also used by some researchers (O’Malley & Chamot, 1990; Vandergrift, 1997, 2003) to gain an understanding of the participants’ underlying processing. As stated by Vandergrift (2003), a think-aloud procedure includes two phases: a training phase and a data collection phase. Whereas the training phase aims at helping participants to know how to think aloud, the data collection phase deals with asking the participants to think aloud and collect data. To initiate the use of the think-aloud technique, participants completed a reading comprehension test extracted from Test of English as a Foreign Language (pretest) and verbalized their thought processes as they did so. Later on, the writing outcome was used as the data to be analyzed. The summary of the data generated from the think-aloud technique was categorized with regard to the key issues on factors affecting reading comprehension proposed by Oxford (1990) and O’Malley and Chamot (1994).

The most important instrument of the study was a Computerized RMSA System which is a computer-based program that includes an inside, outside and the trace interface. The system has already been described in the Introduction Section. The system’s theoretical basis and its subcategories together with other data collection techniques are presented in Table 1.

Table 1.

Subcategories of Three Reading Metacognitive Strategies Using Three Data Collection Techniques in Reading Through Reciprocal Teaching Setting.

		Cue elements in computerized RMSA system	Questions in questionnaire	Checklist for think-aloud
Centering	C1	Click the title	I preview the text title and general structure of the text before reading it	Previewing the text title
	C2	Click the question and instruction	I preview the question and instruction to understand what to do	Previewing the questions
	C3	In the dialogue box they can optionally predict the content	I usually predict the content of the text from the title	Predicting the content
	C4	Click the whole text to see it in 15 seconds	I look quickly through the text to get the main idea or gist.	Skimming
	C5	Either listen to music while reading or just read	I usually decide in advance to pay attention to task not other issues.	Paying attention to only reading
	C6	Highlight or underline while reading	I determine the major points I would pay attention to such as headings and sub-headings, the topic sentence and the text structure.	Paying attention to major points
	C7	Discuss the weak points with teacher	I recall my weak points in reading comprehension and try to comprehend when reading begins.	Think about my weak points in advance
Planning	P1	Discuss the reading problems with teachers	I talk about my reading problems and try to solve them by the aid of teachers or peers	Discussing the reading problems
	P2	Ask teacher questions	I’m not afraid of asking questions.	Asking questions
	P3	Discuss effective strategies with my teacher	I share ideas with teacher about effective strategies we have tried.	Discussing effective strategies
	P4	Click quick glance at text	I scan the text first and concentrate on what I will read.	Scanning
	P5	Click quick look at directions	I read the task before I read the text.	Reading the task before reading the text
	P6	Use notebook within the system	I keep a notebook to write important points on it.	Using a notebook
	P7	Check the time	I allocate some relaxation time to have well-concentration.	Allocating some time for relaxation
	P8	Click automatic versus manual arrangement	I plan before I read because I think it is helpful.	Planning before reading
	P9	Talks about the general content of the passage in advance	I try to guess the whole text.	Guessing the whole text
	P10	Use dictionary within the system	Whenever needed I use dictionary to find the meaning of unknown words.	Using dictionary
	P11	Introduces novel texts to the teacher	Whenever possible I practice reading English texts.	Reading new texts
	P12	Introduces stories	During free times, I usually read English novels.	Reading English novels
Evaluating	E1	Takes notes and discusses difficulties	I write down my significant difficulties in reading comprehension on a note book and try to eliminate them.	Taking notes and writing my difficulties
	E2	Traces left for every referring back to check mistakes	I check my own mistakes and errors and try to find solutions for them.	Checking my mistakes
	E3	Use notebook within the system to write my mistakes on	It is a good idea to use checklists to monitor my errors.	Monitoring my errors
	E4	Discussion with teacher for the difficult points	I enjoyed discussing with my teacher for the difficult points and ex-changing the reading experience to get a more effective reading method to achieve my goal.	Discussing difficult points with teacher
	E5	Use characteristics of good readers by introducing it to peers	I could be able to use the characteristics of a good reader as criteria to evaluate my own reading.	Talking about characteristics of good readers
	E6	Gives ideas about reading performance	I spent time reflecting on my reading performance.	Reflecting on my reading performance

Procedure of Data Collection

The data collection procedure of this study which lasted for 12 weeks, two hours per week started by introducing the methods and procedures of performing the actions in the system. Learners were taught to work with the program at the university computer lab and were advised that the results of the study were part of their mid-term exam. Then learners were given specific user names in order to keep the anonymity in the computerized system. The procedure of data collection was divided into three main stages based on the actions conducted during the study sessions.

Stage I of Data Collection

During the first session, the Preliminary English Test was used to assess the participants’ homogeneity of level (r = 0.82). After performing this test, the RMSA questionnaire was administered to measure participants’ reading metacognitive strategy awareness. During the second session, the reading comprehension section of Test of English as a Foreign Language was employed as the pretest to examine the participants reading comprehension. Immediately after this test, the participants were asked to undertake the think-aloud procedure. They engaged in think-aloud procedure about reading metacognitive strategies after the pretest to avoid its possible effects on pretest scores. This lasted 15 minutes without the learners being interrupted. In general, during the first session of the research, all the participants were trained to use the think-aloud protocol. The researcher used Ericsson and Simon’s levels of verbalization (1993, cited in Nunan & Bailey, 2009) to assist learners in verbalizing their thinking process.

Stage II of Data Collection

The second stage of data collection which lasted for 10 sessions was totally allocated to gathering data using the Computerized RMSA System. All the participants were taught how to use the program and throughout the study sessions, they were supposed to use the system as their class assignment. Here, the participants were encountered with three metacognitive strategies, i.e. centering, planning and evaluating that were embedded within the tasks. The three phases of the system which has already been described are the inside, outside and trace interface. All the interfaces were designed to indirectly find out English as a Foreign Language learners’ RMSA in reciprocal teaching settings. All the participants were indirectly involved in practical side of computerized MSA inventory system. The data collected using this system included participants’ detailed actions which were completely recorded in the trace interface of the system.

Stage III of Data Collection

After collecting the data at Stage II, the researchers administered a convenient reading comprehension test. The test was carried out using the same section of Test of English as a Foreign Language which was employed as the pretest. This was used as a posttest to estimate the effects of Computerized RMSA System on reading comprehension.

Procedure of Data Analysis

The collected data on the basis of the triangulated data collection procedure were analyzed at 3 steps. First, to examine the kind and frequency of RMSA, the data collected using questionnaire, think-aloud procedure and Computerized RMSA System were separately content analyzed. Based on the records of learners’ performance, the researchers coded the RMSA-related actions with reference to the category of metacognitive strategies proposed by Oxford (1990), O’Malley and Chamot (1990), and Mokhtari and Reichard (2002). In fact, they assigned each metacognitive strategy into proper reading metacognitive strategies. When using qualitative techniques, establishing inter-rater reliability is a recognized technique for ensuring the trustworthiness of the study; this is especially true when multiple researchers are involved with coding (McAlister et al., 2017). Here, the researchers used inter-rater reliability to ensure the reliability of the coding system. However, we considered reliability as an evolving system just as mentioned by Syed and Nelson (2015) to overcome the problems which arise by simply focusing on the final product of the coding system.

Then, the researcher described the metacognitive strategies and their use in reading through reciprocal teaching setting. In other words, the researcher interpreted the main ideas proposed based on the description of each reading metacognitive strategy elicited from the questionnaire, think-aloud protocol, and Computerized RMSA System. Finally, to scrutinize the possible effects of computerized RMSA in reciprocal teaching model on reading comprehension, the researchers employed a Matched t-test.

Data Analysis and Results

In this section, the researchers presented the results of analyses carried out to answer the two research questions. Generally, the results are presented as follows:

Students’ perception of strategy use generated using three strategy inventory systems are compared based on the findings of three inventory techniques.

Participants’ progress in reading comprehension which was taught using Computerized RMSA System is presented to scrutinize the effectiveness of the system. This is mainly aimed at analyzing learners’ reading comprehension in reciprocal teaching setting.

Learners’ Perception of Metacognitive Strategy Using Three Metacognitive Strategy Awareness Inventory Techniques

In order to examine and compare learners’ perception of metacognitive strategies using a RMSA Questionnaire, Think-aloud Protocol and Computerized RMSA System, the researcher estimated the frequency of the learners’ performance in three distinct techniques. Generally, both the qualitative data which were gathered using Computerized RMSA System and think-aloud techniques and the quantitative data which were obtained using RMSA Questionnaire were transferred to the scoring checklist prepared by the researcher. Interestingly, participants used distinct types of metacognitive strategies while they were involved in the inventory sessions. Notably, only some tokens of centering, planning and evaluating their learning were reported in the data gathered on the basis of Computerized RMSA System. However, regarding the RMSA questionnaire, the participants were willing to show their awareness in almost all aspects of metacognitive strategies. Also, think-aloud protocol yielded the clarification of only limited types of metacognitive strategies. The researchers summarized the findings of the study generated using three techniques in Table 2. In the case of Computerized RMSA System, trace was content analyzed and the frequency of learners’ use of reading metacognitive strategies was calculated. Considering the RMSA questionnaire, the researcher considered “agree” and “strongly agree” scales as instances of RMSA and use; thus, she added them up to find the frequency of the use of reading metacognitive strategies. Moreover, Think-aloud results were analyzed using the checklist prepared by the researcher and they are also reported in Table 2.

Table 2.

Summary of RMSA Generated Using Three Inventory Techniques in Reading Through Reciprocal Teaching Setting.

		Cue elements in computerized RMSA system		Questions in questionnaire		Checklist for think-aloud
		F	P	F	P	F	P
Centering	C1	16	64%	25	100%	7	28%
	C2	17	68%	23	92%	4	16%
	C3	13	52%	22	88%	9	36%
	C4	10	40%	23	92%	7	28%
	C5	19	76%	25	100%	0	00
	C6	8	32%	16	64%	19	76%
	C7	8	32%	15	60%	3	12%
Planning	P1	15	60%	6	24%	6	24%
	P2	12	48%	19	76%	4	16%
	P3	2	8%	17	68%	2	8%
	P4	18	72%	22	88%	13	52%
	P5	22	88%	22	88%	18	72%
	P6	4	16%	19	76%	5	20%
	P7	16	64%	19	76%	17	68%
	P8	12	48%	25	100%	6	24%
	P9	0	00	19	76%	0	00
	P10	17	68%	25	100%	21	84%
	P11	2	8%	16	64%	0	00
	P12	0	00	15	60%	0	00
Evaluating	E1	18	72%	18	72%	12	48%
	E2	24	96%	24	96%	17	68%
	E3	0	00	14	56%	0	00
	E4	19	76%	18	72%	6	24%
	E5	6	24%	18	72%	0	00
	E6	5	20%	20	80%	10	40%
Total		283	45.28%	482	77.12%	179	28.64%

The data gathered from the Computerized RMSA System revealed that evaluating one’s mistake was the most frequently identified type of reading metacognitive strategies used by the participants. A quick look at the directions was the next frequently used reading metacognitive strategies using Computerized RMSA System. The next frequent strategies recognized here were related to C5, i.e. “either listen to music or read without losing concentration” and E4, i.e. “discussion with the teacher for difficult points”. In fact, 76% of participants in this group used these reading metacognitive strategies. In the case of data gathered using RMSA Questionnaire, surprisingly except for one characteristic of reading metacognitive strategies, the others were selected by more than 50% of the participants. In other words, regarding C1, C5, P8 and P10, all participants selected maximum use of reading metacognitive strategies. Besides, C2, C3, C4, P4, P5, E2 and E6 attracted more than 80 percent of the participants. On the contrary, Think-aloud protocol did not generate frequent use of reading metacognitive strategies. In fact, except for C6, P4, P7, P10 and E2, other reading metacognitive strategies did generate less than 50% of reading metacognitive strategies. There were even some reading metacognitive strategies (C5, P8, P11, P12, E3, & E5) which had not been reported by the participants.

For the sake of valid comparison of the results of reading metacognitive strategies using three data collection techniques, i.e. Computerized RMSA System, RMSA Questionnaire and Think-aloud protocol, the researcher estimated the significance of difference using One-way Analysis of Variance. The descriptive statistics of RMSA generated using questionnaire, think-aloud and computerized system are shown in Table 3:

Table 3.

Descriptive Statistics for Comparison of Reading Metacognitive Strategies Using Three Data Collection Techniques in Reading Through Reciprocal Teaching Setting.

					95% confidence interval for mean
	N	Mean	Standard deviation	Standard error	Lower bound	Upper bound	Minimum	Maximum
Computerized RMSA system	25	11.32	7.39	1.47	8.26	14.37	.00	24.00
Questionnaire	25	19.40	4.45	.89	17.56	21.23	.00	25.00
Think-aloud protocol	25	7.44	6.73	1.34	4.65	10.22	6.00	21.00
Total	75	12.72	8.00	.92	10.87	14.56	.00	25.00

Also, the following table (Table 4) presents the information related to Analysis of Variance Test.

Table 4.

Analysis of Variance Test for Comparison of Reading Metacognitive Strategies Using Three Data Collection Techniques in Reading Through Reciprocal Teaching Setting.

	Some of squares	Df	Mean Square	F	Sig.
Between groups	1861.52	2	930.76	23.28	.000
Within groups	2877.60	72	39.96
Total	4739.12	74

A One-way Analysis of Variance Test indicated that there was a significant difference in RMSA and Use amongst Computerized RMSA System (M = 11.32, SD = 7.39), RMSA Questionnaire (M = 19.40, SD = 4.45) and Think-aloud Protocol (M = 7.44, SD = 6.73), F(2, 72) = 23.28, p < 0.05. The effect size was large (eta squared = 0.39). Turkey post-hoc test showed that the difference between questionnaire and Computerized RMSA System results was significant. The same significance of the difference was obtained regarding difference between RMSA Questionnaire and Think-aloud Protocol results in Reading through Reciprocal Teaching Setting. However, the two other groups, i.e. Computerized RMSA System and Think-aloud did not differ from each other significantly (see Table 5).

Table 5.

Post hoc Test for Reading Through Reciprocal Teaching Setting.

					95% confidence interval
(I) group	(J) group	Mean difference (I- J)	Std. Error	Sig.	Lower bound	Upper bound
System	Questionnaire	−8.08000*	1.78811	.000*	−12.3592	−3.8008
	think aloud	3.88000	1.78811	.083	−.3992	8.1592
Questionnaire	System	8.08000*	1.78811	.000*	3.8008	12.3592
	think aloud	11.96000*	1.78811	.000*	7.6808	16.2392
think aloud	System	−3.88000	1.78811	.083	−8.1592	.3992
	Questionnaire	−11.96000*	1.78811	.000*	−16.2392	−7.6808

The Effect of Computerized RMSA System on Reading Comprehension

Having collected the research data based on reading comprehension tests administered as pre and posttest, the researcher used the Statistical Package for Social Sciences version 20 to analyze the data. A Matched t-test was performed in order to compare reading comprehension of the participants both before and after implementing the Computerized RMSA System. In addition, significance of dispersion of choices (P<.0.05) and the mean of scores were defined.

The descriptive findings of Matched t-test indicate that pretest displayed a somewhat lower mean score for reading comprehension than posttest (20.60 versus 27.04) and the key question was whether this difference reached statistical difference. Then, we have carried out a Matched t-test to compare reading comprehension before and after using the Computerized RMSA System. The results are presented in Table 6:

Table 6.

Results of Matched t-Test for Comparing Reading Comprehension Before and After Using the Computerized RMSA System.

	Mean	Standard deviation	Standard error Mean	95% confidence interval of the difference		T	df	Sig. (2-tailed)
	Mean	Standard deviation	Standard error Mean	Lower	Upper	T	df	Sig. (2-tailed)
Pretest	20.6000	4.27200	.85440
Posttest	27.0400	5.35786	1.07157
Paired Differences	−6.44000	4.09349	.81870	−8.12971	−4.75029	−7.866	24	.000

The results show that there was a significant difference in scores for pretest (M = 20.60, SD = 4.27) and posttest (M = 27.04, SD = 5.35), t(24) = -7.86, p<.05, but the magnitude of the difference in the means was large (Cohen’s d = 1.33), with Computerized RMSA System explaining 13 per cent of the variance in reading comprehension.

Discussion

Despite variations in the views towards the nature of metacognitive strategy inventory instruments, the need for metacognitive strategy inventory has been acknowledged (Yang, 2010) most possibly owing to its direct and indirect influence on language achievement. However, developing a valid instrument and reliable scoring technique is of great concern to many, including Harrison and Vallin (2017).

The findings of this study revealed that unlike the questionnaire results which reported the use of almost all reading metacognitive strategies by the participants, the results of the think-aloud protocol did not generate a satisfactory amount of reading metacognitive strategies. As declared by Mackey and Gass (2005), “in all research that relies on participants’ giving information on their thought processes, one needs to be aware that participants may not be aware of their processes and/or they may not wish to reveal them” (p.85). The lack of metacognitive strategies revealed by think-aloud protocol might also be related to a type of problems suggested by Russo et al. (1989). Russo et al. (1989) pointed out that think-aloud protocols might lead to forgetting and sometimes fabrication of information, thus making it difficult for the learners to express their mental processes as they are. Learners’ thought processes might also be distorted more than necessary (Charters, 2003). Also questionnaires may have had higher reports of RMSs because the participants used strategies that were not detected by the computerized RMSA system. However, this might be the result of dishonest answers proposed by the participants. According to Mackey and Gass (2005), it is difficult to rely on responses to questions when an understanding of the instructions is vague and unclear. Thus, the use of an inventory system with indirect method of data collection would be helpful.

The studies to date have not explicitly used the indirect and semi-direct inventory techniques for studying RMSA and use. The researchers of this study introduced a Computerized RMSA System which can assist researchers and teachers to elicit Metacognitive Strategy Awareness without learners biased ideas which are very common when the questionnaires come to take a role. Also, unlike the think-aloud procedure which puts great energy on learners to actively think about their underlying processes and verbalize them, this system will indirectly and semi-directly lead language learners to generate Reading Metacognitive Strategies.

The findings emerging from the present study substantiated the positive effect of RMSA system on reading comprehension and emphasized the need for the use of semi-direct inventory techniques in the field of metacognitive strategy research. For the purpose of measuring learners’ awareness of metacognitive strategies, most researchers use distinct types of inventory techniques including questionnaire, observation, think-aloud protocols and interviews (Dinsmore et al., 2008; Harrison & Vallin, 2017; Winne & Perry, 2005). Although being the most controversial inventory technique, questionnaires tend to be the most typical data collection instruments (Harrison & Vallin, 2017). Even Harrison and Vallin (2017) only informed researchers of methods of scoring Metacognitive Awareness Inventory. In most studies, the data collection technique is a questionnaire with Likert scales (Adıgüzel & Orhan, 2017; Coskun, 2018; Kana, 2015). According to Harrison and Vallin (2017), Metacognitive Awareness Inventory survived the field of metacognitive strategy research because of their convenience. Experience has shown that relying on an instrument just for its practicality is not logical. Thus, the results of this study can notably present valuable solutions to overcome the shortcomings generated by questionnaires or think-aloud protocols.

The findings regarding the effectiveness of Computerized RMSA System on reading comprehension led support to those of Al-Alwan (2012), Ghaith and El-Sanyoura (2019), Ismaiel and Tawalbeh (2015), and Jayapraba(2013) who reported reading comprehension of language learners increased as a result of teaching metacognition reading strategies. Typically, Jayapraba(2013) was concerned with science classes where metacognitive strategy instructions were most effective in enhancing academic achievement. The study findings are in line with what has been reported by Carretti et al. (2014) who found a positive influence of metacognitive strategy training on reading comprehension.

Although there are a number of studies (Asikcan & Saban, 2018; Dabarera et al., 2014; Furnes & Norman, 2015; Meniando, 2016) which highlighted the positive influence of metacognitive strategy training on reading comprehension, interested researchers should not underestimate other studies with contradictory results. Mehrpour et al. (2012) reported that although learners’ awareness of reading metacognitive strategies increased, their reading performance did not specially and significantly increase as a result of metacognitive strategy instruction. Moreover, Shang (2016) did not find any relationship between strategy instruction or use and reading comprehension.

Conclusion

The results of the qualitative investigation showed that participants’ perception of and verbalizing reading metacognitive strategies seemed to be overestimated in questionnaire results. In comparison, the think-aloud protocol elicited fewer reading metacognitive strategies used by Iranian learners of English as a Foreign Language. The interesting fact refers to the results obtained from the Computerized RMSA System because the participants’ use of reading metacognitive strategies was examined indirectly. Also, a One-way Analysis of Variance showed that there was a significant difference in RMSA and use amongst Computerized RMSA System, RMSA Questionnaire and Think-aloud Protocol, F(2, 72) = 23.28, p < 0.05. Turkey post-hoc test showed that the difference between questionnaire and Computerized RMSA System results was significant. The same significance of the difference was obtained regarding the difference between RMSA Questionnaire and Think-aloud Protocol results. However, the two other groups, i.e. Computerized RMSA System and Think-aloud did not differ from each other significantly. Meanwhile, the Matched t-test analyses of reading comprehension scores after 10 sessions of treatments using Computerized RMSA System revealed a statistically significant difference in the pre and posttest scores.

This study is helpful in the context of foreign language learning; thus, all the characters of the teaching and learning context who are somewhat related to the foreign language teaching situation will benefit from the results of this research. To put it directly, the researcher believes that this study will be beneficial to all the main participants of the language learning and teaching context, i.e. teachers, learners, and other stakeholders in the realm of foreign language acquisition.

In the case of Iranian learners, the researcher is quite certain that this study will bring about some direct advantages by improving their reading comprehension. As language learners are the primary elements of language teaching and learning system, all research activities should provide fruitful implications for them. Presumably, language learners should know in-depth what reading and writing metacognitive strategies are and how such strategies do assist learners to achieve acceptable results. This study was also very important to language learners in that they can implicitly find out about the strategies they use while performing reading and writing tasks.

Like language learners, language teachers are the other major elements in the language teaching and learning system. Meanwhile, the English teachers are invited to reconsider their attitudes to the importance of metacognitive strategies, reciprocal learning, and teaching. The findings stressed the learners’ need for teacher’s indirect implementation of metacognitive strategies at least in the Iranian context. We, as teachers, cannot leave out students on their own to learn without any attention to metacognitive strategy use. What else teachers need to consider is their valuable and effective role in the effectiveness of providing support for learners.

This study focused on metacognitive strategies as remedial strategies for practicing reading comprehension of Iranian learners of English as a Foreign Language; it would be interesting to examine the effect of such cognitive and affective strategies on reading comprehension. Furthermore, there is enough space for carrying out longitudinal studies to find the effect of metacognitive strategies on other language skills. That is, the interested researchers can examine the general and specific effects of reading metacognitive strategies on writing, listening, and even speaking skill. The same study can be carried out with sub-skills such as vocabulary and grammar. Besides, there is the possibility of investigating the study with more than one group.

Supplemental Material

sj-pdf-1-jec-10.1177_0735633120937437 - Supplemental material for Reading-Metacognitive Strategy Awareness and Use in Reciprocal Teaching Settings: Implementing a Computerized RMSA System

Supplemental material, sj-pdf-1-jec-10.1177_0735633120937437 for Reading-Metacognitive Strategy Awareness and Use in Reciprocal Teaching Settings: Implementing a Computerized RMSA System by Elnaz Reshadi-Gajan, Nader Assadi and Hanieh Davatgari Asl in Journal of Educational Computing Research

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Elnaz Reshadi-Gajan is a PhD candidate in TEFL. Her research interests lie in the areas of individual differences, cooperative and collaborative learning and assessment, and metacognitive strategy inventory and instruction.

Nader Assadi received his PhD degree in Applied Linguistics from the Universiti Sains Malaysia, in 2011. Dr Assadi Aidinlouhas published two books, and presented and published tens of papers in international conferences and journals. He has supervised many MA and PhD theses. His main areas of interest are Systemic Functional Linguistics & Grammar, Discourse analysis, Reading & Official Translation.

Hanieh Davatgari Asl received her PhD degree in TEFL from the Islamic Azad University-Tabriz, in 2013. Dr Davatgari Asl is has presented and published several papers in international conferences and journals. She has supervised many MA and PhD theses.

ORCID iD

Elnaz Reshadi-Gajan

Supplemental Material

Supplemental material for this article is available online.

References

Adıgüzel

Orhan

(2017). The relation between English learning students’ levels of self-regulation and metacognitive skills and their English academic achievements, Journal of Education and Practice, 8(9), 115–125.

Al-Alwan

A. F.

(2012). The effect of using metacognition reading strategies on the reading comprehension of Arabic texts. International Journal of Applied Educational Studies, 13(1), 2.

Al-Jarreh

J. M. M.

Mansor

Talafhah

R. H.

Al-Jarreh

J. M.

(2018). A review of research into metacognitive strategies in the process of education: Teaching and learning of English skills. International Journal of Current Research, 10(12), 76832–76838.

Anderson

N. J.

(1991). Individual differences in strategy use in second language reading and testing. Modern Language Journal, 75, 460–472. https://doi.org/10.1111/j.1540-4781.1991.tb05384.x

Anderson

N. J.

(2002). The role of metacognition in second/foreign language teaching and learning. ERIC Digest. ERIC Clearinghouse on Languages and Linguistics.

Anderson

R. C.

Pearson

P. D.

(1984). A schema-theoric view of basic processes in reading comprehension. In P. Pearson

(Ed.), Handbook of reading research (pp. 255–291). Longman.

Asikcan

Saban

(2018). Prospective teachers’ metacognitive awareness levels of reading strategies. Cypriot Journal of Educational Sciences, 13(1), 23–30.

Azevedo

Hadwin

A. F.

(2005). Scaffolding self-regulated learning and metacognition—Implications for the design of computer-based scaffolds. Instructional Science 33(2005), 367–379.

Bannert

(2006). Effects of refection prompts when learning with hypermedia. Journal of Educational Computing Research, 35(4), 359–375.

10.

Becirovic

Brdarevic-Celjo

Sinanovic

(2017). The use of metacognitive reading strategies among students at International Burch University: A case study. European Journal of Contemporary Education, 6, 645–655.

11.

Berg

(2009). Qualitative research methods for the social sciences. Allyn & Bacon.

12.

Bishara

Kaplan

(2018). The relationship of locus of control and metacognitive knowledge of math with math achievements. International Journal of Disability, Development and Education, 65(6), 631–648.

13.

Boulware-Gooden

Carreker

Thornhill

Joshi

R. M. M.

(2007). Instruction of metacognitive strategies enhances reading comprehension and vocabulary achievement of third-grade students. The Reading Teacher, 61(1), 76–77.

14.

Carretti

Caldarola

Tencati

Cornoldi

(2014). Improving reading comprehension in reading and listening settings: The effect of two training programmes focusing on metacognition and working memory. British Journal of Educational Psychology, 84(2), 194–210.

15.

Catts

H. W.

(2018). The simple view of reading: Advancements and false impressions. Remedial and Special Education, 39(5), 317–323.

16.

Chamot

A. U.

(2004). Issues in language learning strategy research and teaching. Electronic Journal of Foreign Language Teaching 1(1), 14–26.

17.

Chamot, A.U. & El-Dniary, P.B. (1999). Children’s learning strategies in immersion classrooms. The Modern Language Journal 83(3), 319–341.

18.

Chapelle

(1996).Validity issues in computer-assisted strategy assessment for language learners. http://lib.dr.iastate.edu/engl_pubs/56

19.

Charters, E. (2003). The use of think-aloud methods in qualitative research: an introduction to think-aloud methods. Brock Education 12(2), 68–82.

20.

Cohen, A.D. (1998). Strategies in learning and using a second language. Longman.

21.

Coskun

(2010). The effect of metacognitive strategy training on the listening performance of beginner students. Novitas-Royal, 4(1), 35–50.

22.

Coskun

(2018). A study on metacognitive thinking skills of university students. Journal of Education and Training Studies, 6(3), 38–46.

23.

Cox

Clark

(1998). The use of formative quizzez for deep learning. Computers and Education, 30(3), 157–167.

24.

Curwen

M. S.

Miller

R. G.

White-Smith

K. A.

Calfee

R. C.

(2010). Increasing teachers’ metacognition develops students’ higher learning during content area literacy instruction: Findings from the read-write cycle project. http://digitalcommons.edu/education_articles/6

25.

Dabarera

Renandya

W. A.

Zhang

L. J.

(2014). The impact of metacognitive scaffolding and monitoring on reading comprehension. System, 42, 462–473.

26.

Dagoc

D. A.

Tan

D. A.

(2018) . Effects of metacognitive scaffolding on the mathematics performance of grade 6 pupils in a cooperative learning environment. International Journal of English and Education, 7(4), 378–391.

27.

Dardjito

(2019). Students’ metacognitive reading awareness and academic English reading comprehension in EFL context. International Journal of Instruction, 12(4), 611–624.

28.

Dinsmore

D. L.

Alexander

P. A.

Loughlin

S. M.

(2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20, 391–409. https://doi.org/10.1007/s10648-008-9083-6

29.

Dixon-woods

Shaw

R. L.

Agarwal

Smith

J. A.

(2004). The problem of appraising qualitative research. Quality and Safety in Healthcare, 13, 223–225.

30.

Djudin

(2017). Using metacognitive strategies to improve reading comprehension and solve a word problem. Journal of Education Teaching and Learning, 2(1), 124–129.

31.

Dornyei

(2003). Questionnaires in second language researches. Lawrence Erlbaum Associates.

32.

Dornyei

(2007). Research methods in applied linguistics. Oxford University Press.

33.

Duffy, S.A., Morris, R.K., & Rayner, K. (1988). Lexical ambiguity and fixation times in reading. Journal of Memory and Language 27, 429–446.

34.

Elleman

A. M.

Oslund

E. L.

(2019). Reading comprehension research: Implications for practice and policy. Policy Insights from the Behavioral and Brain Sciences, 6(1), 3–11.

35.

Ellis

A. K.

Denton

(2010). Teaching, learning and assessment together: Reflective assessments for middle and high school mathematics and science. Eye on Education.

36.

Ericsson

K. A.

Simon

H. A.

(1980). Verbal reports as data. Psychological Review, 87(3), 215–251.

37.

Fazeli

S. H.

(2012). Use of metacognitive English language learning strategies based on personality trait. Theory and Practice in Language Studies, 2(3), 531–539.

38.

Fitrisia

Tan

K. E.

Yusuf

Y. Q.

(2015). Investigating metacognitive awareness of reading strategies to strengthen students’ performance in reading comprehension. Asia Pacific Journal of Educators and Education, 30(1), 15–30.

39.

Flavell

J. H.

(1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911.

40.

Furnes

Norman

(2015). Metacognition and reading: Comparing three forms of metacognition in normally developing readers and readers with dyslexia. Dyslexia, 21(3), 273–284.

41.

Gavora

Vaculikova

, Kalenda, J. Kalman, O., Gombos, P., Swigost, M., & Bontova, A. (2019). Comparing metacognitive reading strategies among university students from Poland, Hungry, Slovakia and Czech Republic. Journal of Further and Higher Education. Advance online publication. https://doi.org/10.1080/0309877X.2019.1614545

42.

Gersten

Fuchs

L. S.

Williams

J. P.

Baker

(2001). Teaching reading comprehension strategies to students with learning disabilities: A review of research. Review of Educational research, 7(1), 279–320.

43.

Ghaith

El-Sanyoura

(2019). Reading comprehension: The mediating role of metacognitive strategies. Reading in a Foreign Language, 31(1), 19–43.

44.

Haider

A. H.

Al-Naqabi

A. K.

(2008). Emirati high school students’ understandings of stoichiometry and the influence of metacognition on their understanding. Research in Science and Technological Education, 26(2), 215–237.

45.

Harrison

G. M.

Vallin

L. M.

(2017). Evaluating the Metacognitive Awareness Inventory using empirical factor-structure evidence. Metacognition and Learning. Advance online publication. https://doi.org/10.1007/s11409-017-9176-z

46.

Heinrich

Wang

(2003). Online marking of essay-type assignments. Proceedings of the World Conference on Educational Multimedia Hypermedia and Telecommunications (Conference session). AACE.

47.

Iserbyti

Elen

Behets

(2010). Instructional guidance in reciprocal peer tutoring with task cards. Journal of Teaching in Physical Education, 29(1), 38–53.

48.

Ismaiel

Tawalbeh

(2015). Effectiveness of a metacognitive reading strategies program for improving low achieving EFL readers. International Education Studies, 8(1), 71–87. https://doi.org/10.5539/ies.v8n1p71

49.

Jacobs

J. E.

Paris

S. G.

(1987). Children’s metacognition about reading: Issues in definition, measurement, and instruction. Educational Psychologist, 22(3), 255–278.

50.

Jayapraba

(2013). Metacognitive instruction and cooperative learning-stratgeis for promoting insightful learning in science. International Journal on New Trends in Education and Their Implications, 4(1), 165–172.

51.

Joreskog, K.G., & Sorbom, D. (1979). Advances in factor analysis and structural equation models. Abe Books.

52.

Kana

(2015). Türkçe öğretmeniadaylarının üstbiliş farkındalık düzeyleri [Investigation the effect of metacognition training on distance education students’ metacognitive awareness levels], Akademik Sosyal Araştırmalar Dergisi, 3(17), 66–81.

53.

Kistner

Rakoczy

Otto

Dignath

(2010). Promotion of self-regulated learning in classrooms: Investigating frequency, quality, and performance. Metacognition and Learning, 5(2), 157–171.

54.

Lee

J. G.

VaPatten

(1995). Making communicative language teaching happen. McGraw-Hill.

55.

Leon-Guerrero

(2008). Self-regulation strategies used by student musicians during music practice. Music Education Research, 10(1), 91–106.

56.

Leung

(2015). Validity, reliability, and generalizability in qualitative research. Journal of Family Medicine and Primary Care, 4(3), 324–327.

57.

Leutwyler

(2009). Metacognitive learning strategies: Differential development patterns in high school. Metacognition and Learning, 4(2), 111–123.

58.

Kam

W. K.

(2011). Mosston’s reciprocal style of teaching: A pilot study in Hong Kong. New Horizons in Education, 29(2), 27–37.

59.

Lincoln

Y. S.

Lynham

S. A.

Guba

E. G.

(2011). Paradigmatic controversies, contradictions, and emerging confluences, revisited. In N. K. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (pp. 97–128). SAGE.

60.

Liu

Sung

Yao

(2014). Computer, intelligent computing and education technology. CRC Press.

61.

Lunenberg

Korthagen

Swennen

(2007). The teacher educator as a role model. Teaching and Teacher Education, 23(5), 586–601.

62.

Lv.

Chen

(2010). A study of metacognitive-strategies-based writing instruction for vocational college students. English Language Teaching, 3(3), 136–144.

63.

Mackey

Gass

S. M.

(2005). Second language research: Methodology and design. Lawrence Erlbaum Associates.

64.

MacNamara

D. S.

(2011). Measuring deep, reflective comprehension and learning strategies: Challenges and successes. Metacognition and Learning, 6, 195–203.

65.

Marzano

R. J.

Pickering

D. J.

Pollock

J. E.

(2001). Classroom instruction that works: Research-based strategies for increasing student achievement. ASCD.

66.

McAlister

A. M.

Lee

D. M.

Ehlert

K. M.

Kajfez

R. L.

Faber

C. J.

Kennedy

M. S.

(2017). Qualitative coding: An approach to assess inter-rater reliability [Paper presentation]. 124th American Society for Engineering Education Annual (ASEE) Conference and Exposition, Columbus, OH, United States.

67.

McBride

D. M.

Dosher

B. A.

(2002). A comparison of conscious and automatic memory processes for picture and word stimuli: A process dissociation analysis. Consciousness and Cognition, 11, 423–460.

68.

McLain

K. V.

Gridley

B. E.

McIntosh

(1991). Value of a scale used to measure metacognitive reading awareness. The Journal of Education Research, 85(2), 81–87.

69.

Mehrpour

Sadighi

Bagheri

(2012). Teaching reading comprehension strategies to Iranian EFL pre-university students. The Journal of Teaching Language Skills (JTLS), 4(1), 107–139.

70.

Meniando

J. C.

(2016). Metacognitive reading strategies, motivation and reading comprehension performance of Saudi EFL students. English Language Teaching, 9(3), 117–129.

71.

Metcalfe

Shimmura

A. P.

(1994). Metacognition: Knowing about knowing. Massachusetts Institute of Technology Press.

72.

Miholic, V. (1994). An inventory to pique students’ metacognitive awareness of reading strategies. Journal of Reading 38, 84–86.

73.

Mokhtari

Dimetrov

D. M.

Reichard

C.A.

(2018). Revising the Metacognitive Awareness of Reading Strategies Inventory (MARSI) and testing for factorial invariance. Studies in Second Language Learning and Teaching, 8(2), 219–246.

74.

Mokhtari

Reichard

C. A.

(2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psychology, 94(2), 249–259

75.

Mokhtari

Sheorey

(2002). Measuring ESL students’ awareness of reading strategies. Journal of Developmental Education, 25(3), 2–10.

76.

Mosston

Ashworth

(2002). Teaching physical education. Benjamin Cummings.

77.

Nunan

Bailey

K. M.

(2009). Exploring second language classroom research. Cengage Learning.

78.

Olson

G. M.

Duffy

S. A.

Mack

R. L.

(1984). Thinking-out-loud as a method for studying real-time comprehension processes. In D. Kieras & M. Just (Eds.), New methods in reading comprehension research (pp. 253–286). Erlbaum.

79.

O’Malley

A. U.

Chamot

J. M.

(1994). The CALLA handbook: Implementing the cognitive academic language learning approach. Addison Wesley Longman.sss

80.

O’Malley

J. M.

Chamot

A.U.

(1990). Learning strategies in second language acquisition. Cambridge University Press.

81.

Oxford

(1990). Language learning strategies: What every teacher should know. Newbury Publishers’ House.

82.

Palincsar

A. S.

Brown

A. L.

(1984). Reciprocal teaching of comprehension-fostering and comprehension monitoring activities. Cognition and Instruction, 1(2), 117–175.

83.

Paterson

A. H.

(2002). What has QTL mapping taught us about plant domestication? New Phytologist, 154(3), 591–608.

84.

Pereira-Laird

J. A.

Deane

F. P.

(1997). Development and validation of a self-report measure of reading strategy use. Reading Psychology, 18(3), 185–235.

85.

Pressley

(2000). What should comprehension instruction be the instruction of? In Kamil

M. L.

Mosenthal

P. B.

Pearson

P. D.

Barr

(Eds.), Handbook of reading research. Erlbaum (pp. 545–561).

86.

Pressley

Afflerbach

(1995). Verbal protocols of reading: The nature of constructively responsive reading. Routledge.

87.

Rahman

(2016). The advantages and disadvantages of using qualitative and quantitative approaches and methods in language “testing and assessment” research: A literature review. Journal of Education and Learning, 6(1), 102–112.

88.

Ravitch

(2010). The death and life of the great American school system: How testing and choice are undermining education. Basic Books.

89.

Razak

N. Z. A.

Gani

N. A. A.

Ithnin

N. H. C.

(2018). Reading metacognitive strategies employed by ESL learners. International Journal of Education and Multidisciplinary Studies, 12(3), 61–73.

90.

Reid

(2002). Designing online quiz questions to assess a range of cognitive skills. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications (Conference session). AACE.

91.

Richards

J. C.

Lockhart

(1996). Reflective teaching in second language classrooms. Cambridge University Press.

92.

Rosenblatt

L.M.

(1978). The reader, the text, the poem: The transactional theory of the literary work. Southern Illinois University Press

93.

Rosenblatt, L.M. (1986). The aesthetic transaction. Journal of Aesthetic Education 20, 122–127.

94.

Russo

J. E.

Johnson

E. J.

Stephens

D. L.

(1989). The validity of verbal protocols. Memory and Cognition, 17, 759–769.

95.

Sallee

M. W.

Flood

J. T.

(2012). Using qualitative research to bridge research, policy, and practice. Theory Into Practice, 51(2), 137–144.

96.

Sarac

Karakelle

(2012). On-line and off-line assessment of metacognition. International Electronic Journal of Elementary Education, 4(2), 301–315.

97.

Schellings

(2011). Applying learning strategy questionnaire: Problem and possibilities. Metacognition and Learning, 6, 91–109.

98.

Schmitt

M. C.

(1990). A questionnaire to measure children’s awareness of strategic reading processes. The Reading Teacher, 43(7), 454–461.

99.

Selamat

Sidhu

G. K.

(2013). Enhancing listening comprehension: The role of metacognitive strategy instruction (MetSI). Procedia—Social and Behavioral Sciences, 90(2013), 421–430.

100.

Shang, H.F. (2016). Online metacognitive strategies, hypermedia annotations, and motivation on hypertext comprehension. Journal of Educational Technology & Society 19(3), 321–334.

101.

Sheikh

Soomro

K. A.

Hussain

(2019). Metacognitive awareness of reading strategies, reading practice and academic attainment of university students. Journal of Education and Educational Development, 6(1), 126–137.

102.

Sim

Holifield

Brown

(2004) Implementation of computer assisted assessment: Lessons from the literature. Association for Learning Technology Journal, 12(3), 215–229.

103.

Syed

Nelson

S. C.

(2015). Guidelines for establishing reliability when coding narrative data. Emerging Adulthood, 3(6), 375–387.

104.

Trapman

van Gelderen

van Schooten

Hulstijn

(2017). Reading comprehension level and development in native and language minority adolescent low achievers: Roles of linguistic and metacognitive knowledge and fluency. Reading and Writing Quarterly, 33(3), 239–257.

105.

Tsai

(2009). The model of strategic e-learning: Understanding and evaluating student e-learning from metacognitive perspective. Educational Technology and Society, 12(1), 34–48.

106.

Turel

McKenna

(2013). Design of language learning software. In B. Zou, M. Xing, C. H. Xiang, Y. Wang, & M. Sun (Eds.), Computer-assisted foreign language teaching and learning: Technological advances (pp. 188–209). Hershey.

107.

Vandergrift

(1997). The comprehension strategies of second language (French) listeners: A descriptive study. Foreign Language Annals, 30, 387–409. https://doi.org/10.1111/j.1944-9720.1997.tb02362.x

108.

Vandergrift

(2003). Orchestrating strategy use: Toward a model of the skilled second language listener. Language Learning, 53(3), 463–496. https://doi.org/10.1111/1467-9922.00232

109.

Veenman

M. V. J.

Hout-Wolters

B. V.

Afflerbach

(2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3–14

110.

Vygotsky

L. S.

(1978). Mind in society: The development of higher psychological processes. Harvard University Press.

111.

Wang

Spencer

Xing

(2009). Metacognitive beliefs and strategies in learning Chinese as a foreign language. System, 37(1), 46–56.

112.

Ward

P. C.

Lee

(2005). Peer-assisted learning in physical education: A review of theory and research. Journal of Teaching in Physical Education, 24(3), 205–222.

113.

Winne

P. H.

Perry

N. E.

(2005). Measuring self-regulated learning. In Boekaerts

Pintrich

P. R.

Zeidner

(Eds.), Handbook of self-regulation (pp. 531–566). Elsevier Academic Press.

114.

Yang

(2010). Developing a reciprocal teaching/learning for collegue remedial reading instruction. Computers and Education, 55(10), 1193–1201.

115.

Young

T. J.

(2015). Questionnaires and Surveys. In Hua

(Eds.), Research methods in intercultural communication: A practical guide (pp.163–180). Willey Blackwell.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB