Abstract
Providing an active learning environment and engaging students in classroom discussion could be quite challenging. This study used a quasi-experimental method to manipulate students’ incentives for participation in group discussion to investigate its impact on students’ learning outcomes. Two research methods courses taught over four years by the author were examined. Eighty samples of students’ online discussions were recorded, transcribed, and analyzed to assess the impact of four different strategies on quantity, quality, and outcome of students’ group discussions. The results showed significant differences in all aspects. The implications of the results for teachers who plan to use group discussion in their courses are discussed and suggestions for future research are offered.
Current issues in online teaching
Realizing the limitations of lecture-based instruction, educators have been looking for more effective ways to help students to play an active role in learning. Providing an active learning environment and engaging students in classroom discussion has been a challenge for educators (Rezaei, 2020; Clinton and Kelly, 2020). One of the main challenges in online teaching is lack of students’ active participation in online discussions (Morawo et al., 2020). Particularly, since the onset of the pandemic, and the migration of traditional courses to online modes of instruction, the challenge has become even more clear. The impersonal nature of the virtual learning environment creates a tendency for detachment and disengagement.
For years, the most common teaching strategy used in classrooms has been lectures. However, lectures are usually considered a one-way communication and lack many of the components of active learning such as critical thinking, self-pacing, and the encouragement of dialog and group discussion (Fredrick and Hummel, 2004). In online courses, lectures are even less desirable. Particularly, faculty struggle with teaching courses in areas such as the social sciences, humanities, and education. These are areas where students are expected to be actively and intellectually engaged in learning, develop attitudes toward the topic, improve their writing and research skills, and become lifelong learners (Langley and Guzey, 2014). Past research uncovers the need for more active learning environment and more engaging teaching strategies.
There are research studies supporting the idea that collaborative methods are helpful in this regard no matter what course is taught (Bennett, 2015). Most of these studies have focused more on self-report measures of engagement, likability, and benefits, rather than a direct measure of the students’ engagement or a robust measure of the outcome of the group discussion (Flosason et al., 2015). In addition, earlier studies in collaborative learning and group work have been mainly conducted in face to face or traditional education (Mutrofin et al., 2017). Researchers who have studied group discussions in online courses have mostly used descriptive or correlational research methods through surveys. Some other investigators have used qualitative methods to explain or to explore group dynamics or students’ feelings in online courses (Bennett, 2015).
The goal of the present study was to examine which of the four ways of conducting online discussion makes it more interactive and more productive. The author aimed to understand how one can improve the quantity and quality of students’ engagement as well as the outcome of students’ group discussion.
Research on collaborative learning
The merits of collaborative learning for adult learners have been well documented in the literature (Dallmer, 2004; Gayman and Jimenez, 2020). The range of benefits includes engaging students in active learning, promoting self-understanding, encouraging critical thinking, improving interpersonal skills, increasing motivation, and learning about teamwork. Therefore, group discussion, as a form of collaborative learning, has become one of the most popular teaching strategies, particularly, in online education and higher education (Choi et al., 2001). As Dixson (1991) noted, group discussion improves discussion skills, raises motivation, and helps students sharpen and test the validity of their ideas. It also helps students develop teamwork skills such as encouraging others, careful listening, goal setting, and gaining the feeling of acceptance and belonging. Educators have mentioned that a vast majority of today’s students will ultimately find works in which they have to interact with other people (Hurren et al., 2006). It is believed that among the four communication skills writing, reading, listening, and speaking, speaking is intuitively the most important and hence teachers should have a special focus on group discussion among students (Tan et al., 2020). Students working in groups, experience increased social support, report higher satisfaction with their learning, and learn better than students working as individuals (Johnson et al., 2007). Collaborative learning procedures have also been shown to enhance student satisfaction with the learning and classroom experience (Rezaei 2015, 2017).
Considering the benefits of collaborative learning, group work, and group discussions, educators have long been looking for best practices in the field. Evidently, not all earlier practices have been successful (Author 2015, 2017). There are some negative reports and most of the negative reports have come from the areas of physical or medical sciences where students work together on a well-defined project and have specific goals to reach. For example, Qamar et al. (2015) reported that “medical students’ discussion intervention” showed poor results in terms of their mean scores in their final professional exam, and their pass rate, and in terms of their perceptions of the course. These authors also reported the results of other studies with similar conclusions, employing self-report evaluations (e.g. Bennett, 2015). Similarly, Flosason et al.’s (2015) study did not show any clear advantages of small-group discussion in terms of learning outcomes. They claimed that the difference in achievement scores in other studies using group discussion could be attributed to a more effective instructor, not the difference between the lecture format and group discussion. They even questioned the validity of the famous research of collaborative learning performed by Crouch and Mazur (2001). Crouch and Mazur had collected years of data to compare lectures with peer instruction or small group discussion methodology and concluded that students in the “peer instruction” method had performed significantly higher than the traditional method. However, Flosason et al. (2015) suggested that the difference might have resulted because more knowledgeable students provided the correct answers to less knowledgeable students.
Furthermore, some educators have claimed that in collaborative activities, some students might slack off or not participate or contribute to the discussions (Bacon, 2005). This is particularly possible if students have a specific partner for the whole semester. It is possible that some students take advantage of their partners and contribute less than expected. Another possible reason for the lack of students’ participation in online discussions is that students do not know what to ask or how to ask questions (Choi et al., 2001). They report that students may refrain from or are hesitant to ask questions in front of their peers, perhaps because of the fear of making mistakes or embarrassing themselves in front of a large audience. As a result, student can neither ask the right questions nor generate productive feedback. Finally, it has been reported that some cultures discourage commenting about other people face to face, especially when there is an important relationship, such as a friendship or a classroom relationship, to be maintained (Jong et al., 2013). More importantly, lack of verbal participation (silence) in group discussion cannot not always be interpreted as lack or of active learning (Masek et al., 2021; Remedios et al., 2008).
This lack of consensus among researchers about the effectiveness of group work and group discussion, raises an important question to be investigated in the current study. The question is how or under what conditions this strategy could lead to more constructive group discussion and better learning outcomes. There have been some attempts to identify factors that improve the quality of students’ group discussion. Some have compared different group sizes, and others have compared online with face-to-face group discussions and other have focused on the role of instructor (Omidire, 2022) or the role of rehearsal for group discussion (Stroud, 2021). One major challenge that has been addressed in the literature, is teachers’ difficulty in assessing the contribution of each group member in a group discussion (Handayani et al., 2019). A main goal of this study was to examine if teacher’s assessment strategies impacted students’ participation in group discussion.
Theoretical framework and research questions
The main research question in the present study is which assessment guideline leads to a more meaningful discussion and better outcome. Researchers have suggested various ways to increase students’ engagement in group discussion or to improve the quality of students’ interactions (Gayman and Jimenez, 2020). Traditionally, the author has followed the guidelines suggested in educational technology textbooks (e.g. Boettcher and Conrad, 2016) recommending the importance of designing grading criteria for students’ participation in group discussion. Boettcher and Conrad (2016) suggested that clear guidelines about what is expected of learners could make a significant contribution to ensuring understanding and satisfaction. Similarly, Horton (2012) recommended that instructors should establish codes of conduct in advance, and should communicate expectations clearly to all students. He suggested that students should receive credit for specific actions that they take in a group discussion. He specifically recommended five criteria to be used in assessment of students’ participation and contributions in group discussions. The factors include substance, timing, originality, tone, and persuasiveness.
In recent years, a substantial body of work has been produced dealing with interaction in language testing, including conversation analysis, as well as other kinds of discourse analysis. The assessment of test-takers’ conversational competence has recently gained a lot of attention in language tests (e.g. Cambridge First Certificate, Cambridge Certificate of Proficiency, College English Test-Spoken English Test, and Public English Test Systems in China). College English Test–Spoken English Test (CET–SET) is a national oral proficiency test which is used for non-English majors in China to validate the match between intended and actual test-taker language. He and Dai (2006) have created a checklist of Interactional Language Functions (ILFs) to evaluate students’ discussions. While IFLs have been used to assess students’ language proficiency, it has yest not been used to measure the quality and the outcome of group discussions in another subject matter.
After several years of using a simple online form (called traditional rubric in this study) to evaluate the amount of students’ participation, the author switched to the implementation of a new set of criteria suggested by Horton (2012) to evaluate both the quantity and the quality of students’ participation since it provides more specific criteria. However, after 1 year of using Horton’s criteria, the author decided to compare it with the more specific IFL criteria to find out which one leads to more interaction, higher quality of interaction, and better learning outcomes.
Considering the difference in the levels of specificity among the traditional assessment criteria, Horton’s guidelines, and IFL’s criteria, the main question in this study is to find out which one improves students’ engagement and their performance in online courses. A second goal of this study was to compare video-based group discussion using video conferencing tools with text-based group discussion using discussion boards. Based on these two questions and regarding the literature the following hypotheses will be tested in this study.
1- Providing students with more specific guidelines (Horton’s and IFL’s criteria) would lead to more engagement and deeper level of conversations among students.
2- Video-based group discussions would lead to more engagement and deeper level of conversations among students in comparison with the text-based online discussion.
Methodology
Procedures
The present study used a quasi-experimental method (nonequivalent groups design with no control group) over a 4-year period to manipulate students’ incentives for participation in group discussion. Each year, a different guideline was given to students to guide them on how their participation would be evaluated by the instructor. In all 4 years, however, the same rubric was used by peer groups to evaluate their group members.
Each year, two sections of two university-level courses were used to collect data for this study, and each year a different strategy was utilized to incentivize students’ group discussions. The first course was an introductory course in qualitative and quantitative research methods (EDP400). For this course, students were required to have weekly group discussions. Each week, five to seven questions were posted by the instructor, and students were asked to answer those questions collaboratively through online group discussions. Each group was told to choose a group facilitator, and the facilitator was supposed to lead the discussion, record the discussion, and send the final answers to the instructor. For the first year of this study, students were asked to use the online discussion board provided by the university to document their discussion. The traditional rubric was given to this group as a guide for their participation in group discussions (named “traditional rubric” in this study). Two grades were given to students. One grade was the group grade (named performance score in this article), which was based on the accuracy of their written responses. The second grade was given for active participation in group discussion (named “peer evaluation” in this article). This grade was given by their peer-group using an online form provided by the instructor. The rubric’s criteria are:
On scale 1–5, rate the level of participation of this student.
On scale 1–5, rate the level of timeliness of this student’s answers.
On scale 1–5, rate the level of accuracy of this student’s answers.
Add any comments, suggestions, or concerns about this student.
The same procedure was used in the second year except that, students were asked to use a video conferencing tool to perform their discussion and were asked to record the video and submit it to the instructor. Students used the same rubric to evaluate their peer’s participation in group discussions.
For the third year, students were again asked to use a video conferencing tool for their group discussion. However, students were told that their group participation will be assessed by the following Horton’s criteria.
Substance: Is the message substantial or trivial. Does it add value to the conversation?
Timing: When did the post occur? Was it posted on time? Was it the first one to contribute an idea or surface a concern?
Originality: Does the post offer a new idea or a new perspective to the conversation? Was it the first to contribute supporting evidence, an alternative interpretation, or an elegant restatement?
Tone: Were there any disagreements? Was the post professional and helpful? Were there any disagreements? If a disagreement, was the statement polite and direct?
Persuasiveness: Did the message offer evidence and logic to support a claim or opinion? Was the evidence from a reliable source?
In the fourth year of the study, students were told that their group participation will be assessed by the following IFL’s criteria.
Agreement: Express agreement with what another speaker has said. “I agree,” or “I was thinking the same thing,” or “That’s a really good point, I was thinking something similar.”
Disagreement: Express disagreement with what another speaker has said. “ I Disagree,” or “I have a different perspective,” or “I see what you’re saying, but I disagree,” or “This is not right.”
Asking for opinions or information: Ask for opinions or information. “Does anyone know about this?” or “What do you think about this?” or “Has anyone read this?”
Challenging: Challenge opinions or assertions made by another speaker by giving countering reasons or evidence. “This is not what the book says,” or “This contradicts the goals of the article,” or “However, the authors say. . ..”
Supporting: Support opinions or assertions made by another speaker by providing more reasons or evidence. “To add on to what you said, the text mentions. . .” or “The results of this article support what you said,” or “I agree with your statements and would like to add on. . .,” “The text supports this argument.”
Modifying OR Compromising: Modify arguments or opinions in response to another speaker or express ideas building on what another speaker has said. “I see, this is what it means,” or “OK, I understand now,” or “let write it this way,” or “Thanks, I do it this way,” or “John’s idea made me think of another point,” or “To add on to your answer. . .”
Persuading: Attempt to persuade another speaker to accept one’s view. “I liked the paragraph where. . .” or “It is important to note that. . .,” or “I think my answer is correct because. . .” or “I believe the instructor wants to see it like this.”
Asking for clarification: Ask for explanations of words, expressions, or opinions that may not have been understood. Making sure you understand others’ opinions before you react, “What do you mean?” or “I don’t understand, could you rephrase that?” or “What does internal validity mean?”
Giving clarification: Give clarification as required by another speaker or correct another speaker’s misunderstanding of one’s own message. “Yes, that’s what I meant,” or “Let me rephrase,” or “What I mean is. . .”
Asking for confirmation: Restate what another speaker has said, or part of it, to confirm his/her own understanding of the message. “Is this what you mean?” or “Tell me if I’m understanding you correctly. . .” or “Is this right?” or “Is everybody OK with my answer?”
Checking for comprehension: Check the listeners’ understanding of the message to find out whether the speaker is understood by others before you give the final answer. “Do you all understand what I mean?” or “Did that make sense?” or “Do you have any questions?”
The traditional criteria were used by students for peer evaluation in all 4 years of the study. However, to compare students’ conversations in this article, the IFL’s rubric (categories) was used. This scale was used for three reasons. First, a single criterion was needed to compare different treatments in the 4-year period. Second, the ILF criteria are more objective and easier to measure. Second, the IFL rubric is more inclusive, and it covers most criteria used in the other two scales.
The second course for which data was collected in this study, was a more advanced course in quantitative research methods (EDP 520). The same four strategies and rubrics were used for this course. The major difference between the two courses was the content and the nature of the questions used for group discussion. For the first course the questions were more factual and conceptual. For example, students were asked to identify the research problem or the hypothesis or to identify dependent and independent variables in the given article. In the second course, questions were mostly analytical and critical. For example, a research scenario was given, and students were asked to identify the threats to internal validity of the research, or they were asked to give suggestions on how to improve measurement in the given research scenario.
Participants and samples
During the last 4 years, each year two sections of the introductory course and two sections of the advanced course were offered by the author. About 25–30 students were enrolled in each section. For each section, five to six groups were formed, five to seven students participated in each group, and there were six to eight required group discussions. After removing the first group discussions, a random sample of one group discussion were selected from each group for data analysis. In order to collect more reliable data, the first group discussions were removed since students were not quite prepared for the discussions and had to learn how to proceed with group discussions and in order to keep consistency among different phases of the study only five discussions were transcribed for each section of each course. Therefore, 80 group discussions were selected and, hence, the total sample size, for all analyses is 80. In the first year of this study, the group discussions were text-based and performed through a discussion board and students’ contributions were documented through their postings on the discussion board. In total, 20 text-based group discussions and 60 video-based group discussions were randomly selected and analyzed for this study.
Variables and measurements
The independent variables used in this analysis were: type of the course (introductory, advanced), discussion mode (text-based, video-based), and type of guideline or rubric (2 years of traditional rubric, 1 year of Horton’s rubric, and 1 year IFL’s rubric). Dependent variables included group’s performance score (the score given to a group based on the accuracy of their answers to discussion prompts), peer evaluation score (average score given to individual students based on the traditional rubric), and satisfaction score (measured through a survey at the end of each semester). Other independent variables included the scores from the subscales of IFL’s rubric.
Descriptive analysis was used to show the frequency of students’ participation as well students’ learning outcome. Analysis of variance (ANOVA) was used to the test the difference in students’ learning outcome under different treatment conditions (first hypothesis).
Results
The analysis started with transcribing the recorded videos. The transcriptions were then reviewed by the author and his assistants to remove all off-task comments (posts). Therefore, what is shown in the third column of Table 1 is the number of on-task posts analyzed in this study. This table shows only the average number of posts per discussion in each section of each course. Ten random group discussions (five from each section) were analyzed for each course. A total of 950 pages of transcripts which included more than 23,000 words were analyzed for this study.
Number of on-task comments and averages of students’ outcomes.
The last three columns of Table 1 show students’ performance under different modes of group-discussion. Students were required to evaluate the level of contributions of each of their group members. The sixth column of Table 1 shows the average peer evaluation score for each course (on a scale of 1–10). The next column shows the average of students’ satisfaction with the experience (on a scale 1–6), and the last column shows the average of students’ performance in group discussions (on a scale of 0–5).
The results of content analysis of students’ conversations during group discussions are presented in Table 2. IFL’s criteria are listed in the first column of the table, and the next columns show the number and percentage of comments (posts) belonging to each category. For example, in the third column, the first number (369) shows that in the first year of the study, for the EDP400 course, an average of 369 comments were posted in the category of agreement and the second number in this table (53) indicates that 53% of all comments posted belonged to the agreement category. The last row of the table showed the total number of comments posted by students in the ten sample discussion videos/text for each course.
The results of students’ conversations in different years of the study.
Inferential statistical analyses were also performed to compare students’ participation and their performances under different conditions. The first t-test analysis compared students’ performance in text-based group discussion with their performance in video-based group discussions. The results showed that students in the video-based group discussion were significantly more satisfied with their experience in comparison with students in text-based group discussions. The same difference was observed in students’ performance score and the total number of comments posted (expressed) by students. However, no significant differences were observed in students’ evaluation of their peer’s participation (contribution) in group discussions.
One way analysis of variance (ANOVA) was conducted to examine differences among groups who used different types of rubrics as a guide for their group discussion. In order to make groups comparable, the text-based group discussion was not included in this analysis. Among the video-based groups, the first group (Year 2) used the traditional rubric (online form) as a guide. In the second year, Horton’s guideline and in the third year third IFL’s criteria were used as guideline to conduct their discussions. The results (Figure 1 and Table 3) showed that students in Horton group scored significantly higher in their correct answers to discussion prompts. However, students in the traditional group rated their peer’s contribution to group discussion significantly higher than students in the Horton and IFL groups. The number of comments (posts) stated by students was significantly higher in the IFL group in comparison with the other two groups, but students’ satisfaction was highest when the traditional rubric was used.
Comparing the outcomes of using three different types of rubrics.
Post hoc ANOVA results comparing performance of students under three rubric conditions .
(p < 0.05; df = 79)
Finally, Horton’s guideline was compared with IFL’s criteria to find which one led to higher student engagement and performance. The results, as reflected in Table 4, showed that there were significant differences between the two groups in all aspects. While IFL’s guideline generally led to more engagement and higher quality conversations in most aspects, Horton’s guideline led to better performance in the discussion outcome as measured by students’ scores, and also it led to higher satisfaction with the experience as well as higher peer ratings.
Comparing Horton’s with IFL’s guideline in their impact on students’ group discussion.
(p < 0.05)
Discussion and conclusion
The present study used a quasi-experimental method to manipulate students’ incentives for participation in group discussion to investigate the impact on students’ engagement. Some educators have recommended that instructors should create a list of expectations and communicate those expectations clearly to all students (Horton, 2012). The results confirmed those recommendations and showed that if students know what they are expected to do, it will help them to focus more during the group discussion and avoid silence or off-task conversations. The results showed significant improvements both in quantity and quality of students’ participation in group discussions.
As discussed in the literature, He and Dai (2006) reported that that only a small number of their students produced the ILFs such as challenging, supporting, modifying, persuading, developing, and negotiating meaning in the discussion (only 6 out of the 144 students negotiated meaning in the discussion). The result of the present study was not consistent with He and Dia’s findings. As reflected in the results section (Table 4), the application of IFL led to significant improvement in quality of students’ discussions as measured by the 11 criteria for high quality group discussion. Most likely, the reason for this inconsistency is that He and Dai (2006) didn’t share the guideline with students and only the instructors had access to the rubric. It should be noted that many students do not know what to ask or how to ask questions, and other students feel unable to say what they mean and are afraid of being wrong (Choi et al., 2001). However, as shown in this study, if students know how they can participate and how their participation will be evaluated, they will participate.
A significant change was observed when more detailed or more rigorous guidelines (rubrics) were given to students to follow during their group discussions. The results showed that, while the simpler criteria led to more student satisfaction, the more demanding criteria, or the more specific criteria led to more participation and to a higher quality conversation. As shown in Tables 3 and 4, using the IFL’s criteria inspired students to talk more, challenge each other, provide more support for their claims, ask for more evidence/information from their peer group, compromise more, modify their statements if needed, persuade other students to respond, and make sure others understand their viewpoints. It was observed that the more specific guidelines also increased students’ scores on the assignment.
However, more rigorous guidelines were associated with lower peer ratings of students’ contribution to group discussions. It seems that students in the first 2 years of the study did not know how to evaluate their peer’s participation and, as a result, rated their peers equally high. The variance of peer group ratings was much higher in the last 2 years of the study, indicating students knew how to better evaluate their peer’s participation by using a more specific tool.
Another obvious change in students’ participation was observed when students were given the option to use video conference tools instead of discussion board. In year 1, students were required to use a text-based discussion board, but in the second year they used Zoom for their video-based discussion. As shown in Table 3, the number of on-task comments jumped from average of 80 to an average of 370 comments per discussion. This was quite remarkable. Furthermore, students’ performance, as measured by their scores on written final answers, also increased significantly. These findings suggest video conference tools may be a promising way to improve students’ participation, and future research should further investigate this.
Earlier studies had indicated that some courses are more appropriate for group discussions than the others. It was suggested that synthesis tasks require students to discuss ideas and theories, while the application tasks ask the group to apply the learning theory to solve a particular learning problem. For example, Jonassen (1997) predicted that when the task is synthesis, groups collaborate significantly more. Similarly, Tin (2003) discussed that in closed convergent tasks, where only one outcome is expected or is true, the participants need to converge toward a single goal. In summary, divergent tasks have greater potential for a higher level of students’ discussion, but convergent tasks lead to a more evenly distributed amount of work among group members. One of the courses examined in this study was an introductory course for which questions for discussions were at lower levels of Bloom’s taxonomy, mainly focusing on remembering, understanding, and applying. The second course was a higher-level course, and questions for discussions were at higher levels of Bloom’s taxonomy, including analysis, evaluation, and creation. The goal was to see if the quantity and quality of discussions were different in these two courses. The results showed that students in the higher-level course were more satisfied, but no other significant differences were observed in terms of number of comments, or the quality of comments posted by students. This was quite unexpected since the literature had indicated that more divergent topics lead to higher levels of discussion. Perhaps, since the same guideline (rubric) was given to both classes, students in both courses were equally impacted and inspired to contribute. It is also possible that despite the expectations of the author, the difficulty level of the questions had not been very different for students in those two courses.
Comparison of the impact of Horton’s and IFL’s guideline on level and quality of students’ discussion also revealed some significant differences. As expected, IFL guideline led to higher students’ engagement in group discussions in most of the 11 criteria. However, Horton’s guideline led to higher performance on the ask, more students’ satisfaction, and higher peer ratings. It is remarkable that in two out of 11 categories, students in the Horton group outperformed the IFL group (supporting, persuading, and asking for opinions or information). A closer look at IFL’s and Horton’s criteria shows that these are the three areas where these two guidelines overlap. This is an important finding indicating Horton’s criteria leads to more participation and better performance when they are used. However, since Horton’s guideline does not cover other criteria of high-quality conversation, it lacks the rigor offered by IFL and, therefore, falls behind in areas that are not addressed in it.
Therefore, Horton’s guideline is more impactful in areas that it covers, but it needs some improvements to cover the other criteria of high-quality conversation. It is recommended that, in the future, a combination of the two guidelines be examined for its effectiveness. In such a study, the researchers may combine the impactful wording in Horton’s guideline with the additional criteria covered in IFL to see how they work together. The most important implication of this research for instructors is that if they want their students to actively participate in group discussion and to have authentic and productive conversation, there must be some sort of accountability (assessment) of their contribution. Furthermore, teachers’ expectations for high quality conversations need to be communicated clearly, precisely, and ahead of time to their students.
As discussed in the introduction, some students may interpret contribution in terms of quantity rather than quality. As a result, they try to make as many posts as they can or talk as much as possible rather than making relevant contributions to the discussion. Consequently, this may lead to domination of the whole discussion by a few students, and lack of challenge from the rest. A second issue reported in earlier studies (Swain, 2001) was that some shy or introverted students with low self-confidence, don’t participate that much in discussions, particularly, in online discussions, where they can hide behind their off-cameras. The results of this study showed that if instructors use clear and specific criteria for evaluating students’ participation, not only the overall participation improves significantly, but also the diversity of participants increases. In the early years of the study, only a few students dominated most of the discussions, but when Horton and IFL guidelines were given as criteria, more and more diverse groups of students participated. Particularly, the criteria for challenging each other was quite effective in this regard.
Some research studies had indicated that students spend only a minority of their meeting time engaging with content (Summers and Volet, 2010). The results of the present study showed, however, that only 30% of students’ talking was off task, and these off-task discussions happened mostly before the actual task started. Therefore, another implication of the present study is that teachers should not assume that group assignments will necessarily give rise to substantial engagement if the criteria of high-quality discussion are not clearly communicated with them. Another claim reported in earlier studies was the idea that students may communicate more directly and more bravely when writing instead of talking (Eisele, 2013). Still, the results of this study showed that the video-based discussion led to much more conversation. Therefore, the assumption that since text-based discussion is time-independent and allows time for more communication was not validated by the present study.
Overall, it could be concluded that it is quite useful to use either Horton’s guideline or IFL’s checklist to engage students in authentic group discussions, and to increase the quality of their discussions. However, the current study had several limitations that warrant discussion. First, data were collected on different cohorts of students and the “treatments” applied there were different in different years. Particularly, the data for the last year of the study were collected during the COVID-19 pandemic, when students had more experience with online discussions than earlier years. Therefore, time is an important confounding variable to be noted in the interpretation of the results because there is a chance that the difference seen is just the result of comparing different cohorts. Furthermore, in video-based conversations there were some non-verbal communications or some silent participations that were not reflected in the transcripts analyzed for the study. For example, nodding heads as a sign of agreement or shaking heads as a sign of disagreement was not reflected in the data. Also, some comments were removed from data analysis because the researcher could not fit them into any of the categories. Finally, this study focused only on two research methods courses which were very similar. The results might have been quite different if other courses, particularly courses in STEM field, were examined.
Footnotes
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
