Pre-Service Teachers’ Perceptions of Websites That Feature Research-Based Practices

Abstract

Research-based practices generally improve student outcomes but are not regularly implemented with fidelity in classrooms. Websites that feature research-based practices may be used to improve pre-service teachers’ understanding of the practices; however, little is known about pre-service teachers’ experiences with or perceptions of these websites. The purpose of this study was to evaluate and explore pre-service teachers’ perceptions of content related to one effective strategy, self-monitoring, after reviewing three websites that feature research-based practices (i.e., IRIS, Ci3T, and Intervention Central) to which they were randomly assigned. Results indicate that the 168 participating U.S. pre-service teachers increased their perceived understanding of self-monitoring after visiting the websites. In addition, participants identified features of the websites (i.e., videos, real-world examples, implementation guides, assessments, quizzes, resources, and organization) they perceived as helping them understand self-monitoring.

Keywords

classroom behavior classroom behavior management general education classroom special education classroom

Research-based practices (RBPs) are instructional practices indicated to improve student outcomes through high-quality experimental studies (Cook & Odom, 2013). Despite the benefits of implementing RBPs with fidelity (e.g., improved student outcomes), a research-to-practice gap exists in which RBPs are not regularly implemented with fidelity in schools and classrooms (Cook & Farley, 2019; Gage et al., 2017). Teacher preparation is an important time to foster understanding of RBPs and how to use them so that educators begin their teaching careers with the knowledge needed to optimize learner outcomes. Pre- and in-service educators frequently access websites to enhance their understanding of instructional practices (Hott et al., 2019; Opfer et al., 2016). However, many of these websites feature practices that are not validated by high-quality research (Test et al., 2015). Although some websites that explicitly feature RBPs have been developed, little is known about how educators perceive these websites and their content or whether visiting them affects pre-service teachers’ (PSTs’) perceived understanding of and ability to use RBPs. Such information can be used to refine and develop web-based resources featuring RBPs. Therefore, the purpose of this preliminary study is to examine (a) the perceptions of PSTs regarding features of three popular websites (i.e., Ci3T, Intervention Central, IRIS) featuring RBPs and (b) the effects of visiting these websites on PSTs’ perceived understanding of a targeted RBP and perceived knowledge of how to use the RBP.

When seeking out information on instructional practices, educators typically prefer resources they find trustworthy, usable, and accessible (Carnine, 1997). However, Carnine posited that educators tend not to perceive traditional research-based information (e.g., research publications) as trustworthy, usable, or accessible because, respectively, (a) research often occurs in controlled situations that do not reflect the realities of classroom teachers (Boardman et al., 2005), (b) research publications seldom include ready-to-use guidelines or materials, and (c) teachers often cannot access journal articles that promote RBPs (Fleming & Cook, 2022). Websites such as Teachers Pay Teachers and Pinterest allow teachers to upload instructional resources for other educators to download and use (see Beahm et al., 2019). Teachers may perceive content on these websites as accessible, trustworthy, and usable because it (a) is developed by teachers, (b) includes ready-to-use content, and (c) can be accessed at minimal cost. Indeed, teachers report that they frequently use websites such as Teachers Pay Teachers and Pinterest (Hott et al., 2019; Opfer et al., 2016). However, evaluations of these and similar websites suggest that much of the content is not research-based and likely ineffective (Polikoff, 2019; Test et al., 2015).

Web-based resources represent a potentially powerful tool to supplement the training of PSTs during their teacher preparation programs, which increasingly are being shortened and condensed, making it difficult to provide extended instruction on RBPs (Chesley & Jordan, 2012; Freeman et al., 2014). Fortunately, educational scholars and organizations have developed websites that explicitly target RBPs. For example, Ci3T (ci3t.org), Intervention Central (www.interventioncentral.org), and IRIS (iris.peabody.vanderbilt.edu) each provide free access to ready-to-use resources to support the implementation of RBPs.

One RBP featured on all three websites is self-monitoring (SM). Evidence suggests that SM is an effective practice that has been shown to improve students’ behavior and academic outcomes (McDougall et al., 2017). SM involves systematically observing one’s own behavior, setting behavioral goals, and recording and reviewing one’s own data (Cooper et al., 2020; Rafferty, 2010). In addition, SM is designed to reduce teachers’ workloads by teaching students to monitor their own behavior. Although many teachers report using SM to support their students, implementation fidelity is often low, which may jeopardize behavioral or academic improvements (Briesch et al., 2014).

IRIS, supported by the Office of Special Education Programs Technical Assistance Network, features modules for educators to learn about RBPs, including SM (https://iris.peabody.vanderbilt.edu/module/sr/cresource/q2/p04/), which are developed in collaboration with education researchers and experts (Bransford et al., 1999). The IRIS Module on self-regulation includes a page specific to SM that (a) lists the benefits of SM for students with and without disabilities (supported by references to research), (b) describes five steps for implementing SM, (c) provides a 3-min animated video of a teacher implementing SM with a student, and (d) concludes with an activity assessing understanding of steps for implementing SM with feedback. In addition, a link to blank tally sheets that can be printed for students to self-record their behavior is provided. Research suggests that IRIS modules help teachers learn new information regarding effective practices and are incorporated into many special education teacher preparation programs (Sayeski et al., 2015).

The Comprehensive, Integrated Three-Tiered Model of Prevention (Ci3T, 2023; https://www.Ci3T.org/pl) was developed by the Ci3T Strategic Leadership Team to support safe and supportive school environments and has been found to improve teachers’ self-efficacy, reduce burnout, and support teachers and students in improving student outcomes (Lane et al., 2020; Oakes et al., 2021). The Ci3T website provides a Tiered Intervention Library that features Tier-2 and Tier-3 interventions as well as low-intensity strategies, including SM, that are supported by research as effective. Content on SM includes (a) a short video of an expert describing the practice; (b) downloadable documents such as an introduction to SM and its six critical steps, a 38-slide PowerPoint presentation overviewing SM and how to implement it, a guide to additional resources, and a worksheet for implementing the six steps of SM; and (c) downloadable materials to be used in implementing SM such as an implementation checklist, social validity assessments, student self-recording forms, and a goal-setting form.

Intervention Central (https://www.interventioncentral.org/behavioral-intervention-modification) provides teachers with resources for RBPs related to Response to Intervention and Common Core State Standards, which are organized by topic (e.g., communication tools, self-management, bully prevention). This website was developed by Jim Wright, a school psychologist and administrator who consults with schools on issues such as behavioral interventions for struggling students. The Intervention Central page on SM includes a brief overview of SM, including its benefits; a description of seven critical steps for implementation (with references to supporting research); and links to materials that educators can use to implement SM, such as student self-recording forms (i.e., behavior rating scale, behavior checklist, frequency-count form), beep recordings at various fixed intervals, and an example reinforcer survey.

Web-based resources can improve educators’ competency, knowledge, and skills regarding RBPs (Rakap et al., 2015). Thus, websites like Ci3T, Intervention Central, and IRIS have the potential to support PSTs’ understanding of RBPs and how to use them, especially if PSTs perceive the websites and their contents to be trustworthy, usable, and accessible. However, little is known about whether and how accessing websites that feature RBPs increases PSTs’ perceived understanding of RBPs and how to use them, or how PSTs perceive features of these websites. Therefore, the research questions for this study are:

Research Question 1 (RQ1): Do PSTs’ perceived (a) understanding of SM and (b) knowledge of how to use SM increase after visiting the Ci3T, Intervention Central, and IRIS websites, and does change in these outcomes differ between the websites?

Research Question 2 (RQ2): Does the number of core elements of SM included in PSTs’ descriptions of practice increase after visiting the Ci3T, Intervention Central, and IRIS websites, and does change in this outcome differ between websites?

Research Question 3 (RQ3): What are PSTs’ perceptions of the usability of Ci3T, Intervention Central, and IRIS websites, and are there differences in perceptions between websites?

Research Question 4 (RQ4): What features of the Ci3T, Intervention Central, and IRIS websites do PSTs perceive as most helpful for understanding SM and what features would they like to add to increase their understanding?

Method

Participants

After receiving institutional review board approval, we asked faculty members at four different universities in different regions (i.e., Pacific, Midwest, and East Coast) of the United States to participate in the study. Eight university instructors teaching general and special education courses agreed to participate. Instructors invited PSTs in their classroom management and introduction to special education courses to participate by sharing the survey link and explaining they could complete the survey at any time during the semester. If students submitted a completed final page of the survey to their instructor, they received extra credit in their class. Of 251 PSTs invited to participate, 231 opened the survey. One-hundred sixty-eight PSTs completed initial items were randomly assigned to a condition, completed most of the survey items, and were included in the analytic sample (67% completion rate). Each participant was randomly assigned to one of the three conditions by Qualtrics when they opened the survey. The 63 PSTs who opened the survey but were excluded from analyses did not provide consent (n = 1), did not respond to pretest questions (n = 30), or did not respond to post-intervention items (n = 32). Of those 63 PSTs, 15 had been assigned to the IRIS condition, five to the Intervention Central condition, 12 to the Ci3T condition, and 31 dropped out before being assigned to a condition. Five additional participants were excluded from analyses of RQ2 and RQ3 because they did not respond to items related to those research questions (see Supplemental Material Figure S1).

Fifty-one participants in the analytic sample were assigned randomly to the IRIS condition, 65 to the Intervention Central condition, and 52 to the Ci3T condition. Variability across conditions is due to differences in random allocation to conditions and differential attrition (see Supplemental Material Figure S1). Overall, most participants were female (89.3%), white/Caucasian (62.5%), seeking elementary general education licensure (51.2%), between 18 and 24 years of age (69.1%), and had some experience working with school-age students (95.2%; see Supplemental Material Table S1). Chi-square tests revealed no significant differences between conditions on gender, race, licensure, experience working with children, or training in behavior management. There were no significant differences in reported familiarity with SM between groups, χ² (8, N = 168) = 2.18, p = .97, with most respondents reporting to be either familiar with (48.2%) or unsure of their familiarity with (35.1%) SM.

Survey Development, Items, and Administration

Before developing the survey, we asked seven teacher educators what websites they use in their teacher preparation classes or recommend to PSTs. Six websites were nominated by multiple teacher educators: Ci3T, IRIS, What Works Clearinghouse, Intervention Central, Autism Focused Intervention Resources & Modules, and Evidence Based Intervention Network. We then examined each website to determine which (a) provided research support for practices featured on the website (e.g., included citations to supporting research), (b) featured SM, and (c) were freely accessible. Three websites met these criteria and were used in this study: Ci3T, Intervention Central, and IRIS. Before developing the survey, we reviewed items assessing the social validity of educator websites featuring RBPs in previous research (see Mandracchia & Sims, 2020). Then, we examined how each website presented SM to ensure items applied meaningfully to all conditions. We asked four teachers, each with over five years of teaching experience, and two special education professors to pilot the initial draft of the survey and provide feedback. Based on their feedback, we clarified phrasing identified as potentially confusing but did not add or remove any items (e.g., changed “I understand the strategy self-monitoring” to “I understand what self-monitoring is,” changed the 10-point scale for recommending the website to a colleague to the same 7-point scale used to rate the likelihood of using the website again).

Pre-intervention questions assessed participants’ demographics, experience working with school-age children and youth, and amount of training in behavior management. Participants also rated their agreement that (a) “I understand what SM is” and (b) “I know how to implement SM with a student in the classroom” from strongly disagree (1) to strongly agree (5). We also asked participants to describe SM in their own words using one to two sentences. Participants were then directed to visit a randomly assigned website (i.e., Ci3T, Intervention Central, IRIS) using a direct link to the SM content on the website and to read, listen to, and engage with (e.g., respond to, interact with activities) the resources on the website. We did not provide specific directions beyond this, as we were interested in examining typical experiences of visiting a website to find out about SM. Participants reported spending an average of 19.9 min on their assigned website (SD = 11.8, range = 2–86), with no differences between websites, F(2,165) = 0.57, p = .55.

After participants interacted with the website, they returned to the survey and completed the same questions regarding perceived understanding of SM and perceived knowledge of how to implement SM and were again asked to describe SM in one to two sentences. Then, participants rated their agreement that the website was easy to navigate, and that specific website features helped them understand SM from strongly disagree (1) to strongly agree (5). The first author created a list of each website’s key elements or features. The third author reviewed this list to check for accuracy. See Table 3 for a list and brief descriptions of these features. Participants next rated, from 1 (extremely unlikely) to 7 (extremely likely), (a) “How likely are you to recommend the website to a colleague?” and (b) “how likely would you be to use a website like this to learn more about different behavior management strategies?” Finally, participants were asked two open-ended questions: “What features of the website most helped you understand SM and why?” and “What features would you have liked to add to the website to increase your understanding of SM and why?”

Data Analysis

To address RQ1, because data were not normally distributed, we used Wilcoxon signed-ranks tests to examine differences between pre-and post-test ratings of perceived (a) understanding of SM and (b) knowledge of how to implement SM across and within websites. We used Kruskal–Wallis tests to examine differences in changes in the perceived understanding and knowledge for implementing SM between website conditions.

To address RQ2, we created a rubric to assess the presence of three critical elements of SM presented on each website in participants’ definitions: Students (a) assess their own behavior, (b) record their own behavior, and (c) set behavior goals. Description scores ranged from 0 to 3, representing the number of critical elements included in the description. The purpose of the rubric was to minimize subjectivity in scoring. The first two authors each scored half of the descriptions. The third and fourth authors each double-coded 30% of descriptions to estimate interrater reliability, which was calculated to be 81% (number of agreements divided by the total number of descriptions). Because data were normally distributed, we used t-tests to examine differences in the number of core SM elements in descriptions after visiting websites within conditions and differences in change in core SM elements in participants’ descriptions (at pre- and post-test) between conditions.

To address RQ3, we calculated descriptive statistics of participants' ratings of the likelihood of visiting a similar website in the future, the likelihood of recommending the website to colleagues, and the perceived ease of navigating the website. We used Kruskal–Wallis tests to examine differences between conditions. When Kruskal–Wallis tests were significant, Dunn post hoc tests with the Holm method of p-value adjustment were used to determine between which conditions differences existed.

To examine RQ4, we (a) quantitatively analyzed (using descriptive statistics, see Table 3) participants’ ratings of the helpfulness of website features and (b) qualitatively analyzed participants’ narrative responses to the open-ended questions. In addition, to clarify how the features of the websites that participants rated differed from one another, we provided brief descriptions of each rated website feature in Table 3 when the content is not obvious. For the qualitative analysis, the first author undertook the first stage of coding, which involved exploring participants’ comments to develop a general understanding of the dataset and develop themes based on the frequency of words and phrases in the responses (Creswell & Creswell, 2017). Then, responses were organized according to key phrases and grouped according to previously established themes. Data were summarized and counted across participants (Hannes, 2011). Using investigator triangulation (Patton, 1999), the second and third authors reviewed the established themes and frequency counts to confirm accuracy.

Results

Perceived Understanding of Self-Monitoring (RQ1)

Post-test mean ranks for perceived understanding of SM, Z = 9.09, p < .01, r = .70, and perceived knowledge of how to implement SM, Z = 9.98, p < .01, r = .77, were significantly higher than pre-test mean ranks across conditions as well as within each condition (see Table 1). A Kruskal–Wallis test indicated no significant difference between conditions for change in pre- to post-test ranks for perceived understanding of SM, H(2) = 4.14, p = .12. However, a significant difference was identified between conditions for change in pre- to post-test mean ranks regarding perceived knowledge for implementing SM, H(2) = 28.00, p < .05. Post hoc analyses revealed significant differences between (a) IRIS and Ci3T conditions, with change in ranks being greater in the IRIS condition (p < .05), and (b) IRIS and Intervention Central conditions, also with change in ranks being greater for the IRIS condition (p < .05).

Table 1.

Pre- and Post-Test Comparisons of Perceived Understanding of and Knowledge to Implement Self-Monitoring.

Name of Website/PSTs’ Perception	Pre-test	Post-test	Z	r
Name of Website/PSTs’ Perception	Median (SD)	Median (SD)	Z	r
IRIS Center
Understanding of SM	4 (1.09)	5 (0.76)	5.08**	0.71
Knowledge to implement SM	3 (1.27)	4 (0.89)	5.23**	0.73
Intervention Central
Understanding of SM	4 (1.08)	5 (0.81)	5.70**	0.71
Knowledge to implement SM	3 (1.17)	4 (0.89)	6.48**	0.80
Ci3T
Understanding of SM	4 (0.93)	5 (1.07)	5.44**	0.75
Knowledge to implement SM	3 (0.66)	4 (0.71)	5.87**	0.81

Note. r is a measure of effect size calculated by Z/sqrt n. Range for each measure is 1–5. PST = preservice teacher; SM = self-monitoring.

p < .01.

Core Feature of Self-Monitoring in Participants’ Descriptions (RQ2)

Among the 153 (n = 46 Ci3T, n = 61 Intervention Central, n = 46 IRIS) participants who provided pre- and post-test descriptions of SM, post-test description scores (M = 1.50), reflecting the number of critical elements of SM in participants’ descriptions of the practice, were significantly greater than pre-test description scores (M = 0.62) across conditions, t(296.42) = 8.70, p < .05. Description scores were also significantly higher at post-test for respondents in each condition: Ci3T condition pre-test M = 0.46, post-test M = 1.43; t(5.62) = 84.57, p < .05; Intervention Central condition pre-test M = 0.60, post-test M = 1.62; t(5.81) = 88.87, p < .05; and IRIS condition pre-test M = 0.75, post-test M = 1.47; t(4.01) = 114.96, p < .05). There was no difference in change in description scores between conditions, F(2,150) = 1.84, p = .34. Across all conditions, 52 (33%) participants included self-assessment in their pre-test descriptions, with more than twice as many (n = 108, 70%) including it in their post-test descriptions. Thirty-two (21%) participants included self-recording in their pre-test descriptions, whereas 82 (53%) included it in their post-test descriptions, representing more than a two-fold increase. Ten (7%) participants included goal-setting in pre-test descriptions, whereas 37 (24%) included it in post-test descriptions, a greater than three-fold increase (see Supplemental Table S2 for numbers across conditions).

Perceptions of the Website (RQ3)

Descriptive statistics related to RQ3 can be found by website in Table 2. Mean ranks of ease of navigation differed significantly across conditions, H(2) = 8.44, p < 05. Post hoc analyses revealed significant differences between (a) the Intervention Central and Ci3T conditions, with Intervention Central being higher (p < .05), and (b) the IRIS and Ci3T conditions, with the IRIS scores being higher (p < .05). Mean ranks for reported likelihood to recommend the websites also differed significantly, H(2) = 7.03, p < .05. Ratings for the IRIS and Ci3T conditions differed significantly, with IRIS being higher (p < .05); none of the other comparisons was statistically significant. No significant difference between websites was identified for mean ranks of reported likelihood to use a similar website (p = .08).

Table 2.

Participants’ Perceptions of Websites.

Feature	M	Median	SD	Range
IRIS Center (51 participants)
Ease of navigation	4.38	4	0.67	2–5
Likelihood to recommend	6.10	6	0.81	4–7
Likelihood to use similar website	6.42	7	0.71	4–7
Intervention Central (65 participants)
Ease of navigation	4.35	4	0.68	2–5
Likelihood to recommend	5.69	6	0.96	3–7
Likelihood to use similar website	6.13	6	0.81	4–7
Ci3T (52 participants)
Ease of navigation	3.88	4	0.99	2–5
Likelihood to recommend	5.64	6	1.01	3–7
Likelihood to use similar website	6.07	6	0.91	4–7

Note. The rating scale for ease of navigation was (1) strongly disagree to (5) strongly agree and the rating scale for likelihood to recommend and use was (1) extremely unlikely to (7) extremely likely.

Website Features (RQ4)

Descriptive statistics for participants’ ratings of the helpfulness of website features for understanding how to use SM are provided in Table 3. For the qualitative analysis, we identified five themes reflecting website features about which participants most frequently commented in their responses to features that were most helpful in understanding SM and what features would they most like to add to increase understanding of SM. Using frequency counts, we determined how many statements were made and what percentage of the 43 participants responding to the open-ended questions made a statement about each theme.

Table 3

Ratings of Helpfulness of Website Features for Understanding How to Use Self-Monitoring.

Features	M (SD)
IRIS Center
The section on “collecting baseline data” [which includes a sample graph of a student’s baseline behavior and a mix of bulleted and narrative written content] helped me understand how to use SM. (n = 48)	4.25 (0.60)
The section on “teach SM procedures” [which includes an image of tally sheet, a link to additional sample tally sheets, and a step-by-step breakdown of teacher behaviors] helped me understand how to use SM. (n = 48)	4.29 (0.58)
The video [a 3-min narrated video with animated depictions of a teacher and students] on the website helped me understand how to use SM. (n = 48)	4.35 (0.70)
The activity (SMP with Zach) [an activity with multiple-choice questions on five key steps for implementing SM with feedback provided for each response] helped me understand how to use SM. (n = 48)	4.23 (0.75)
Intervention Central
The section on “defining behavior targets to self-monitor” [which includes 2 bulleted lists of common behavior to target for increasing and decreasing] helped me understand how to use SM. (n = 62)	4.26 (0.75)
The section on “choosing method for recording SM” [which includes bulleted descriptions of 3 formats for collecting SM data and links to examples of each format] helped me understand how to use SM. (n = 62)	4.37 (0.61)
The section on “choosing an SM schedule” [which includes bulleted information on 5 options for when to schedule SM during the school day] helped me understand how to use SM. (n = 62)	4.27 (0.63)
The section on “deciding on a monitoring cue” [which includes bulleted information on four options for monitoring cues and a link to access free “beep tapes”] helped me understand how to use SM. (n = 61)	4.18 (0.74)
The link to the blog helped me understand how to use SM. (n = 56)	4.13 (0.85)
Ci3T
The PowerPoint presentation [a 38-slide presentation on SM] helped me understand how to use SM. (n = 49)	4.33 (0.72)
The “Implementation” and “Treatment integrity” checklists helped me understand how to use SM. (n = 48)	4.15 (0.68)
The resource guide [which includes a list of articles, websites, and books on self-monitoring] helped me understand how to use self-monitoring. (n = 48)	4.10 (0.72)
The sample SM checklists [1 for elementary, 1 for secondary] helped me understand how to use SM. (n = 48)	4.42 (0.65)
The “self-instruction” and “goal setting” sections [blank forms for generating self-instruction statements and goals for students who are SM] helped me understand how to use SM. (n = 48)	4.25 (0.67)

Note. Participants only rated the features of the website they visited. The rating scale was (1) strongly disagree to (5) strongly agree. SM = self-monitoring.

Theme 1: Videos

Thirty participants (70% of respondents; 21 from the IRIS condition, 7 from Ci3T, and 2 from Intervention Central) mentioned that the videos were helpful in understanding SM, and 30 participants (70% of respondents) also expressed that they wanted more videos. No video demonstrated a real teacher implementing SM in an actual classroom. The video on IRIS was an animated representation, and the video on Ci3T was a coach providing an implementation overview. One participant in the IRIS condition stated, “Overall, the most helpful was the video as I could visualize what it would look like in practice.” Alternatively, one participant who visited Intervention Central suggested, “I wish there could have been videos or pictures. . . . Having pictures or videos could have been a little more helpful because I would have been able to see examples.” Whereas some participants indicated the general importance of videos, a few made specific recommendations about what they would like to see in the videos. One participant mentioned they would like “to add a video to show an example of SM and how it can be used in a classroom.” Overall, participants indicated that watching videos of SM being implemented in a classroom facilitates their understanding. Some mentioned they wanted videos that walked them through the steps of SM, whereas others noted that seeing students’ SM in a classroom would help them feel more confident in implementation.

Theme 2: Real-World Examples

Eighteen participants (42% of respondents; 14 from the IRIS condition and 4 from Intervention Central) mentioned that real-world examples helped them understand how to implement SM, and 38 participants (88%) stated they would like more examples of teachers using SM. Real-world examples on the websites included a video of a cartoon teacher using SM with a student and an interview with a teacher explaining how they use the practice in their classroom, and a written example of how a teacher used the practice. One Intervention Central participant noted, “I appreciated the examples in each section since they help to transfer the experience to a real-life scenario. I am saving the link to continue using it.”

Some participants noted they would like more examples of teachers using SM in the classroom. For example, “Maybe more examples of self-monitoring in action,” and “I would maybe like an example of a specific situation and how to remediate it using self-monitoring in elementary school.” Several participants indicated they would like examples of SM used in a variety of situations, such as with different behaviors or in different settings. Other participants requested examples of how to introduce SM to students: “I would have liked more strategies for how to convince students that self-monitoring is important, effective, and useful,” and another said it would be helpful to see “ideas on how to obtain a student’s willing cooperation, particularly if the student is reluctant to participate.” In addition, some participants wanted examples of SM being poorly implemented to showcase what not to do.

Some participants stated they would like to learn about student and teacher perspectives. “I would like to add a student’s perspective to see how they navigate,” and “Hearing from teachers’ experience can be a good addition. They can share their own experiences and techniques with self-monitoring.” Finally, some participants mentioned video examples that correspond with step-by-step guides would provide valuable, complementary demonstrations of real-life examples. One participant recommended “an example of the different steps/components walked through with a real-life example of an example behavior that a student could self-monitor.” Overall, participants found examples beneficial when learning about SM and suggested including examples of SM being used in various settings, testimonials from students and teachers regarding SM, videos with step-by-step explanations, and examples of SM being incorrectly implemented.

Theme 3: Implementation Steps

Twenty-five participants (58% of respondents; 10 from IRIS, 10 from Ci3T, and 5 from Intervention Central) stated that step-by-step breakdowns helped them understand how to implement SM. One IRIS participant said, “The breakdown of the steps to self-monitoring was the most helpful because it broke the concept into five parts and explained each step in detail.” Another who visited Intervention Central felt positively about the site “because of how they broke down the steps and included bullet points.” Several participants from Intervention Central mentioned that breaking down steps prevented them from being overwhelmed with information.

Theme 4: Other Resources

Several participants mentioned other resources they found or believed would be helpful. Sixteen participants (37% of respondents) from Ci3T indicated how useful the PowerPoint and printable resources were in explaining SM. For example, one PST who visited the Ci3T website noted, “the PowerPoint helped me because it gave me a visual I would be able to go back to if need be.” Seven PSTs assigned to the IRIS website noted a multiple-choice activity to practice implementing SM was helpful, with one participant stating that, “If I misunderstood what was expected, there was feedback given to me on why the choice was appropriate or not.” Twelve participants (28%) indicated they wanted more activities, three requested the addition of quizzes, and two indicated a desire for more printable materials.

Theme 5: Website Organization

Twenty participants (47% of respondents; 16 from Intervention Central, 2 from IRIS, and 2 from Ci3T) mentioned that the organization of the website was easy to follow and facilitated learning about SM. Eighteen participants (42%) stated that they appreciated the conciseness of their website and how the different tabs and headings made it easy to navigate. For example, one participant who visited Intervention Central said, “The breakdown of everything with the number and bullet points helped keep everything organized which also kept my thoughts organized.” One participant in the IRIS condition said, “I like how we can expand the tabs [and] the clear and concise way it is written was really effective.” Twenty participants (42%) also provided suggestions on ways to improve website organization. Five (11%) requested more visuals and pictures to help them navigate the website and three (7%) mentioned that the text was small and hard to read. One participant suggested, “A summary section where key points are highlighted” would be helpful and another mentioned, “I think it would be nice to have the sections at the top available to click on rather than scrolling down to see each one.”

Discussion

Results indicate that PSTs reported relatively high and increased perceived understanding of and perceived knowledge for implementing SM after visiting all three websites. Perceived knowledge for implementing SM increased significantly more in the IRIS condition compared with the other conditions. In open-ended responses, participants in the IRIS condition frequently noted the value of the website’s video and multiple-choice practice activity for implementing SM, which may have led to increased perceptions of knowing how to implement the practice. Both the cartoon video and the activity demonstrated SM being implemented in a classroom. As participants recommended real-world examples for helping to understand website content, it is possible that these features led to increased perceptions of knowing how to implement SM.

Participants most frequently mentioned videos as a website feature they felt was helpful and that they wanted to see more of on the websites. This is consistent with quantitative ratings that the video was most highly rated feature on IRIS. Video examples of teachers modeling RBPs have been shown to improve confidence and implementation fidelity by, for example, helping educators analyze areas of their behavior that need adjusting to maximize intervention effectiveness (Sherin & van Es, 2009). Therefore, content creators may want to include videos that provide multiple video models of actual teachers implementing targeted interventions.

The second most frequently recommended website feature was real-world examples, which aligns with previous findings that teachers prefer to learn about instructional practices from other teachers (Beahm et al., 2021). As such, featuring real-world examples may increase PSTs’ perceptions of trust and usability of website content. Websites featuring RBPs can include real-world examples in many ways, such as videos featuring teachers implementing the practice (Borko et al., 2011), written explanations from practitioners on what they have done when a student is unresponsive to a practice, and audio clips of practitioners discussing how to implement with fidelity. Videos and real-world examples, the two most requested website features by participants, can be integrated with one another by featuring videos with real-world examples of targeted RBPs.

Participants also recommended step-by-step guides on implementing SM to improve their understanding. This is consistent with research showing that breaking down complex tasks into manageable steps helps individuals learn, retain, and implement information (Brown et al., 2016). Indeed, the checklist on Ci3T, which features a step-by-step breakdown of SM, was the most highly rated feature across all websites. As participants mentioned the benefit of revisiting content, it may be helpful to create checklists and other resources (e.g., PowerPoints) that can be downloaded and referenced after visiting the website. Participants also reported that organizational features such as bold headings, large font, and clear directions made navigating the websites and finding information to understand the material easier. These responses align with suggestions made by experts in website development, such as using bold colors and pictures that attract attention (Beahm & Bradshaw, 2023). Intervention Central and IRIS, the two websites with the highest ratings on ease of navigation, both use headings that include bulleted information about SM. Presenting information in manageable chunks (e.g., bulleted lists, step-by-step guides) is consistent with principles of Universal Design for Learning and can be readily incorporated into website design to facilitate understanding (O’Hanlon, 2005).

Implications and Recommendations for Research and Practice

Given findings of increased perceived understanding of SM and knowing how to implement SM after visiting the three websites, teacher educators may want to consider incorporating freely accessible websites that promote RBPs into teacher preparation (e.g., Ci3T, Intervention Central, IRIS). Zimmermann and colleagues (2022) reported that PSTs mostly use Google’s search engines to find information on instructional practices. Therefore, content creators might follow recommendations for search engine optimization to promote websites that feature RBPs (see Gudivada et al., 2015). In addition, as teachers frequently use Teachers Pay Teachers and Pinterest to find instructional practices (Opfer et al., 2016), content creators might consider posting their content on these frequently visited websites for free downloading to possibly encourage PSTs to visit their website to learn about RBPs.

Teacher educators may consider providing PSTs a list of websites that promote RBPs so that PSTs have easy access to trustworthy, accessible resources. Moreover, teacher educators might consider teaching PSTs how to identify trustworthy websites and how to use the resources on those websites to support effective implementation of targeted RBPs. Online resources can be used as supplemental resources in university classrooms to provide PSTs with additional information regarding targeted RBPs. For example, instructors could assign a small group of PSTs to become experts on an RBP (e.g., SM) that they later demonstrate to the entire class. The PSTs could use the websites to learn additional information, watch videos, and increase their understanding of the practice before demonstrating the RBP. By engaging PSTs with trustworthy websites featuring RBPs in these and other ways, teacher educators can provide exposure and practice using these easily accessible resources that PSTs can use to support their instruction in future years when they are in their own classroom.

When developing a website, content creators may want to consider incorporating real-world examples, videos, implementation guides, activities, and quizzes, as PSTs indicated these features helped them understand the content when visiting websites. For example, one option for combining implementation guides, real-world examples, and videos is to overlay directions on a video explicitly describing the steps a teacher is using when implementing SM. Incorporating real-world examples and implementation guides provides PSTs with usable and trusted models. In addition, online activities and quizzes can be used to engage educators in active learning. Content creators may also consider including specific features for early career teachers, such as thorough descriptions of RBPs and a focus on practicing implementation of their core elements.

Researchers may wish to expand on this study by conducting component analyses in which website features are systematically varied to evaluate which combination of features has the largest effects on PSTs’ knowledge and behavior related to implementing targeted instructional practices with fidelity. Researchers should also examine perceptions of websites that feature RBPs for specific populations of educators (e.g., special education teachers, paraprofessionals, educators of color). The effects of using online content for in-service educators should also be examined in future research. In addition, future research should assess the effects of using websites to support the implementation of RBPs by pre- and in-service teachers. For example, researchers could evaluate PSTs’ implementation fidelity of an RBP before and after engaging with a website. Although we evaluated perceived understanding and implementation know-how, assessing actual implementation would provide more information regarding the effectiveness of using a website to support the implementation of RBPs. Finally, future researchers might also evaluate how participants engage with websites by using a “think-aloud” protocol that involves participants narrating their thoughts while interacting with a website (Fonteyn et al., 1993). This protocol may be especially useful for exploring how different groups of educators perceive specific types of resources on websites.

Limitations

One important limitation of the study is, outside of participants’ descriptions of SM, outcome data, including rating of understanding, were self-reported perceptions, which are subject to response bias. Although we attempted to minimize response bias by informing participants of their anonymity, it is possible that some participants provided socially desirable responses. In addition, we relied on self-report to measure how long participants spent on the websites, which may have been inaccurate, and we did not assess how actively participants engaged with the websites. We used a rubric to evaluate the number of key elements that were present in participants’ descriptions of SM before and after visiting websites but did not specifically ask respondents to include key elements in their descriptions. A more direct evaluation of understanding of SM (e.g., a multiple-choice quiz) may have yielded different results. Although there was no significant difference in time reportedly spent on the websites between conditions, the range in reported time is substantial (2–86 min). Although participants rated their understanding of SM significantly higher after visiting the websites, we did not examine whether such perceived increases resulted in improved practice. This is an especially important concern given the short period of time most participants engaged with the websites. In addition, we used self-reported time spent on the website and permanent products (i.e., responses to questions about specific website content) as proxy measures for treatment integrity. Future researchers could directly measure how long participants spend on the website as a whole and on specific sections of the website to evaluate treatment integrity.

In addition to invited PSTs who chose not to participate, 32 additional PSTs dropped out after random assignment to condition, raising the possibility of non-response bias. It is possible, for example, that PSTs who fully participated were more open to learning about SM from websites featuring RBPs than those who did not. In addition, attrition after randomization was not even across conditions (23% attrition rate in IRIS condition, 7% for Intervention Central, and 19% for Ci3t). It is possible that the differential attrition may reflect differences in participants’ responses to the websites and influenced results. Although we collected demographic information regarding participants’ experience with behavior management training and working with children, we did not measure participants’ program year or if they had completed any teaching practica, factors which may be related to how they interacted with and learned from the websites. Moreover, the sample size was relatively small for each condition. It is possible that results would differ with a larger and more representative sample.

Conclusion

Although educators can access multiple websites that feature RBPs, to our knowledge, researchers had not previously evaluated PSTs’ perceptions of these websites. In this study, PSTs (a) increased perceived understanding SM and perceived knowledge for implementing SM and (b) included more key elements of SM in their explanations of the practice after interacting with one of three websites (i.e., Ci3T, Intervention Central, IRIS). Participants recommended videos, real-world examples, implementation guides, and other resources such as self-assessments as features of websites that did and would help them understand SM. More research is needed to understand which website features cause improved knowledge and behavior.

Supplemental Material

sj-docx-1-rse-10.1177_07419325231217522 – Supplemental material for Pre-Service Teachers’ Perceptions of Websites That Feature Research-Based Practices

Supplemental material, sj-docx-1-rse-10.1177_07419325231217522 for Pre-Service Teachers’ Perceptions of Websites That Feature Research-Based Practices by Lydia A. Beahm, Nina R. Mandracchia, Bryan G. Cook and Austin H. Johnson in Remedial and Special Education

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Lydia A. Beahm

Bryan G. Cook

Supplemental Material

Supplemental material for this article is available at

References

Beahm

L. A.

Bradshaw

C. P.

(2023). Adapting evidence-based professional development models for online delivery and scale-up to practitioners in applied settings. In Evans

S. W.

Owens

J. S.

Bradshaw

C. P.

Weist

M. D.

(Eds.), Handbook of school mental health: Innovations in science and practice (3rd ed., pp. 493–512). Springer.

Beahm

L. A.

Cook

B. G.

Cook

(2019). Proceed with caution: Using web-based resources for instructing students with and at risk for EBD. Beyond Behavior, 28(1), 13–20. https://doi.org/10.1177/1074295619836076

Beahm

L. A.

Yan

Cook

B. G.

(2021). Where do teachers go for behavior management strategies? Education and Treatment of Children, 44(3), 201–213. https://doi.org/10.1007/s43494-021-00046-2

Boardman

A. G.

Argüelles

M. E.

Vaughn

Hughes

M. T.

Klingner

(2005). Special education teachers’ views of research-based practices. The Journal of Special Education, 39(3), 168–180. https://doi.org/10.1177/00224669050390030401

Borko

Koellner

Jacobs

Seago

(2011). Using video representations of teaching in practice-based professional development programs. ZDM Mathematics Education, 43(1), 175–187. https://doi.org/10.1007/s11858-010-0302-5

Bransford

J. D.

Brown

A. L.

Cocking

R. R.

(1999). How people learn: Brain, mind, experience, and school. National Academy Press.

Briesch

A. M.

Briesch

J. M.

Mahoney

(2014). Reported use and acceptability of self-management interventions to target behavioral outcomes. Contemporary School Psychology, 18(4), 222–231. https://doi.org/10.1007/s40688-014-0016-8

Brown

F. E.

McDonnell

J. J.

Snell

M. E.

(2016). Instruction of students with severe disabilities (8th ed.). Pearson.

Carnine

. (1997). Bridging the research-to-practice gap. Exceptional Children, 63(4), 513–521. https://doi.org/10.1177/001440299706300406

10.

Chesley

G. M.

Jordan

(2012). What’s missing from teacher prep. Educational Leadership, 69(8), 41–45.

11.

Comprehensive, Integrated Three-Tiered Model of Prevention. (2023). Contact us. https://www.ci3t.org/contact

12.

Cook

B. G.

Farley

(2019). The research-to-practice gap in special education. In Bateman

D. F.

Yell

M. L.

(Eds.), Current trends and legal issues in special education (pp. 110–125). Corwin.

13.

Cook

B. G.

Odom

S. L.

(2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79(2), 135–144. https://doi.org/10.1177/0014402913079002021

14.

Cooper

J. O.

Heron

T. E.

Heward

W. L.

(2020). Applied behavior analysis. Pearson.

15.

Creswell

J. W.

Creswell

J. D.

(2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage.

16.

Fleming

J. I.

Cook

B. G.

(2022). Open access in special education: A review of journal and publisher policies. Remedial and Special Education, 43(1), 3–14. https://doi.org/10.1177/0741932521996461

17.

Fonteyn

M. E.

Kuipers

Grobe

S. J.

(1993). A description of think aloud method and protocol analysis. Qualitative Health Research, 3(4), 430–441. https://doi.org/10.1177/104973239300300403

18.

Freeman

Simonsen

Briere

D. E.

MacSuga-Gage

A. S.

(2014). Pre-service teacher training in classroom management: A review of state accreditation policy and teacher preparation programs. Teacher Education and Special Education, 37(2), 106–120. https://doi.org/10.1177/0888406413507002

19.

Gage

N. A.

MacSuga-Gage

A. S.

Crews

(2017). Increasing teachers’ use of behavior-specific praise using a multitiered system for professional development. Journal of Positive Behavior Interventions, 19(4), 239–251. https://doi.org/10.1177/1098300717693568

20.

Gudivada

V. N.

Rao

Paris

(2015). Understanding search-engine optimization. Computer, 48(10), 43–52. https://doi.org/10.1109/MC.2015.297

21.

Hannes

(2011). Chapter 4: Critical appraisal of qualitative research. In Noyes

Booth

Hannes

Harden

Harris

Lewin

Lockwood

(Eds.), Supplementary guidance for inclusion of qualitative research in cochrane systematic reviews of interventions. Version 1 (updated August 2011). Cochrane Collaboration Qualitative Methods Group. http://cqrmg.cochrane.org/supplemental-handbook-guidance

22.

Hott

B. L.

Dibbs

R.-A.

Naizer

Raymond

Reid

C. C.

Martin

(2019). Practitioner perceptions of algebra strategy and intervention use to support students with mathematics difficulty or disability in rural Texas. Rural Special Education Quarterly, 38(1), 3–14. https://doi.org/10.1177/8756870518795494

23.

Lane

K. L.

Oakes

W. P.

Allen

G. E.

Lane

N. A.

Jenkins

A. B.

Lane

K. S.

Messenger

M. L.

Buckman

M. M.

Fleming

K. K.

Romine

R. E. S.

(2020). Improving educators’ knowledge, confidence, and usefulness of a Comprehensive, Integrated, Three-Tiered (Ci3T) Model of Prevention: Outcomes of professional learning. Education and Treatment of Children, 43(3), 279–293. https://doi.org/10.1007/s43494-020-00021-3

24.

Mandracchia

N. R.

Sims

W. A.

(2020). Development of the Usage Rating Profile-Web Resource (URP-WR): Using assessment to inform web resource selection. Computers in the Schools, 37(4), 269–291. https://doi.org/10.1080/07380569.2020.1835388

25.

McDougall

Heine

R. C.

Wiley

L. A.

Sheehey

M. D.

Sakanashi

K. K.

Cook

B. G.

Cook

(2017). Meta-analysis of behavioral self-management techniques used by students with disabilities in inclusive settings. Behavioral Interventions, 32(4), 399–417. https://doi.org/10.1002/bin.1491

26.

Oakes

W. P.

Lane

K. L.

Royer

D. J.

Menzies

H. M.

Buckman

M. M.

Brunsting

Cantwell

E. D.

Schatschneider

Lane

N. A.

(2021). Elementary teachers’ self-efficacy during initial implementation of Comprehensive, Integrated, Three-Tiered Models of Prevention. Journal of Positive Behavior Interventions, 23(2), 93–105. https://doi.org/10.1177/1098300720916718

27.

O’Hanlon

. (2005, April 7–10). Adapting online instruction for a learning-disabled audience [Conference session]. ACRL, 12th National Conference, Minneapolis, MN, United States. www.ala.org.o’hanlon05.pdf

28.

Opfer

V. D.

Kaufman

J. H.

Thompson

L. E.

(2016). Implementation of K–12 state standards for mathematics and English language arts and literacy. RAND.

29.

Patton

M. Q

. (1999). Enhancing the quality and credibility of qualitative analysis. HSR: Health Services Research, 34(5), 1189–1208.

30.

Polikoff

. (2019). The supplemental curriculum bazaar: Is what’s online any good? Thomas B. Fordham Institute.

31.

Rafferty

L. A

. (2010). Step-by-step: Teaching students to self-monitor. TEACHING Exceptional Children, 43(2), 50–58. https://doi.org/10.1177/004005991004300205

32.

Rakap

Jones

H. A.

Emery

A. K.

(2015). Evaluation of a web-based professional development program (Project ACE) for teachers of children with autism spectrum disorders. Teacher Education and Special Education, 38(3), 221–239. https://doi.org/10.1177/0888406414535821

33.

Sayeski

K. L.

Hamilton-Jones

(2015). The efficacy of IRIS STAR legacy modules under different instructional conditions. Teacher Education and Special Education, 38(4), 291–305. https://doi.org/10.1177/0888406415600770

34.

Sherin

van Es

E. A.

(2009). Effects of video club participation on teachers’ professional vision. Journal of Teacher Education, 60(1), 20–37. https://doi.org/10.1177/0022487108328155

35.

Test

D. W.

Kemp-Inman

Diegelmann

Hitt

S. B.

Bethune

(2015). Are online sources for identifying evidence-based practices trustworthy? An evaluation. Exceptional Children, 82(1), 58–80. https://doi.org/10.1177/0014402915585477

36.

Zimmermann

Engel

Mayweg-Paus

(2022). Pre-service teachers’ search strategies when sourcing educational information on the Internet. Frontiers in Education, 7, Article 976346. https://doi.org/10.3389/feduc.2022.976346

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB