Professional Development on Data-Based Individualization: A Mixed Research Study

Abstract

Data-based individualization (DBI) is a process of collecting and analyzing data on students’ response to intervention and then making intervention adaptations accordingly. Although this process can lead to better student outcomes, very few teachers are trained in the components of DBI, particularly in relation to behavior. Improving practice requires not only ongoing professional development but also understanding about how teachers’ experiences in training can lead to better outcomes. Within the context of implementing a behavior intervention, the purpose of this study was to evaluate how participating in ongoing professional development on DBI affects teachers’ perceptions of themselves in relation to the DBI framework over time. Using a convergent, parallel mixed-methods research design, we evaluated the conceptual understanding, self-efficacy, and perceptions associated with DBI before, during, and after professional development of 16 general and special education teachers. Data analysis indicated teachers reported significant improvements in all three areas over time. Qualitative data indicated active practice and collaboration with other professionals contributed to these improvements. Key findings, limitations, and future directions are discussed.

Keywords

progress monitoring assessment individualized interventions professional development teacher perceptions qualitative methods

Students with, or at risk of, emotional and behavioral disorders (EBDs) exhibit complex academic and behavioral issues. Yet, many teachers may not have the experience, knowledge, confidence, or skills to effectively provide the evidence-based instruction and intervention necessary to meet these students’ needs (Jones, 2009). For beginning special educators, effective behavior management can be difficult, but it is an even more formidable task for general educators (Beam & Mueller, 2017; Bruhn, Freeman, Hirn, & Kern, 2018). Special educators, for example, report feeling more confident and prepared than general educators to work with students with challenging behaviors (Beam & Mueller, 2017). This is highly problematic given the majority of students with, or at risk for, EBD receive instruction in general education classrooms (Forness, Kim, & Walker, 2012).

This problem is further exacerbated by several factors. First, educators of students with challenging behavior typically have fewer years of teaching experience. Second, many of those students receive instruction from paraprofessionals who may have no training (Bradley, Doolittle, & Bartolotta, 2008). Third, many state accreditation policies do not require training in evidence-based classroom management practices for preservice teachers (Freeman, Simonsen, Briere, & MacSuga-Gage, 2014). Teachers often report having limited knowledge of research-based behavioral practices, and in turn, they have limited experience implementing these practices (Jones, 2009; Stormont, Reinke, & Herman, 2011). With minimal exposure to behavior interventions and assessment, it is not surprising teachers report feeling unprepared to manage classroom behavior (Chesley & Jordan, 2012) and, hence, many leave the field (Ingersoll, 2001). On the contrary, teachers’ sense of self-efficacy related to classroom management may serve as a protective factor against burnout and attrition from the profession (Aloe, Amo, & Shanahan, 2014). Thus, there is a need to provide effective training to educators to improve their understanding of and self-efficacy with the use of research-based practices for managing behavior proactively and responding to problem behavior with appropriate supports.

Data-Based Individualization and Professional Development

In addition to using effective behavioral interventions, teachers need to evaluate whether students are responding and make decisions accordingly. This can be done using data-based individualization (DBI; Kern & Wehby, 2014; Stecker, Fuchs, & Fuchs, 2005). DBI is a systematic process of formatively and summatively evaluating the effects of a targeted intervention to guide subsequent intervention programming. Research suggests DBI has resulted in improved academic (e.g., reading, math, spelling; Stecker et al., 2005) and behavioral outcomes (Bruhn, Rila, Mahatmya, Estrapala, & Hendrix, 2018). It is predicated on consistent, accurate implementation of a targeted research-based intervention and ongoing data collection and analysis. In the DBI process, data are gathered continually on the student’s behavior to determine if the intervention is effective. Then, the interventionist (e.g., a teacher or a student support team) uses the data to make intervention adaptations. If a student is responsive, the intervention may be faded to promote maintenance and generalization. If a student is nonresponsive, the intervention may be intensified or modified to increase the likelihood of success. If intensifying intervention is ineffective, further assessment may be conducted.

Typically, measuring behavioral progress has been done through direct observations and Direct Behavior Rating (DBR). Regardless of the type of behavioral data collected, teachers require training and practice to become efficient, accurate, and independent in progress monitoring. Yet, using data to make decisions on a consistent and ongoing basis can be complex and cumbersome, particularly for educators who are not adequately trained in this process (Wayman, 2005). School personnel also view data analysis as overly labor-intensive, which may inhibit implementation of data-based decision-making as is required in the DBI process (Wayman, 2005). For teachers who are not trained in using data to make individualized decisions, professional development (PD) and data coaches with expert-level knowledge have been recommended (Nichols & Singer, 2000; Stormont et al., 2011). However, PD on data collection and analysis has a history of being inadequate, though it is crucial to the sustainability of any data initiative (Armstrong & Anthes, 2001). For example, traditional PD conducted at the district level may involve teachers from different schools gathering in a central location to learn about a new strategy (Bruce, Esmonde, Ross, Dookie, & Beatty, 2010). Some have characterized this type of top-down knowledge dissemination activity as “sit and get” PD, where teachers sit and listen to a presenter, but receive no practice, action planning, or follow-up (Chappuis, Chappuis, & Stiggins, 2009). This type of PD focuses on disseminating knowledge rather than developing skills through ongoing practice and coaching (Bruce et al., 2010; Dunn, Airola, Lo, & Garrison, 2013). Teachers need more than traditional one-shot “sit and get” training sessions, and instead, can benefit from training situated in a classroom context that allows for continuous practice through “sustained iterative cycles of goal setting, planning, practice, and reflecting” (Bruce et al., 2010, p. 1599). This is particularly important given the relation between teachers’ sense of self-efficacy about data-based decision-making, its implementation, and in turn, student outcomes (Dunn et al., 2013). Specifically, when teachers lack self-efficacy about certain strategies, they are less likely to implement them or persist when confronted with challenges which can be especially detrimental as it relates to students with, or at risk for, EBD who need evidence-based, data-driven interventions.

Purpose

To meet the needs of students with, or at risk for, EBD, teachers must have the knowledge, self-efficacy, and skills to implement interventions and evaluate their effectiveness. This process of intervention and evaluation may be accomplished through a DBI framework. However, teachers lack adequate training in nearly all components (e.g., implementing evidence-based interventions, data collection, data analysis) of this framework (Freeman et al., 2014; Stormont et al., 2011). To address the lack of teacher knowledge, limitations of current practice, and follow the research-based recommendations for delivering high-quality PD, we created a PD model for training teachers in DBI (hereafter referred to as DBI-PD; see Method section for description). Our DBI-PD model is based on PD literature supporting the use of two critical components: (a) authentic learning opportunities and (b) building teacher self-efficacy (Bruce et al., 2010; Dunn et al., 2013). Providing authentic learning opportunities involves three major elements: (a) learning situated in a classroom context, (b) engaging in active practice, and (c) building collaboration. Building teacher self-efficacy involves assessing and addressing prerequisite skills and knowledge, providing opportunities for feedback and discussion, providing real-world examples, and embedding direct experience. As part of a larger study (Bruhn, Rila, et al., 2018), we found teachers were able to implement DBI with integrity and, in turn, students significantly improved their behavior. Given these findings, we were further interested in understanding these teachers’ experiences in PD and how that affected their perceptions and implementation. Thus, using data collected as part of the larger study on student outcomes, we used a concurrent, parallel mixed research design (Creswell & Clark, 2017; Johnson & Onwuegbuzie, 2004) to investigate our central research question: What are teachers’ perceptions related to DBI and how do these perceptions change over time from participating in DBI-PD? We explored this research question by analyzing the alignment between teachers’ quantitative self-assessments and their qualitative, anecdotal reports about DBI and DBI-PD.

Method

Participants and Setting

This study took place in a Midwestern U.S. school district serving 20 elementary schools (K–sixth grade), three middle schools (seventh–eighth grade), two high schools (ninth–12th grade), and one alternative high school (ninth–12th grade). Total enrollment was about 14,000 students. Approximately 36% of students received free or reduced lunch, 9% received special education services, and 9% were English language learners. In this noncategorical state, students receiving special education services were deemed “eligible individuals” rather than identified by an Individuals with Disabilities Education Act (IDEA) disability category. Districtwide ethnicity percentages were 58.4% White, 18.7% Black, 11.5% Hispanic, 6.1% Asian, and 5.3% identified as other.

Balancing the resources allocated for this study with the sample size recommendations for having sufficient “information power” for qualitative analysis, we sought to recruit about 15 participants (Malterud, Siersma, & Guassora, 2016). Following a districtwide email about the opportunity to participate in DBI-PD, 16 elementary teachers from six different schools signed up for DBI-PD and consented to participate in the study. The 16 teachers comprised two males, 14 females, with 12 White, three Black, and one identified as Other. Four teachers were special education teachers and the remaining 12 were general education teachers who taught third, fourth, fifth, or sixth grade. All teachers had degrees in education (highest degrees earned: bachelor’s = 4, master’s = 10, and doctorate = 2), and, on average, the teachers had 14.6 years of experience (range = 3–35 years). Because DBI involves implementing and adapting a behavioral intervention with students, each teacher nominated a student with persistent challenging behavior in their class to participate as well. Students had to meet at least one of the following criteria: (a) high rates of off-task behavior and poor academic performance, (b) numerous office discipline referrals or behavior screening score indicating risk, (c) Individualized Education Program (IEP) with behavior goals, or (d) an EBD diagnosis (Bruhn, Rila, et al., 2018). The nominated student received a behavioral intervention in the participating teacher’s class. Two teachers implemented intervention with the same student in different classes (e.g., art and music), and one teacher implemented intervention with two different students in two different classes. The remaining teachers implemented the intervention with only one student in only one class (see Bruhn, Rila, et al. [2018] for a full description of the behavior intervention and student outcomes).

Procedures

We obtained institutional review board (IRB) approval from the university and school system prior to intervention. Teachers registered online to participate for the DBI-PD, which consisted of five sessions totaling 16 hr over the course of the 2016–2017 school year. All trainings took place on the university campus and were led by the principal investigator (PI; i.e., first author). Teachers provided consent prior to the beginning of the first session. Consistent with PD focused on providing authentic learning opportunities and building teacher self-efficacy, at the beginning of each training session, we provided participants a list of objectives related to content, implementation, and evaluation. Content objectives referred to the information covered in that day’s training, what participants were expected to learn, and the opportunities for practice that would be provided that day. Implementation objectives referred to tasks teachers would complete both during the training and in their classrooms prior to the next training session. Evaluation objectives referred to the assessments participants would complete during that day’s training session that were related to the overall goals of the research study.

Session 1

Session 1 lasted 5 hr and occurred in November 2016. Prior to the session, participants completed the DBI Self-Assessment (DBI-SA; see Measures section). Content included (a) the relation between academic difficulty and challenging behavior, as well as how teachers and students tend to respond; (b) an overview of self-determination as it relates to behavior; and (c) detailed information and research on self-monitoring. Teachers watched a video on how to use the iPad self-monitoring app they would be implementing in their classrooms. This served as the standard Tier 2 intervention for use within the DBI framework. Next, the PI taught teachers a process for using self-monitoring interventions. Teachers then generated a list of students who met criteria for participation and practiced writing definitions of target and replacement behaviors. Teachers independently explored the app and practiced entering information in the settings screen (e.g., teacher name, student name, interval length, goal, behaviors). At the end of Session 1, the PI reviewed the procedures related to consent forms for parents and assent forms for students. Once the teacher obtained parental consent, she or he met with students in private to explain the study and allow the student to assent. Teachers brought signed parental consent and student assent forms to Session 2 of DBI-PD.

Session 2

Teachers attended Session 2 one month after Session 1 and it lasted 5 hr. At the beginning of Session 2, the PI reviewed the purpose of the DBI-PD and discussed objectives for the session. Next, the PI reviewed self-monitoring and technology content from Session 1 before introducing new content focusing on the DBI framework. DBI was explained in a broad context and then in more detail within the context of the Tier 2 intervention, self-monitoring. Steps included (a) implementing the self-monitoring intervention, which included steps for training the student via teaching, modeling, and practice, as well as a discussion of fidelity; (b) monitoring student progress (i.e., collecting and analyzing graphed data); (c) adapting intervention (e.g., intensifying intervention for nonresponders and fading intervention for responders); and (d) conducting functional behavior assessments for nonresponders.

To develop the skills and self-efficacy of teachers, they practiced using two case studies. Each case study included a description of the student, teacher, school context, problem behaviors, and self-monitored replacement behaviors. Also, teachers were provided baseline data to visually analyze a bar graph and calculate a baseline average. Then, they established an initial behavioral goal and designed self-monitoring intervention procedures (e.g., interval length, frequency of feedback). They were provided intervention data via a bar graph to analyze visually for responsiveness. Based on their interpretations, they made decisions on how to adapt the intervention (e.g., increase goal, decrease interval length) and this process repeated. The whole group discussed each case study and teachers had the opportunity to ask questions. After case study practice, teachers spent time designing an initial draft of the self-monitoring intervention for their assented student. They named and defined examples and nonexamples of the problem behavior of their assented student. Next, they named and defined replacement behaviors (i.e., the behaviors they wanted the student to perform instead of the problem behaviors) in written format. The PI modeled how to program replacement behaviors into the self-monitoring iPad app, which the teachers did next. The PI had teachers write down on a form the self-monitoring intervention procedures including the instructional activity or time intervention would be in place, interval length (i.e., time between self-monitoring occurrences), and whether feedback and/or reinforcement would be provided (and if so, when and what, respectively). Finally, the PI modeled baseline data collection using the app. Teachers practiced using the app and asked questions. They were instructed to start collecting baseline data after winter break (January) and bring the baseline data to the next training session. They created a tentative calendar for collecting baseline data and intervention data. Finally, they completed the Intervention Rating Profile-15 (IRP-15; Witt & Elliot, 1985; reported in Bruhn, Rila, et al., 2018) to gauge their opinion about the self-monitoring intervention they designed.

Session 3

Session 3 occurred 6 weeks later in January. Prior to this 2-hr session, teachers collected 5 days of baseline data using the app. The PI began the session by providing an overview of new content, implementation, and evaluation objectives for Session 3, as well as a brief review of previous content. Teachers analyzed the graphed baseline data in the app, calculated the average, and wrote the average on their DBI form. Next, they reviewed the initial draft of the self-monitoring intervention procedures and made any changes on paper. The PI modeled how to customize the app. Teachers used the app to set initial student goals and program interval length. Finally, the PI discussed the logistics of implementation and modeled how to train the student on self-monitoring. To meet implementation objectives, teachers determined dates for training the student, beginning implementation, and examining data; which were to occur prior to Session 4. Finally, teachers completed the DBI-SA as a mid-point measure. Between Sessions 3 and 5, research assistants observed intervention implementation fidelity of each teacher/student pair once per week and provided performance feedback or answered questions, as needed. The PI provided remote coaching through email reminders and helpful implementation tips weekly.

Session 4

The fourth DBI-PD session occurred 3 weeks after the third session and lasted about 2 hr. The PI described session objectives and then reviewed treatment fidelity and how it could be used in conjunction with student data to make intervention decisions. Teachers then examined their DBI forms, where they had documented intervention procedures. Next, they reviewed how to make adaptations for responders and nonresponders. Teachers worked in small groups to examine data graphed in the app to determine whether the student’s behavior was stable or variable and whether the student was meeting his or her daily goal. Based on this analysis, teachers decided whether to make an adaptation to intervention and recorded it on the DBI form. Analysis and adaptations were discussed first in small groups and then as a large group. Finally, teachers completed the first five questions of an 11-item open-ended questionnaire. After completing the questionnaire, teachers met in small groups to discuss answers to the questionnaire and make any edits to their own answers. The session culminated in a large group discussion of the five questions. Between Sessions 4 and 5, teachers continued to implement the self-monitoring app, analyze data every 3 to 5 days, make appropriate adaptations based on responsiveness, and document those adaptations on their DBI form. Research assistants and the PI continued to observe weekly, answer questions, and send email reminders, as needed.

Session 5

The final session occurred 4 weeks later and lasted 2 hr. After session objectives were presented, the PI briefly reviewed self-monitoring interventions and the DBI framework, including an overview of functional behavior assessment for nonresponders. Teachers completed a summative analysis of their intervention results. In small groups, they presented their data and concluded whether self-monitoring within a DBI framework was effective for their student. Extending from Session 4, teachers completed the last six questions of the questionnaire, shared answers and edit responses, and then discussed responses with the whole group. Finally, teachers completed the DBI-SA and the IRP-15.

Measures

Data-Based Individualization Self-Assessment

Teachers completed the DBI-SA at the beginning of the first training session (Time 1), end of the third training session (Time 2), and at the end of the last training session (Time 3). The DBI-SA is a researcher-created tool for teachers to rate themselves on a Likert-type scale about their understanding of specific concepts, their self-efficacy in implementing these concepts within the classroom, and the usability of these concepts in the classroom (Lane, Menzies, Bruhn, & Crnobori, 2011). The conceptual understanding subscale asks teachers to rate their understanding of multiple terms associated with DBI on a scale of 0 to 3 (0 = I do not understand this concept, 1 = I understand this concept a little, but probably not enough to explain it to others, 2 = I understand this concept, 3 = I understand this concept and could explain it to others). The self-efficacy subscale asks teachers to rate their ability in using various DBI components in the classroom on a scale of 0 to 3 (0 = I do not have the ability to use this in my classroom yet, 1 = I may be able to use this in my classroom, but I need more training, 2 = I can use this in my classroom, 3 = I can definitely use this in my classroom and help others use this as well). The usability and feasibility subscale asks teachers to rate the usability and feasibility of components of DBI on a scale of 0 to 3 (0 = This is not useful and/or practical in my classroom, 1 = This has the potential to be useful and/or practical in my classroom, 2 = This is useful and practical in my classroom, 3 = This is useful and practical in my classroom and I would recommend it to other teachers).

We adapted the self-assessment from a tool designed to assess teachers’ knowledge, confidence, and use of function-based interventions (Lane et al., 2015), which has demonstrated strong internal consistency (.94–.95) and interrater reliability (.97–.98). Adapted items on this version included student self-monitoring of behavior, technology-based self-monitoring, DBI, behavioral progress monitoring, maintenance programming, behavioral data collection, behavioral data analysis, and behavioral goals. The purpose of adapting items on the self-assessment tool was to ensure the wording and the content being assessed aligned with the wording and content provided in the DBI-PD. Mean scores were calculated for each item at each time point. A composite score for each subscale was created at each time point by summing the scores on each item, with possible scores ranging from 0 to 24 and higher scores indicating greater conceptual understanding, self-efficacy, and perceived usability/feasibility. Cronbach’s alpha was calculated for each DBI-SA subscale at each time point. Alpha values for conceptual understanding were .82 at Time 1, .93 at Time 2, and .90 at Time 3. Alpha values for self-efficacy were .82 at Time 1, .86 at Time 2, and .84 at Time 3. Alpha values for usability and feasibility were .81 at Time 1, .90 at Time 2, and .76 at Time 3.

Open-ended questionnaire

During training Sessions 4 and 5, teachers completed one half of an open-ended questionnaire, respectfully. The full questionnaire contained 11 questions about teachers’ individual perceptions of training, knowledge, self-efficacy, usability, and feasibility related to DBI. For example, teachers described (a) the extent to which they understood how to determine student responsiveness to intervention using the app and make adaptations accordingly, (b) how aspects of the PD contributed to their knowledge and self-efficacy, (c) advice they would offer other teachers related to DBI, (d) the extent to which they would feel comfortable training others in the DBI process or implementing DBI with another student, and (e) how useful and practical DBI was in their classrooms.

Each half of the questionnaire took about 10 min to complete. Teachers completed the questionnaire individually. Once all individuals completed the assigned questions for that session, they shared and discussed their answers with the three to four people sitting at their table. Teachers could edit their answers during discussion. After 10 min of sharing in their original table groups, teachers regrouped with teachers from other tables to share and edit again. Finally, at the end of 20 min, the PI led a large group discussion of responses to the questionnaire. During this time, teachers could edit their answers again, if needed. The purpose of having multiple discussions was to increase the likelihood teachers would provide thorough responses to the questions as teachers shared ideas and opinions. Teachers turned their completed questionnaire forms into the PI.

Research Design and Analysis

We used a mixed research concurrent-parallel design to describe and evaluate the alignment between numerical ratings of teachers’ perceptions of DBI and their qualitative responses to open-ended questions about DBI and the DBI-PD (Creswell & Clark, 2017). This involved collecting and analyzing two independent strands of quantitative and qualitative data to obtain, “different but complementary data on the same topic” (Morse, 1991, p. 122). We analyzed data using a pragmatic approach (Feilzer, 2010), which requires thorough analysis of each data set and then an integration of the data sets to improve the validity of the results and offer a richer description of the phenomena, in this case, the impact of ongoing PD on teachers’ learning and experiences with the DBI framework (Desimone, 2009).

The quantitative data set used in this study was the DBI-SA comprising numerical ratings of teachers’ self-reported conceptual understanding of the DBI, self-efficacy with DBI, and DBI usability/feasibility. We performed multivariate analysis of variance (MANOVA) to examine main effects of time across the teacher ratings. MANOVA tests were performed on the listwise sample size (n = 14). We did not impute missing data from two participants because the missing scores were not at random. Given the dependent variables comprised several related subscales, the use of MANOVA helped to account for the linear dependence among the variables and guard against Type I error (Huberty & Morris, 1989; O’Brien & Kaiser, 1985). The data met statistical assumptions necessary to conduct inferential statistics. The dependent variables of interest were correlated with coefficients ranging from r = .14 to .82, p < .05, and the scores on each subscale demonstrated a normal distribution with no extreme outliers, and skewness and kurtosis within the acceptable range. The sample size was sufficient to detect effect sizes of 0.37 at power = 0.80. We ran a total of four MANOVA tests with the following configurations of dependent variables: (a) composites of the domains, (b) eight items under conceptual understanding only, (c) eight items under self-efficacy only, and (d) eight items under usability and feasibility only. To account for the number of tests we performed, we used a corrected experiment-wise p value of .0125 (p = .05/four tests) to assess significance.

The qualitative data set consisted of teachers’ written responses to the open-ended questionnaire. Teachers’ responses were copied, verbatim, into Excel by one research assistant and checked for 100% accuracy by a second research assistant. The research team (PI, two graduate assistants, and two research staff) performed thematic analysis of the responses using a two-cycle coding process (Saldaña, 2015) to create categories that emerged from the qualitative data. First, the research team, as a group, read all responses for one question to develop an initial set of codes. The coding was guided by a priori categories related to the domains (i.e., conceptual understanding, self-efficacy, usability/feasibility) and items assessed in the DBI-SA. After the discussion, research team members independently coded a set of responses. Each team member added codes to the initial codebook and then came back together to discuss the emerging themes until there was a point of saturation. Codes and themes were refined, consolidated, and clarified collaboratively by the team. After this second discussion, each team member recoded their set of responses using the revised codebook.

Responses were assigned to research team members using a response matrix that captured the number of teachers (n = 16) and number of questions (n = 11) represented in the qualitative data. To establish inter-rater reliability, the PI assigned each research team member a set of responses to code that crossed teachers and questions to ensure that at least 75% of the responses were double-coded. We evaluated inter-rater reliability via several procedures. First, the four coders were each assigned to code the responses from three questions and four teachers individually, based on the codebook developed collaboratively by the team. Next, each coder was paired with another and switched their coding assignments to compare for reliability. Given the exploratory nature of the data and the number of codes developed for questions and teachers, we determined reliability by the proportion of agreement between the pairs, as well as across the coding team and all text (Kurasaki, 2000). Intercoder agreements across the question themes ranged from 0.69 to 1.0 (M = 0.90). Intercoder agreements across the teacher themes ranged from 0.62 to 1.0 (M = 0.81).

We collected quantitative and qualitative data concurrently, but analyzed data using separate processes. The quantitative and qualitative data held equal status with the goal of the data analysis strategy being data comparison (Onwuegbuzie & Corrigan, 2014). After the independent analysis of each data source, the findings were brought to the full research team to identify key themes and findings from each. We compared the key findings to corroborate the results. When discrepancies (negative cases) occurred, we discussed possible reasons for the misalignment between the quantitative and qualitative self-reports of teacher perceptions of both DBI and the DBI-PD. Often, this involved situating the findings in the “spatial and temporal context” (Feilzer, 2010, p. 12) of the study to understand what and how the quantitative and qualitative data represent the phenomena. In using quantitative and qualitative data sets, we triangulated the data to gain a greater “breadth and depth of understanding” of the central research question (Johnson, Onwuegbuzie, & Turner, 2007, p. 123) related to teachers’ perceptions of themselves in relation to the DBI framework over time.

Results

In terms of composite-level for the subscales—conceptual understanding, self-efficacy, and usability and feasibility—the MANOVA tests indicated an overall significant difference between the means at the different time points for all three, Pillai’s trace = 1.12, F = 10.72, df = 6, p < .001. The univariate F tests showed significant differences in the means across time for conceptual understanding, F = 71.52, df = 2, p < .001; self-efficacy, F = 50.13, df = 2, p < .001; and usability and feasibility, F = 35.93, df = 2, p < .001. MANOVA findings for the specific linear trends and item-level differences within each subscale are described below and in Table 1.

Table 1.

Data-Based Individualization Self-Assessment Results.

	Conceptual understanding				Self-efficacy				Usability and feasibility
Items	T1M (SD)	T2M (SD)	T3M (SD)	F	T1M (SD)	T2M (SD)	T3M (SD)	F	T1M (SD)	T2M (SD)	T3M (SD)	F
Composite	13.27(4.33)	21.44(3.42)	22.07(3.13)	82.19** 1 < 2, 3	12.07(4.25)	20.31(3.28)	22.40 (2.32)	116.82** 1 < 2, 3	12.73(3.63)	18.38(3.95)	22.31(2.27)	129.74** 1 < 2 < 3
Student self-monitoring of behavior	1.60(0.63)	2.81(0.40)	2.93(0.26)	63.71** 1 < 2, 3	1.47 (0.52)	2.75(0.45)	2.80(0.41)	64.29** 1 < 2, 3	1.73(0.8)	2.31(0.70)	2.88(0.34)	33.97** 1 < 2 < 3
Technology-based self-monitoring	1.40(0.91)	2.75(0.45)	2.80(0.41)	44.85** 1 < 2, 3	1.27(1.80)	2.56(0.51)	2.80(0.41)	33.47** 1 < 2, 3	1.33(0.49)	2.25(0.78)	2.88(0.34)	82.79** 1 < 2 < 3
Data-based individualization	1.40(0.74)	2.73(0.46)	2.73(0.46)	58.91** 1 < 2, 3	1.33(0.82)	2.50(0.52)	2.73 (0.46)	39.39** 1 < 2, 3	1.47(0.64)	2.27(0.70)	2.75(0.45)	42.21** 1 < 2, 3
Behavioral progress monitoring	2.00(0.66)	2.81(0.54)	2.73 (0.59)	15.68* 1 < 2, 3	1.87 (0.92)	2.69(0.48)	2.93(0.26)	22.75** 1 < 2, 3	1.73(0.70)	2.44(0.51)	2.81(0.40)	23.40** 1 < 2 < 3
Maintenance programming	0.80(1.01)	2.19(0.83)	2.40(0.83)	45.47** 1 < 2, 3	0.93 (0.80)	2.00(0.73)	2.60(0.63)	78.00** 1 < 2 < 3	1.20(0.86)	1.94(0.77)	2.56(0.63)	25.37** 1 < 2 < 3
Behavioral data collection	1.93(1.03)	2.75(0.58)	2.80(0.56)	14.90* 1 < 2, 3	1.73(0.88)	2.63 (0.62)	2.80 (0.41)	23.40** 1 < 2, 3	1.67(0.72)	2.31(0.70)	2.69 (0.60)	19.12* 1, 2 < 3
Behavioral data analysis	1.93(0.70)	2.75(0.45)	2.87(0.35)	39.00** 1 < 2, 3	1.53(0.83)	2.56(0.63)	2.93 (0.26)	39.39** 1 < 2, 3	1.70(0.65)	2.44(0.51)	2.88 (0.50)	52.91** 1, 2 < 3
Behavioral goals	2.20(0.78)	2.81(0.40)	2.80(0.41)	11.64* 1 < 2, 3	1.93(0.70)	2.62(0.62)	2.80 (0.41)	23.40** 1 < 2, 3	1.90(0.66)	2.56(0.51)	2.88(0.34)	39.99** 1 < 3

Note. For parsimony, F value represents within-subject contrast for the linear effect only. Numbers in bold represent significant pairwise comparisons.

p < .01. **p < .001.

Conceptual Understanding

Quantitative

The MANOVA showed a significant linear trend, F = 82.19, df = 1, p < .001, and quadratic effect, F = 45.39, df = 1, p < .001, for the conceptual understanding composite. Post hoc tests using Bonferroni correction indicated that Time 1 means were significantly lower than Time 2 and Time 3 means (p < .001); there was no significant difference between Time 2 and Time 3 means. The same pattern emerged for the linear trend on the eight items on the DBI-SA. All items, except for maintenance programming, also had significant quadratic effects (p < .01).

Qualitative

Overall, nine teachers (56.25%) reported favorable perceptions of their conceptual understanding across all related questions, whereas the remaining teachers had mixed responses in their report of conceptual understanding of DBI. Two primary themes emerged from questions related to conceptual understanding: (a) making decisions based on intervention data and (b) helpful aspects of the PD series.

When making decisions based on intervention data, 15 teachers reported understanding how to make data-based decisions; one replied “somewhat,” and zero said “no.” When asked what they looked for when making data-based decisions, most teachers (68.75%; n = 11) indicated student performance in relation to their goal as the primary criterion. For example, Teacher 7 stated “I looked to see how consistently the goal was met. In addition, I checked the graph to see how much above the goal the student data was in order to adjust [the] next steps.” Teacher 5 specifically stated that establishing a reasonable goal, based on baseline data, was important for measuring student performance during intervention: “I understood that I should use the average of the baseline data to make a reachable goal for the student.” The teacher further discussed increasing the interval length once the student reached a high performance level: “and then, as I saw him responding to intervention (his percentage was between 80% and 100%), I chose to increase the interval time. I increased it again to see if he could consistently reach his goal.” Teacher 16, the only teacher to reply “somewhat” to the question, was unsure of what decisions to make when the student was unresponsive: “I didn’t know for sure if I should just continue with increasing the goal or maintaining the goal and trying to focus on one area of emphasis.”

Teachers identified aspects of PD that contributed to their conceptual understanding. One-hundred percent (n = 16) of the teachers indicated practicing with the self-monitoring app during PD and/or in the classroom as the most helpful. For example, Teacher 9 reported, “I think learning about it and getting to practice it in the classroom really strengthened my understanding.” The second most frequently mentioned helpful component of the PD series was collaborating with other teachers, either during PD sessions or at school (43.75%; n = 7). For example, Teacher 16 explained,

[I] enjoyed the discussion with colleagues with similar and different [behavioral] concerns. We never get this kind of time! [I] enjoyed troubleshooting and gained new strategies or ways to look at the situation. [I] liked the end session where we reviewed ways to set the class up for positive behavior success. [It] motivated me to make some post spring break changes to the end of the year.

Other frequently mentioned components of the PD that contributed to increased understanding were reviewing principles of behaviorism, applying concepts from PD to students in classrooms, working through case studies during PD, and the overall explanations and presentations used during PD. Teacher 7 stated, “The in-person training and hands-on experience with the app provided a concrete understanding.”

Self-Efficacy

Quantitative

The MANOVA showed a significant linear trend for self-efficacy, F = 116.82, df = 1, p < .001. Given our corrected p value, the quadratic trend was not considered to be significant, F = 7.48, df = 1, p = .02. Post hoc tests with Bonferroni correction revealed that Time 1 means were significantly lower than Time 2 and Time 3 means (p < .001); there were no significant differences between Time 2 and Time 3 means. This linear trend was found in the MANOVA examining differences at the item level, except with maintenance programming.

Qualitative

Overarching themes related to self-efficacy centered on (a) implementation of the DBI framework using the iPad app for data collection and the self-monitoring intervention, and (b) experience with the training. Within the first theme, teachers’ responses related to the implementation of the DBI framework included two subthemes: fidelity of implementation and confidence. Results from the open-ended questionnaire indicated eight teachers (50%) said they were able to implement data-based decisions with fidelity, one was unsure, two did not answer, three said they were not able to implement with fidelity. Teacher 15 said, “I feel comfortable doing this without support, but I have not had to yet.” Teacher 7 noted, “I think I was able to do it with fidelity. The student has been a good responder. I think if the student was more sporadic, I could have seen it being more difficult to make decisions.” One teacher (3), however, stated, “Fidelity was not as good as I had planned (feedback). There is nothing anyone could have done—it is just chaotic during transition.”

In terms of the confidence subtheme, 13 of the 16 teachers (81.25%) indicated positive feelings about their data-based decisions. Most teachers’ answers about feeling confident in their decisions were related to how their students were responding to intervention. For instance, Teacher 2 stated, “I feel confident in the changes because of what the student has exhibited thus far. He has been very consistent with staying on task and completing assignments.” Similarly from Teacher 7, “I feel the decisions have been valuable. When we reflect on meeting the goal, you can see the student beam with pride.” Another teacher (9) stated, “I felt my decisions were correct because the student was still able to reach the goal even when it was changed.” Only a couple of teachers (3 and 16) indicated some regret with their data-based decisions, stating, “I think I rushed upping the goal after a week of success,” and, “I wish I would have started with a lower goal to help the student experience success, and then gradually increased the goal.”

To gauge teachers’ experience with the training and how that related to self-efficacy, teachers reported on whether they would feel comfortable training others in DBI and self-monitoring. Consistent with the subtheme of confidence, 12 teachers (75%) indicated they would feel confident training others. No one said they would feel uncomfortable training others. Instead, they indicated hesitation and a partial level of comfort. Of those stating they would feel comfortable training others, responses included, “Yes, I would! I think learning about it and getting to practice it in the classroom really strengthened my understanding (Teacher 9).” Similarly, Teacher 7 said, “Yes, I would. The in-person training and hands-on experience with the app provided a concrete understanding.” Teacher 1 said, “Yes, because our training was very thorough and we have the PowerPoints to help guide it.” One response (Teacher 8) was predicated on student outcomes: “Yes, since my kiddo was very responsive, I feel more confident to share my experience.” One teacher (5) who displayed less confidence stated,

I would feel comfortable showing someone how to use the app, select an appropriate student, and track data because I felt solid on my choices in these areas. I would not feel comfortable answering other questions that teachers may have, sharing the research that informs this process, or setting an end goal because I do not have a solid understanding of these concepts.

Relatedly, teachers were asked, after having been through the training series, whether they would feel comfortable implementing this intervention within the context of DBI with another student. All but one teacher indicated they would like to implement with another student. The one teacher (13) who said no, instead said, “Not in such a large group setting, maybe in a more controlled setting.” Some teachers answering the question referred to the self-monitoring app, and not the DBI process or framework. For example, a teacher (7) said, “Yes, it was a good resource for empowering the kids to self-monitor behavior.” Others referred to both the intervention and the DBI process: “Yes—been through it and feel comfortable with the process (Teacher 10),” and “Absolutely! I feel comfortable with the process and implementation (Teacher 9).” Others indicated a desire to implement what they learned with other students: “Definitely! With the huge progress he made, I would use it with all students struggling with behavior (Teacher 12),” and “Yes, I would like to try it on someone with more intense behavior issues (Teacher 11).”

Usability and Feasibility

Quantitative

The MANOVA tests indicated a significant linear trend for usability and feasibility, F = 129.74, df = 1, p < .001. The quadratic effect was nonsignificant, F = 0.50, df = 1, p = .49. Post hoc tests with Bonferroni correction found that Time 1 means were significantly lower than Time 2 and Time 3 means (p < .001), and Time 2 means were significantly lower than Time 3 means (p < .001). There was variability in the linear trend when looking at the eight items on the DBI-SA for this domain.

Qualitative

To gauge teachers’ perceptions of usability and feasibility, teachers were asked about how using the app and DBI fits within the classroom. Themes focused on the ease of implementation and practicality of using the app.

Related to the ease of implementation, all teachers responded with comments focusing on the self-monitoring app, rather than the DBI process. Eight teachers (50%) expressed positive comments (e.g., “easily,” “seamlessly”) about classroom fit, whereas six (37.5%) had mixed reviews (e.g., “struggled at first,” “on a normal day it goes well, but normal days aren’t often,” “most of the time flowed fairly smoothly”). Teacher 1 had a negative response saying, “[It was a] difficult—tough class. Hard to manage other student issues at the same time,” and another teacher provided no response. Again, when reporting challenges or struggles, teachers focused on the app use during classroom instruction rather than the DBI process of examining data and making adaptations. For example, Teacher 3 said, “Most of the time things flowed fairly smoothly. But, finding time to give effective feedback to one student while monitoring several students in transition and providing instruction was often difficult.” Similarly, Teacher 6 stated,

It interrupted the flow slightly. My student takes her time to score herself but she is scoring with fidelity and cares about doing well. Other students notice the iPad and ask what it is about but they are respectful and get back to work. Sometimes I have to stop working with a group or student to score.

On the contrary, Teacher 10, said, “Seamlessly . . . at this point. My particular students involved in this project are familiar with different types of monitoring regarding behaviors. I have a para to help support me in collecting data.” Teacher 12 agreed, “It fits in easily and is very accessible no matter what we are doing in the classroom. I kept the same routine while he/we learned to implement the program. Very user friendly.”

Similar to the results about the ease of implementation, when asked about how useful and practical the app and DBI were in the classroom, teachers continued to focus on the app rather than the DBI process. Eleven teachers (68.75%) noted how useful, practical, and easy the app was to use. Teacher 11 stated, “Great, useful, easy, convenient. Motivating to student. Gave student status. Good attention time with student.” Though, several mentioned it may be easier to use during independent work time than during whole group instruction. Only one teacher mentioned DBI stating, “[I] would like to know more about how to taper students from the program once they have maintained the goal level for a significant period of time.”

Discussion

Previous research has indicated (a) many teachers are not adequately equipped to manage classroom behavior due to lack of experience and training, which may cause them to leave the field (Bradley et al., 2008; Freeman et al., 2014; Ingersoll, 2001); (b) teachers’ self-efficacy in managing classroom behavior may prevent attrition from the field (Aloe et al., 2014); and (c) many teachers are not trained in using data to make decisions about behavior (Stormont et al., 2011). Improving teachers’ knowledge, skills, and self-efficacy in managing challenging behavior and making data-informed decisions requires effective PD that is ongoing and embeds practice through authentic learning experiences with supportive feedback (Bruce et al., 2010; Lane et al., 2015). The need to expand and improve the methods by which teachers’ experiences with PD are documented also is central to improving practice (Desimone, 2009). Thus, the purpose of this study was to describe and evaluate teachers’ experiences in a PD model designed to provide an authentic learning experience and build self-efficacy related to implementation of DBI for students with challenging behavior.

Key Findings

Over the course of the school year, 16 elementary teachers participated in five PD sessions. Between DBI-PD sessions, participants practiced their skills in their classroom. Through this ongoing process of in-person PD followed by real classroom experience, we quantitatively assessed how teachers’ perceptions of their own conceptual understanding, self-efficacy, and usability/feasibility related to DBI changed over time. These results provide a sense of how teachers perceived their experience and related outcomes. Teacher self-assessment measures are a common quantitative tool used to examine teachers’ perceptions of their knowledge and skills (Ross & Bruce, 2007). Concurrently, we analyzed how teachers responded to open-ended questions related to each domain (i.e., conceptual understanding, self-efficacy, usability/feasibility) to further understand what aspects of the DBI framework and DBI-PD were most salient in the teachers’ experiences. The qualitative responses helped to provide a more descriptive context for teachers’ self-assessments on the DBI-SA. By collecting both quantitative and qualitative data from teachers, we were able to identify points of corroboration or contradiction (Johnson & Onwuegbuzie, 2004) within teachers’ evaluations of DBI and the DBI-PD. Combining teachers’ quantitative and qualitative responses allowed us to compare results from two types of data and describe how and why teachers’ perceptions of conceptual understanding, self-efficacy, and usability/feasibility changed over time (see Table 2).

Table 2.

Data Comparison.

DBI-SA themes	T1	T2	T3	Post hoc trend	Qualitative data
DBI-SA themes	T1	T2	T3	Post hoc trend	Congruent data	Discrepant data
Conceptual understanding	13.27 M 4.33 SD	21.44 M 3.42 SD	22.07 M 3.13 SD	1 < 2, 3	“I think learning about it and getting to practice it in the classroom really strengthened my understanding.” (Teacher 9) “The in-person training and hands on experience with the app provided a concrete understanding.” (Teacher 7)	“I didn’t know for sure if I should continue with increasing the goal and trying to focus on one area of emphasis.” (Teacher 16)
Self-efficacy	12.07 M 4.25 SD	20.31 M 3.28 SD	22.40 M 2.32 SD	1 < 2, 3	“I feel confident in the changes because of what the student has exhibited thus far. He has been very consistent with staying on task and completing assignments.” (Teacher 2) “I feel the decision have been valuable. When we reflect on meeting the goal, you can see the student beam with pride.” (Teacher 7)	“Fidelity was not as good as I had planned (feedback). There is nothing anyone could have done—it is just chaotic during transition.” (Teacher 3) “I would not feel comfortable answering other questions that teachers may have, sharing the research that informs this process, or setting an end goal because I do not have a solid understanding of these concepts.” (Teacher 5)
Usability and feasibility	12.73 M 3.63 SD	18.38 M 3.95 SD	22.31 M 2.27 SD	1 < 2 < 3	“It fits in easily and is very accessible no matter what we are doing in the classroom. I kept the same routine while he/we learned to implement the program. Very user friendly.” (Teacher 12) “Great, useful, easy, convenient. Motivating to student. Gave student status. Good attention time with student.” (Teacher 11)	“[It was a] difficult—tough class. Hard to manage other student issues at the same time.” (Teacher 1) “Most of the time things flowed fairly smoothly. But, finding time to give effective feedback to one student while monitoring several students in transition and providing instruction was often difficult.” (Teacher 3)

Note. DBI-SA = Data-Based Individualization Self-Assessment.

Data analysis indicated significant increases in teachers’ conceptual understanding and self-efficacy from prior to training (Time 1) to after the third training session (Time 2). However, there were no significant increases from Time 2 to Time 3. This may be explained by the fact that content and case study practice was front-loaded in the training sessions. That is, most exposure to content occurred in the first three training sessions. It is possible that by the last PD session, there was not much room for growth in teachers’ knowledge or confidence. As it relates to gains in conceptual understanding, teachers’ open-ended responses to the questionnaire further corroborates this, as many reported practice during trainings and working through case studies contributed to their gains in knowledge. An area for future inquiry could include examining how implementation fidelity changes as content knowledge and self-efficacy improve.

Interestingly, comments of teachers about what contributed to their self-efficacy not only focused on their training, but also on how students responded to intervention. For many teachers, it appeared that if they saw improvements in their students’ behavior, then they felt confident they were making the correct data-based decisions about how to implement and adapt intervention. To some degree, this contradicts statistical findings indicating no significant increases in self-efficacy from Time 2 to Time 3, though overall self-efficacy was high and, overall, there were significant improvements in students’ behavior (Bruhn, Rila, et al., 2018). Actual classroom implementation with a student occurred between Time 2 and Time 3, which is when teachers would have had a chance to observe changes in students’ behavior. Yet, quantitative data indicate the major increase in self-efficacy occurred prior to implementation. By situating the quantitative findings with the qualitative themes that emerged from the open-ended responses collected after DBI implementation, we gleaned a closer look into how teachers perceived their self-efficacy in the context of actual practice. Without the qualitative results, we may have presumed from the DBI-SA findings that the latter part of the PD was not as effective in improving teachers’ self-efficacy. When juxtaposed with the timeline of the PD and qualitative responses, however, we noted the distinction between self-reported self-efficacy and practice-based self-efficacy (Tschannen-Moran & Hoy, 2001).

Mostly, teachers indicated they would feel comfortable training other teachers in DBI, and implementing intervention and using DBI with another student. These responses correspond positively with conceptual understanding and self-efficacy scores on the DBI-SA indicating teachers understood the concepts, could explain them to others, had the ability to use the concept in their classroom, and could help others use the concept. In the future, researchers could examine the extent to which teachers who have completed DBI-PD are able to effectively train other teachers, much like in a “train the trainers” model of PD. Further, long-term follow-up with participants could provide important information about the feasibility, sustainability, and generalizability of DBI implementation.

Unlike conceptual understanding and self-efficacy, teachers made significant improvements in their perceptions of the usability and feasibility of DBI components across all three time points, indicating steady linear growth over time. We hypothesize this significant growth occurred more slowly than growth in other domains because teachers needed firsthand experience in their classrooms—not just case study practice during PD—to determine whether DBI implementation was practical and useful in their classrooms. This practice did not occur until after Time 2. However, analysis at the item level of the DBI-SA in conjunction with the qualitative responses indicates this finding is a bit more nuanced.

First, in response to the open-ended questions, nearly all teachers frequently reported their perceptions about usability and feasibility in relation to the self-monitoring intervention app, whereas only one teacher directly addressed DBI. This is important because the intervention and DBI are two separate concepts, but they are used concurrently. Item-level analysis of the DBI-SA questions indicated significant differences between all three time points for the concepts of “student self-monitoring of behavior” and “technology-based self-monitoring,” which both correspond to the intervention; whereas there was no significant growth between Time 2 and Time 3 (when it would be expected) on the items “data-based individualization,” “behavioral data collection,” and “behavioral data analysis,” which correspond directly to the data-based decision-making process in DBI. Thus, it appears after only practicing during three PD sessions, teachers felt collecting and analyzing data and then individualizing an intervention was useful and feasible in the classroom. It was not until after they had the chance to implement the self-monitoring intervention app in the classroom that they felt the intervention was feasible and would recommend it to others. To some degree, this suggests participants correctly view intervention and DBI as separate concepts, but it is also likely the phrasing of open-ended questions could be improved to ensure responses focus on only one concept at a time. We address this in the discussion of limitations. Finally, these findings also highlight the importance of practice and experience within the classroom in contributing to teachers’ perceptions. This is not surprising given the research indicating the use of authentic learning experiences is a critical component to effective PD (Bruce et al., 2010; Dunn et al., 2013).

Limitations

Although teachers improved perceptions of their own understanding, self-efficacy, and usability/feasibility related to DBI, and their experience with DBI-PD activities appears to have contributed to these improvements, findings are limited by several methodological and procedural issues. First, though teachers reported improvements in their understanding of DBI-related concepts, we did not include a direct measure of their knowledge. Thus, we cannot conclude if teachers truly learned the content of the PD, or if they just perceived that learning occurred. This is in contrast with other studies demonstrating both perception and actual knowledge may be significantly improved via effective, practice-based PD (Lane et al., 2015). Future studies on the effects of DBI-PD on teacher knowledge should include a pre–post assessment of participant knowledge.

Second, this study was descriptive rather than experimental. Potential future studies could include a control condition in which teachers receive no PD, receive PD on another topic, or receive PD on the same topic but using different PD principles. For instance, the DBI-PD model used in this study was conducted in-person and allowed participants to apply their learning in their own classroom with one of their students. Comparison models might involve web-based PD with telecoaching or more traditional “sit and get” PD.

Third, this study was part of a larger study examining the quantitative effects of using DBI on the behavior of elementary students (Bruhn, Rila, et al., 2018). Although the data collection, analysis, and interpretation for this study involved quantitative and qualitative data sources, the origins of the mixed-methods work were primarily quantitative, which also reflect the primary methodological training and orientation of the study’s authors. Mixed research studies should be conceived as coming from their own epistemological orientation where the incorporation of quantitative and qualitative methods leads to meta-inferences about the research question (Johnson & Onwuegbuzie, 2004; Onwuegbuzie & Corrigan, 2014) rather than simply the use of two separate methods in one study (e.g., multimethod; Creamer, 2017).

Lessons learned

Given our limited experience with qualitative data, we recognize at least two lessons learned, and thus areas for improvement to ensure a rigorous, fully mixed research design. In future studies involving open-ended questionnaires, we recommend including a qualitative researcher in the development of questions and protocols. That is, only after coding participants’ responses did we realize our questions may have been too complex in that they addressed multiple topics (e.g., How did the app and DBI fit into the structure and flow of your classroom?). We anticipate paying greater attention to the content validity of the questions and simplifying the language so that respondents have a clearer understanding about the phenomena we are exploring. As evidence of this, we highlight our poorly worded question: How useful and practical was the app and DBI for use in your classroom? As previously stated, participants focused mainly on the app instead of DBI. Breaking that question into two would have allowed us to explore both the intervention app and DBI separately without the inadvertent exclusion of one or the other. Or, had we developed an interview protocol for teachers, instead of using open-ended responses, we could have prompted teachers to evaluate the app or DBI more, based on their responses to the original question. In addition, we did not write the IRB application to allow for member checks, as is best practice for establishing rigor in qualitative analysis. In the future, we would include member checks to determine whether participants agreed with our interpretation of their responses. This limitation is tempered by the fact that all responses were written by participants, entered into a database verbatim by one research assistant, and checked for reliability by a different research assistant. We kept notes on our process as an audit trail. Had teachers been interviewed individually and their responses then transcribed, a member check would be imperative to establish the trustworthiness of findings. This last point underscores the need to include a qualitative researcher on any mixed research study to more fully support the analysis and interpretation of data.

Conclusion

Despite these limitations, this study contributes to a body of literature suggesting effective PD can improve content knowledge through active practice in authentic learning environments and should include collaboration and feedback with other professionals (Lane et al., 2015). By participating in PD guided by these effective principles, teachers made significant improvements in their conceptual understanding of DBI, self-efficacy related to using DBI, and perceptions of the usability and feasibility of DBI implementation. Qualitative responses allowed for a richer, more nuanced, understanding of quantitative findings. Furthermore, this study illuminates PD practices such as ongoing practice during training and in the classroom that led to improvements in students’ behavior (Bruhn, Rila, et al., 2018). By better understanding how teachers experienced DBI-PD, researchers and practitioners can deduce which PD components were the most salient while adapting other PD components to increase the likelihood of improving the behavior of students who receive a data-based, individualized intervention within the DBI framework.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

Funding for this study was provided by the Center for Educational Transformation (500-14-2540-00000-18190200-30-01).

References

Aloe

A. M.

Amo

L. C.

Shanahan

M. E.

(2014). Classroom management self-efficacy and burnout: A multivariate meta-analysis. Educational Psychology Review, 26, 101–126.

Armstrong

Anthes

(2001). How data can help. American School Board Journal, 188(11), 38–41.

Beam

H. D.

Mueller

T. G.

(2017). What do educators know, do, and think about behavior? An analysis of special and general educators’ knowledge of evidence-based behavioral interventions. Preventing School Failure, 61, 1–13.

Bradley

Doolittle

Bartolotta

(2008). Building on the data and adding to the discussion: The experiences and outcomes of students with emotional disturbance. Journal of Behavioral Education, 17(1), 4–23.

Bruce

C. D.

Esmonde

Ross

Dookie

Beatty

(2010). The effects of sustained classroom-embedded teacher professional learning on teacher efficacy and related student achievement. Teaching and Teacher Education, 26, 1598–1608.

Bruhn

A. L.

Freeman

Hirn

Kern

(2018). High-leverage practices: Feedback. In McLeskey

Maheady

Billingsley

Brownell

Lewis

T. J.

(Eds.), High-leverage practices for inclusive classrooms (pp. 107–119). Routledge.

Bruhn

A. L.

Rila

Mahatmya

Estrapala

Hendrix

(2018). Analyzing the effects of data-based, individualized interventions using multilevel modeling and visual analysis. Journal of Emotional and Behavioral Disorders. Advance online publication. doi:10.1177/1063426618806279

Chappuis

Stiggins

(2009). Supporting teachers. Educational Leadership, 66, 56–60.

Chesley

G. M.

Jordan

(2012). What’s missing from teacher prep. Educational Leadership, 69, 41–45.

10.

Creamer

E. G.

(2017). An introduction to fully integrated mixed methods research. SAGE.

11.

Creswell

J. A.

Clark

V. L.

(2017). Designing and conducting mixed methods research. SAGE.

12.

Desimone

L. M.

(2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38, 181–199.

13.

Dunn

K. E.

Airola

D. T.

Garrison

(2013). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. The Journal of Experimental Education, 81, 222–241.

14.

Feilzer

M. Y.

(2010). Doing mixed methods research pragmatically: Implications for the rediscovery of pragmatism as a research paradigm. Journal of Mixed Methods Research, 4(1), 6–16.

15.

Forness

S. R.

Kim

Walker

H. M.

(2012). Prevalence of students with EBD: Impact on general education. Beyond Behavior, 21, 3–9.

16.

Freeman

Simonsen

Briere

D. E.

MacSuga-Gage

A. S.

(2014). Pre-service teacher training in classroom management: A review of state accreditation policy and teacher preparation programs. Teacher Education and Special Education, 37, 106–120.

17.

Huberty

C. J.

Morris

J. D.

(1989). Multivariate analysis versus multiple univariate analyses. Psychological Bulletin, 105, 302–308.

18.

Ingersoll

(2001). Teacher turnover and teacher shortages: An organizational analysis. American Educational Research Journal, 38, 499–534.

19.

Johnson

R. B.

Onwuegbuzie

A. J.

(2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26.

20.

Johnson

R. B.

Onwuegbuzie

A. J.

Turner

L. A.

(2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112–133.

21.

Jones

M. L.

(2009). A study of novice special educators’ views of evidence-based practices. Teacher and Teacher Education, 32, 101–120.

22.

Kern

Wehby

J. H.

(2014). Using data-based individualization to intensify behavioral interventions: Implementation example. Teaching Exceptional Children, 46, 45–53.

23.

Kurasaki

K. S.

(2000). Intercoder reliability for validating conclusions drawn from open-ended interview data. Field Methods, 12, 179–194.

24.

Lane

K. L.

Menzies

Bruhn

A. L.

Crnobori

(2011). Managing challenging behaviors in schools: Research-based strategies that work. Guilford Press.

25.

Lane

K. L.

Oakes

W. P.

Powers

Diebold

Germer

Common

E. A.

Brunsting

(2015). Improving teachers’ knowledge of functional assessment-based interventions: Outcomes of a professional development series. Education and Treatment of Children, 38, 93–120.

26.

Malterud

Siersma

V. D.

Guassora

A. D.

(2016). Sample size in qualitative interview studies: Guided by information power. Qualitative Health Research, 26, 1753–1760.

27.

Morse

J. M.

(1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120–123.

28.

Nichols

B. W.

Singer

K. P.

(2000). Developing data mentors. Educational Leadership, 57(5), 34–37.

29.

O’Brien

R. G.

Kaiser

M. K.

(1985). MANOVA method for analyzing repeated measures designs: An extensive primer. Psychological Bulletin, 97, 316–333.

30.

Onwuegbuzie

A. J.

Corrigan

J. A.

(2014). Improving the quality of multiple methods research reports in the field of human resource development and beyond: A call for rigor as an ethical practice. Human Resource Development Quarterly, 25, 273–299.

31.

Ross

J. A.

Bruce

C. D.

(2007). Teacher self-assessment: A mechanism for facilitating professional growth. Teaching and Teacher Education, 23, 146–159.

32.

Saldaña

(2015). The coding manual for qualitative researchers. SAGE.

33.

Stecker

Fuchs

L. S.

Fuchs

(2005). Using curriculum-based measurement to improve student achievement: Review of research. Psychology in the Schools, 42, 795–819.

34.

Stormont

Reinke

Herman

(2011). Teachers’ knowledge of evidence-based interventions and available school resources for children with emotional and behavioral problems. Journal of Behavioral Education, 20, 138–147.

35.

Tschannen-Moran

Hoy

A. W.

(2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783–805.

36.

Wayman

J. C.

(2005). Involving teachers in data-driven decision making: Using computer data systems to support teacher inquiry and reflection. Journal of Education for Students Placed at Risk, 10, 295–308.

37.

Witt

J. C.

Elliot

S. N.

(1985). Intervention rating profile-15: Acceptability of classroom intervention strategies. In Kratochwill

T. R.

(Ed.), Advances in school psychology (pp. 251–288). Lawrence Erlbaum.