Special Educators and Data Recording

Abstract

This study examined the effects of delayed recording on the accuracy of data recorded by special educators serving students with high- or low-incidence disabilities. A multi-element design was used to compare the accuracy of data recorded across three conditions: (a) immediately after a student’s target behavior occurred, (b) immediately after the conclusion of the instructional lesson, and (c) 3 hr after the conclusion of the instructional lesson. Results indicated special educators achieved higher levels of agreement recording data immediately after a student’s target behavior occurred and immediately after the conclusion of the instructional lesson.

Keywords

data recording delayed recording special educators disabilities

Federal mandates require special educators to develop individualized education programs (IEP) containing measurable goals and objectives as well as a statement of how students’ progress toward meeting their goals and objectives is being measured (Individuals with Disabilities Education Act [IDEA], 34 C.F.R. § 300.347[a][7]). Federal mandates also require special educators to provide effective and efficient instruction leading to student success in school (No Child Left Behind Act, 2001; Simonsen et al., 2010; Vannest, Temple-Harvey, & Mason, 2009). To do so, special educators must use evidence-based practices and justify their instructional decisions based on data (Boardman, Arguelles, Vaughn, Hughes, & Klingner, 2005; No Child Left Behind Act, 2001). Data allow special educators to meet IDEA requirements by providing evidence illustrating student progress and documenting the effectiveness of instructional practices (Babkie & Provost, 2004; Boardman et al., 2005). Specifically, data play a role in the development of evidence-based practices and are used to assess student achievement (McDuffie & Scruggs, 2008).

Despite considering it important, special educators often fail to engage in data recording due to difficulties recording data within the classroom, feeling overwhelmed at the varying student ability levels, frustration teaching while concurrently managing student behaviors, and aggravation with the lack of time available (Boardman et al., 2005). However, to accurately document student progress, teachers must regularly engage in data recording (Brown, Lehr, & Snell, 2006; Horner, Sugai, & Todd, 2001). When data are not recorded regularly and accurately, teachers may make incorrect conclusions about student progress and intervention effectiveness (Kennedy, 2005). One method for accurately measuring student progress is immediate direct observation recording (Farlow & Snell, 1989; Lewis-Palmer, Sugai, & Larson, 1999; Munger, Snell, & Loyd, 1988) that requires observing and recording a student’s behavior immediately after it occurs (Kennedy, 2005). Unfortunately, many teachers find immediate direct observation recording too time-consuming and a task that interferes with the fluidity of their teaching (Jones, 2009; Munger et al., 1988; Walton, 1985).

Studies examining special educator accuracy in recording student performance data are few (Logan, 1991). Two studies examined whether special educators could record data accurately to reflect students’ performance after completion of the task: Logan (1991) and Taber-Doughty and Jasper (2012). Logan first examined whether special educators could accurately record data on their students after the completion of a target behavior. He suggested that immediate direct observation recording was not necessary when recording the behaviors of students with more severe disabilities as their skill acquisition was slower and they tended to demonstrate greater behavioral constancy. He hypothesized that teachers could accurately document student performance at some point after the completion of the task. In this dissertation study, four special educators recorded data at three different time periods (e.g., 10 min, 30 min, end of the day) on student behaviors. Results indicated data were most accurate when recorded immediately following instructional sessions that were 10 or 30 min in length. When teachers waited until the end of the day to record data, accuracy decreased. Logan concluded the teachers in the study did not need to engage in simultaneous data recording due to the consistency in students’ behavior. He noted their data were accurate when the delay between behavioral occurrence and data recording was 10 to 30 min in length.

More recently, Taber-Doughty and Jasper (2012) examined the effects of delayed recording on the accuracy of data recorded by three special educators on the behaviors of students with mild and moderate disabilities during three different time periods (e.g., immediately after the behavior occurred, at the end of the school day, at the start of the following school day). The authors hypothesized that inconsistencies in student behavior would require teachers to engage in immediate direct observation recording to obtain accurate data. Indeed, findings suggested data recorded during the time period immediately after the target behavior occurred were more accurate than those recorded at the end of the school day or the following school day. However, limitations noted that findings applied only to students with mild and moderate disabilities. In addition, the number of behaviors each teacher was asked to remember was another limitation. Teachers were asked to record data on three target behaviors per student; thus, the need to remember several behaviors possibly affected the accuracy of the teachers’ data. Finally, although immediacy in data recording was most accurate, teachers reported this was the most difficult and least feasible time period for which to record data (Taber-Doughty & Jasper, 2012).

There is a need to determine feasible data recording methods that result in the most accurate data and are easy for teachers to use. Unfortunately, those teachers who choose not to engage in data recording may encounter issues such as lacking the information necessary to work successfully with students and the ability to determine instructional programs that work best (Walton, 1985). Another potential issue is that without data, teachers may be unable to accurately assess changes in students’ learning and behavioral performance. Therefore, it is important teachers’ instructional decisions be based on data as they allow them to accurately evaluate student progress and make appropriate instructional decisions (Farlow & Snell, 1989; Utley, Zigmond, & Strain, 1987).

To date, little published research examines whether teachers can accurately record student learning and behavioral performance data at some point after behavioral completion rather than via immediate direct observation recording (Logan, 1991). Therefore, the purpose of this study was to replicate and expand on previous investigations (Logan, 1991; Taber-Doughty & Jasper, 2012) examining the effects of delayed recording on the accuracy of data recorded by special educators. However, unlike previous investigations, this study evaluated the length of time between student performance and teachers’ data recording to determine whether the delay had an impact on the accuracy of data recorded by each teacher. Four special education teachers recorded data on student participants with disabilities. Participants included students from high- and low-incidence disability popu-lations and the analysis assessed the accuracy of data recorded during three conditions. The research question was as follows:

Research Question 1: When comparing data by special education teachers on their students with disabilities, in which condition will they record the most accurate data: (a) immediately after the student’s target behavior occurred, (b) immediately after the conclusion of the instructional lesson, or (c) 3 hr after the conclusion of the instructional lesson?

Method

Participants

Four teachers (pseudonyms) were chosen based on the following criteria: (a) each held a current teaching license in special education, (b) presently taught in middle or junior high school, (c) were willing to participate, and (d) agreed to follow all data recording requirements. In addition, eight students (pseudonyms) were selected to participate based on the following: (a) had a primary diagnosis of a mild intellectual, learning, or emotional/behavioral disability (i.e., high-incidence), or a moderate/severe/profound intellectual disability (i.e., low-incidence) or autism; (b) attended secondary school as a member of a participating teacher’s class; (c) indicated willingness to participate; and (d) received parent/guardian consent for students’ participation. Table 1 provides a description of each student’s disability and behaviors targeted for data recording.

Table 1.

Description of Each Student’s Targeted Behavior.

Student	Disability	Targeted behavior	Method of data recording	Teacher responsible
Angel	EBD	Noncompliance (i.e., refusal to participate in class after receiving a directive from the teacher)	Event recording for every instance observed	Ms. Ebaka
Cale	LD, ADHD	Crying (i.e., tears in eyes or streaming down face)	Event recording for every instance observed	Ms. Ebaka
April	ASD, speech	Activity refusal (i.e., refusal to participate in class activities)	Event recording for every verbal prompt given to participate in class	Ms. Abernathy
Micah	ASD, speech	Schoolwork refusal (i.e., refusal to complete schoolwork)	Event recording for every verbal prompt given to begin or continue schoolwork	Ms. Abernathy
Andrew	EBD, speech	Schoolwork refusal (i.e., refusal to complete schoolwork)	Event recording for every verbal prompt given to begin or continue schoolwork	Ms. Kristic
Dustin	EBD, LD	Talk-outs (i.e., Speaking without permission during instruction)	Event recording for every verbal prompt given to not speak without permission	Ms. Kristic
Kenya	MID, communication	Chest touching (i.e., inappropriate touching of her breasts)	Event recording for every instance observed	Ms. Howard
Scott	ASD, speech	Eye poking (i.e., poking or attempting to poke other people in their eyes)	Event recording for every instance observed	Ms. Howard

Note. EBD = emotional/behavior disability; LD = learning disability; ADHD = attention deficit hyperactivity disorder; ASD = autism spectrum disorder; MID = mild intellectual disability.

Ms. Ebaka

Ms. Ebaka was a fourth-year special educator at a junior high school. She taught basic mathematics and language arts to seventh- and eighth-grade students with high-incidence disabilities for four periods per day. Each class period was 50 min in length. She indicated she recorded daily academic and social skills data on each of her students. She printed progress reports weekly and evaluated students’ grades for trends.

For the present study, Ms. Ebaka recorded daily data on Angel and Cale. Angel’s targeted behavior for data recording was noncompliance; specifically, refusing to participate in class after receiving a directive from her teacher. If Angel did not comply with Ms. Ebaka’s directive within 1 min, an instance of noncompliance was documented. Cale’s behavior targeted for data recording was crying in class. Examples included tears filling in eyes or streaming down face with or without sound. This was operationally defined with an onset–offset time period. For example, one instance of crying occurred when a tear(s) left the eye, lasted for at least 5 s, and after it ended for at least 5 s. An instance was documented each time Cale cried during the class period.

Ms. Abernathy

Ms. Abernathy was a first-year special educator at a middle school. She taught life/functional skills to fifth- and sixth-grade students with moderate or severe intellectual disabilities in a self-contained classroom setting; her class periods were 30 min in length. Prior to this study, she indicated she was not recording data and noted she was “still trying to gain an understanding of how to run the classroom.” Due to the variability of the students’ behaviors, Ms. Abernathy recorded data on the number of verbal prompts she gave April and Micah to cease their targeted behavior. April’s targeted behavior was activity refusal; specifically, refusing to participate in class activities. Similarly, Micah’s targeted behavior was schoolwork refusal; specifically, refusing to complete schoolwork during class. Therefore, Ms. Abernathy documented each verbal prompt she gave April to begin or continue participation in the class activity and each verbal prompt she gave Micah to begin or continue her schoolwork. A verbal prompt consisted of Ms. Abernathy prompting the students and waiting 5 s for the students to comply. If the students did not comply with Ms. Abernathy’s verbal prompt within 5 s, an instance was documented on the data recording sheet. An instance was documented for each verbal prompt.

Ms. Kristic

Ms. Kristic was in her 15th year as a special educator at a middle school. During her 50-min class periods, she taught mathematics, language arts, social studies, and science to fifth- and sixth-grade students with high-incidence disabilities. At the time of this study, she and her paraeducators recorded student learning and behavior performance data using daily point sheets tied to a behavioral level system used in her classroom. For this study, Ms. Kristic recorded data on the number of verbal prompts she gave Andrew and Dustin to cease their targeted behaviors. Andrew’s targeted behavior was schoolwork refusal, specifically, refusing to complete schoolwork. Dustin’s targeted behavior was talk-outs, specifically, speaking without adult permission during whole-group instruction. Thus, Ms. Kristic documented the number of verbal prompts she provided Andrew to facilitate his engagement in schoolwork completion and each verbal prompt she gave Dustin to obtain adult permission before speaking out. A verbal prompt consisted of Ms. Kristic prompting the students and waiting 5 s for the students to comply. If the students did not comply with Ms. Kristic’s verbal prompt within 5 s, an instance was documented on the data recording sheet. An instance was documented for each verbal prompt.

Ms. Howard

Ms. Howard was in her 12th year as a special educator at a junior high school. She taught functional skills in a self-contained classroom to seventh- and eighth-grade students with low-incidence disabilities; her class periods were 30 min in length. Ms. Howard regularly recorded data on the students in her classroom using permanent products and direct observation methods. For this study, Ms. Howard recorded student learning performance data on Kenya and Scott. Kenya’s behavior targeted for data recording was inappropriate touching of her upper chest area (i.e., breasts). Scott’s behavior targeted for data recording was poking other people in their eyes. Ms. Howard documented each instance she observed Kenya and Scott engaged in their target behaviors during the class period.

Settings

This study was conducted in two middle/junior high schools in a suburban Midwestern city. Comparison sessions took place within each teacher’s classroom where students were observed during the same instructional periods each school day. Ms. Ebaka and Ms. Kristic taught instructional periods 50 min in length while Ms. Abernathy and Ms. Howard taught instructional periods 30 min in length. Teachers were required to record data on behaviors that occurred during whole-group instruction. The first author served as an investigator and was present during each comparison session to (a) ensure teachers followed the data recording schedule for each student’s target behavior and (b) act as an independent observer.

Six students and three adults were present in Ms. Ebaka’s classroom during comparison sessions. The room was arranged with eight student desks positioned in front of an interactive white board (IWB) that was used during group instruction. Near the back was the teacher’s desk with a small table in which small group instructional activities took place. Four computers and a bookcase filled with books and novels were located in the front of the room for student use.

Ten students and five paraeducators were present in Ms. Abernathy’s classroom. The room contained 3 tables in the shape of a semi-circle with 10 chairs used for small-group instruction, a microwave, 2 bookcases, cabinets, and 5 computers. Ten student desks were positioned in front of an IWB that was used during whole-group instruction. A divider separated a small section of the room that was designated as the time-out area.

In Ms. Kristic’s classroom, there were 15 students and 3 additional adults. Her classroom was divided into three rooms. The primary classroom contained a large table with 8 chairs, a lounge area with beanbag chairs, a computer area, and 25 student desks positioned in front of an IWB. Ms. Kristic used the IWB during group instruction. There were two adjacent smaller rooms used for small group instruction and time-out.

Ms. Howard’s class consisted of 10 students and 4 adults. Students sat in desks facing an IWB. The classroom also contained a lounge chair, bookcase, couch, table with 10 chairs, and a swing. Adjacent to the main classroom was a full kitchen and a separate private bathroom.

Variables

The dependent variable was the level of accuracy with which each teacher recorded data. Accuracy was measured by comparing teacher-recorded data to that recorded by the investigator using the videotaped observations and was defined as the percentage of agreement between teacher- and investigator-recorded data. To be considered accurate, each teacher needed to achieve an 85% level of agreement with the investigator’s data. This level of agreement is considered acceptable in single-subject research (Kennedy, 2005). To control for the impact of previous knowledge teachers had about their students, each teacher recorded data on student behaviors not previously gathered.

To evaluate the effects of delayed recording on data accuracy, three independent variables were randomly alternated to determine the condition teachers would record student data most accurately. A random number generator was used to develop a sequence for each condition to ensure each occurred equally and in a randomized order. These conditions were (a) immediately after the student’s target behavior occurred, (b) immediately after the conclusion of the instructional lesson, and (c) 3 hr after the conclusion of the instructional lesson. Each condition was evaluated to determine which time period allowed teachers to record data that were accurate.

Data Recording

Prior to the study, target behaviors were selected and operationally defined to ensure greater accuracy in data recording. Event recording was used by the teachers to document the number of times each student exhibited a target behavior or the number of teacher prompts required for a student to engage in on-task behavior (see Table 1 for a list of each student’s target behavior). In addition, the investigator acted as an independent observer and recorded frequency data on the number of times each student exhibited a target behavior or the number of prompts the teacher provided to students to engage in on-task behavior. Event recording was selected due to its ease in use and accuracy for capturing data related to the targeted behaviors (Kennedy, 2005).

All instructional periods (i.e., sessions) for each teacher were video recorded for interobserver agreement (IOA) purposes. These videotapes were used by two trained observers to verify IOA. IOA was necessary to ensure the reliability of the results of this study (Kennedy, 2005).

Research Design

A multi-element design was used to illustrate the impact of delayed recording on the accuracy of data recorded on student performance by teachers during three different conditions. Multi-element designs allow for the evaluation of the differential effects of two or more treatments that are alternated in quick succession using a single participant (Kennedy, 2005). This design was selected as it allowed for an immediate comparison between the three conditions to determine whether one resulted in greater accuracy than another (Barlow & Hersen, 1984).

Using a random number generator, the investigator randomized under which condition teachers would record data on student behaviors. This randomization reduced the likelihood of teachers predicting the order in which conditions would occur (Barlow & Hersen, 1984). Each teacher recorded data for two conditions each day according to a randomized sequence. Teachers were assigned only one condition during an instructional period. For example, a teacher could record data for Condition A during one instructional period and Condition C for another instructional period in the same day.

Procedures

Pre-comparison

One week prior to beginning the pre-comparison sessions, a video camera was placed in each teacher’s classroom to reduce the novelty effect of its presence on students. Teachers were instructed on data recording procedures for each condition and practiced recording frequency data on a sample student not included in this study. The investigator was present during each pre-comparison session and provided verbal feedback when each teacher recorded data correctly or incorrectly. Data agreement of 100% was required to begin comparison.

Comparison

Teachers recorded data on student performance across 15 days with 10 sessions for each condition and the conditions were randomized. The investigator acted as an independent observer during all sessions of each condition and used event recording to record data on student performance. In addition, the investigator’s presence ensured each teacher recorded data during the designated condition. All sessions were video recorded to verify IOA.

Social Validity

Social validity was evaluated through an investigator-created, paper–pencil questionnaire using open-ended questions. Prior to the study, teachers were provided a questionnaire asking them to describe their current data recording procedures, success of IEP objectives, student achievement, their thoughts regarding the study, and which data recording condition they thought would be most accurate. Each was also provided a questionnaire following comparison. Teachers were surveyed to determine which data recording condition they thought was most accurate, whether they would continue to record data during that condition, and their likes and dislikes about data recording. In addition, teachers were informally interviewed to discuss topics not covered on the questionnaire to allow them to express any additional thoughts about their experiences.

IOA and Treatment Fidelity

To ensure reliability, IOA data were recorded across 33% of comparison sessions for each condition by the investigator and two trained independent observers simultaneously yet independently from one another. The observers viewed randomly selected videotapes to document target behaviors or number of prompts per student. IOA was calculated by dividing the number of agreements by the number of agreements plus disagreements and then multiplying by 100. Agreement for Angel and Cale (Ms. Ebaka), Kenya and Scott (Ms. Howard), and Dustin (Ms. Kristic) was 100% for all sessions in which data were recorded. Agreement for April (Ms. Abernathy) ranged between 90% and 100% with a mean of 96.6%. Finally, agreement for Micah (Ms. Abernathy) and Andrew (Ms. Kristic) ranged between 80% and 100% with mean scores of 90% and 93.3%, respectively, for all sessions in which data were recorded.

The investigator created a procedural reliability checklist containing a task list of comparison steps to ensure each teacher correctly recorded data and recorded them during the correct designated condition. Treatment fidelity data were recorded during 30% of the comparison sessions for each condition for each teacher to ensure trustworthiness and consistency of the comparison implementation. The percentage of implementation was calculated by dividing the number of correctly implemented steps by the number of steps possible and then multiplying by 100%. Treatment fidelity results ranged from 89% to 100% for each teacher across students and conditions with mean scores of 96.3% for Ms. Ebaka, 94.5% for Ms. Allen and Ms. Howard, and 100% Ms. Kristic.

Results

A comparison of mean differences revealed each teacher achieved higher levels of agreement recording data during Conditions A (immediately after the behavior occurred) or B (immediately after the conclusion of the instructional lesson) for at least one of their students. Results were not consistent across teachers or across conditions. Table 2 illustrates the level of agreement for each teacher’s data per condition. Table 3 describes the frequency of behaviors observed by each teacher.

Table 2.

Level of Agreement by Condition for Each Teacher/Student Pair.

Teacher and students	Condition A	Condition B	Condition C
Ms. Ebaka
Angel	79.2% (100/25–100)	70% (100/0–100)	60% (100/0–100)
Cale	90% (100/0–100)	100%	100%
Ms. Abernathy
April	64.4% (97/0–100)	53.5% (55/0–100)	57.7% (72/0–100)
Micah	86.7% (100/0–100)	55.9% (58.5/0–100)	42.4% (35/0–100)
Ms. Kristic
Andrew	48.3% (58.5/0–100)	69% (75/50–100)	66.3% (75/0–100)
Dustin	90% (100/0–100)	70% (100/0–100)	90% (100/0–100)
Ms. Howard
Kenya	78% (97/0–100)	95.2% (100/67–100)	85.7% (86.5/50–100)
Scott	89.3% (100/0–100)	93.4% (100/75–100)	70% (100/0–100)

Note. First number represents the level of agreement. Within the parenthesis, the first number represents the median and the second number represents the range for each condition. For Cale, median and range were 100 during Conditions B and C. Condition A = immediately after target behavior occurred; Condition B = immediately after the instructional lesson; and Condition C = 3 hr after the conclusion of the instructional lesson.

Table 3.

Behavioral Frequency by Teacher Participant.

	Teacher participant (student participant)
Session	Eubank (Angel)	Eubank (Cale)	Abernathy (April)	Abernathy (Micah)	Kristic (Andrew)	Kristic (Dustin)	Howard (Kenya)	Howard (Scott)
1	1/2 (50)	0/0 (100)	2/2 (100)	2/10 (20)	2/1 (50)	12/12 (100)	0/0 (100)	3/3 (100)
2	1/0 (0)	0/0 (100)	3/2 (67)	6/6 (100)	0/0 (100)	6/6 (100)	14/15 (93)	13/14 (93)
3	2/1 (50)	0/0 (100)	1/1 (100)	15/15 (100)	2/3 (67)	0/0 (100)	1/1 (100)	0/0 (100)
4	0/0 (100)	1/1 (100)	0/0 (100)	2/3 (67)	1/0 (0)	0/0 (100)	5/3 (60)	0/0 (100)
5	3/2 (67)	0/0 (100)	1/0 (0)	2/12 (17)	3/3 (100)	0/0 (100)	5/6 (83)	0/0 (100)
6	0/0 (100)	0/0 (100)	5/3 (60)	4/3 (75)	0/3 (0)	0/0 (100)	9/12 (75)	11/14 (79)
7	3/3 (100)	0/0 (100)	2/0 (0)	3/3 (100)	0/0 (100)	0/1 (0)	35/30 (86)	0/0 (100)
8	0/0 (100)	0/0 (100)	3/0 (0)	20/0 (0)	0/0 (100)	0/0 (100)	0/0 (100)	0/0 (100)
9	1/1 (100)	0/0 (100)	2/0 (0)	6/5 (83)	0/0 (100)	3/0 (0)	5/5 (100)	0/0 (100)
10	2/0 (0)	0/0 (100)	0/2 (0)	5/4 (80)	0/0 (100)	0/0 (100)	2/1 (50)	0/0 (100)
11	1/0 (0)	0/0 (100)	0/0 (100)	11/11 (100)	0/0 (100)	5/5 (100)	25/27 (92)	0/0 (100)
12	1/0 (0)	0/0 (100)	35/33 (94)	11/11 (100)	0/0 (100)	5/5 (100)	16/20 (80)	0/0 (100)
13	0/0 (100)	0/0 (100)	1/1 (100)	12/12 (100)	4/5 (80)	0/4 (0)	6/8 (75)	5/4 (80)
14	0/0 (100)	0/0 (100)	3/3 (100)	17/30 (57)	2/5 (40)	5/0 (0)	16/16 (100)	0/0 (100)
15	0/0 (100)	0/0 (100)	3/3 (100)	3/3 (100)	3/10 (30)	0/0 (100)	1/1 (100)	0/0 (100)
16	1/0 (0)	0/0 (100)	1/3 (33)	3/3 (100)	6/0 (0)	0/0 (100)	2/2 (100)	0/0 (100)
17	0/1 (0)	0/0 (100)	1/0 (0)	3/1 (33)	3/1 (33)	0/0 (100)	6/6 (100)	1/0 (0)
18	0/0 (100)	0/0 (100)	0/0 (100)	1/0 (0)	1/1 (100)	0/0 (100)	24/21 (87)	2/2 (100)
19	0/0 (100)	0/0 (100)	0/0 (100)	0/1 (0)	0/0 (100)	0/0 (100)	2/1 (50)	1/1 (100)
20	0/0 (100)	0/0 (100)	2/1 (50)	0/1 (0)	1/2 (5)	0/0 (100)	37/35 (96)	0/0 (100)
21	0/0 (100)	0/0 (100)	2/0 (0)	5/6 (83)	1/2 (50)	0/0 (100)	3/3 (100)	0/0 (100)
22	4/1 (25)	0/0 (100)	0/0 (100)	8/2 (25)	1/2 (50)	0/0 (100)	0/0 (100)	0/0 (100)
23	0/0 (100)	0/0 (100)	4/1 (25)	2/0 (0)	0/2 (0)	0/0 (100)	4/4 (100)	0/0 (100)
24	0/0 (100)	0/0 (100)	2/4 (50)	4/4 (100)	2/4 (50)	0/0 (100)	0/0 (100)	2/2 (100)
25	2/2 (100)	0/0 (100)	0/0 (100)	7/3 (43)	0/2 (0)	2/0 (0)	0/0 (100)	6/8 (75)
26	0/0 (100)	1/1 (100)	1/0 (0)	0/0 (100)	1/2 (50)	0/0 (100)	15/16 (94)	0/0 (100)
27	0/2 (0)	0/1 (0)	2/0 (0)	4/2 (50)	7/6 (86)	0/0 (100)	2/3 (67)	0/3 (0)
28	0/0 (100)	0/0 (100)	0/0 (100)	0/0 (100)	0/0 (100)	0/0 (100)	2/2 (100)	0/0 (100)
29	0/0 (100)	0/0 (100)	0/0 (100)	6/12 (50)	2/0 (0)	0/0 (100)	0/0 (100)	0/1 (0)
30	0/0 (100)	0/0 (100)	10/13 (77)	3/2 (67)	1/1 (100)	0/0 (100)	2/0 (0)	1/0 (0)

Note. First number represents investigator’s data and the second number represents the teacher’s data. Level of agreement as percent of agreement in parenthesis.

Ms. Ebaka

Ms. Ebaka recorded data on Angel’s refusal to participate in class and Cale’s crying. When examining data recording sessions on Angel’s target behavior, Ms. Ebaka recorded accurate data for 19 sessions with 31.5% (n = 6) occurring during Condition A, 37% (n = 7) occurring during Condition B, and 31.5% (n = 6) occurring during Condition C. In addition, Ms. Ebaka recorded accurate data for 29 sessions in which she observed Cale’s target behavior with 31% (n = 9) occurring during Condition A, 34.5% (n = 10) occurring during Condition B, and 34.5% (n = 10) occurring during Condition C.

Based on the average for each condition, none of the data recorded for Angel were considered accurate (i.e., greater than 85% agreement); however, all data recorded for Cale were considered accurate. Ms. Ebaka achieved her highest level of agreement recording data immediately after Angel’s target behavior occurred (Condition A; = 79.2%; median = 100; range = 25–100). When examining the level of agreement for data recorded on Angel’s behavior immediately after the conclusion of the instructional lesson (Condition B; median = 100; range = 0–100), 70% agreement was achieved while 60% agreement was achieved 3 hr after the conclusion of the instructional lesson (Condition C; median = 100; range = 0–100). The mean agreement on the number of behaviors exhibited by Cale was 90% immediately after his behavior occurred (Condition A; median = 100; range = 0–100) and 100% immediately after the conclusion of the instructional lesson (Condition B; median = 100) and 3 hr after the conclusion of the instructional lesson (Condition C; median = 100).

Ms. Abernathy

Ms. Abernathy recorded the number of verbal prompts she gave April to participate in activities and Micah to complete her schoolwork during class activities. When examining data recording sessions on April’s target behavior, Ms. Abernathy recorded accurate data for 14 sessions with 43% (n = 6) occurring during Condition A, 28.5% (n = 4) occurring during Condition B, and 28.5% (n = 4) occurring during Condition C. In addition, Ms. Abernathy recorded accurate data for 11 sessions in which she observed Micah’s target behavior with 73% (n = 8) occurring during Condition A, 9% (n = 1) occurring during Condition B, and 18% (n = 2) occurring during Condition C.

Based on the average for each condition, none of the data recorded for April were considered accurate (i.e., greater than 85% agreement). In addition, only the data recorded for Micah during Condition A were considered accurate. Ms. Abernathy achieved her highest level of agreement recording data immediately after April’s target behavior occurred (Condition A; = 64.4%; median = 97; range = 0–100). When examining the level of agreement for data recorded on April’s behavior immediately after the conclusion of the instructional lesson (Condition B; median = 55; range = 0–100), 53.5% agreement was achieved while 57.7% agreement was achieved 3 hr after the conclusion of the instructional lesson (Condition C; median = 72; range = 0–100). For Micah, Ms. Abernathy achieved her highest level of agreement recording data immediately after the target behavior occurred (Condition A; = 86.7%; median = 100; range = 0–100). When examining the levels of agreement for data recorded immediately after the conclusion of the instructional lesson (Condition B; median = 58.5; range = 0–100), 55.9% agreement was achieved while 42.4% agreement was achieved 3 hr after the conclusion of the instructional lesson (Condition C; median = 35; range = 0–100) for Micah.

Ms. Kristic

Ms. Kristic recorded the number of verbal prompts she gave Andrew to complete his schoolwork and Dustin to obtain adult permission before speaking out. When examining data recording sessions on Andrew’s target behavior, Ms. Kristic recorded accurate data for 13 sessions with 23% (n = 3) occurring during Condition A, 38.5% (n = 5) occurring during Condition B, and 38.5% (n = 5) occurring during Condition C. In addition, Ms. Kristic recorded accurate data for 25 sessions in which she observed Dustin’s target behavior with 36% (n = 9) occurring during Condition A, 28% (n = 7) occurring during Condition B, and 36% (n = 9) occurring during Condition C.

Based on the average for each condition, none of the data recorded for Andrew were considered accurate (i.e., greater than 85% agreement). In addition, only the data recorded for Dustin during Conditions A and C were considered accurate. When examining data recorded immediately after Andrew’s behavior occurred (Condition A; median = 58.5; range = 0–100), 48.3% agreement was achieved while the highest level of agreement occurred immediately after the instructional lesson (Condition B; = 69.0%; median = 75; range = 50–100). Ms. Kristic achieved an agreement level of 66.3% 3 hr after the conclusion of the instructional lesson (Condition C; median = 75; range = 0–100). For Dustin, Ms. Kristic achieved the highest level of agreement immediately after his target behavior occurred (Condition A; = 90%; median = 100; range = 0–100). The mean agreement on the number of behaviors exhibited by Dustin was 70% immediately after the conclusion of the instructional lesson (Condition B; median = 100; range = 0–100) and 90% 3 hr after the conclusion of the instructional lesson (Condition C; median = 100; range = 0–100).

Ms. Howard

Ms. Howard recorded data on Kenya’s touching of her upper chest area and Scott’s target behavior of poking other people in the eyes. When examining data recording sessions on Kenya’s target behavior, Ms. Howard recorded accurate data for 21 sessions with 28.5% (n = 6) occurring during Condition A, 43% (n = 9) occurring during Condition B, and 28.5% (n = 6) occurring during Condition C. In addition, Ms. Howard recorded accurate data for 23 sessions in which she observed Scott’s target behavior with 39% (n = 9) occurring during Condition A, 30.5% (n = 7) occurring during Condition B, and 30.5% (n = 7) occurring during Condition C.

Based on the average for each condition, data recorded for Kenya during Conditions B and C, and data recorded for Scott during Conditions A and B were considered accurate (i.e., greater than 85% agreement). For Kenya, the level of agreement immediately after her target behavior occurred (Condition A; median = 97; range = 0–100) was 78% while 95.2% agreement (i.e., accurate) was achieved immediately after the instructional lesson (Condition B; median = 100; range = 67–100). When examining the level of agreement for data recorded on Kenya’s behavior 3 hr after the instructional lesson, 85.7% agreement was achieved (Condition C; median = 86.5; range = 50–100). For Scott, the level of agreement was 89.3% immediately after his target behavior occurred (Condition A; median = 100; range = 0–100). When examining the level of agreement for data recorded immediately after the instructional lesson, 93.4% agreement was achieved (Condition B; median = 100; range = 75–100) while 70% agreement (i.e., not accurate) was achieved 3 hr after the instructional lesson (Condition C; median = 100; range = 0–100).

Social Validity

Prior to the study, all teachers reported they thought the condition immediately after the behavior occurred would produce the most accurate data. Specifically, Ms. Ebaka stated these data would be “fresh in her mind” and allow her “to remember them more accurately.” Ms. Kristic reported, “so many things happen in a day and after some time goes by it is hard to remember what all happened that day.” When asked about their concerns regarding recording data during the different conditions, Ms. Kristic reported she did not feel she would “be able to get it all done in the midst of a busy day.” In addition, Ms. Abernathy mentioned time as a concern, stating, “I don’t want to take time away from teaching to record data.” Despite concerns, all teachers reported data recording as beneficial. Ms. Howard noted data recording will “allow me to see patterns in behaviors so that I can think about ways to change it.”

Following comparison, teachers were surveyed to determine which data recording condition they thought was most accurate, whether they would continue to record data during that condition, and their likes and dislikes about data recording. All reported they thought the condition immediately after their student’s target behavior occurred produced the most accurate data. When asked whether they would continue using this method of data recording in the future, some seemed unsure. Ms. Ebaka reported, “If there is a specific behavior I’m targeting, I will.” However, Ms. Kristic reported this condition was too difficult for her to use because “it takes a lot of time and I don’t have a lot of time.” All teachers expressed the importance of data recording and their need to improve their current methods. Overall, each indicated that Condition B (immediately after the instructional lesson) was their preferred condition.

Discussion

This study examined the effects of delayed recording on the accuracy of data recorded by four special educators. All four teachers met the necessary 85% agreement level for their data to be considered accurate for at least one student during either Conditions A or B. Unfortunately, three teachers (i.e., Ms. Ebaka, Ms. Abernathy, and Ms. Kristic) did not record accurate data for their other student during these same conditions. To be considered accurate enough to make databased decisions, each teacher’s data needed to be within 85% agreement of the videotaped observations to be considered acceptable (Kennedy, 2005). For three of the students in which data were not considered accurate, teachers achieved a range of 48.3% to 79.2% data agreement. Thus, additional training in data recording is necessary to increase consistency in accurate data recording.

When comparing the data each teacher recorded during the three conditions, each had higher levels of agreement among some conditions than others. For instance, Ms. Ebaka (Cale), Ms. Kristic (Andrew), and Ms. Howard (Kenya and Scott) successfully recorded data during Condition B for one of their students. In addition to achieving higher levels of agreement during Condition B, Ms. Ebaka (Cale), Ms. Kristic (Dustin), and Ms. Howard (Kenya) each experienced success during Condition C with one student. Unfortunately, not all teachers consistently recorded accurate data for both of their students across conditions.

There are several possible reasons why these inconsistencies in data recording may have occurred. It is possible that Ms. Ebaka, Ms. Kristic, and Ms. Howard’s previous teaching experience with related data recording contributed to their success during Conditions B and C. Each was a veteran teacher with more than 8 years of teaching and data recording experience. In contrast, Ms. Abernathy was in her first teaching year. While she experienced the most success during Condition A, she reported it was difficult to remember how students performed when the delayed recording was in place and she was not required to record data immediately. Interestingly, all teachers completed the same graduate program and received training in data recording from the same faculty member, although at different times. All of these factors suggest that Ms. Abernathy’s lack of experience with teaching and data recording likely affected her level of accuracy. Furthermore, these factors suggest that more immediate data recording is necessary for those special educators new to the field.

It is also very likely that the frequency of each student’s target behavior affected the accuracy of data that each teacher recorded. For instance, Ms. Ebaka (Cale) and Ms. Kristic (Dustin) both recorded data on students who had low frequencies of target behaviors. The low frequency of the target behaviors may have made them easier for the teachers to remember and document. It is also possible that changes in Cale’s and Dustin’s medication affected the accuracy of data that each teacher recorded. At the beginning of this study, Cale frequently exhibited his target behavior. However, toward the middle of the study, his medication was changed and his target behavior decreased to almost zero levels. Alternatively, Dustin exhibited his target behavior infrequently throughout the entire study. However, on two occasions, Dustin failed to receive his medication before coming to school and his target behavior increased. Thus, when Dustin and Cale experienced an increase in their target behavior, it was apparent (i.e., salience) to their teachers as typically their target behaviors were nonexistent. Similarly, Ms. Howard may have been affected by the salience of Scott’s target behavior—eye poking. This target behavior may have been so prominent that Ms. Howard was more likely to remember it occurring. Thus, salience and the low frequency of target behaviors may have had an impact on the teachers’ ability to record accurate data and attributed to higher levels of agreement during each condition. On the contrary, many teachers recorded data on students who had higher frequencies of target behavior. For example, Ms. Ebaka (Angel), Ms. Abernathy (April and Micah), and Ms. Kristic (Andrew) did not record consistent data for their students who exhibited target behaviors at higher frequencies. Consequently, the high frequency of the target behavior may have made it difficult for teachers to track the data they observed; thus, affecting the accuracy of the data each teacher recorded and attributing to lower levels of agreement during each condition.

Conclusion, Implications, Limitations, and Future Directions

Although immediate direct observation recording is espoused as the most accurate data recording method, teachers did not consider it the most feasible particularly because of the barriers they face when using this data recording method (Boardman et al., 2005; Taber-Doughty & Jasper, 2012). Prior research provided evidence to suggest special educators can engage in accurate delayed recording; however, the delay between behavior occurrence and recording should be 30 min or less (Logan, 1991; Taber-Doughty & Jasper, 2012). Similarly, in this study, some teachers experienced success engaging in delayed recording for at least one of their students. However, similar results were not observed during delayed recording conditions (i.e., Conditions B and C) for all teachers with both of their students, except Ms. Howard. The inconsistencies among teachers suggest that the results were not sufficient or accurate enough (i.e., greater than 85%) to warrant use for databased decision making. Although delayed recording may have the potential to address the barriers teachers often cite as reasons why they fail to engage in data recording, additional research is necessary to confirm the reliability of this data recording method (Boardman et al., 2005).

As previously discussed, there are a few possible issues that may have affected the findings of this study as well as teachers’ ability to record data consistently. For one, the salience and frequency of students’ target behaviors may have had an impact on the accuracy of data recorded and the level of agreement achieved by each teacher during each condition. The investigator made attempts to address these variations in student behavior by having each teacher record student performance data on a single target behavior per student in an effort to decrease the likelihood of student behaviors affecting teacher accuracy (i.e., salience, behavior is easy to remember). Future research should consider choosing target behaviors that are less prone to salience and do not pose difficulties for teachers to maintain high levels of treatment integrity and reliable data recording (e.g., moderate frequency behaviors). Choosing these target behaviors may involve working closely with the teacher or, perhaps, the investigator completing independent observations of students to determine target behaviors that might be appropriate for use in the study. Another possible issue included changes in medication for two students. Although this is the nature of applied research, future research should make attempts to select student participants who are less likely to experience environmental changes such as changes in medication, daily schedules, and absences. The use of student participants who will exhibit consistent behaviors throughout the duration of the study may have a significant impact on and, possibly, strengthen the findings of the study.

Several limitations were identified. One is the method used to calculate IOA, gross calculation. Using this method, the number of agreements was divided by the number of agreements plus disagreements and then multiplied by 100. Therefore, there is a possibility that the independent observers documented the same number of behaviors during a given instructional period and achieved 100% agreement. However, they may not have observed the same exact behaviors. In the future, this study should be replicated using interval agreement or occurrence and nonoccurrence agreement to ensure an accurate representation of IOA (Kennedy, 2005).

Another limitation is the lack of generalizability. While only four teachers served as participants in this study, findings were consistent across all four (i.e., higher levels of agreement during Conditions A and B) for at least one of their students, but not for both. Due to the small number of participants and the inconsistency of the results, it is difficult to determine whether the findings apply to the larger population. In addition, the homogeneity of the teacher sample makes it difficult to generalize findings beyond the specific group of participants used in this study (Barlow & Hersen, 1984). Therefore, future research should systematically replicate this study to verify whether teachers can engage in delayed data recording and obtain an accurate representation of student performance, and generalize findings (Kennedy, 2005).

A final limitation includes threats to internal validity. Attempts were made by the investigator to avoid a confounding variable. Data were recorded during alternating conditions to avoid order effects and prevent teachers from predicting the order in which conditions occurred (Barlow & Hersen, 1984). Unfortunately, not all teachers recorded data at an acceptable level of accuracy (i.e., agreement). Particularly, the data recorded by Ms. Ebaka, Ms. Abernathy, Ms. Kristic, and Ms. Howard were not considered acceptable for one of their students. Thus, the findings of this study should be interpreted with caution.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Babkie

A. M.

Provost

M. C.

(2004). Teachers as researchers. Intervention in School and Clinic, 39, 260–268.

Barlow

D. H.

Hersen

(1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). Elmsford, NY: Pergamon Press.

Boardman

A. G.

Arguelles

M. E.

Vaughn

Hughes

M. T.

Klingner

(2005). Special education teachers views of research-based practices. The Journal of Special Education, 39, 168–180.

Brown

Lehr

Snell

M. E.

(2006). Conducting and using student assessment. In Snell

M. E.

Brown

(Eds.), Instruction of students with severe disabilities (7th ed.; pp. 73–121). Upper Saddle River, NJ: Pearson.

Farlow

L. J.

Snell

M. E.

(1989). Teacher use of student performance data to make instructional decisions: Practices in programs for students with moderate to profound disabilities. Journal of the Association for Persons With Severe Handicaps, 14, 13–22.

Horner

R. H.

Sugai

Todd

A. W.

(2001). “Data” need not be a four-letter word: Using data to improve school wide discipline. Beyond Behavior, 11, 20–26.

Jones

M. L.

(2009). A study of novice special educators’ view of evidence-based practices. Teacher Education and Special Education, 32, 101–120. doi:10.1177/0888406409333777

Kennedy

C. H.

(2005). Single-case designs for educational research. Boston, MA: Pearson Education.

Lewis-Palmer

Sugai

Larson

(1999). Using data to guide decisions about program implementation and effectiveness. Effective School Practices, 17, 47–53.

10.

Logan

(1991). An analysis of the accuracy of student performance data recorded by teachers under three conditions (Unpublished doctoral dissertation). Georgia State University, Atlanta.

11.

McDuffie

K. A.

Scruggs

T. E.

(2008). The contributions of qualitative research to discussions of evidence-based practice in special education. Intervention in School and Clinic, 44, 91–97.

12.

Munger

Snell

M. E.

Loyd

B. H.

(1988). How often do you need to collect student performance data? A study of the effects of frequency of probe data collection and graph characteristics on teachers’ visual inference. Retrieved from ERIC database. (ED310560)

13.

No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002).

14.

Simonsen

Shaw

S. F.

Faggella-Luby

Sugai

Coyne

M. D.

Rhein

. . . Alfano

(2010). A school wide model for service delivery: Redefining special educators as interventionists. Remedial and Special Education, 31, 17–23. doi:10.1177/0741932508324396

15.

Taber-Doughty

Jasper

A. D.

(2012). Does latency in recording data make a difference? Confirming the accuracy of teachers’ data. Focus on Autism and Other Developmental Disabilities, 27, 168–176. doi:10.1177/1088357612451121

16.

Utley

B. L.

Zigmond

Strain

P. S.

(1987). How various forms of data affect teacher analysis of student performance. Exceptional Children, 53, 411–422.

17.

Vannest

K. J.

Temple-Harvey

K. K.

Mason

B. A.

(2009). Adequate yearly progress for students with emotional and behavioral disorders through research-based practices. Preventing School Failure, 53, 73–83.

18.

Walton

W. T.

(1985). Educators’ responses to methods of collecting, storing, and analyzing behavioral data. Journal of Special Education Technology, 7, 50–55.