Advancing the Discussion About Systematic Classroom Behavioral Observation,a Product Review of Tenny,J. (2010). eCOVE Observation Software. Pacific City,OR

Materials

The eCOVE (Tenny, 2010) system requires the use of a laptop computer (Mac or PC). The program can also be synchronized with an iPhone, iTouch, iPad, or personal digital assistant (PDA). Reports may be transferred from one device to another via e-mail. The eCOVE software version 2.5.1.2 can be installed using a CD-ROM. Alternatively, anyone can download a free 30-day trial of any edition of eCOVE from www.ecove.net; if the software is purchased, the software license code will eliminate the 30-day time limit. Within the CD, the manual provides clear illustrations of how to use various features of the system and videos are available for how to use each of the observation templates on the previous basic version. Brief videos, provided by the developer, of how to use the latest version are provided online. No technical manual for the eCOVE system could be found; therefore, the reliability and validity statistics are unknown. The following information would be helpful in a technical manual for a systematic observation system: the level of independent observer agreement, concurrent validity of specific behavior observation categories (e.g., on-task behavior) with either other behavioral observation systems or with related behavioral rating subscales, treatment sensitivity (Hintze, 2005), and discriminative validity, such as accurately distinguishing between ADHD–predominantly inattentive type and ADHD–combined type (McConaughy et al., 2009).

New and Improved in the Latest Version (2.5.1.2)

There have been many improvements since the first version of the eCOVE software. The latest version (2.5.1.2) is associated with a completely different opening page, which is easier to navigate and includes more options as well as allowing the user to model different classes by arranging visual representations of the different students according to their current seating chart; the classes can be saved in the database for subsequent observations. Although the first version of eCOVE (1.4.5) includes many of the types of observations found in the new system, the most recent version includes new observational tools such as the following: tracking student hand raising, interruptions of the class by a student, student learning style, physical tics, please or thank-you statements, teacher feedback support, teacher praise rate and type, time spent on reading, speaker in discussion (enables one to calculate the percentage of time that each group member speaks), teacher response to misbehavior, and connecting content to a child’s background. Hand raising and speaker in discussion were the only templates for observing specific types of behavior as they unfold that were added since version 2.4.1. However, the latest version is the only one that provides choice scale observations, which are checklists that the observer can use to rate the quality of the classroom environment on a number of different factors (e.g., whether clear and positive rules are posted on the wall). In addition, in the most recent version, one can create an unlimited number of observational templates.

The new version allows the user to specify the status of the individual being observed (i.e., teacher, student, gender, classroom) and the user assigns a unique identification number to each student and teacher to ensure privacy. The later versions (2.4.1 and 2.5.1.2) have a specific section designed to enter attributes for each of the students and teachers, so during specific student observations, the individual’s specific attributes are displayed on the screen. For example, the observer can note the individual’s primary language, behavior type, Individual Education Plan (IEP) type, eligibility, math, and reading level, as well as the attributes of teachers.

In addition, the latest versions include interval timers and a descriptive tool set that organizes the variety of observation types. The two recent versions allow the observer to work with multiple observation types at one time, which enables observers to combine strategic observation types. For instance, one can track teacher eye contact and types of teacher utterances in conjunction with student on-task coding. However, when one is using timers to track behaviors for more than one individual (e.g., student and teacher), one needs to be aware that the timer for an individual stops once the observations are switched to the next individual. The timer can be resumed once the observation on the original individual resumes. If the observer wants to track the behavior of multiple students simultaneously, eCOVE allows one to track the number of behaviors exhibited, such that one can calculate the percentage of intervals in which multiple children exhibit a certain behavior.

The latest version of eCOVE allows one to make group comparisons among students with a variety of attributes. For example, one can compare students with disabilities with the rest of the classroom on any of the observation tools.

Intended Users

The eCOVE behavioral observation system can be used by applied psychologists, counselors, consultants, school administrators, English Language Learning (ELL) educators and special education teachers. In fact, there are separate versions for administrators and special educators, whereas psychologists and counselors are offered a general version. There is also a separate version for ELL classrooms.

Reviewers’ Use of eCOVE

Both reviewers familiarized themselves with the eCOVE system and practiced using it while independently observing classroom scenes from movies or television shows. The first reviewer observed the first two episodes of Teach, a reality show on A&E (Patel, 2010a, 2010b) in which the actor Tony Danza decided to become a teacher in a low socioeconomic status high school in Philadelphia. Tony’s instructional consultant and principal explained to him that he talks too much, failing to let the students respond. To examine this, the first author used the eCOVE system to find that Tony indeed talked the vast majority of the time during his first few periods of class and rarely provided any wait time for student’s to respond to his questions. This was associated with high levels of off-task behavior on the part of students. The second reviewer compared a running narrative of classroom scenes from Kindergarten Cop (Grazer & Reitman, 1990), starring Arnold Schwarzenegger, with a behavioral observation summary from the same 12 min of footage. She found that a global impression of her narrative was congruent with the results from the computer-generated eCOVE summary report. The Kindergarten Cop was good at maintaining eye contact (95% of the time) with his class, but used verbal corrections over 1.5 times more frequently than praise. The narrative revealed that many of those verbal corrections entailed yelling. Fortunately, one can record important time-stamped narrative comments with the eCOVE system while using various systematic observation tools. If the reviewers were to consult with the Kindergarten Cop during his 1st week as a teacher, one goal for improvement would likely be improving the praise to correction ratio from 1/1.5 to 3/1 (Wheldall, 2005).

Child Observation With eCOVE

One can meticulously follow the on-task versus off-task behaviors of children with the eCOVE system. Both on-task and off-task behaviors are broken down by various subtypes (e.g., looking, working, responding to the teacher, out of seat, talking to peers). Although many researchers and practitioners have focused on on-task behavior as a unitary construct, different types of on-task behaviors have different correlations with teacher ratings of student attention and certain types of on-task behavior are more likely in different classroom situations (Lauth et al., 2006). Tracking on-task and off-task behaviors may be useful for psychologists dealing with referrals for inattentive or hyperactive behaviors. For example, during behavioral consultation, on-task behavior can be used to monitor a child’s progress during the baseline and different intervention phases (Froiland, 2011b). Following different types of on-task behaviors may reveal which type of on-task behavior is most sensitive to a particular treatment. One can also track the extent to which students are displaying positive nonverbal signs, such as smiling or positive gestures, which could be useful for tracking the progress of children who also have internalizing concerns. In addition, social skills can be tracked with eCOVE, such as hand raising and appropriately saying “Please” and “Thank you.” There is also an observation scheme for tracking the rate of verbal or physical tics per minute, which could be valuable for referrals involving symptoms of Tourette’s syndrome or for monitoring this potential side effect of psychostimulants (Varley, Vincent, Varley, & Calderon, 2001).

Because this latest version of the software allows the user to combine multiple observation types, strategic combinations of observations can help one get a more comprehensive idea of the child’s behaviors. For example, one can track both on-task coding and compliance with directions. Because teachers often exaggerate the ratings of children’s attention problems when the student is oppositional (Stevens, Quittner, & Abikoff, 1998), measuring both on-task behavior and compliance with directions (i.e., compliance vs. verbal, physical, or passive noncompliance) could help the observer to discern which would be the best primary intervention target. Observers can use these data to better inform interventions aimed at making the environment more conducive to student engagement.

The observer can also track student’s hand raising while conducting an observation on another student (or teacher). This tool calculates the percentage of possible times that each student raises their hand; students with hyperactivity and impulsivity tend to forget to raise their hand and wait for their turn to speak. If many children in the class forget to raise their hands, appropriate entry into the classroom discussion could be taught to the whole class, whereas if only one or two students struggle with this, appropriate hand raising could be one of many targets for an individualized intervention.

Observing Teachers

eCOVE provides a variety of structured ways to observe teachers and provide useful data for teacher consultation. For instance, one could use the Teacher Talk observation to discover that a teacher talks 90% of the time, students talk 7% of the time, and no one is talking 3% of the time. Because students often learn more and feel more engaged when they are able to discuss the information, part of the intervention could involve the teacher providing students more time to answer a question, rather than answering his or her own questions when students do not respond immediately. The teacher’s implementation of this technique could be followed using the Wait Time observation, which looks at the average elapsed time before the student responds or the teacher begins to talk again. To promote thoughtful student participation, the teacher could also be encouraged to use higher level questioning, which could be tracked using the eCOVE observation entailing Bloom’s Taxonomy (e.g., looks at the use of questions that promote reflective evaluation, analysis, application, and synthesis vs. the mere recall of information; Mayer, 2002). This is just one example of how this system could be employed in teacher consultation.

Psychologists can use the Talking out of Turn (which looks at how the teacher responds to children who verbally interrupt the discussion or lecture) or Response to Behavior observations to track how a teacher responds to children’s behavior, which could be useful in measuring the implementation fidelity of a classroom behavioral intervention. Likewise, the percentage of the teacher’s provision of different types of praise can be monitored (e.g., praising efforts, behavior, or attitude) as well as how frequently the different types of praise are provided per minute. One can also examine the extent to which the teacher makes eye contact with the audience versus looking at notes, the screen, and so on. In addition to specific observation structures, this version of the software includes brief objective checklists, so that one can record whether or not established classroom environment factors are in place, such as student work featured on the walls and positively stated class rules that are visible in a prominent location. Rather than relying on one’s memory of good instructional practices, such checklists may aid the observer in identifying whether a variety of important instructional strategies are being utilized by the teacher.

Tracking the teacher’s directly observed behaviors allows the consultant to help teachers to objectively examine their teaching methods. This information could assist psychologists during teacher consultation because teacher behaviors can be tracked over time and monitored for improvement. Tracking a teacher’s behaviors can coincide with tracking student’s behaviors to see if improvements in the student’s behavior correlate with enhanced instructional and classroom management methods.

Although many of the behavioral observation templates within the eCOVE system are supported to some extent by behavioral or educational research, the learning styles template is not supported. One can analyze the extent to which teachers provide activities that have one of the following emphases: visual, auditory, tactile, manipulative, and kinesthetic. Yet, the research on the validity and utility of learning styles suggests that they are not a part of evidence-based instruction (Landrum & McDuffie, 2010). However, coding the percentage of activity in the classroom that has a kinesthetic component may be germane for children with attention problems. This is because physical exercise may enhance cognitive functioning and provide an opportunity for attention restoration (Halperin & Healey, 2011). In particular, one qualitative case study found that the student displayed less off-task behavior during activities that entailed kinesthetic activity (Wheeler et al., 2009), and others have recommended that teachers provide students with brief physical activity breaks to restore their attention levels (Cooper & Bilton, 2002). Therefore, in cases in which the psychologist believes that physical activity could be beneficial in promoting concentration, tracking the percentage of class time that allows for kinesthetic learning could be useful.

In version 1.4.5 of eCOVE, the attention to gender template enables the observer to determine the extent to which teachers engage in the following practices with boys versus girls: questioning, extended questions, praise/acknowledgement, and error correction. For the purposes of observations for children with attention disorders, this specific tool should be reinstated in future versions due to the fact that boys are overrepresented among ADHD referrals; most referrals are made by teachers, and there is now evidence that many teachers have a gender bias in determining whether or not a referral is necessary (Sciutto, Nolfi, & Bluhm, 2004). The most recent versions of eCOVE allow one to make a gender comparison once the observation is completed. Once one is ready to create a report, one can create comparisons between groups. Although this feature is useful, one can only examine gender comparison once the observation is over instead of being able to visually track attention to gender throughout the observation. The reviewers felt that this comparison by gender report feature was somewhat hard to find, thus making this capability much less salient than before. The eCOVE attention to gender template could be used to find out if boys are getting praised for good behavior less or if girls are being ignored more when they are off-task. Let us say that a psychologist receives a referral for a boy in one classroom who is off-task roughly 30% of the time. In the initial observation, the psychologist also observed that a randomly selected male peer was off-task 12% of the time. Yet, the psychologist noticed that a nonreferred girl was off-task 28% of the time. The psychologist then uses the attention to gender observation template and finds that boys are being corrected (e.g., “Pay attention or you will lose your recess today!”) more for off-task behavior in the classroom, whereas the off-task behavior of girls is largely ignored. In this chimerical classroom, members of both genders rarely receive praise for answering questions well, treating others well, exhibiting diligence, or completing work. However, boys receive more rhetorical questions (e.g., “What do you think you are doing?”), which are perceived of as controlling by children and decrease motivation (Froiland, 2011a), and girls receive more extended questions about educational content. In this scenario, the psychologist may help the teacher devise an intervention that could include the following: using descriptive and genuine praise to encourage the on-task behavior of boys and girls in the class; asking boys more extended questions about the curricular content and using fewer rhetorical questions; either using planned ignoring for the off-task behavior of boys and girls or consistently correcting boys and girls, in which case the psychologist could provide consultation regarding effective ways of delivering warnings and consequences.

Discussion

Structured classroom behavioral observations often play an important role in assessing and treating children with attention and hyperactivity problems. This review indicates the eCOVE software is likely to be helpful to practitioners interested in measuring a wide variety of student and teacher behaviors that are important for children with attention problems and difficulties that are often comorbid. Although a technical manual or other research examining the reliability and validity of the eCOVE is necessary, many of the eCOVE behavioral observation tools are likely to be quite useful, based on recent scientific findings in the attention disorders literature.

In consideration of recent research on behavioral observations for children with attention problems, the eCOVE system has a number of observation tools that are germane to the assessment of children who struggle to pay attention. For instance, On-Task Coding allows one to analyze the type of on-task behavior a child displays, which is important because teachers may respond more favorably to inconspicuous on-task behaviors than other forms of on-task behaviors (Lauth et al., 2006). On-task classroom behavior is a particularly important measure, distinguishing children with ADHD from those who do not have ADHD (Rapport et al., 2009). Furthermore, on-task behavior is helpful in tracking the progress of children with attention problems in response to interventions (Froiland, 2011b). Because ODD and ADHD are often comorbid (APA, 2000) and oppositional behavior may inflate teacher ratings of inattention (Stevens et al., 1998), the Compliance With Directions observation scheme may help psychologists to determine which behavior issue is more prominent for a particular child. Although many of the eCOVE observation tools have a connection to sound research, the learning styles observation tool should not be used because the research has not supported the validity of learning styles or teaching that focuses on learning styles (Landrum & McDuffie, 2010). Although the latest version of eCOVE has the feature of comparing groups on multiple attributes (e.g., gender) when creating a report, this version no longer has the specific attention by gender observation tool. The specific attention by gender tool makes the relevance of gender more salient, which is important because teachers have a proclivity to refer more boys than girls for attention problems, even when behaviors are similar (Sciutto et al., 2004). Overall, the eCOVE system has many observation tools that are important for measuring children’s attention-related behavior; however, an examination of the research suggests that the attention to gender observation tool should be reinstated and the learning styles template should be eliminated.

With a variety of structured observations for examining teacher behavior, the eCOVE behavioral observation system highlights the importance of examining the classroom environment. One can examine how frequently children are being praised by the teacher for good behavior, the percentage of questions asked that promote high-order thinking, how the teacher responds to student misbehavior, and other teacher behaviors that may be important for promoting the attention and self-regulation of students. In general, psychologists serving children with attention problems may want to increase their use of systematic behavioral observations because the classroom environment can promote or hinder attention (Lauth et al., 2006; Wheeler et al., 2009). Furthermore, systematic behavioral observations can be used to help examine whether the classroom instruction is at a pace and level that will provide an optimal intellectual challenge for the student, so as to prevent inattention related to boredom (due to easy tasks) or sheer frustration (due to tasks that are too difficult).

Potential consumers may be more confident about the utility of this product if various forms of reliability and validity are addressed in future studies involving the eCOVE, such as the level of agreement among independent observers, concurrent validity of specific behavior observation categories (e.g., on-task behavior) with related behavioral rating subscales, and treatment sensitivity (Hintze, 2005). Furthermore, there is now evidence that behavioral observations can establish discriminative validity (McConaughy et al., 2009). Thus, future research could investigate whether eCOVE observation data, collected by observers blind to children’s diagnoses, can be used to accurately differentiate between children with ADHD–combined type and ADHD–predominantly inattentive type, and children who have normal levels of attention.

Once practitioners become familiar with this comprehensive observation system, they will likely find that they save time. In particular, the computer-generated calculations of behavioral statistics and graphs of progress can save time. Precious time can also be saved by deciding from a menu of observation targets on one’s computer or iPhone, rather than scrambling to find the right reproducible hard copy of an observation form or exhausting time creating one’s own observation system for each unique case. One of the poignant strengths of the eCOVE is the range of child and teacher behaviors that can be observed with well-structured formats and relatively straightforward recording procedures.

Conclusion

The eCOVE behavioral observation could help psychologists effectively monitor the behavior of children with attention problems and the germane behavior of their teachers. There are many observation tools within the eCOVE system that are well suited to assessing children with attention problems; however, it is important that researchers examine the reliability and validity of the eCOVE before firmer conclusions can be made. Due to the fact that many practitioners use systematic behavioral observation during the assessment/consultation process (Wilson & Reschly, 1996) independent research on this system is warranted.