Evaluation of Remote Versus Face-to-Face Observation of Teacher Candidates in an Alternative Certification Program

Abstract

The University of Kentucky provides an alternative certification program to prepare teacher candidates in moderate and severe disabilities. Observations occur every semester that candidates are in the 2-year program and must be conducted all across Kentucky, often many miles away from campus. University researchers placed cameras in the classrooms of teacher candidates and measured reliability of teacher observations conducted face-to-face and remotely. The researchers also surveyed perceptions of the teacher candidates and university observers and analyzed the cost-effectiveness of the two observation methods. The results indicated that the observers achieved similar results on an observation form regardless of the observation method used. The remote observations were the most cost efficient of the two methods both monetarily and in the time commitment needed by the observers. The findings include the perceptions of the teacher candidates and university observers for benefits and challenges of each observation method.

Keywords

teacher education remote observation rural special education alternative certification

Special education teacher shortages in the United States have been a persistent problem that has negatively impacted the nation being able to fulfill the obligations of the Individuals with Disabilities Education Act (2004; Cook & Boe, 2007; McLeskey, Tyler, & Flippin, 2004; Thornton, Peltier, & Medina, 2007). The U.S. Department of Education Office of Postsecondary Education (2017) published a report on teacher shortages for every state in the United States. Data were presented on shortages by academic area from 1990 to 2017. The results showed that all states have reported teacher shortages in multiple academic disciplines for more than 25 years. For the latest data reported (2016–2017 and 2017–2018), the report indicated special education teacher shortages for all states except Georgia, Vermont, and New Mexico. Shortages are even more dire in rural school districts where recruiting and retaining qualified teachers is difficult (Bailey & Zumeta, 2015; Sundeen & Wienke, 2009) and are compounded by issues including geographic separation, highly diverse student populations, poverty, lower wages, and less access to high-quality professional development (Johnson, Ohlson, & Shope, 2018; Sundeen & Wienke, 2009). In a report published by the Education Commission of the States, Aragon (2016) reported national data indicating that teacher shortages in special education and rural regions have been persistent across 13 years of data from 1999 to 2012.

Similar to national shortages, Kentucky has reported teacher shortages for students with severe disabilities for each year since 1990 except for the year of 1998 to 1999 (U.S. Department of Education Office of Postsecondary Education, 2017). This lack of qualified personnel severely limits the ability of the schools to provide a quality, free appropriate public education for this population of students with high-intensity needs. Therefore, Kentucky, like other states, established alternative routes to teacher certification programs to increase the number of certified special education teachers (Childre, 2014; Humphrey & Wechsler, 2007; Rosenberg & Sindelar, 2005). The University of Kentucky (UK) operates such a university-based alternative certification program to prepare teachers of school-aged students with moderate and severe disabilities (MSD) in a response to the shortages of special education teachers.

In this program, individuals who hold a bachelor’s degree in any discipline and are accepted into the program at UK, are allowed to teach on a temporary provisional certificate issued by the Kentucky education professional standards board (EPSB) while they receive training for 2 years. UK trains teacher candidates across the entire commonwealth due to the distance education capability of the program. Therefore, the university is challenged to provide high-quality distance education including advising services, delivery of course content, and supervision of teaching. Distance education has been used effectively for a number of years to train rural special education teachers with various technologies used for delivering content (Ludlow & Collins, 2016). In fact, because of special education shortages, universities have been using distance learning technologies for a number of years (Collins, Baird, & Hager, 2009; Glomb, Midenhall, Mason, & Salzberg, 2009; Ludlow & Duff, 2009) to prepare special educators in an effort to meet the demand of hard-to-fill special education positions. At UK, course content in the alternative certification program in MSD is delivered for two academic years. All coursework, except for two courses, is offered in a synchronous online format at times after the school day. Teacher candidates join the course synchronously using Zoom software (https://zoom.us/) from their home computer alongside classmates taking the class in person on campus. Two technology courses (instructional technology and assistive technology) that require hands-on experiences are conducted during the summer months and teacher candidates are required to attend those courses in person on campus. They are conducted in an accelerated format so that each class is completed in 2 weeks.

While the teacher candidate is enrolled in the program, the EPSB in response to a Kentucky Administrative Regulation (16 KAR 9:080; http://www.lrc.ky.gov/kar/016/009/080.pdf) requires the school district employing the student and UK to maintain a mentoring agreement. Within this agreement, the school district assigns a mentor teacher to the alternative certification teacher candidate. Then, the UK conducts 5 hr of observation of the teacher candidate providing instruction in the classroom, the mentor teacher conducts 5 hr of observation, and UK and the district agree on who will conduct an additional 5 hr of observation. The mentoring agreement also describes the role of the mentor teacher in providing in-class and out-of-class support and a process for maintaining regular communication between UK and the district on the progress of the teacher candidate. Each year when the student applies for a new temporary provisional certificate, the mentoring agreement with an update on student progress is submitted to the EPSB with the application. Teacher candidates in the program are observed by university personnel every semester, except for summer semesters, as part of the requirements of the program.

Given that UK allows teacher candidates working anywhere in Kentucky to enter the program, the observation component of the program is taxing on the resources of the department. Because of the geographic distance of many of the teacher candidates, faculty must devote many hours in travel time and are only able to observe candidates teaching in their classrooms 2 times a semester. This limits the frequency of observations that university supervisors can provide despite the knowledge that providing frequent observations with immediate feedback is necessary for creating change in teaching behavior (Scheeler, Congdon, & Stansbery, 2010; Scheeler, McAfee, Ruhl, & Lee, 2006; Scheeler, Ruhl, & McAfee, 2004). To assist with the need to conduct observations of teacher candidates living and working many miles away from campus, UK purchased cameras to install in the classrooms of the candidates living more than 45 miles away from campus. These cameras were to be used by university supervisors, physically located on campus, with the capability to control the video (i.e., pan, tilt, zoom) and to hear audio to observe teacher candidates in the classroom. However, a Kentucky Administration Regulation (16 KAR 9:080; http://www.lrc.ky.gov/kar/016/009/080.pdf) requires that the observers of teacher candidates in alternative certification programs must be physically present in the classroom when conducting observations. Therefore, UK requested a waiver with the EPSB to conduct a pilot study to allow a university faculty member to observe the candidate through the use of a video camera placed in the teacher candidates’ classrooms. The waiver request was approved by the EPSB with the stipulation that the first observation would be carried out on a face-to-face basis; subsequent observations could be conducted remotely. Face-to-face observations would resume if an observation indicated that the candidate was not performing in accordance with program expectations. The EPSB also asked UK to conduct an independent evaluation of the effectiveness of the remote observation cameras and compare it to face-to-face observations. Therefore, the UK team (first three authors) contacted an independent evaluator (IE, fourth author) to design and conduct an evaluation of the remote observation cameras used to supervise alternative certification teacher candidates.

The research questions generated by the team included the following:

Is the evaluation method reliable regardless of the format (i.e., can independent observers get similar results face-to-face and remotely)?

What are the perceptions of the teacher candidates when being observed face-to-face versus remotely?

What are the perceptions of the observers when conducting face-to-face versus remote observations?

Is the use of the remote observations cost effective?

Method

Evaluation Tools

To answer the research questions, the UK team and the IE developed an evaluation tool for this study to assess teacher behaviors during the observation: MSD Field Placement Observation Form. The IE developed two survey tools to assess perceptions of the teacher candidates and UK observers: Survey for Observed Teacher Candidates and Survey for UK Observers.

MSD field placement observation form

The original version of the observation form was developed by the UK team based on research-based strategies, competencies, and high leverage practices (Council for Exceptional Children, 2015; McLeskey et al., 2017; Westling, Fox, & Carter, 2015). The IE reviewed the form and made additions based on the Autism and Low Incidence Classroom Observation Tool (ALCOT; Pennington, 2018, 2017). The team and IE then met, added additional indicators, modified the rating codes, and clarified definitions for each indicator that observers would be recording. In addition, specific teacher candidate behaviors (i.e., opportunities to respond [OTRs], attention to positive/negative behaviors) were added to the form. The final observation form, shown in the Appendix, was divided into two sections: classroom components and teacher candidate behaviors. The classroom components were divided into four sections: environmental supports, student support, instruction/curriculum, and assessment/data collection with 16 indicators. The teacher candidate behaviors were divided into two sections: OTRs and attention to positive/negative behavior.

Survey protocol for observed teacher candidates

The IE, with feedback from the UK team, developed a survey protocol that was given to the teacher candidates following observations. The questions were based on research and surveys related to remote observations by Heafner, Petty, and Hartshorne (2011); Heafner, Petty, and Hartshorne (2012); and Wash, Bradley, and Beck (2014).

The form contained five closed-ended questions asking the following: (a) From which observation do you feel you received the most useful (informative) feedback? (b) Which type of observation do you feel allowed for a more authentic view of teaching and learning in your classroom (more representative of what happens in your classroom everyday)? (c) Which type of observation created more distraction for your students? (d) Which type of observation was more convenient for you? and (e) Which type of observation do you prefer? Response options were face-to-face, remote, no difference. Three open-ended questions also were included: (a) Briefly describe any perceived positive outcomes of the remote observations. (b) Briefly describe any challenges you encountered during or as a result of the remote observations. (c) Do you feel that remote observations are an effective way to observe your classroom?

Survey protocol for university observers

The IE also designed a similar survey to obtain information from the observers. Questions included on the observers’ survey included the following: (a) During which type of observation do you feel you gave more detailed (quality) feedback? (b) Which type of observation do you feel allowed for a more authentic view of teaching and learning? (c) Which type of observation was more convenient for you? and (d) Which type of observation do you prefer? Response options included face-to-face, remote, no difference. Three open-ended questions included the following: (a) Briefly describe any perceived positive outcomes of the remote observations. (b) Briefly describe any challenges you encountered during or as a result of the remote observations. (c) Do you feel that remote observations are an effective way to observe classrooms?

Remote Observation Equipment

In 2010, UK began working to find a system that would be easy to use in the field and include a camera that would pan, tilt, and zoom (Hager, Baird, & Spriggs, 2012). Based on the successes and failures of these initial observations, UK settled on the use of Logitech ConferenceCam CC3000e cameras (http://info.logitech.com/cc3000e-partners-us.html). These cameras were sturdy and easy to install in the teachers’ classrooms. Each camera cost approximately US$900 and came with a tabletop camera, a hub to plug in all cables, and a tabletop microphone. The university set up the cameras, so the connections were color coded and it took no more than 5 min to plug them into a computer. At the initial installation, a technician was present in the school district and on the UK campus to make sure the connection was working and to override firewalls if needed. The two computers then connected using Skype for Business^® and the teacher candidate initiated a call to connect the two computers. Once connected via Skype for Business, the university supervisor could immediately begin controlling the camera using the desktop sharing software to control the camera that was connected to the computer. The supervisor was able to pan, tilt, or zoom the camera in the classroom remotely using the software while physically being located at the university. If the supervisor had difficulty hearing or seeing, the camera and microphone could easily be moved closer to the teacher.

Participants and Settings

Teacher candidates

Four teacher candidates participated in the evaluation. Each teacher was enrolled at UK in the Alternative Certification (Option 6) MSD program. All candidates were female and taught in rural school districts: two in middle schools and two in high schools. The closest school to the UK campus was 70 miles (approximately 1 hr 30 min) away. The farthest school from the UK campus was 200 miles (approximately 3 hr 15 min) away.

UK observers

Two UK supervisors participated. Each observer was a tenure-line faculty member in special education at UK. Both had been employed as faculty members for 5 years and had extensive experience observing and providing feedback on teaching behaviors to both undergraduate and graduate teachers in face-to-face formats.

Data Collection

MSD field placement observation form

This form, shown in the appendix, was used to collect data during observations by the university supervisors (first two authors). The university supervisors contributed to the development of the observation form, so they were highly familiar with each item. They developed the operational definitions and discussed how each item would be scored during the observations. They then conducted three practice observations in local classrooms using student teachers not associated with the study. The observers independently scored the form and debriefed following each observation to compare scores. If disagreements occurred, the observers refined the operational definitions. The third observation resulted in acceptable interobserver agreement reliability of 80% or better on each section of the form, after which the observers began observations for the study. In addition, following each observation conducted for the study, the observers discussed any disagreements and refined definitions of items prior to the next observation.

Data were collected with each teacher candidate simultaneously and independently (i.e., one supervisor was observing in the classroom [face-to-face], one supervisor was observing remotely [using camera]) from UK. The teachers were told that to collect data for the study, both observers would be collecting data for the teacher observation at the same time. For the classroom components (i.e., environmental supports, student support, instruction/curriculum, assessment/data collection), the observers synchronized when the observation would begin and independently watched the classroom scoring each indicator. Then, for the data collection on teacher behaviors (i.e., OTRs, attention to positive/negative behaviors), the observers planned an observation time to score each of those (e.g., there was a 10 min period of time when the observers would begin and end the scoring of the teacher behaviors and collect frequency data on those behaviors).

Data from each observer were used to calculate inter-observer agreement (IOA). After each teacher candidate was observed, one of the observers would scan and e-mail the completed forms to the IE for review. IOA was calculated using a point-by-point formula for classroom components (i.e., [number of indicators of agreement/number of indicators agreed + number of items disagreed] × 100) and gross method for teacher behaviors, as these were frequency counts (i.e., [smaller of two observers’ counts/larger of counts] × 100).

Survey protocol for observed teacher candidates

Upon verification that a teacher candidate observation had received both face-to-face and remote observations at least once, the IE sent an e-mail with the survey attached. During the course of their time in the alternative certification program, all candidates had received both face-to-face and remote observations using the camera and had also experienced post-observation conferences both face-to-face and using the camera. In response to the survey, they could reflect on the totality of their experiences receiving each of the observation formats during their time in the program. Each of the teacher candidates completed the survey and e-mailed the results back to the IE. Once the IE received the results, she sent a return e-mail requesting a follow-up telephone call to give teacher candidates an opportunity to clarify or expand on any of their answers.

Survey protocol for observers

Upon completion of all four observations, the IE sent a survey to each of the UK observers. The observers returned their results through e-mail.

Results

MSD Field Placement Observation Form

The observation form was divided into two sections: classroom components and teacher behaviors. The classroom components were divided into four sections: environmental supports, student support, instruction/curriculum, and assessment/data collection, with 16 indicators. The teacher behaviors were divided into two sections: OTRs and attention to positive/negative behavior. IOA was calculated for each component. The mean IOA across observations ranged from 50% to 100% on each indicator with an overall mean of 87.5%. The IOA percentages on all components are shown in Table 1.

Table 1.

Interobserver Agreement of Observers on Components of the Observation Form.

Component	Mean IOA across observations
Environmental supports
Can observe all areas of the classroom	100%
Visual schedules/calendars are present	100%
Signals to indicate when transitions are approaching	100%
Use of AT/modifications is evident	100%
Staff members are engaged in instruction/preparation activities when not on a scheduled break	50%
Student support
Students have a means to respond	50%
T communicates appropriately with students (responds to student initiations: positive and respectful) Student engagement: all students are engaged in instructional activities (e.g., working with a para, working on individual tasks) for the duration of the observation	100% 100% 100%
Instruction/curriculum
Rules reviewed in positive manner	100%
Gains/maintains student attention	100%
Implements procedures accurately	75%
Specific/descriptive praise used	75%
Culminates lesson appropriately	100%
Reviews objectives, previews upcoming activities, check schedule	100%
Instructional materials are age appropriate	100%
Assessment/data collection
Records direct observation data	100%
Provides students with opportunity to self-assess	50%
Total mean agreement	87.5%

Note. IOA = inter-observer agreement.

IOA data also were collected on the teacher candidate behaviors of providing OTRs, providing attention to positive student behaviors, and providing attention to negative student behaviors. Across teachers, the observers had a mean 82.3% IOA (range = 65%–100%) on providing OTRs, a mean 94.6% (range = 80%–100%) on providing attention for positive behaviors, and a mean 87.5% (range = 50%–100%) for providing attention to negative behaviors.

Survey Protocol for Observed Teacher Candidates

Closed-ended responses

The four teacher candidates each completed a written survey through e-mail. The results of the five closed-ended survey questions were mixed across participants and are shown in Table 2.

Table 2.

Number of Responses to Closed-Ended Questions for Teacher Candidates and Observers.

	Response options
Survey question	FTF	Remote	No difference
Teacher candidate responses
1. From which type of observation do you feel you received the most useful (informative) feedback?	n = 1		n = 3
2. Which type of observation do you feel allowed for a more authentic view of teaching and learning in your classroom? (more representative of what happens in your classroom everyday)	n = 1		n = 3
3. Which type of observation created more distraction for your students?	n = 1	n = 1	n = 2
4. Which type of observation was more convenient for you?		n = 1	n = 3
5. Which type of observation do you prefer?		n = 3	n = 1
University observer responses
1. During which type of observation do you feel you gave more detailed (quality?) feedback?			n = 2
2. Which type of observation do you feel allowed for a more authentic view of teaching and learning?			n = 2
3. Which type of observation was more convenient for you?		n = 2
4. Which type of observation do you prefer?	n = 2

Note. FTF = face-to-face.

Open-ended responses

Teacher candidates provided responses for perceived positive outcomes, challenges, and effectiveness of remote observations. In terms of positive outcomes, one teacher candidate remarked on the advantage of increased flexibility the remote cameras provided by saying,

The flexibility to have an observation at any time is a very positive outcome of utilizing remote observations. If I would have had a problem with a student or wanted some input on observing student behaviors or a lesson, the remote observation could have been quickly set up for my professors at UK to observe or another peer . . . to observe.

Two teacher candidates liked the remote observations and reported it created less of a distraction. One candidate wrote,

I really enjoyed the remote observation. It creates less of a distraction for my students and personally, it does not make me feel as nervous. It is almost like there is no one in the room and we could carry on throughout the day as normal without having a “visitor” in the room.

Another candidate wrote, “There wasn’t an extra person in the room which can sometimes lead to my students getting off task. The remote observation allowed for my students and I to be observed without them being aware someone else was there.”

In terms of challenges, one teacher candidate indicated that there were no challenges, and two made comments related to difficulty setting up the camera. For example, one candidate wrote, “The only challenges we faced were trying to get the camera setup. It took a few tries, but we finally were able to get every kink worked out and it worked perfectly with our past observation.” One additional comment related to the need to move the camera so that the observer could see the back of the room and the candidate said, “The only difficulty with the remote observation was accurately seeing the back of the room where our group area is without moving the camera closer. The camera was small, however, so it was easily moved.”

When asked whether the remote observation was an effective way to observe teaching, all comments indicated that this method was an effective way to observe. The teacher candidates indicated that they “received the same quality of observations with both,” that “some of my most helpful and insightful feedback on utilizing systematic instruction . . . was provided from remote observations,” and that “remote observation makes it more convenient for the observer and the times and dates of the observation.”

In follow-up telephone interviews, the IE was able to talk with three of the four teacher candidates and allow them to expand on their survey responses. In the comments from the phone interviews, candidates discussed the positive aspects of remote observation indicating that they really liked the remote observations “as an option” that they “felt a little less stressed with camera than with face-to-face” observation of someone sitting in the classroom and that their “students were not distracted” by the camera. Positive aspects of face-to-face observations were mentioned by the teacher candidates. One teacher indicated she “felt she was getting a little more attention when the observers were in the room,” that the post-observation session occurred “right after the face-to-face observation,” and that she “got to know” UK observers better in a face-to-face format. Some challenges related to remote observation included comments related to difficulty getting the camera set up at first, difficulty with the placement of the camera, and difficulty of the observer seeing the “data sheets.” More than one comment related to the challenge of having the post-observation conference held at a later time (e.g., during planning period or a couple of days after the observation) rather than immediately after the observation. Challenges related to face-to-face observations were that the observations were “more nerve wracking with someone in the classroom,” and the post-conference observations had to take place before the UK observer left so it was necessary for the teacher candidate to “stop what she was doing.” One teacher candidate recommended that the first observation should be face-to-face so that the candidate could “get to know the observer first.”

Survey Protocol for Observers

The two observers each completed a written survey through e-mail. UK observers responded to each survey question in the same way, as shown in Table 2. For the open-ended questions, they responded about positive outcomes, challenges, and effectiveness of remote observations.

In terms of positive outcomes, they wrote that the remote observations allowed for “more authentic observation” as “you can . . . observe the classroom any time,” and observations can occur at a time that is the “best for the teacher and classroom.” The use of the camera allows for more frequent observations because the observer can join at any time of the day and “the time of driving would not be needed.” Both observers appreciated being able to use the technology to conference with the candidates following the observation.

When writing about challenges, both observers remarked about the challenges of finding the right technology that would work between schools and the university and have a “pan, tilt, and zoom” feature. They also talked about how the microphone picked up all noises in the classroom, so it could be difficult to hear if the classroom was noisy. Also, one principal would not allow the technology to be installed in the classroom, so if remote observations were to be used in classrooms in the future, time would need to be devoted to obtaining permission to use the cameras in each individual school or classroom, with the threat that some administrators may not give permission at all.

Both observers agreed that remote observation was an effective way to observe classrooms. They wrote that they could “see as much remotely as . . . live,” that the “pros outweigh any cons,” and that “the ability to more frequently observe students during times of the day that they need assistance outweighs the challenges.”

Costs of Observations

The cost of each observation can be calculated monetarily and by time (e.g., number of hours) spent traveling. The initial cost of each camera was US$900. The cameras were set up at the schools during the initial face-to-face observation. At the end of the teacher candidates’ program, the camera was returned to UK either by observers picking it up or through a mail/shipping service. The original box with protective packing materials was used to return the camera, and the university coordinated payment for the mail service.

Table 3 shows approximate costs per remote observation. Hotel rates and meal reimbursement were calculated for schools that were over 2 hr (one-way) from UK using state rates. The total number of travel hours for one observation at each of the four schools was approximately 17.5 hr. The total of monetary expenses for one observation at each of the four schools was approximately US$671.34. Calculation of the approximate cost for each face-to-face observation ranged between US$39.00 and US$315.78 per observation.

Table 3.

Cost of Face-to-Face Observation.

School	Distance (roundtrip-time)	Distance (roundtrip-miles)	Travel costs (mileage)	Travel costs (per diem food and/or hotel)	Total monetary cost per roundtrip
1	3 hr	140 miles	US$54.60	NA	US$54.60
2	5 hr	264 miles	US$102.96	US$159.00	US$261.96
3	6.5 hr	402 miles	US$156.78	US$159.00	US$315.78
4	3 hr	100 miles	US$39.00	NA	US$39.00

Discussion

School districts in remote areas have persistently struggled with recruitment and retention of certified teachers (Aragon, 2016; Berry, Petrin, Gravelle, & Farmer, 2012) that is complicated by nationwide teacher shortages (Sutcher, Darling-Hammond, & Carver-Thomas, 2016; Thornton et al., 2007). Therefore, it is important for universities and school districts to identify factors that contribute to retaining teachers of students with severe disabilities. Several studies have identified self-efficacy as one factor in retaining teachers (Blackburn & Robinson, 2008; Schaefer, Long, & Clandinin, 2012). Having frequent observations of teaching with targeted coaching can affect teachers’ beliefs of their own effectiveness, thus emphasizing the need for universities to find a way to provide these observations to those who are geographically isolated from the faculty (Kaufman & Ring, 2011). UK’s alternative certification program in MSD uses distance education capabilities to deliver content to alternative certification teacher candidates living in rural areas, but has experienced challenges in providing high-quality and frequent observations of classroom teaching due to the geographic distance of some teacher candidates. Therefore, it was important to evaluate the effectiveness of remote observation to ensure that those observations were of similar effectiveness and provided similar authentic views of the candidates’ teaching as compared to face-to-face observation. Considerations often expressed related to conducting observations of rural teacher candidates include (a) the ability to achieve reliable observation of teacher behaviors if remote observation is used, (b) the preferences and perceptions of teachers and observers related to the effectiveness and logistics of accomplishing observations, and (c) the costs associated with different methods of observation.

One consideration when using remote observation is ensuring that an observer can accurately see and hear all the activities in a classroom with the use of a camera. Some rural districts have challenges with adequate bandwidth strength to use remote observation cameras like were used in this investigation. This was not a challenge in Kentucky as the bandwidth in all schools, even those in rural regions, was adequate to operate the system. This system used a desktop microphone which did pick up audio from the entire classroom. Because the microphone was able to be moved, it could be moved closer to the teacher if needed; however, it was necessary to conduct observations during a time when there was not much extraneous noise occurring in the classroom, so the teacher and students could be heard. The use of the camera system, however, did result in acceptable rates of reliability on the classroom indicators, OTRs, attention to positive behaviors, and attention to negative behaviors. These results indicate that both evaluation methods were reliable. Although the observers obtained 100% agreement on 11 of the 16 indicators across the four observations, three of the indicators received a 50% agreement. These were (a) staff members engaged when not on a scheduled break, (b) students have a means to respond, and (c) teacher provides student with an opportunity to self-assess. Of these, we do not have enough data to tease out for certain whether or not the method of observation resulted in disagreements in scoring. Preliminarily, however, for the self-assess indicator, the same observer scored the indicator lower regardless of the method of observation indicating that the disagreement may have been related to the observers defining the indicator differently rather than the method of observation. For the indicator of having a means to respond, the remote observer scored the item higher on two of the four observation tools, but the same observer conducted both remote observations, so we do not know whether the disagreements were related to the observation method or a difference in definition between observers. The only indicator that may be related to the method of observation was the indicator of staff members engaged when not on a break. For this indicator, both observers conducting face-to-face observations (regardless of who the observer was) rated the item lower than the remote observer. It is possible that although more data would be needed to confirm, the face-to-face observer was able to more accurately observe all staff members in the classroom as the remote observer had to rely on the panning of the camera to observe the entire classroom, which did move more slowly than an individual observing live.

A second consideration with remote observation is determining whether teacher candidates and observers deem them as being just as effective and convenient as when being observed face-to-face. According to the teacher survey results, a majority (3 out 4 teacher candidates) indicated that there was no difference between the type of observation related to informative feedback, authentic view of teaching, and convenience. Teacher candidates were split between which type of observation created more distraction for their students (1 face-to-face, 1 remote, 2 no difference). None of the teacher candidates reported that face-to-face observations were more convenient (1 remote, 3 no difference). Three teacher candidates preferred remote observations, while one reported no preference.

Through open-ended responses, teacher candidates indicated that they liked the remote observations for the flexibility of scheduling and minimal distractions for both students and teacher candidates. Some of the teacher candidates mentioned difficulty with technology, although the difficulties seemed limited and easily fixed. Follow-up telephone interviews revealed that teacher candidates seemed to like meeting with UK faculty observers initially as a way to get to know them, but enjoyed the flexibility of using the remote camera as a method for post-observation conferencing. The results of the survey indicate that the teacher candidate perceptions of the remote observations were positive. They saw benefits to the remote observations and did not see differences in the quality of the observations when the faculty member was at a remote site.

The observer survey results, similar to teacher responses, indicate that both observers agreed that there was no difference in quality of feedback or authentic view of teaching when observing remotely or face-to-face. Both observers agreed that the remote observations were more convenient; however, both preferred face-to-face. While observers preferred face-to-face observations because they could achieve a holistic view of the classroom and the community and develop rapport, the majority of the teachers preferred a remote observation. This may be primarily related to the nervousness the teachers felt when being observed and having another individual visiting in the classroom. The observers responded that the main benefits of the remote observations relate to the frequency and flexibility with which observers can see the classroom. Remote observations allow observers to join classes at multiple times during the day if needed or at the most convenient time for teacher candidates. It also can allow for more frequent observations as the challenges of driving time are removed. Teachers similarly reported the advantage of the flexibility of camera observations. As with the teacher candidates, the observers indicated some difficulty with the technology, but agreed that the pros outweighed the cons.

Finally, a third consideration with remote observation relates to the cost comparison of the methods. Although the cameras required an initial investment, they produced the results that the observers needed to complete reliable observations. UK observers highly valued having cameras that allowed them to control the view of the camera remotely so they could adequately see all areas of the classroom. This may have contributed to the high levels of reliability that were achieved by the two observers, although this remains an empirical question. Depending on the distance of the teacher’s school from UK, the monetary difference alone may decide cost-effectiveness. However, the real cost benefit is in time. The use of the remote cameras for observation eliminates the down time that observers spend in the car (and possibly in a hotel overnight). The use of the remote observations also significantly cuts down on the time it would take for an observer to get to a teacher’s classroom for immediate assistance (e.g., if there is an immediate situation the teacher would like observed, the UK faculty member could be in the classroom remotely in a matter of minutes). Results of the cost-effectiveness analysis revealed that the remote observations were more effective than continuous face-to-face observation.

The results of this evaluation are similar to those obtained by other researchers. Heafner et al. (2011) conducted a comparison of synchronous remote using live classroom software with face-to-face observations of teacher candidates. They found that although the two observation methods were not equal, they were comparable in evaluating teacher effectiveness and growth. Heafner et al. (2012) conducted another study where they reported the perceptions of the university observers when observing the teaching of graduate interns using synchronous remote software. The observers reported the benefits of the remote observations as saved time, flexibility, convenience, and authentic views of teaching. All of these were similar to comments made by UK observers. Similar challenges mentioned by UK observers also were reported by the Heafner et al. participants including limitations in the audio of the remote technology (i.e., microphone picked up extraneous noise), and the lack of ease of creating a personal rapport when meeting with a teacher candidate only in remote formats as evidenced by the preference of face-to-face observations by UK observers.

Limitations and Future Research

Limitations of this evaluation are the small number of teacher candidates and university observers. Although a small number of teacher candidates in rural areas with MSD is to be expected given the low incidence nature of the population and that the candidates were attending one of numerous teacher preparation programs available in the state, studies with larger samples of participants would create generalizable results, which is not possible with this study. In addition, the university observers were limited in sample size and were the same individuals who were conducting the remote and face-to-face observations for the university. Therefore, there was a possibility that their responses could have been biased given that the outcomes could affect the future observation method used in their teacher preparation program.

An endorsement of the observation form and use of remote observations should be guarded as the need for additional studies comparing observation methods is needed with larger sample sizes so that generalizable results can be found. In addition, this evaluation only included two methods of observation, synchronous camera observation versus face-to-face. Many additional observation methods have emerged as technology has advanced. These various methods could be compared and would contribute to the literature in this area (e.g., asynchronous observation using video recording, live versus delayed feedback, written versus audio feedback, time stamped feedback versus feedback not time stamped).

Conclusion

The need for teacher preparation programs to be creative in solving the observation needs of their programs is evident, especially in light of national teacher shortages and the geographic separation of some teacher candidates from their universities. The technology used in this evaluation was simple to install and worked well across four different classrooms located in rural areas in Kentucky. Given the results, other university supervisors may consider remote observation as a viable alternative to meeting the needs of their teacher candidates. Supervisors should continue to work with school districts and state agencies to gain approvals for the use of remote observation as a solution for providing frequent feedback for teacher candidates separated geographically from the university.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Aragon

(2016). Teacher shortages: What we know. Denver, CO: Education Commission of the States. Retrieved from https://www.ecs.org/wp-content/uploads/Teacher-Shortages-What-We-Know.pdf

Bailey

T. R.

Zumeta

(2015). How states can help rural LEAs meet the needs of special populations. In Gross

Jochim

(Eds.), SEA of the Future Series: Vol. 4. Uncovering the productivity promise of rural education (pp. 40–53). Seattle, WA: Center on Reinventing Public Education.

Berry

A. B.

Petrin

R. A.

Gravelle

M. L.

Farmer

T. W.

(2012). Issues in special education teacher recruitment, retention, and professional development: Considerations in supporting rural teachers. Rural Special Education Quarterly, 30, 3–11.

Blackburn

Robinson

(2008). Assessing teacher self-efficacy and job satisfaction of early career agriculture teachers in Kentucky. Journal of Agricultural Education, 49, 1–11.

Childre

A. L.

(2014). Preparing special educators highly qualified in content: Alternative route certification for unlicensed teachers in rural Georgia. Rural Special Education Quarterly, 33, 23–32.

Collins

B. C.

Baird

C. M.

Hager

K. D.

(2009). The University of Kentucky distance education program in moderate and severe disabilities. Rural Special Education Quarterly, 28, 30–40.

Cook

E. E.

Boe

L. H.

(2007). National trends in the source of supply of teachers in special and general education. Teacher Education and Special Education, 30, 217–232.

Council for Exceptional Children. (2015). What every special educator must know: Professional ethics and standards. Arlington, VA: Author.

Glomb

Midenhall

Mason

Salzberg

(2009). Reducing isolation through regional mentors and learning communities: A way to support rural learners. Rural Special Education Quarterly, 28, 31–35.

10.

Hager

K. D.

Baird

C. M.

Spriggs

A. D.

(2012). Remote teacher observation at the University of Kentucky. Rural Special Education Quarterly, 31, 3–8.

11.

Heafner

T. L.

Petty

T. M.

Hartshorne

(2011). Evaluating modes of teacher preparation: A comparison of face-to-face and remote observations of graduate interns. Journal of Digital Learning in Teacher Education, 27, 154–164.

12.

Heafner

T. L.

Petty

T. M.

Hartshorne

(2012). University supervisors’ perspectives of the remote observation of graduate interns. Journal of Computing in Higher Education, 24, 143–163.

13.

Humphrey

D. C.

Wechsler

M. E.

(2007). Insights into alternative certification: Initial findings from a national study. Teachers College Record, 109, 483–530.

14.

Individuals with Disabilities Education Improvement Act, Pub. L. 108-446 U.S.C. (2004). Retrieved from https://sites.ed.gov/idea/files/20060814-Part_B_regulations.pdf

15.

Johnson

Ohlson

M. A.

Shope

(2018). Demographic changes in rural America and the implication for special education programming: A descriptive and comparative analysis. Rural Special Education Quarterly, 37, 140–149. doi:10.1177/8756870518771381

16.

Kaufman

R. C.

Ring

(2011). Pathways to leadership and professional development: Inspiring novice special educators. Teaching Exceptional Children, 43(5), 52–60.

17.

Ludlow

B. L.

Collins

B. C.

(Eds.). (2016). Online in real time: Using Web 2.0 for distance education in rural special education. Morgantown, WV: American Council on Rural Special Education.

18.

Ludlow

B. L.

Duff

M. C.

(2009). Evolution of distance education at West Virginia University: Past accomplishments, present activities, and future plans. Rural Special Education Quarterly, 28, 9–17.

19.

McLeskey

Barringer

M.-D.

Billingsley

Brownell

Jackson

Kennedy

. . . Ziegler

(2017, January). High-leverage practices in special education. Arlington, VA: Council for Exceptional Children & CEEDAR Center.

20.

McLeskey

Tyler

N. C.

Flippin

S. S.

(2004). The supply and demand for special education teachers: A review of research regarding the chronic shortage of special education teachers. The Journal of Special Education, 38, 5–21.

21.

Pennington

(2018). Autism and low incidence classroom observation tool. Verona, WI: Attainment Company.

22.

Pennington

R. C.

(2017). An IEP for me: Program improvement for teachers of students with moderate to severe disability and autism spectrum disorder. Rural Special Education Quarterly, 36, 146–154.

23.

Rosenberg

M. S.

Sindelar

P. T.

(2005). The proliferation of alternative routes to certification in special education. The Journal of Special Education, 39, 117–127.

24.

Schaefer

Long

J. S.

Clandinin

S. J.

(2012). Questioning the research on early career teacher attrition and retention. Alberta Journal of Educational Research, 48, 106–121.

25.

Scheeler

M. C.

Congdon

Stansbery

(2010). Providing immediate feedback to co-teachers through bug-in-ear technology: An effective method of peer coaching in inclusive classrooms. Teacher Education and Special Education, 33, 83–96. doi:10.1177/0888406409357013

26.

Scheeler

M. C.

McAfee

J. K.

Ruhl

K. L.

Lee

D. L.

(2006). Effects of correct feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher Education and Special Education, 29, 12–25. doi:10.1177/088840640602900103

27.

Scheeler

M. C.

Ruhl

K. L.

McAfee

J. K.

(2004). Providing performance feedback to teachers: A review. Teacher Education and Special Education, 27, 396–407. doi:010.1177/088840640402700407

28.

Sundeen

T. H.

Wienke

W. D.

(2009). A model for recruiting and preparing new special educators: Mining an untapped potential. Rural Special Education Quarterly, 28, 3–10.

29.

Sutcher

Darling-Hammond

Carver-Thomas

(2016). A coming crisis in teaching: Teacher supply, demand, and shortages in the U.S. Palo Alto, CA: Learning Policy Institute. Retrieved from https://learningpolicyinstitute.org/product/coming-crisis-teaching

30.

Thornton

Peltier

Medina

(2007). Reducing the special education teacher shortage. The Clearing House, 80, 233–238.

31.

U.S. Department of Education Office of Postsecondary Education. (2017). Teacher shortage areas nationwide listing 1990-1991 through 2016-2017. Retrieved from https://www2.ed.gov/about/offices/list/ope/pol/tsa.pdf

32.

Wash

P. D.

Bradley

Beck

(2014). Remote classroom observations with preservice teachers. SRATE Journal, 24, 58–64.

33.

Westling

D. L.

Fox

Carter

E. W.

(2015). Teaching students with severe disabilities (5th ed.). Boston, MA: Pearson.