Abstract
We use student-level administrative data from Georgia to examine how teacher characteristics affect student learning. Specifically, we examine students’ normalized end-of-course test scores for a required high school economics course. We develop two models. First, we examine returns to activities often linked to teacher effectiveness—experience, advanced degrees, National Board Certification, and curriculum-specific in-service workshops. Second, we investigate the importance of having teachers share characteristics with their students. Similar to many studies before us, we find little systematic link between any of the observable teacher characteristics and better student outcomes once we control for unobservable teacher characteristics. Of note, we do find that female students tend to perform better with female teachers than with male teachers. Overall, the results highlight the difficulty in relying on just one specific measure of teacher quality to ensure student performance.
Introduction
The teaching profession holds a special place in society. The shaping of young minds is more important now, if possible, than ever before because global markets now demand and reward intellectual contributions more handsomely than physical ones (see World Bank, 2016, or Psacharopoulos & Patrinos, 2004). And, as states and local school boards scrutinize education budgets, it becomes more important to get the most out of the remaining dollars allocated to schools. Hiring and retaining the best teachers is paramount. Unfortunately, it is difficult to systematically analyze what makes for a good teacher. It has always been difficult to untangle the many confounding elements that go into quality teaching. Personal characteristics combine with investment in human capital which, in turn, combines with the school environment to create an educational experience for students. And student characteristics can either amplify or conceal the efforts of teachers and school administrators when one looks at specific outcome measures to determine the quality of teaching. But, in today’s educational environment, people—taxpayers and parents, primarily—want to be able to quantify what makes for good teaching to reward effective teachers and release ineffective ones.
Although not directly an issue of teacher quality, some research suggests that the combination of student and teacher characteristics may have an impact on teacher effectiveness. It is an open question as to whether or not students do better when they have a teacher who shares the same gender or race. A concern over a lack of diversity in some areas, particularly science, technology, engineering, and mathematics (STEM) areas, has led education programs to encourage women and minorities to train for teaching jobs. Some (e.g., Hoffmann & Oreopoulos, 2009, and Ozogul, Johnson, Atkinson, & Reisslein, 2013, focus on gender, whereas Ehrenberg & Brewer, 1994, Cherng & Halpin, 2016, and Egalite, Kisida, & Winters, 2015, address matching on race) have postulated that by encouraging women and minorities to fill teaching jobs in these areas, girls and minority students would likely benefit.
Our data allow us to examine both of these issues in the high school economics course. First, we add to the literature in an examination of student outcomes on a state-mandated, standardized test in economics that follows a required high school economics course in Georgia public schools. Our data match student test scores to rich student-level administrative data and match student observations to teacher data. We analyze the impact of teacher characteristics, including acquisition of advanced degrees, National Board Certification, and years in service as well as attendance at in-service workshops intended to provide teachers with economic content knowledge specifically designed to help them teach to the Georgia curriculum. In our attempts to model teacher effectiveness, we control for student characteristics and apply school-level fixed effects to control for differences between schools that are unlikely to change over the time span of our data. We also employ teacher-level fixed effects to investigate the importance of unobserved, time-invariant teacher characteristics. Our findings are largely consistent with past findings. We find some evidence that early teacher experience improves teacher effectiveness and that graduate degrees confer some positive returns to the teachers who obtain them. Similarly, we find that the National Board Certification is correlated with stronger student test scores in economics. There is also some evidence for benefits to in-service learning. However, when we control for teacher and school fixed effects, most of the benefits of measurable facets of teacher quality become statistically no different from zero. That is to say, there is not an identifiable policy variable available upon which one might rationally base teacher salary in our analysis. It appears that unique invariant characteristics underlie most of the differences between the teachers in our sample.
On the issue of matching student characteristics to teacher characteristics, we do find limited support for the hypothesis that students thrive when they share personal characteristics with their teacher. In particular, female students do seem to do better with female teachers. Racial matching, however, does not seem to significantly improve student outcomes under almost all the cases we can examine with our data.
Our study is presented as follows. The next section provides a literature review that examines past findings about what makes for an effective teacher and why some believe it is desirable to have teachers who share certain characteristics with their students. The subsequent section describes our data. The fourth section provides an empirical model with which we analyze our data. In the section that follows, we provide an analysis of our results and then conclude.
Prior Literature
Any discussion concerning the best way to allocate a school budget should consider what teacher characteristics to reward based on what affects the quality of instruction. There is, however, little agreement on this subject. Research in the field of education has failed to put its collective finger on any one characteristic that systematically identifies good teachers. Economists have, likewise, failed to provide any policy recommendations that would unequivocally improve teaching.
There is a common list of characteristics that researchers examine—primarily because they are relatively easy to observe. States have used them, at least in part, to determine teacher salaries under the assumption that they correlate with teacher effectiveness. This list includes years of experience, advanced degrees obtained, and teacher certification at both the state and national level. In our investigations, we add content-specific in-service training to the list. Most research that attempts to explain student gains in knowledge utilize some combination of these input measures to control for the quality of teachers.
Years of Experience
Most school systems reward teachers for years of experience. In Georgia, teachers receive increases in their pay grade almost every year from their third year to their eighth year, and then every other year until they complete 20 years of service. Paying based on years of service assumes that years in service is a proxy for a steady accumulation of human capital gained over time. Very few studies concur with this assumption. Bhai and Horoi (2019) examine twins in North Carolina elementary schools. They find positive returns to teacher experience in both reading scores and math scores. The gains are initially steep then taper off after 5 years. Rockoff (2004) finds evidence that years of experience confer fairly small but steady increases in teacher effectiveness for reading comprehension for elementary school–age children. But, he does not find similar evidence for teachers of mathematics. The consensus appears to be that after the first few years of service, teachers gain little, if any, additional effectiveness that can be attributed directly to time served. Kane, Rockoff, and Staiger (2006) and Clotfelter, Ladd, and Vigdor (2007a, 2007b) find increases in teacher effectiveness in the early years of the teacher’s career but decreasing gains (but still positive and statistically significant) as experience increases. Rivkin, Hanushek, and Kain (2005) find that during the first 5 years mathematics teachers gain some effectiveness as they gain experience. But afterward, there is no evidence of additional effectiveness. Harris and Sass (2011) find that for elementary and middle school teachers, productivity increases initially, but gains wane after the first 5 years. Grissom and Strunk (2012) take this as evidence for front-loading teacher salaries to reward them for early gains in effectiveness and provide an incentive to retain teachers who demonstrate classroom effectiveness. Continually increasing pay based solely on accumulated time in the classroom, however, is not supported.
Degrees Obtained
A second formulaic way in which teacher pay is determined is to incorporate degrees earned as a factor of base pay. The standard human capital models hold that additional years of education should translate into additional productivity. However, as more school districts began to reward additional degrees in the pay schedule, more degrees became available through more avenues for teachers. Cebula, Mixon, and Montez (2015) take this assumption at face value and find in an ordinary least squares (OLS) regression positive impact of a school district increasing the proportion of teachers holding advanced degrees. But, not all degrees are of equal value. Ehrenberg and Brewer (1994) report that the selectivity of a teacher’s undergraduate institution positively affects student progress. They attribute the impact on the underlying quality of the teacher and assert that the selectivity of the undergraduate institution is an indicator of the verbal skills or intellect of the teacher. Clotfelter et al. (2007b) find that teachers with undergraduate degrees from more competitive schools are marginally more effective in the classroom than those with undergraduate degrees from noncompetitive schools. In general, the research on the subject has increasingly found advanced degrees, in and of themselves, provide little guarantee of increases in teacher effectiveness. For example, looking at the proportion of teachers in fourth- to seventh-grade classrooms with advanced degrees, Rivkin et al. (2005) find no statistically significant evidence that an increase in the proportion of teachers with advanced degrees improves student performance on either math or reading scores. Clotfelter et al. (2007b) report very similar results when looking at individual teachers. Bhai and Horoi (2019) find the effect of advanced degrees on teacher effectiveness to be either negative or not statistically significant, depending on the model specification. In short, the research indicates that obtaining advanced degrees does not generally improve teacher performance and is a poor candidate for systematic pay increases.
Certification
One approach to acknowledging teacher quality is to reward a teacher once he or she has obtained external certification. Certification can occur at the state or national level. Goldhaber and Brewer (2000) find evidence that teachers with standard (state-level) certification perform better in the math and science classroom as compared with teachers with no certification. Kane et al. (2006) find virtually no impact from state-level certification. Primary among certifying organizations is the National Board for Professional Teaching Standards (NBPTS), which offers National Board Certification for teachers who can demonstrate a mastery of their craft. Bhai and Horoi (2019) find that the students of National Board–certified teachers outperform their peers significantly on standardized math tests. They fail to find a statistically significant impact of Board Certification on elementary students’ reading comprehension scores. Clotfelter et al. (2007b) isolate the effectiveness of National Board Certification as an ex post measure of teacher quality. They find that it does tend to identify better teachers—that is the students of teachers who are NBPTS-certified outperform their peers in math and reading by a statistically significant amount.
In-Service Training
An additional avenue for many teachers to build human capital, short of earning additional degrees or certification, is to attend workshops that offer specific subject content. In-service training is one way teachers can acquire such training. As our study focuses on economics teachers, we look at the in-service learning opportunities provided by the Georgia Council on Economic Education (GCEE). GCEE offers teacher workshops on topics directly linked to Georgia’s Performance Standards in economics. We can add this additional teacher variable because we have access to data that link the teachers’ workshop attendance history to the student end-of-course test (EOCT) scores. There is evidence from past research that GCEE workshops help teachers in the high school economics classroom. Swinton, De Berry, Scafidi, and Woodard (2007, 2010), Swinton, Scafidi, and Woodard (2012), and Swinton and Scafidi (2012) all find that the students of teachers who attend particular individual topic workshops score better on Georgia’s EOCT than students of teachers who have not attended such workshops. In this study, we take a broad approach by identifying teachers who have attended any GCEE workshops without making any distinctions among which ones they attended.
One of the problems with relying on observable differences in teacher efforts to accumulate human capital is that there may be systematic differences between teachers who seek out ways to improve their teaching performance and those who do not. Without a means for addressing this potential misspecification problem, a model of teacher effectiveness is likely to overstate the importance of observed characteristics such as advanced degrees and National Board Certification. To account for this possibility, many studies incorporate a teacher fixed effects approach to modeling student outcomes. Rockoff (2004), Rivkin et al. (2005), and Clotfelter et al. (2007a, 2007b) each incorporate this approach. It is important to keep in mind that because the differences in teachers’ years of experience, advanced degrees, certification status, and in-service training efforts are what determine the values of teacher fixed effects coefficients, care must be taken in the interpretation of the coefficients on the various years-of-experience dummy variables. Adding a vector of dummy variables that represent unobserved yet invariant teacher characteristics that correlate to student outcomes is likely to absorb some of the apparent impact of observed efforts teachers make to improve their classroom effectiveness.
While specific teacher characteristics are important, so is the learning environment itself. Heck (2007) focuses on various school characteristics that affect student outcomes. He pays special attention to the organizational principles schools use. But, because in many cases, ours included, the time frame covered by data is too short to observe variation within schools, there is little to be gained by examining the different school qualities that may vary across schools. Therefore, similar to most of the other studies cited here, we incorporate school-level fixed effects to capture school specific qualities that may either aid or hinder student learning.
Finally, it is important to capture the differences students bring to the classroom. Student characteristics such as gender and race have consistently been shown to affect estimated test scores. To isolate the impact of differences in teacher characteristics on students, two different impacts must be considered. First, one must control for students’ preexisting knowledge. If a pretest in the same subject is available, it can serve as a hallmark to identify what a student understands prior to taking a class and allows the research to focus on the teacher value-added or gains in knowledge in the classroom. Both Rivkin et al. (2005) and Clotfelter et al. (2007a, 2007b) have access to tests in the same subject areas taken prior to the class outcome they investigate. At other times, a test score in a closely related subject may be sufficient. Evans, Swinton, and Thomas (2015) show that geometry test scores are better predictors for student performance in the economic classroom than algebra test scores. Clark, Scafidi, and Swinton (2011) show that when geometry EOCT scores are used as controls for student prior ability and for student’s peers’ ability, they are both statistically significant and economically significant. Studies such as Ballard and Johnson (2004) and Swinton et al. (2012) have also shown the efficacy of using geometry test scores to control for prior aptitude in economics. Mendez-Carbajo, Mixon, and Asarta (2018) show that a student who has taken some economics is likely to comprehend math better. Therefore, we control for preexisting student aptitude in economics by using a previous EOCT in geometry. We eliminate observations where the geometry EOCT is taken either in the same semester or after the economics EOCT.
It is also important to control for student differences that might contribute to the learning process in general. Educational production functions (e.g., Bosshardt & Watts, 1994) include characteristics that may influence a student’s ability to learn, such as gender, race, or measures of a student’s available economic resources, or any other variables that may improve or impede a student’s ability to acquire knowledge. In this study, we include each student’s EOCT score in geometry, gender, race (White, Black, Asian, Hispanic, Native American, or mixed race), access to free or reduced-price lunch, and disability status.
Matching student and teacher characteristics
As schools cast around for ways to improve the learning environment, some researchers have suggested that, in addition to having teachers with more experience and more training, schools should try to hire teachers who better mirror their students—Schools should make efforts to hire more women and more minorities. We have an opportunity to investigate the link between student and teacher pairings on gender and a few ethnic categories.
Both links have been studied fairly extensively using the National Education Longitudinal Study of 1988, which provides a rich matching of student and teacher characteristics, and Tennessee’s Project Student Teacher Achievement Ratio (STAR) data, which incorporates random classroom assignment to investigate the effects of class size on student outcomes. Recent interest in improving STEM education has led to greater interest in recruiting teachers from a wide swath of the potential teacher population. Gender has been studied more thoroughly than the impact of racial differences, particularly in the context of STEM subjects and in economics. These subjects have received more scrutiny because traditionally there have been fewer women entering into these fields.
Female Students and Female Teachers
Bettinger and Long (2005) examine the role women play as mentors to future female economists in colleges. They find that female instructors make up a far smaller percentage of economics faculty than the percentage of female students majoring in economics. They find that having more women in underrepresented departments can increase the likelihood of female students taking more classes within those departments. Dee (2007) offers a careful analysis of the National Education Longitudinal Study of 1988 in which he concludes that there is some reason to be concerned about the gender pairing of teachers with students. In his analysis, female teachers appear to exert a negative influence on male students in English courses and both male and female students in mathematics. He suggests that he finds these results in part because he controls for student fixed effects. He suggests that male students who struggle in school are more likely to be assigned to male teachers. Therefore, the lack of effect found in studies such as Ehrenberg, Goldhaber, and Brewer (1995) is due to misspecification issues. Hoffmann and Oreopoulos (2009) find weak evidence for gender playing a role in performance or persistence in a major in college. Interestingly, they also note that their results are in part driven by males performing worse in classrooms led by a female. While Asarta, Butters, and Thompson (2014), in an examination of results from a national online economics knowledge competition, find that female students scored better if they had female teachers than female students who had male teachers, they also find male students with female teachers performed better than male students with male teachers. Therefore, they conclude that they find no evidence for a peer mentoring effect. Finally, Emerson, McGoldrick, and Siegfried (2018) find no link between the choice of a female student choosing to major in economics and having female professors.
Minority Students and Minority Teachers
For years, it was speculated that minority students would benefit if they were taught by someone who shared their ethnic background. Testing such a hypothesis has proven difficult due to specification problems. Dee (2004) makes use of the random assignment of the STAR data set to provide fairly strong evidence that both White and African American students benefit from being paired with a teacher of the same race. Other researchers have largely failed to find the same effect. For example, Howsen and Trawick (2007) reconsider Dee’s findings using the same data. Once they control for student innate ability (as measured by a cognitive skills index available within the data set), they find that the importance of matching on race becomes statistically irrelevant to the model. Ehrenberg and Brewer (1994) use the degree to which a school’s teacher racial makeup matches its students’ racial makeup but find no link to student performance.
In this study, we take advantage of knowledge of both teacher and student characteristics that allow us to further investigate the potential advantages or disadvantages of attempting to match teacher characteristics to student characteristics. We find some positive advantages to gender matching but none for racial matching.
Data
As part of its response to No Child Left Behind legislation (Public Law 107-110, 115 Stat. 1425) and in part due to Georgia’s A+ Education Reform Act of 2000 (O.C.G.A §20-2-281), the Georgia Department of Education (GaDOE) instituted a series of high school subject EOCTs. All public high school students must take these tests to graduate. The tests are high-stakes tests in that they constitute 15% of the course they represent. 1 For this study, we focus on two of these tests—high school geometry and economics. The economics EOCT represents our dependent variable. The geometry EOCT represents a control variable for student achievement prior to taking economics. While we do not observe student economic knowledge before they take their high school economics class, past studies (e.g., Ballard & Johnson, 2004; Clark et al., 2011; Evans et al., 2015; Swinton et al., 2012) have demonstrated that mathematics knowledge (geometry in particular) is a good predictor of future success in economics.
Georgia’s Department of Education links these test results to rich administrative student data. The student data include the student’s gender, race (categorized as Asian, Black, Hispanic, native American, White, or mixed race), economic status (whether or not a student receives a free or reduced-price lunch), and disability status (whether or not the student suffers from one or more of a number of physical or mental disabilities). The student data are also linked to a teacher of record for each EOCT.
Teacher data also come from the GaDOE. We use data from the Certified Personnel Index (CPI). The CPI provides teacher information including the teacher’s gender, age, race (categorized as White, Black, Hispanic, or other race), degrees earned, whether or not the teacher is National Board-Certified, and years of experience as described by the teacher’s salary step (see the appendix for salary step definition). While we also know what school a teacher teaches in, we do not use school-specific characteristics. Instead, we utilize a school-level fixed effects approach as explained in the description of the model. We cannot use both as the time period of the study is too short for most of the school characteristics to change in any significant fashion.
Finally, we include information as to whether or not teachers have attended in-service workshops conducted by the GCEE. Swinton et al. (2010) show that the students of teachers who have attended GCEE workshops tend to perform better on the economics EOCT than other students, other things being equal. GCEE provides in-service workshops to well over a thousand teachers each year. The workshops are provided at no cost to teachers and cover all facets of the state curriculum (for a more detailed description of the nature of the workshops, see Swinton et al., 2010). GCEE has kept computerized records of all teachers who have participated in their workshops since 1990. We match the teachers in the CPI data set to GCEE data. We then create a dummy variable that is equal to 1 if the teacher of record has taken any GCEE workshop and 0 otherwise. In the data set, 42.6% of teachers have attended GCEE in-service workshops at some point.
We utilize 4 years of data covering the fiscal years 2006 through 2009. Although we have access to all public school students in Georgia, our data have a few limitations. First, we use only those student observations that we can match geometry EOCT scores that were prior to their economics EOCT scores. We do not have matches for all students for a couple of reasons. In the early years of the testing environment, those who took the economics EOCT may have had geometry before the various EOCTs were first administered. Furthermore, not all schools schedule economics after geometry. There are a few schools that teach economics in the freshman or sophomore years. In these cases, a prior geometry EOCT score is not available. Second, if we do not have data on the characteristics or credentials of a student’s teacher, then that student’s observation is not used.
As a result, we have 2,237 teachers in the data set matched to 197,886 student observations. Rather than use raw economics EOCT scores as our dependent variable we normalize the scores by constructing z scores. The z scores for each test score is equal to
A summary of the student and teacher data is shown in Table 1.
Summary Statistics.
Note. EOCT = end-of-course test.
The majority of the students in the sample are female (53%), whereas the majority of the teachers in the data are male (67%); 47% of the students and only 16% of the teachers in the data are non-White. A total of 34% of the students in the sample received free or reduced-price lunches. The majority of the teachers in the data have an advanced degree (62%), but less than 2% of teachers are National Board-Certified.
Analytical Framework
In this section, we put forth two empirical models. In the first model, we test for evidence of returns to teacher investment in their own abilities. In the second model, we test for evidence that matching particular teacher characteristics, race and gender in particular, to student characteristics improves student performance.
Model 1: Teacher Human Capital
Similar to Rockoff (2004), Rivkin et al. (2005), and Clotfelter et al. (2007a, 2007b), we investigate returns to teacher investment in human capital while focusing on student-level data. In contrast to the above-mentioned studies, which all examine general student reading and mathematics standardized competency test results, we focus on student performance on a standardized test of economic knowledge (the state-mandated EOCT in economics). All Georgia public school students must take this high-stakes exam upon completing a mandatory high school economics class. We model the contributions of student gains in knowledge by accounting for student characteristics, time-invariant school and teacher characteristics, and, most importantly, changes in teacher characteristics.
One of the often confronted problems with modeling the contributions of teachers to student learning is controlling for student knowledge prior to their experience with the teacher. While Rockoff (2004) does not control for prior student knowledge directly, Rivkin et al. (2005) and Clotfelter et al. (2007a, 2007b) do so in different ways. Both studies have measures of achievement in the same test area (math and reading) from previous tests. That is, their data track students’ progress on a succession of tests in the two subject areas of interest. Rivkin et al. (2005) focus on the change in test scores using a first differences approach. That is, they measure the impact of school, teacher, and student characteristics on the change of test scores. In contrast, Clotfelter et al. (2007a, 2007b) use previous test scores as a control variable on the right-hand side of their model. Admittedly, this approach is not perfect—such measures may be endogenous to the student’s learning process—it is a generally accepted method of controlling for student prior knowledge. The Rivkin et al. approach is not appropriate in our case because, in our data, students do not take an economics test prior to the EOCT test result we observe. Instead of prior test scores in the same subject, economics, we use a related measure of prior student achievement, namely, previous EOCT scores in geometry. Therefore, our approach is akin to the Clotfelter et al. approach. Past studies (e.g., Ballard & Johnson, 2004; Evans et al., 2015; Swinton et al., 2012) have shown that math achievement (geometry in particular) is a strong predictor of subsequent success in economics classes. Therefore, we present a modified “value-added” model in which the student’s normalized test score in the mandated geometry EOCT represents the student’s aptitude for economics prior to taking the economics course.
Other factors that have consistently been demonstrated to affect student achievement are student characteristics such as gender, race, disability status, and poverty status. We control for these characteristics with a series of student demographic indicator variables. Also of interest are the characteristics of the schools in which students learn. If these characteristics change over time (e.g., see Heck, 2007, which focuses on school-level resources), quantifying the changes is important. Over a short period of time, however, these characteristics are unlikely to change with enough regularity to allow us to pick up the effects of variations within schools. Therefore, we use a fixed effects approach to capture variations in school characteristics between schools.
Of primary interest in this study are teacher characteristics that contribute to student learning of economics. There are two general types of characteristics we consider—those that demonstrate sufficient change within the time of the study to allow us to quantify the impacts of the changes and those that are time invariant (at least for the time of the study). These variables are the number of years of service, whether or not the teacher has an advanced degree, whether or not the teacher is National Board-Certified, and whether or not the teacher has attended any in-service economics workshops with the GCEE. We have found in previous studies (Swinton et al., 2007, 2010; Swinton & Scafidi, 2012; Swinton et al., 2012) that teacher attendance to such workshops has a positive effect on student learning. With the exception of workshop attendance, each of the variables listed has been linked, in one form or another, to teacher pay. To control for time-invariant teacher characteristics, we include teacher fixed effects dummy variables.
The base model takes the following form:
EconEOCTi is the normalized test score of student i on the economics EOCT and GeomEOCTi is the student’s normalized EOCT score in geometry. The vector of student characteristics (Studenti) includes a dummy variable for gender (= 1 if the student is female), a series of indicator variables for race (Asian, Black, Hispanic, Native American, and mixed race with White as the omitted variable), poor (= 1 if receives free or reduced-price lunch), and disability status (= 1 if the student is categorized as disabled). The VTeacherjt vector contains teacher characteristics that vary with time. There are four different teacher characteristics that we use to determine the accumulation of human capital. First, we quantify experience with 13 separate pay step qualifications. Each pay step represents an additional 1 or 2 years on the job (see the appendix for a definition of the pay steps). We use pay steps as opposed to years of service because it is desirable to compare the points at which a teacher is automatically given a raise with his or her incremental effectiveness at that same juncture. Second, we include a dummy variable that describes whether or not a teacher has earned an advanced degree. Third, we use a dummy variable that signifies whether or not the teacher has earned National Board Certification. Clotfelter et al. (2007a, 2007b) find evidence that earning National Board Certification is positively correlated with teacher effectiveness (although it is difficult to determine whether National Board Certification is simply a way for better teachers to signal that they are more committed to teaching). Finally, we include a fourth teacher characteristic that is relevant for the specific subject of economics. We know whether teachers have attended any in-service workshops conducted by the Georgia Council of Economic Education. We include this information with a dummy variable in the model.
Some characteristics do not change during the time spanned by the data we use. NVTeacherj represents a vector of teacher fixed effects dummy variables. While it is important to control for unobserved characteristics such as individual teacher motivation, including a fixed effects dummy variable for each teacher often subsumes the impact of variables meant to demonstrate the effectiveness of different activities the teacher might undertake to improve his or her effectiveness. For example, a teacher dedicated to her craft not only may be a better teacher in general, but also may be more likely to pursue National Board Certification. In those studies (e.g., Clotfelter et al., 2007a, 2007b; Rivkin et al., 2005; Rockoff, 2004) that use teacher fixed effects, most find that the measurable student impact of activities meant to develop human capital diminish. NVSchooln represents a vector of n school fixed effects dummy variables to capture the impact of invariant school characteristics that affect student learning. While other work has found that specific school characteristics do matter, our time frame is too short to observe any meaningful change in any of these characteristics. Therefore, an examination of the importance of within-school effects is not possible. To show the importance of both teacher-level and school-level fixed effects, we first present results without either, and then we present results with teacher-level fixed effects but not school-level fixed effects. Following that, we control for school-level fixed effects but not teacher-level fixed effects. Finally, we present results that control for both simultaneously.
Model 2: Teacher Demographics
Because we also have some invariant teacher information, gender and race in particular, we can look at the effect of teacher demographics on student learning. Therefore, we model student performance including teacher demographics. Whereas Dee’s (2004) work provides evidence for the hypothesis that students respond well to teachers who happen to look similar to them, others such as Howsen and Trawick (2007) have raised issues with Dee’s findings. So, we also include interaction terms that indicate whether or not the teacher and student share the same gender or race. The second model takes the following form:
In this model, the vector of teacher variables, Teacherj, represents indicator variables for whether or not the teacher is above 60 years, whether or not the teacher is Black or Hispanic (White being the omitted variable), and whether or not the teacher is female. The vector TeacherjStudenti represents a series of interaction dummy variables that indicate whether or not student i and teacher j share similar characteristics. The vector consists of seven separate dummy variables that are meant to capture the effect of female students with female teachers, Black students with Black teachers, Hispanic students with Black teachers, Asian students with Black teachers, Black students with Hispanic teachers, Hispanic students with Hispanic teachers, and Asian Students with Hispanic teachers. The other vectors remain the same as in the first model.
We chose to focus on the teacher–student interactions listed above, rather than all possible combinations, due to concerns about small cell sizes for some other combinations of students and teachers. We dropped all students from the following categories: Native American or mixed race. We also dropped any observations for which the teacher was identified as being anyone other than White, Black, or Hispanic. For some of these interactions, the cell sizes dropped into the single digits and we are not comfortable drawing conclusions based on so few observations. 2
We present the results in two ways. 3 We first present the results omitting the teacher–student interaction variables. This acts as a baseline from which we compare the next set of results. In the second set of results, we include the interaction terms as described.
Results and Discussion
The results of the estimation of our teacher human capital model are presented in Table 2. The first column of results in Table 2 contains the estimated parameters of a simple regression that does not include teacher or school-level fixed effects. When teacher and school-level fixed effects are not included in the model, we find that students who have a teacher who has attended a workshop score roughly 12% of a standard deviation higher on their economics EOCT exam. We also find that students whose teachers have advanced degrees or are nationally certified perform better than other students. We find very mixed effects for experience; as measured by our pay step variables, some steps are statistically significant and positive, whereas others steps are statistically significant and negative, and many are not statistically different from zero. The positive parameters appear in the earlier years of teachers’ careers, whereas the negative results dominate the later years.
Model 1: Teacher Credentials.
Note. FE = fixed effects; EOCT = end-of-course test.
Statistical significance at the 10% level. **Statistical significance at 5%. ***Statistical significance at 1%.
All of the above results are consistent with earlier findings when researchers do not control for unobserved teacher or school characteristics. Of course, the exclusion of school- and teacher-level fixed effects from the base model used to estimate the first column of parameters means that effects of differences in school environment or difference in teacher characteristics that we do not observe are forced into the differences we do observe. To the extent that the unobservable characteristics affect the other characteristics (e.g., a school that values Board Certification encourages all teachers to pursue certification or the more conscientious a teacher is about her performance, the more likely she is to attend an in-service workshop) our initial results could be biased. To address that omission, we add school-level fixed effects (as presented in the second column of results), then teacher-level fixed effects (as presented in the third column of results), and finally both school- and teacher-level fixed effects (as presented in the final column of results).
In all the iterations presented in Table 2, we find that female, non-White, poor, and learning-disabled students perform worse than their counterparts. We also find that students who perform better on the geometry EOCT are likely to perform significantly better on the economics EOCT. The inclusion of school- and/or teacher-level fixed effects has little or no impact on the importance of student characteristics in determining student outcomes on the economics EOCT.
As is evident from the results presented in the final column of Table 2, after including both school- and teacher-level fixed effects, we find markedly different effects of teacher credentials on student scores on the economics EOCT. The effect of workshop attendance on student performance becomes statistically insignificant. The effect of a student’s teacher earning an advanced degree does not appear to be a statistically significant determinant of student performance on the economics EOCT. A teacher becoming nationally certified appears to decrease a student’s performance on the economics EOCT by roughly 11% of a standard deviation. And, once we control for school- and teacher-level fixed effects, students with teachers who move up a pay step often can be expected to perform worse than those who do not. This effect is statistically significant for 11 of the 13 pay steps.
The change in sign and significance observed for these teacher credential variables appears to be driven by the inclusion of the teacher-level fixed effects, as evidenced by the change in effects when comparing the second column of results (the addition of school fixed effects) with the third and fourth columns of results (the addition of teacher-level fixed effects and the inclusion of both school- and teacher-level fixed effects). The finding that teacher credentials generally have either no effect or, in some cases, a negative effect on student performance on the economics EOCT after including teacher-level fixed effects suggests that a teacher’s demographics may be a significant determinant of student performance. Concentrating on teacher characteristics also offers us the opportunity to examine the hypothesis that there is no advantage to matching student characteristics with teacher characteristics. That is, within the limits of the data, we can see whether there are advantages for female students to be paired with female students and whether some non-White students perform better when paired with teachers of a particular race. We present the results of our analysis of the effect of teacher characteristics in Table 3.
Model 2: Teacher Characteristics and Teacher/Student Interactions.
Note. FE = fixed effects.
Statistical significance at the 10% level. **Statistical significance at 5%. ***Statistical significance at 1%.
The first column of parameter estimates in Table 3 presents the results of our analysis of the effect of teacher characteristics on student performance on the economics EOCT after controlling for school-level fixed effects, but excluding student–teacher characteristic interactions. The coefficient estimates for differences in student characteristics are largely the same as previously reported. When looking at the teacher characteristics, we find that having a teacher who is above 60 years significantly decreases student performance on the economic EOCT. We also find that having either a Black or Hispanic teacher reduces student performance, all other things being the same. Students with a female teacher earn scores on the economics EOCT that are roughly 7% of a standard deviation higher than students with a male teacher.
It is possible that students of different races and genders may perform differently based on the race or gender of their teacher. For example, it is possible that a female student may perform differently if her teacher is also female. To address this possibility, we interacted teacher and student race and gender variables. The second column of results in Table 3 contains our parameter estimates of the updated model. These parameter estimates must be interpreted carefully. When we add interaction terms, we are comparing impacts relative to implicit interactions that are not presented.
We find that female students who have female teachers score 4.3% of a standard deviation higher on the economics EOCT than do male students with male teachers. This difference is statistically significant and is similar in magnitude to one third to one half of the (negative) effect of being poor as presented in the various columns in Table 2. Interestingly, there appears to be little advantage to matching students to teachers of similar racial background. Black students matched with Black teachers seem to gain no statistically significant advantage (or disadvantage) over White students with White teachers. The same is true for Hispanic students matched with Hispanic teachers. We do not present the interaction of Asian students with Asian teachers due to a lack of observations. Of the other interactions we examine, Asian students who have Hispanic teachers score 16% of a standard deviation better on the economics EOCT than do White students with White teachers. The other interactions do not have statistically significant coefficients.
The inclusion of these teacher and student characteristic interactions also changes the interpretation of the effects of our teacher female, teacher Black, and teacher Hispanic variables, as well as the student race and gender variables. The interpretation of the coefficient on our teacher female variable is now the effect of a female teacher on a male student (his score will be roughly 5% of a standard deviation higher) relative to a male teacher with a male student. Likewise, the interpretation of the student female variable is that female students who have male teachers will score 16% of a standard deviation lower than male students with male teachers. So, female teachers have a positive influence on both female and male students relative to male teachers.
The interpretation of the coefficients on teacher Black and teacher Hispanic changes to the effects of those teachers on White students relative to White teachers. So, a White student with a Black teacher will, on average, perform 0.06 SD worse on the economics EOCT than that student would with a White teacher all else held constant. Likewise, a White student would be expected to perform 0.18 SD worse with a Hispanic teacher than with a White teacher. The coefficients on the student race variables also change such that they should be interpreted as the effect on the economics EOCT performance of those students having White teachers relative to the effect on White students of having White teachers. As these parameters are all statistically significant and negative, we are left with a complex picture of the impact of matching students and teachers along racial characteristics. White students perform worse on the economics EOCT when their teacher is non-White than when their teacher is White. Non-White students perform worse on the economics EOCT when their teacher is White than non-White. But, there seems to be no advantage to matching non-White students with a teacher of their same race. There also seems to be no disadvantage to matching a non-White student with any non-White teacher of a different race.
Conclusion and Suggestions for Future Research
Before considering any potential policy implications of our study, let us consider the importance of considering what we cannot observe. Our first model shows the important implication of including fixed effects to control for both school and teacher characteristics we cannot observe or that do not vary over time. Without school and teacher fixed effects, we would be likely to attribute some of the observed student success on the economics EOCT to factors often considered to improve teacher performance, such as attainment of graduate degrees, National Board Certification, in-service workshop attendance, or years of experience. Once we account for unobservable differences in schools and teachers, the impact of these observable efforts largely disappears or in some cases become negative.
Once we control for these fixed and unobservable characteristics, we are left with a practical problem. If observable characteristics do not provide statistically significant improvements in student performance, can they still be used to reward teachers or attract better teachers into the profession? We offer a two-part answer to this. First, the results of our base line regression in Model 1 (Table 2) can provide a starting point. Consider in-service workshop attendance, advanced degree attainment, and National Board Certification together. All of them not only seem to improve teacher impact but also seem to be efforts undertaken by teachers who may be better anyway. This is based on the observation that when unobserved teacher characteristics are controlled for, the impact of these efforts becomes either statistically insignificant or negative. That seems to imply that the efforts could be rewarded but the school board would have to ensure that the quality of the activity is monitored. Once teacher pay is tied to efforts taken outside of the control of the school board, there is an incentive for entrepreneurs to attempt to capture this rent. For example, less than scrupulous schools might offer graduate degree programs without offering much in the way of additional skills. So-called diploma mills capitalize on the need for teachers to quickly obtain graduate degrees. Therefore, when school boards can certify the quality of graduate degrees or in-service workshops, they can use the attainment of the degrees or attendance at the workshops as indicators of underlying teacher quality. The popularity of National Board Certification is that it offers a standardized process that can be used to infer a level of teacher commitment. Unfortunately, the current practice of the vast majority of schools (Nittler, 2018) is to offer additional pay for advanced degrees without regard to the quality of the institution granting the degree.
Turning our attention to teacher tenure, it seems clear that linking pay to time in the classroom provides a poor incentive after the first seven pay steps in our study and even earlier in other studies. The consensus, therefore, seems to be that it is beneficial to load retention-based pay in the front end of a teacher’s career. If their pay is sufficient to retain them early in their career, they can continue to build human capital via the other paths studied herein. But, attempts to change the practice of linking teacher pay directly to tenure have all but failed. As a case in point, the pay grade levels based on tenure used to analyze the Georgia teacher data in this study are the same as they were over a decade ago.
When it comes to gender, our results suggest that there is an advantage to encouraging more women to teach economics in our high schools. One third of Georgia’s economic teachers are female. More than 50% of students are female. Both male and female students benefit from having a female teacher. To the extent that teachers can be rewarded for student performance, female teachers may be encouraged to choose economics as a discipline. But, whatever the mechanism, there is a benefit to be had by encouraging more women to enter the discipline at the high school level. Currently, the economics profession is struggling to attract women into the profession. Emerson et al. (2018) highlight the difficulty in recruiting young women into the field. But, they focus on students who intend to become professional economists. Little or no research has been done examining how one recruits education students to choose economics as their area of expertise. This is an area ripe for additional investigation.
There is not as clear a message to be gleaned concerning encouraging non-White teachers into the discipline. While there is evidence that non-White students may perform better with non-White teachers than with White teachers, there is also evidence that White students may perform worse with non-White teachers. This seems to be a juncture at which we hesitate to engage in social engineering. Therefore, we defer with the less than bold observation that there seems to be enough going on here to warrant additional research—that is, we cannot dismiss the importance of considering the alignment of teachers of different races with students of different races. But, at the same time, our study does not provide a clear path to an overall improvement in student performance based on such alignments. We leave the discovery of that path, if it exists, to other researchers.
This being said, racial-based gaps in education outcomes have long been a focus of education policy in the United States. Reforms such as the No Child Left Behind Act of 2001 and the long list of revisions to that Act, including the Race to the Top program that was part of the American Recovery and Reinvestment Act of 2009, explicitly target racial discrimination and achievement gaps. In some cases, funding was tied to changes in those achievement gaps. As mentioned previously, one proposed method of shrinking those gaps is a change in the makeup of teachers. Organizations such as the Ford Foundation Center for Social Justice have implemented grant programs designed to develop more minority teachers over the past few decades. The Pathways to Teaching Careers Program funded by the DeWitt Wallace-Reader’s Digest Fund found some success recruiting minority teachers into the profession. According to Moss (2016), even though the federal government and half of the states have programs to recruit minority teachers—primarily to serve in underserved areas—an overwhelming number of these teachers burn out quickly and leave the profession. Ingersoll and May (2016) emphasize the gap between minority teacher turnover and White teacher turnover as well. If it is desirable to change the racial makeup of the classroom, more attention needs to be paid to issues of retention. Otherwise, resources expended toward recruitment of minority teachers will be wasted.
Footnotes
Appendix
| Teacher salary step | Years of experience |
|---|---|
| E | 0, 1, 2 |
| 1 | 3 |
| 2 | 4 |
| 3 | 5 |
| 4 | 6 |
| 5 | 7 |
| 6 | 8 |
| 7 | 9, 10 |
| L1 | 11, 12 |
| L2 | 13, 14 |
| L3 | 15, 16 |
| L4 | 17, 18 |
| L5 | 19, 20 |
| L6 | 21+ |
Acknowledgements
We wish to thank Ben Scafidi for significant contributions during the initial stages of research, Lixia Zhang for her diligent research assistance, and two anonymous referees for their helpful comments during the review process. This research was funded by grants from the United States Department of Education Office of Innovation and Improvement through the Council on Economic Education Excellence in Economic Education Grant (IS-1102889) and Georgia College. Finally, we thank the Georgia Department of Education for sharing its student administrative data and expertise, which made this study possible.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by grants from the U.S. Department of Education, Office of Innovation and Improvement, through the Council on Economic Education Excellence in Economic Education Grant (IS-1102889) and Georgia College.
