Matthew Steinberg's research shows evaluations based on observing teachers in the classroom often fail to meaningfully assess teacher performance.
Jeff Frantz, Penn GSE Associate Director of Communications
frantzj@upenn.edu | 215-898-3269
*Note for TV and radio: The University of Pennsylvania has an on-campus ISDN line and ready access to a satellite uplink facility with live-shot capability.
Teachers may not have a level playing field when it comes to one of the most commonly used tools for high-stakes job performance evaluations — classroom observation of teachers’ instructional practice.
New research from the University of Pennsylvania Graduate School of Education (Penn GSE) and the American Institutes for Research (AIR) shows evaluations based on observing teachers in the classroom often fail to meaningfully assess teacher performance. The study, published this week in Educational Evaluation and Policy Analysis, significantly contributes to the ongoing policy debate over when and how teachers should be evaluated.
Researchers Matthew Steinberg, from Penn GSE, and Rachel Garrett, from AIR, found that the achievement of students entering a teacher’s classroom accounts for much of a teacher’s evaluated performance. This means that prior academic achievement of students is a significant predictor of teacher success in the high-stakes evaluation system.
“When information about teacher performance does not reflect a teacher’s practice, but rather the students to whom the teacher is assigned, such systems are at risk of misidentifying and mislabeling teacher performance,” Steinberg and Garrett wrote.
Among Steinberg and Garrett’s findings:
Based on their results, Steinberg and Garrett caution that using observation-based measures for high-stakes teacher accountability without understanding and accounting for classroom composition will skew results, with potentially significant consequences.
“The misidentification of teachers’ performance levels has real implications for personnel decisions, and fundamentally calls into question an evaluation system’s ability to effectively and equitably improve, reward, and sanction teachers,” wrote Steinberg and Garrett.
Steinberg and Garrett reviewed data from a previous study that looked at six school districts over two years, including the New York City Department of Education. This Measure of Effective Teaching study randomly matched teachers to classrooms, differentiating between grade and if a teacher is a subject matter generalist or specialist.
The study comes as schools across the country have stepped up efforts to evaluate teachers. By the start of the 2014–2015 school year, 78 percent of states and 85 percent of the largest school districts and the District of Columbia had implemented teacher evaluation reform.