From the Scholars Strategy Network, written by Nicholas D. Hartlep, Christopher M. Hansen and Brian R. Horn, Illinois State University

 

States and school districts across the United States are trying to decide which teachers are effective in elementary and secondary-school classrooms. Approaches based on student gains in test results are currently in favor, but they are not very useful for predicting who is likely to do well from the moment new teachers are hired — and they do not provide information that schools need to help teachers do better. Principals and administrators end up with crude classifications of teachers as effective or ineffective. Such results can inform choices to retain or fire teachers, but provide little guidance for hiring and training them. Measurements of teacher dispositions and attitudes can help solve this problem.

The spread and limits of “Value Added Modeling”

Currently, an approach to evaluating teacher effectiveness called Value-Added Modeling is spreading across the country at the urging of federal government authorities. This is a statistical approach that tracks each teacher’s contribution to student learning over the course of the school year using results from repeated student testing. Most value-added models used in education compare students’ annual standardized test scores over multiple years to assess student progress in fundamental academic skills like reading and math. Then they use the results — improvements, decline, or stagnation in student scores — as a measure of the effectiveness of individual teachers and schools.

As of May 2014, public school systems in 35 states and the District of Columbia required that student achievement as measured by state reading and mathematics tests be used as a significant fact — even the most significant factor in teacher evaluations. States and school systems are acting to implement this kind of Value-Added Modeling at the urging of the U.S. Department of Education, whose Race to the Top program requires that states applying for funds implement systems that include student test scores as part of the measurement of teacher effectiveness. The idea is to use effectiveness measures to influence decisions about teacher pay and retention.

Despite increasing numbers of states adopting this approach, a number of statistical and professional associations, as well as educational research agencies, believe that Value-Added Models are insufficiently comprehensive. Although they can roughly classify teachers as “highly effective,” “effective,” and “ineffective,” they cannot pinpoint the more or less effective pedagogical practices of teachers. For instance, Value-Added Modeling cannot inform school administrators whether a particular teacher whose students make substantial test gains uses methods sensitive to students’ cultural backgrounds or believes that students from poor families can learn. In practice, school administrators must do more than retain or fire teachers. They have to hire promising new teachers and serve as instructional leaders for their staff, designing professional development programs. Beyond test scores, they can draw upon classroom observations and surveys of students, parents and others.

Finding and developing resilient, effective teachers

The hiring of teaching staff is the first opportunity schools and school districts have to positively affect student achievement, but test-based measures are not useful for novice teachers, for those new to a state or those who have not taught a particular grade at a single school long enough for consistent data to be collected. Previous research suggests that it is less costly to recruit new teachers with the right dispositions as well as abilities than it is to try to weed out ineffective teachers after the fact. One scholar who has championed well-informed teacher selection is Dr. Martin Haberman, who developed measurements based upon decades of research in New York, Chicago and Milwaukee Public Schools. In his view, truly effective teachers are people not defeated by conditions in troubled schools. To find such teachers, Haberman developed a “Star Teacher Pre-Screener” to measure 10 skills and dispositions:

• Persistence predicts propensity to work with children who present learning and behavioral problems on a daily basis, without giving up on them during the school year.
• Organization and Planning gets at how and why effective teachers plan, as well as their ability to manage complex classroom arrangements.
• Values Student Learning measures willingness to make student learning the highest priority.
• Theory to Practice taps the respondent’s ability to see the practical implications of generalizations as well as the broader concepts embodied in specific practices.
• At-Risk Students predicts the likelihood that the respondent will be able to connect with and teach students of all backgrounds and levels.
• Approach to Students predicts how the respondent will attempt to relate to students.
• Survive in Bureaucracy predicts whether the respondent can teach in a large organization.
• Explains Teacher Success measures how the respondent understands good teaching and whether the criteria the respondent uses are relevant to teaching in schools.
• Explains Student Success deals with the criteria the respondent uses to determine students’ success and whether these are relevant to students in schools.
• Fallibility probes how the teacher anticipates dealing with mistakes in the classroom.

Collecting this kind of dispositional information about applicants for teaching positions — and also from members of the existing teaching staff — can make it possible for school administrators to hire the most promising people and devise targeted training programs to enhance the capacities of existing teachers. Despite current efforts to use student test scores as an overriding measure of teacher effectiveness, there is a more pressing need for useful information. To improve schools in the United States, we need to go beyond labeling and denigrating many teachers as failures, by finding people who have the best chance to succeed in the classroom and devising long-term professional development programs to nurture best practices. In that way, we can steadily improve the entire teaching workforce in communities across the United States.

Research for this brief was drawn from the authors’ own ongoing research and “Ensuring Teacher Quality: A Continuum of Teacher Preparation and Development,” published in November 2000.

Related research: In a 2011 Harvard and Columbia study, “Predictive Effects of Teachers and Schools on Test Scores, College Attendance and Earnings,” economist Gary Chamberlain seeks to quantify the short- and long-term impact of fourth- through eighth-grade teachers by observing them in multiple classrooms. Chamberlain considers three student outcomes: average score on a math or reading test given near the end of the school year, fraction of the students in a given class attending college at age 20, and average income at age 28. By holding constant additional relevant variables, he is able to estimate predictive effects of teacher quality.

 

The author is a member of the Scholars Strategy Network, where this post originally appeared.

Keywords: youth, higher education, value-added teaching, research brief