In an effort to put the strongest possible teachers in its classrooms, Los Angeles Unified School District (LAUSD) adopted a district-level screening system, the Multiple Measure Teacher Selections Process (MMTSP). FutureEd spoke with researchers Katharine Strunk and Paul Bruno about their recent paper showing a link between an applicant’s screening assessment and later employment performance.
How did you get involved in this project?
We have had a long-standing partnership with LAUSD where we try to understand aspects of their policies and practices, including their human capital practices. They implemented the MMTSP in hopes of getting stronger teachers into the district and asked us to partner with them to see if the effort has been effective.
Can you describe the new screening process?
Previously, LAUSD did a basic screening to ensure that applicants' credentialing was in place. Then the district interviewed applicants and shared a list of eligible teachers with principals. Under the MMTSP system, the district tried to collect more data about applicants, standardize the evaluation process across applicants, and link the screening process to district priorities.
What were those priorities?
An important priority for the district was aligning screening to teacher evaluation criteria. At about the same time that the new screening system was put in place, LAUSD adopted a new teacher evaluation system in which teachers are evaluated on the basis of the district's Teaching and Learning Framework. This framework includes many specific elements related to teachers' planning of instruction, classroom management, ethical professional conduct, and so on. The new screening assessments are deliberately intended to align with specific elements in the framework.
What are the components of the MMTSP screening tool?
There are eight different screening scores—for an interview, a sample lesson, a writing sample, references, GPA, subject matter licensure exams, background, and preparation.
Most of those are self-explanatory but what constitutes the background and preparation scores?
Up to three background points are given to candidates who have certain prior LAUSD experience like participation in the district's career ladder program, in which the non-teaching staff (like classroom aides) are encouraged to earn a credential; have a master’s degree or higher; have specific prior leadership experience in the military or other organizations, or are recruited through Teach For America. Then an applicant can get two preparation points if they attended a college or university highly ranked by U.S. News & World Report, can demonstrate evidence of prior teaching effectiveness, or majored in the subject they would be teaching.
Did the screening process predict teacher success?
We found that overall screening performance was predictive of teachers' contributions to student achievement, teacher evaluation outcomes, and attendance, but not predictive of a teachers’ retention in their school or the district.
Did any of this surprise you?
Similar work has found that screening performance predicts teacher attrition, so we were a little surprised there was not a significant relationship with retention in our study. And if school staffing operates differently in different districts or screening assessments capture different information about applicants, then maybe we should expect results to vary across settings. We can try to learn from those differences.
Were any of the screening components particularly connected to teachers’ success?
Performance on the sample lesson was related to both value-added contributions that teachers make to student achievement and to their own evaluation outcomes. Professional references, GPA scores, and subject matter knowledge scores were also connected to teachers' evaluation outcomes, as well as to their attendance.
What did you learn about the background points, for such things as prior LAUSD experience, a master’s degree, prior leadership or a Teach for America connection?
These are issues we'd like to learn more about. In some cases, receiving points for these characteristics is associated with greater teaching effectiveness. However, we aren't able to see exactly why applicants earned the points—was it for their Teach for America background or their background in the district—so we can't say exactly which attributes are associated with future effectiveness. We're working with the district to explore these characteristics in more detail.
What do your findings say about the quality of most teacher screening in public education?
We can't speak directly to the quality of teacher screening in schools and districts generally. There is other research suggesting that in many cases teacher hiring is frequently rushed and that screening is often pretty minimal. But there are also probably a lot of differences between districts and between administrators in the same district. Some districts are also trying to increase the use of data in their hiring processes, and our results in Los Angeles suggest that collecting more and better information about prospective teachers might be a good idea.
Are you aware of any other districts in the country that are adopting a similar system?
We know of at least two, Spokane and Washington, D.C., that have done something similar, where there is published research about the work.
Can the MMTSP system be used to make school placements that play to teachers' strengths?
There is probably a way to help principals understand how to look at the data and select teachers who might be the best fit for their schools. But these results are predictive, not definitive, so there is no way to say for certain that a teacher will perform in accordance with their screening data. The other point to remember is that at this time LAUSD does not give any of the data to the principals. They are only given the names of applicants who have passed the cut point and made it on to the eligibility list.
Has LAUSD made any adjustments to the criteria or process?
Not yet. But the district is thinking about potential changes based on what it is learning through our research partnership.
How might this screening tool be used more effectively in the future?
There may be ways that screening components can be weighted differently since some are more predictive than others. At the same time, reweighting certain criteria to better predict a particular outcome often means that other outcomes can't be predicted as well. So it's important for districts to think carefully about what they're looking for in teachers and about potential trade-offs if they focus on just one or two aspects of teacher quality.