Curriculum and standards are key research topics for Morgan Polikoff, a University of Southern California professor and FutureEd senior fellow. So when Mattel’s philanthropic arm approached him and his research partners about developing curriculum to teach science standards using Mattel’s Hot Wheels cars and tracks, they were in. The researchers developed a hands-on curriculum and professional development lessons teaching basic physics using the popular toys, then conducted a randomized controlled trial in about 60 fourth-grade classrooms in a California school district comparing student learning under the project-based and traditional textbook based instruction over three weeks. We recently talked with Polikoff about the project, known as Speedometry.
What are you trying to teach kids with this curriculum?
We’re trying to teach kids both science content—the science content in the fourth grade standards that’s relevant to our curriculum are energy transfers, what kinetic and potential energy are, the transfer from potential to kinetic like when a car goes down a ramp, and then the transfer of energy when objects collide. Those are the science concepts. We’re also teaching them science processes. They have to do investigations. They have to report on their findings. They have to keep track of data. So it’s both content and process, which aligns with the way that the Next Generation Science Standards are already organized.
Why did you choose fourth grade?
Kindergarten and fourth grade were the two best grades in terms of the standards alignment, primarily the science standards. We chose fourth grade to do the randomized experiment because it’s hard to measure kindergartners. It’s hard to measure what they know, and it’s hard to measure their emotions. And so we weren’t so confident that we could do that at scale.
How do your Speedometry curriculum and instruction differ from traditional curriculum and teaching?
The dominant instructional mode in elementary science classrooms is, for lack of a better word, pretty ‘traditional.’ Students or teachers may read about science, teachers may do demonstrations or show videos, or students may do relatively constrained data collection or analysis activities. This unit is very focused on actually doing science. Students pose questions, gather data, answer the questions, and share their results. We use something called the “5E model” that guides students through this process of doing science.
Did you find that this sort of hands-on, project-based curriculum helped children learn science?
We found really promising results. We found that kids learn science—their knowledge on an aligned assessment increased about half of a standard deviation (significant increases in performance on 14 of the 20 items on the test). We found that kids’ positive emotions (e.g., excitement) about science increased and their negative emotions (e.g., boredom) decreased. Their interest in science (e.g., their willingness to want to do this or other science activities again) was also increased. We had an educational psychologist on the project who measured both academic outcomes and these non-academic or social-emotional outcomes.
What did you find with the teachers who worked with this? Elementary school teachers are often not science specialists.
We did follow-up surveys of our teachers. They reported that the Speedometry curriculum was easy to use. We also did fidelity-of-implementation checks with all the teachers. We wanted to see if they were doing what they said and what their curriculum said they should be doing on a given day, and we found very high fidelity of implementation.
A lot of them said they had been used to teaching science in a particular way, a more traditional way. It was maybe more lecture-oriented, not so much students exploring. A number of teachers commented that they were nervous that students couldn’t do this kind of hands-on work.
Could the students handle it?
Teachers were very impressed at how students were able to do it, and then they said they wanted to do more of that kind of teaching. A number of teachers commented on how they felt it expanded their idea of what students would do.
How does this project align with the Next Generation Science Standards?
The Next Generation of Science Standards are organized around content, around science processes, and around cross-cutting themes. They’re sort of like a three-dimensional structure, and so certainly anyone would say that the NGSS are really focused on students actually doing science. And that really was the inspiration for the way that we designed our curriculum. It was about students doing science, again developing hypotheses and testing them.
Was there any concern that this was a commercial product—Hot Wheels—inserted into an academic environment?
Absolutely. And a few of our teachers expressed concerns. I mean the toys do come in a box that says Hot Wheels Speedometry. There are videos. There’s a definite push to make Hot Wheels more of an educational toy. And I think that’s in large part because parents like to buy educational toys. But Mattel had nothing to do with the creation of the curriculum, the research questions, the design, the analysis, or the dissemination.
Obviously there’s some strategy here from Mattel. But they’ve contributed over a million dollars to the research, the independent research. We can publish whatever we want.
Hot Wheels is generally considered a boy’s toy. Were you concerned that this experiment would be more effective for boys than girls?
We looked and we found no differences between boys and girls. We found no differences between English learners and non-English learners. And we found no differences between students with and without disabilities on any of the outcome measures.
There is a concern that there aren’t enough women in STEM, that there aren’t enough women at Google or other tech companies. Could this sort of learning help bridge that gap?
I think it’s hard to see how it would help close the gap because the effects were the same for boys and girls. But the effects certainly weren’t worse for girls, as many people expected that they would be.
There are a lot of reasons for gender gaps in those fields. But certainly I think students’ classroom experiences from the very beginning have to be one of the reasons. And so we want to make our math and science classrooms places where girls feel like they can be successful.
This effort concentrates on curriculum. In a lot of ways, educational reform seems to be about everything but curriculum.
Laying my cards on the table, I think it’s almost tautologically obviously that the in-school
thing that matters most for student learning is the stuff that happens between teacher and student in the classroom. And that has to do with both curriculum, what is taught, and how it’s taught, and how well it’s taught. And there has been a focus on standards, right?
That’s true.
But in general, that focus on standards hasn’t really taken the next step. Which is to say, “Okay. Here are the standards. And here are some curriculum materials you can use.” My personal opinion about the standards movement is that it hasn’t really reached inside the classroom that much because there’s still so many layers in between the standards and what actually gets implemented.
Publishers make interpretations of standards. They put out these curriculum materials. And we have some evidence that a lot of the curriculum materials in the U.S. aren’t really that good. Then there’s the school districts that have to make adoption decisions about these curriculum materials. And I’ve recently been doing work where I’ve been interviewing school district leaders about how they make these decisions. And the simple truth is that regardless of what criteria the district says they use to make the decision, it’s pretty much unrelated to the actual outcome of what they adopt. So districts I think are kind of grasping at straws to try and know what even to adopt.
What’s the solution?
I personally feel like curriculum is a very powerful lever for change. I think the Engage New York curriculum exemplifies that because that’s material that was just put out there for free. And certainly it costs money to make. But it’s very widely used. It’s the most common elementary math curriculum in the entire state of New York. It’s in the top five most common elementary math curricula in California. And so to me I think that that is a promising way to go.
Do you think project-based learning, like Speedometry can make a difference?
We think it’s promising. I mean it’s a small little intervention. What’s great about it is it’s available for free online. Anyone can go download it. Mattel has already given away more than 40,000 of these kits. And it’s been adopted at the whole district level in a number of large districts in California and Texas.