Journalist Paul Tough has published several books over the past decade exploring ways to improve the life chances of children born into poverty. In his new book, The Years that Matter Most: How College Makes or Breaks Us, he examines the role of higher education in social mobility in the U.S. today. He writes at length about the gatekeepers to the nation’s most selective colleges and universities. FutureEd Director Thomas Toch spoke with Tough about one of the most influential of them, the College Board, the organization that administers the SAT admissions test.
Toch: Your book is about the role of higher education in social mobility, especially who gets to go to the nation’s most selective schools, the 275,000 or so students in the nation’s top universities and liberal arts colleges, which you note provide the greatest economic and social advantages to students. You write extensively about the gatekeepers to these valuable seats. One of them is the College Board, the nonprofit organization that administers the SAT, the college admissions test created in the 1920s. What’s the origin of the SAT and how did it become such an influential filter for the nation’s most selective schools?
Tough: The SAT was originally conceived to bring a new type of student to New England colleges and universities, which at the time were a pretty small collection of elite institutions that were mostly educating rich kids from prep schools. The idea was that the SAT would identify smart kids from around the country who would disrupt the aristocratic hierarchy of those colleges.
The majority of the nation’s 4,500 two- and four-year degree-granting colleges don’t have selective admissions and thus don’t put as much stock in SAT and ACT scores, right?
Well, there are less-selective and even non-selective institutions where test scores are important in determining who gets placed in different programs—whether, for example, you’re assigned to developmental math or admitted to regular college math. But as a gatekeeper of who is admitted, the tests play a bigger role in selective institutions.
With the College Board, the owner of the SAT, we’re talking about a nonprofit organization, but one, you write, with $1.3 billion in net assets and a CEO, David Coleman, who makes $1.5 million a year.
The College Board has two different personalities, two different identities. One is a genuine nonprofit that is designed to serve colleges and students. And especially in the last few years, that side of the College Board has been concentrating on trying to rethink the question of equity in higher education admissions through a variety of different interventions.
But then there’s a side of the College Board that behaves much more like a regular for-profit company. It has a product to sell, the SAT, and a single big competitor—not unlike Coke with Pepsi. And those two products, the SAT and the ACT—again, not unlike Coke and Pepsi—aren’t that different. The fight to win customers and market share makes the College Board and ACT, Inc. behave essentially like for-profit companies. And so those two mandates that co-exist within the College Board don’t always fit together easily.
The College Board’s CEO, David Coleman, who’s a Rhodes Scholar who played a central role in writing the Common Core K-12 reading standards, has sought, as you mentioned, to enhance the organization’s role as an engine of social mobility, as an organization that promotes greater equity in higher education. What are some of the things that he’s done and how successful has he been?
Early in his presidency—he was hired in 2012—he talked about three changes he wanted to make to shift the role of the SAT and to make college admissions fairer. The first was redesigning the SAT to make it less complicated and tricky and more aligned with what students were learning in high school.
The second was to replicate on a larger scale an experiment that the economists Caroline Hoxby and Sarah Turner had done to send out informational packets to tens of thousands of high-achieving, low-income students throughout the country to encourage them to apply to more-selective institutions, which got strong results in its experimental form. The third was to partner with Khan Academy, the online learning system, to offer free SAT test prep to any student who wanted it, especially kids who couldn’t afford the kind of expensive test prep that rich kids were using.
You suggest in the book that Coleman and his colleagues have been less than forthright in reporting the disappointing results of a couple of those initiatives, including the project to send information packets to low-income students to get more of them to apply to top schools. What was going on there?
It took me six years to write this book, so I was able to follow the evolution of these different interventions over time. And yes, the packets did not succeed in the way the College Board intended.
They sent out the first batch of replication packets in 2013. A year or two later, they were seeing indications that the replication wasn’t having the same positive effect on student behavior that Hoxby and Turner had reported in their original experiment. But they spent a few more years deciding how to respond to that news. The first time that they shared with the public the fact that [the Hoxby-Turner study] wasn’t replicating well was just a few months ago, after they had already ended the replication project.
You write that there was a similar response to the College Board’s much-touted partnership with Khan Academy to provide free SAT tutoring to students, another attempt to level the college-admissions playing field. Was the College Board trying to mislead the public?
The results from the Khan Academy partnership are not necessarily bad news. But they’re complicated news. The College Board found that students from different demographic groups achieved the same score gains if they used Official SAT Practice for the same amount of time—and they touted that result. But they also found that those significant impacts came only when students used Khan Academy for a long time—20 hours. And 99 percent of students who were invited to use Official SAT Practice weren’t using it that much, which the College Board didn’t report. They also highlighted the result that black students were using Official SAT Practice for a few more minutes, on average, than white students. But in every other instance, traditionally advantaged groups—Asian students, male students, students from families with money and whose parents were well-educated—were using it more than less-advantaged students. They didn’t mention that in the initial rollout of their results.
As a result, the initiative, well-intentioned though it was, widened rather the narrowed the score gap between advantaged and disadvantaged groups, exactly the opposite of what the College Board hoped for.
That’s right. Students from groups who were already doing well on the SAT practiced more and gained more of an advantage over their disadvantaged peers. Again, that doesn’t necessarily mean Official SAT Practice is a bad thing. One analogy that I’ve heard people at Khan Academy use is to public libraries. Even if rich kids are using public libraries more than low-income kids, it doesn’t mean public libraries are a bad thing. It can be good to have opportunities that are free that anyone can use. That may be true. But it’s not the case the College Board and Khan Academy made back in 2017. Instead, they talked in glowing terms about how Official SAT Practice was leveling the playing field for disadvantaged students.
When you’re testing out a new program, and it doesn’t work the way you intended, it doesn’t mean you should never do it again. It means you should learn from and share the results and try and figure out what happened and why. But that has not happened with Official SAT Practice, because the College Board has not been transparent with the data. The organization points out that it did eventually release a little bit of that demographic data a year and a half later, in the fall of 2018—in a report they did not do a lot to publicize. And that’s true. But to me, there was a clear disconnect between the celebratory way they talked about the results in media conference calls early on, and the quiet data release later that showed the story was more complicated.
In your book you point to a third example of the tension between the College Board’s stated commitment to leveling the college-admissions playing field and its business interests. About half of the top liberal arts colleges in the US News rankings no longer require applicants to submit SAT or ACT scores for admissions. They’ve gone “test optional,” relying on students’ GPAs instead of test scores in admissions. College Board has attacked the test-optional trend on the grounds that relying on GPAs hurts low-income students. But you write that the College Board’s claim that SAT scores help low-income students in admissions while GPAs hurt them is “simply untrue.”
The College Board says it doesn’t attack test optional, that it’s happy for colleges to choose to not use the tests. What I saw in my reporting was a campaign mixing social science with public relations to try to cast doubt on the test-optional movement, a point of view expressed in various conference panels and speeches and especially in a scholarly book that Johns Hopkins University published in 2018 that was overseen by three employees or former employees of the College Board.
You call it a “book-length advertorial,” akin to the research published by the tobacco industry on the causes of lung cancer.
That book, and especially one paper in it about the demographics of grade inflation, was designed to cast doubt on the idea that high school grades were, first, a good predictor of how well students will do in college and, second, a factor that when given more weight in admissions often leads to more balance between well-off students and low-income students.
Most people who study testing would agree with both of those ideas, but they are problematic for the College Board. The board uses the fact that there has been grade inflation in the nation’s high schools in recent decades to argue that grade inflation is benefiting advantaged groups more than disadvantaged groups, and thus that high school grades are no longer a fair measure of what students can do. The SAT, the organization argues, is a more accurate measure and also a measure that gives more advantages to low-income students and other disadvantaged groups.
The paper [by College Board researchers Michael Hurwitz and Jason Lee] was sent to a lot of education journalists. It was written about in all sorts of places. It continues to be written about. It created a narrative that’s used now by anyone arguing against test-optional admissions: There is rampant grade inflation; rich kids are taking advantage of it more than poor kids; and that’s why we can’t rely on high school grades any more. We need the SAT and the ACT.
But when you look carefully at that paper, it does not support those claims. Yes, it shows that there has been grade inflation, but it also shows that grade inflation has taken place at a roughly equal rate among different demographic groups—and that the area where there really are new gaps between groups is in their SAT scores. The SAT scores of advantaged groups have been going up over the past couple of decades, and the scores of disadvantaged groups have trended down.
So the College Board was suggesting in public more or less the opposite of what its own research was finding.
To me, the disconnect goes back to the College Board’s split personality. David Coleman and others there do want to have a fairer college-admissions system. They wanted these interventions to work. But when they didn’t work the way they hoped, I think they were torn between their desire to inform the public and a desire to protect the brand.
You suggest in the book that the press has been an enabler of some of the College Board’s misrepresentations.
The education press has a really hard job, and I think the College Board, in the way that it has presented data, has made it harder. I had six years to study this data and to talk to experts about it. You don’t have that luxury if you are a daily reporter.
I should say, too, that there are many educational reporters who’ve done incredible reporting on the College Board, and I relied on them a lot in my book. I think of Nick Anderson of the Washington Post, especially. So I don’t want to criticize the education press. I think the College Board’s strategy of focusing only on certain data when it releases results makes it more likely that journalists on deadline are going to report the data the way the College Board presents it to them. That has led to some misleading articles. Journalists need to demand a greater level of detail from the College Board.
You point out that the College Board was struggling financially when David Coleman became CEO seven years ago, that it was losing market share to the ACT. One response, you write, was to convince states to use the SAT as a high school achievement test under the accountability requirements of the federal Every Student Succeeds Act. The strategy turned the company around, you say, though many testing experts question the use of the SAT to hold high schools accountable.
Yes, there were two campaigns going on at once within the College Board during the years I was reporting my book. One was to change either the perception or the reality of how much the SAT benefited advantaged kids over disadvantaged ones. The other was to win the competitive battle with the ACT test that the College Board was suddenly losing for the first time in its history. One campaign was very public, and one was very private. A lot of what I report on in the College Board chapter of the book has to do with the inherent conflict between those two campaigns.
One of the things that I found fascinating in reading Nick Lemann’s book, The Big Test, on the origin of college admissions testing, is that the ACT was invented to do what both tests are now competing to do, right, which is reflect what is taught in high school. And the whole point of the SAT in its earlier days was that it wasn’t connected to the high school curriculum. It was an aptitude test, its creators argued. “You can’t study for the SAT. You can’t learn to get better at the SAT. It measures something you’re born with or you’re not.” Of course, that wasn’t true. You could always study for the SAT. But that was the argument. One of David Coleman’s challenges is that he’s now portraying the SAT as a tool that does more or less the exact opposite of what it was originally intended to do, or originally presented to do.
The College Board just announced that it is going to revamp its so-called adversity index, the recently created measure of student social and socioeconomic backgrounds, again designed to level the playing field.
Like a lot of things that the College Board has done, it started with good intentions and some good ideas. It is certainly a good idea for admissions people to understand the context that each student brings to the SAT and to their application. And it’s good for them to understand that SAT scores differ by family background. But I think they know that already. That’s why some colleges are going test-optional: they see how closely SAT scores correlate with family income.
You don’t need an index with tons of different data points to demonstrate that. SAT scores correlate with family income, clearly and consistently, and have done so for decades. Colleges should give less weight to SAT scores if they want to admit more low-income students. They don’t need a landscape score to do that.
The research shows, does it not, that students who are admitted without submitting test scores do more or less as well as students with scores?
Data I have seen does suggest that. I write about that specifically at DePaul University, where there is really good data showing that is true.
You focus in the book on the 275,000 or so seats in the nation’s most selective schools. That’s a small fraction of the nearly 17 million students attending undergraduate colleges and universities this fall. Should we be focusing on a wider range of students in our work to increase social mobility?
I would challenge your premise a little bit. I hope I’m giving a broad picture of higher education in the book. There’s one chapter, in particular, Chapter 7, where I write about students who are not going to or aspiring to those more selective institutions. And in some ways, it’s the most important chapter in the book.
We need two separate solutions in higher education. In more-selective institutions, the problem is mostly one of admissions. Decisions are made in ways that make it harder for low-income kids to get into the most mobility-enhancing institutions. There really should be a more-level playing field that would allow more lower-income kids to get to those institutions.
But we also need a much, much better set of options for low-income kids who are coming out of high school who maybe didn’t love school, who aren’t focused on spending four more years sitting in a classroom and reading books and doing math. We just don’t have good options for those kids. And we’ve spent the last couple of decades withdrawing funding from public higher education at a moment when all the signs from the economy and from the labor market indicate that young people, especially young people from disadvantaged backgrounds, need more education, more skills, more credentials to get jobs that can support a decent, middle-class life. That’s the point I’m trying to make in the final chapter of the book: We all need to think differently about public higher education.
You conclude that in public higher education the key drivers are the quality of instruction, the extent to which professors can engage students in the classroom, and then the extent to which institutions can connect students—especially those from low income families, first generation college-goers, students of color—more closely to the institutions themselves, enhancing their sense of themselves as learners and as legitimate, valued members of learning communities.
That’s right. There are important, innovative programs that can help more students succeed at universities of all different selectivity levels. I write about a few of them in the book, and your description of them is exactly right: they are mostly about making students feel more welcome and connected, giving them more of a sense of belonging. That sometimes means a very different kind of teaching and advising than most universities have done in the past.
But in a significant way, the variations in graduation rates among different institutions are also a question of resources. One reason so many low-income students are not completing their degrees is that they can’t afford the tuition. Another is that they can’t afford to jump through the hoops they have to jump through because their states aren’t funding their public higher education well enough. Their institutions just can’t afford the programs and resources that will make it easier for students to succeed. So, yes, there are cool new programs that I think institutions can introduce, but the main thing we need to do is to invest more resources into public higher education. If we do that, it will make it easier in every possible way for disadvantaged students to succeed.
After traveling widely across the higher education landscape for six years, do you believe that higher education can in fact be a more meaningful engine of opportunity?
Absolutely. I mean, I go back and forth on any given day—or on any given page of the book—between being pessimistic and being hopeful. But in that final chapter, I look back at various moments in American history, especially the GI Bill era and the High School Movement of the early 20th century, when the United States, in a way that we are not doing today, came together and decided to value public education and, in some cases, public higher education as something that benefited all of us collectively.
I think that’s what we need to do again. There have been changes in our economy and our politics and our culture that have pushed us away from that way of thinking about education as a common good, a collective good. But I don’t think that it’s a foreign idea in this country. It’s certainly something that plenty of other countries seem to be able to do. If we are able to reclaim that idea of a public higher education system that benefits us all, we can make changes that will lead to a much fairer and better higher education system.
Paul Tough is a contributing writer to The New York Times Magazine and the author of How Students Succeed: Grit, Curiosity, and the Hidden Power of Character. We offered the College Board an opportunity to respond to this interview. The organization directed us to its public response to Tough’s book.