On Monday, I talked with departing Institute of Education Sciences Director Mark Schneider, who just wrapped up his six-year term. In our conversation, he argued for newer and better research centers at IES, along with a heightened commitment to producing timely and accessible reports. Well, as anyone who knows Mark well can attest, he almost always has more to say. I thought I’d reach back out and see if he had anything else he wanted to get off his mind. Here is Part Two of our conversation.
—Rick
Rick: On Monday, you mentioned that Marguerite Roza, Emily Oster, and Sean Reardon are doing good, serious work on education topics and releasing it in a timely fashion. Can you say a bit about what they’re doing right and why NCES isn’t meeting that need?
Mark: As noted on Monday, the most telling example is the collection of school-level finance data. Not only are these data required by law, they are also among the most important pieces of information needed to understand the distribution of resources and to relate detailed expenditure data to school outcomes. But despite years of pressure—from Jim Blew, who headed the policy division of ED during the last few years of the DeVos era, from me, and from many others—NCES has never produced timely district financial data and has yet to produce complete school-by-school financials. District financials are released more than two years after the close of the fiscal year, and school-by-school financial pilot data dating back to FY18 have not yet been released. Looking ahead, NCES has promised to release its first set of complete FY22 school-level financials at the end of 2024—again, over two years after the close of the fiscal year. Assuming NCES meets its promise—a risky bet, given past missed deadlines—this would be “fast” by government standards—but far too slow to meet real-world needs.
Rick: What’s responsible for this?
Mark: Much of the problem is that NCES uses an antiquated approach to data collection, issuing a uniform survey that doesn’t match up to different state systems and then waiting for all submissions before releasing datasets. In contrast, teams from Georgetown, Brown, Stanford, and others grab data directly from the source and then convert them into a more standardized format. Teams then release the state-by-state files as they are available instead of holding it all until the very last state has a clean file. The process is far faster. For instance, Georgetown’s NERD$ site—run by Roza—is posting FY23 school-by-school financial files for some states in as few as six to eight months after the close of the fiscal year. With the quick turnaround, district leaders can, for example, marry the finance data with Stanford’s SEDA performance data and use it to inform decisions during the next budget season.
Rick: OK. So, what’s some of this other work that you flagged?
Mark: Similarly, Emily Oster’s work on school closures during the pandemic was released faster than NCES’s and its coverage was more comprehensive. In part because of NCES’s lagging performance, I was able to get additional money from Congress to set up the School Pulse Survey, which has finally given NCES a tool for gathering close to real-time information about conditions in schools. Fair warning, though: NCES calls it “experimental,” which means they are lying in wait to encumber it with more and more statistical tests that will likely delay the release of the data.
Oster and Sean Reardon have also been far ahead of NCES in gathering and releasing detailed data on student performance. Reardon’s Educational Opportunity Project presents detailed and timely information about COVID-related learning loss and recovery. Oster is also working on releasing more detailed and extensive data on student assessment from all states. To this point, they have released data through 2023. In contrast, NCES’s EdFacts promised the 2022 data in December 2023, but it’s not been released yet. Here’s an additional wrinkle: Oster did her work with one salaried employee and a team of undergrads. EdFacts costs around $13 million per year.
Rick: I’ve been struck that, during Secretary Cardona’s tenure, political appointees have gotten far more involved in the release of NAEP results and used the releases to promote administration talking points more than they have in past administrations. How concerned should we be about IES getting caught up in our partisan divides?
Mark: I was both lucky and happy to serve during both the Trump and Biden administrations. Although appointed by Trump, I have found that most of my dealings with senior leaders appointed by Biden have been quite good. But will that political distance and professionalism hold in a second administration of either major candidate? The signs are not good. For instance, IES has an advisory board called the National Board of Education Sciences, or NBES. When the Obama administration was on its way out the door, it tried to fill NBES with their appointees. However, the commissions were never fully executed, and the Trump administration refused to honor the unfinished appointments. Both administrations were within their legal rights to take these actions—but we will still need to see what political ramifications follow.
When I assumed the IES director’s role about halfway through the Trump administration, there were few people on the board, all holdovers from the Obama administration. It was very hard to get the White House to pay attention to the open seats, and I was only able to get three people, all high quality, appointed to the board. At the end of 2020, on his way out the door, Trump appointed a whole bunch of people to the board, many of whose qualifications raised eyebrows. When Biden took office, those commissions had not been finalized, leaving the board consisting of the three people I was able to get appointed. One morning, each of them received an email saying that they had to resign by the end of the day or be fired. The administration then appointed 14 people to the 15-person board. I fear that if the Republicans win the presidency this fall, they will fire the board and replace it with people more to their liking, making what should be a nonpartisan science board highly politicized. I can’t help but wonder if that will lead to the politicization of the director’s tenure.
Rick: Looking outside of IES, it certainly feels to me like the professional education research organizations have become increasingly ideological over the years. Is that a valid criticism? If so, what might help?
Mark: I agree. In 1996, mathematics professor Alan Sokal published the paper “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity” as a hoax to show the shallowness of postmodern critical theory. When he wrote it, I’m sure he was hoping to put an end to that line of work. But, as James Meigs notes in a 2021 Commentary piece: “Sokal meant his essay as a parodic warning. Twenty-five years later, it appears that the Sokal Hoax was actually an instruction manual.” Rick, as you recall, for years, as the AERA annual conference approached, you would go through the program and identify what you considered the most bizarre takes on education. I’m not sure that you could do this today, since you’d have to publish just about the entire program.
Rick: Oh, man, that takes me back. Yeah, I eventually gave that up. I’d hoped that shame could help discourage some of the sillier stuff, but it felt, like you note, that the silly stuff won out. Why is that?
Mark: Education research is highly ideologically driven. If DEI has affected more established and rigorous sciences, which it has, then what can we expect from a far weaker scientific field such as ours? As Meigs noted about Sokal’s paper turning into an instruction manual, I guess your lists could be viewed as a guide to the future work in education.
Rick: Here’s a final question. If you could leave your successor at IES with one piece of advice, what would it be?
Mark: My daughter told me that she learned this parable while in business school.
A new executive is meeting with the executive she is replacing and asks for advice. The outgoing exec says, “I left three envelopes in the desk drawer. Open them sequentially as crises emerge.”
Sure enough, at the inevitable first crisis, the new exec opened the first envelop, which read: “Blame your predecessor.” At the second crisis, the exec opened the second envelop, which read: “Apologize and swear to do better.” Then the third crisis showed up, and the third envelop read: “Write three letters.”
When I pass the baton, I hope that I have gotten IES a little further down the track than people expected. But the race is long, there’s a lot of work left unfinished, and the demand for accountability and innovation won’t go away any time soon.
Oh, and finally, the next director should pray that we don’t have another pandemic. While there are many reasons we should hope never to see a pandemic again, from a narrow perspective, COVID punched a two-year hole in my tenure, limiting what I was able to accomplish.
