I asked them if it was really true they all use the What Works Clearinghouse.
“Yeah,” Arnie said. “Though, for a research geek like me”—the other two energetically nodded—“the WWC offers a lot less evidence about ‘what works’ than I would’ve expected.”
“What do you mean?” I asked.
“Well, they’ve published a dozen ‘practice guides’ for educators in the past decade, but I don’t know how helpful they really are. During the pandemic, I went to their 2019 guide on using technology. Turns out it was for postsecondary—they didn’t have one for K–12. I checked it out anyway. It had five recommendations, none with ‘strong’ evidence. The first was to ‘use communication and collaboration tools to increase interaction’ among students and teachers. Even that was just based on the researchers’ ‘rationale’—not on evidence. If I want to hear what professors think about ed tech, I can go to Twitter.”
“Hmmm,” I said.
“Now, I’m a middle school math teacher,” Arnie continued, “so I wanted to see what they advised in their ‘improving algebra knowledge’ guide. The first two sentences? ‘Informal language often relies on superficial features such as the position of symbols on the page, rather than on the underlying mathematical operations. Informal language can introduce misconceptions and confusion during standardized assessments where precise language is used, adding unnecessary complexity by giving students another idea to understand.’
“Talk about unnecessary complexity! It sounds like a memo co-written by a statistics professor and a McKinsey consultant. I’m a teacher, man. I don’t have time for that.”
“But did you find a research-backed strategy?” I asked.
“Not really. The guide said there wasn’t really any evidence for the recommendations,” Arnie said. “For instance, when it came to using ‘language that reflects mathematical structure,’ the guide said that no research met the WWC’s standards but that the panel ‘believes’ it’s a good idea. Overall, it said two of the three recommendations were based on ‘minimal evidence,’ which means ‘the panel cannot point to a body of evidence that demonstrates the practice’s positive effect on student achievement.’”
“What did you do then?” I asked.
“I went to their most recent practice guide for math, on intervention in the early grades. I figured that might have some practical tips. But the ‘evidence-based’ advice was stuff like ‘provide systematic instruction,’ teach ‘clear and concise mathematical language,’ ‘use number lines,’ and ‘assign word problems.’ I mean, this is stuff most student teachers could tell you. If a teacher needs ‘evidence’ to know they should use clear language, we’ve got bigger problems.”
I asked Gertrude if she was more upbeat about the WWC. “Oh yeah,” she laughed. “I mean, everyone is more upbeat than Arnie. I think it’s got some good stuff.”
“Can you give me an example?” I asked.
“I teach 3rd grade, and I think there’s a lot there for me,” she said. “Take the brand-new guide on behavioral interventions in K–5. The very first recommendation—backed by ‘Tier 1’ evidence, mind you—is to ‘co-establish, model, and teach clear expectations for student behavior.” There’s evidence-based advice urging teachers to ‘model expectations’ and ‘provide feedback.’ Arnie can grump if he wants, but I can’t imagine how we’d know this without the WWC!”
“What did you think of the evidence?” I asked.
“So helpful! It reassured me that something like ‘providing feedback’ wasn’t just some kooky idea,” Gertrude said.
