When I started writing about YouCubed in 2017, I said they were “sloppy” aboout research. Partly this was because they have a habit of talking about cognitive science and neuroscience in unscientific ways, e.g. talking about “brain sparks” or “blocked working memory.” It’s like if a doctor told you that Asprin would make your neurons vibrate, you might consider that a sign of things to come.
But the bigger sloppiness was their interpretation of a study by Jason S. Moser. The study found that people improved their performance on some task after they made mistakes, the reason being that they noticed the mistakes and improved. In the hands of YouCubed, this Moser study was cited as saying “the brain sparks and grows when we make a mistake, even if we are not aware of it,” i.e. precisely the opposite of what it says.
I don’t want to write the same thing over and over again, because that’s boring, but I do want to add that it’s not just about their interpretation of existing research. The research YouCubed itself releases is often extremely misleading, or so shoddy that it’s hard to even talk about it as research.
There is a particularly clear example in a recent release, “Raising Expectations and Achievement: The Impact of Two Wide Scale De-Tracking Mathematics Reforms.” The authors, Jo Boaler and David Foster, want to show that eliminating tracking leads to better outcomes for kids.
The state of California used to allow 8th Graders to take a variety of courses: Pre-Algebra, 8th Grade Math, Algebra 1, Geometry, or Algebra 2. Students would take a test at the end of the year in whatever course they were in. Which is rather chaotic, sure, but this is California we’re talking about, my understanding is if 100 people sign a piece of paper they have to pick a name out of a hat and then that person has to be governor. California is a bit chaotic.
Anyway, California wanted to do something about this, so they made everybody take a class called “8th Grade Math,” and then they all took the same test.
Well, first of all, are you surprised that scores were better on average after they made everyone take the same test? Would you have predicted it?
I’m not surprised, though I wouldn’t have predicted it. That’s because the tests are totally different. I don’t have any clue what the Geometry test in California is like! Why would I have an opinion about whether students would do better on that or this 8th Grade test? And even if they did, what would that even show? Maybe the 8th Grade test is easier and kids who used to take Algebra 2 aced it.
Anyway, as it turns out, scores went up after this “de-tracking” and Boaler and Foster call this “Study 2” in support of de-tracking.
There are other publications that are less funny, but no less flawed. In particular, there is a MOOC paper where 100s of people dropped out of YouCubed’s intervention, and the study just ignores it. But when hundreds of people are dropping out of your intervention…if hundreds of people stop taking a drug in the middle of the trial, you need to ask some questions, questions like “why did they stop taking it?” and “did their hair grow back after they stopped?” and so on.
So, what can I say, YouCubed and Boaler are not producing legitimate research.
Which, as I often point out, wouldn’t be so concerning on its own. This is the world we live in. People misuse research, literally all the time.
But YouCubed’s entire shtick is that they are research-based. They have risen to influence by beating the drum of research with particular enthusiasm. Their mission mentions research not once, but three times: “Our main goal is to inspire, educate and empower teachers of mathematics, transforming the latest research on maths into accessible and practical forms. We know from research how to teach mathematics well and how to bring about high levels of student engagement and achievement but research has not previously been made accessible to teachers.” Research, research, research.
All this while Jo Boaler has become the closest math education in the US has to a celebrity. (You’ve seen #JoOnAStick, perhaps?) And of course, part of what makes her an important voice in education is that she is a researcher, who does research, and can speak for research.
So, what is going on?
I don’t know. It’s clearly all connected to the first big controversy in Boaler’s career surrounding the Railside study, but I don’t really understand the full trajectory. Clearly, Boaler now sees the world of math education in terms of conflict, frequently highlighting the fact that her agenda has opponents, signing every YouCubed email with “Viva La Revolution [sic].” Was this tendency towards conflict a result of the wars of Railside, or one of their causes?
Either way, it’s been four years since I started writing about YouCubed and I’m feeling ready to go further than “sloppy.” For whatever reason, YouCubed as an organization frequently produces or cites research in ways that don’t show what they claim to show. As a result, I wouldn’t trust the organization (or Boaler) to make any research-based claim.
And the issues will continue to grow until the mathematics education community decides that it isn’t OK to mislead people about research.