All the numbers from 1 to 500, ranked from worst to best by a middleschooler

500. 328

499. 149

498. 373

497. 55

496. 137

495. 18

494. 59

493. 452

492. 40

491. 470

490. 107

489. 379

488. 37

487. 165

486. 97

485. 377

484. 236

483. 252

482. 275

481. 479

480. 366

479. 297

478. 122

477. 465

476. 49

475. 154

474. 187

473. 191

472. 139

471. 235

470. 365

469. 315

468. 392

467. 414

466. 299

465. 117

464. 415

463. 425

462. 475

461. 48

460. 440

459. 173

458. 193

457. 142

456. 199

455. 229

454. 249

453. 329

452. 412

451. 218

450. 492

449. 43

448. 408

447. 311

446. 248

445. 150

444. 96

443. 306

442. 459

441. 146

440. 141

439. 123

438. 376

437. 319

436. 182

435.  60

434. 206

433. 177

432. 240

431.  28

430. 476

429. 389

428. 372

427. 214

426. 382

425. 225

424. 125

423. 444

422. 51

421. 127

420. 447

419. 8

418. 53

417. 342

416. 473

415. 274

414. 65

413. 321

412. 56

411. 245

410. 287

409. 257

408. 496

407. 375

406. 347

405. 417

404. 190

403. 468

402. 24

401. 438

400.  5

399. 266

398. 353

397. 16

396. 327

395. 4

394. 323

393. 22

392. 352

391. 91

390. 396

389. 212

388. 131

387. 215

386. 317

385. 129

384.  20

383. 442

382. 436

381. 99

380. 367

379. 82

378. 75

377. 103

376. 144

375. 100

374. 197

373. 270

372. 209

371. 201

370. 114

369. 37

368. 394

367. 58

366. 395

365. 437

364. 159

363. 259

362. 54

361. 243

360. 264

359. 439

358. 261

357. 185

356. 232

355. 289

354. 482

353. 135

352. 330

351. 304

350. 455

349. 296

348. 81

347. 38

346. 357

345. 364

344. 241

343. 90

342. 308

341. 341

340. 361

339. 242

338. 356

337.  3

336. 94

335. 344

334. 36

333. 47

332. 461

331. 128

330. 124

329.  88

328.  1

327. 318

326. 359

325. 272

324. 489

323. 178

322. 31

321. 222

320. 217

319. 325

318. 499

317. 490

316. 14

315. 413

314. 349

313. 401

312. 410

311. 407

310. 466

309. 83

308. 474

307. 405

306. 160

305. 106

304. 157

303. 210

302. 431

301. 138

300. 383

299. 307

298. 443

297. 427

296. 11

295. 195

294. 380

293. 45

292. 340

291. 110

290. 397 & 332 (tied)

288. 402

287. 263

286. 220

285. 497

284. 186

283. 370

282. 181

281. 80

280. 203

279. 21

278. 108

277. 424

276. 345

275. 445

274. 430

273. 227

272. 116

271. 120

270. 233

269. 246

268. 133

267. 313

266. 111

265. 386

264. 115

263. 136

262. 416

261. 400

260. 487

259. 72

258. 130

257. 262

256. 155

255. 278

254. 258

253.  9

252. 388

251. 277

250. 104

249. 354

248. 140

247. 87

246. 19

245. 238

244. 265

243. 471

242. 419

241. 143

240. 89

239. 355

238. 310

237. 491

236. 467

235. 500

234. 483

233. 93

232. 303

231. 368

230. 171

229. 305

228. 333

227. 198

226. 221

225. 109

224. 10

223. 448

222. 219

221. 456

220. 433

219. 42

218. 267

217. 384

216. 387

215. 404

214. 284

213. 205

212. 71

211. 337

210. 46

209. 273

208. 312

207. 166

206. 35

205. 32

204. 67

203. 480

202. 70

201. 226

200. 163

199. 92

198. 294

197. 26

196. 41

195. 271

194. 231

193. 230

192. 486

191. 44

190. 134

189. 86

188. 324

187. 216

186. 423

185. 79

184. 282

183. 50

182. 288

181. 29

180. 348

179. 293

178. 30

177. 322

176. 291

175. 78

174. 290

173. 298

172. 68

171. 172

170. 334

169. 161

168. 283

167. 279

166. 255

165. 485

164. 228

163. 162

162. 343

161. 204

160. 350

159. 84

158. 421

157. 478

156. 477

155. 25

154. 145

153. 192

152. 446

151. 435

150. 180

149. 385

148. 339

147. 168

146. 113

145. 360

144. 390

143. 23

142. 295

141. 434

140. 256

139. 351

138. 454

137. 39

136. 184

135. 371

134. 411

133. 276

132. 381

131. 251

130. 253

129. 281

128. 406

127. 73

126. 247

125. 76

124. 280

123. 292

122. 62

121. 374

120. 27

119. 460

118. 362

117. 472

116. 77

115. 112

114. 176

113. 331

112.  2

111. 495

110. 309

109. 52

108. 418

107. 320

106. 183

105. 151

104. 57

103. 61

102. 391

101. 174

100.  7

99. 268

98. 148

97. 175

96. 152

95. 300

94. 101

93. 484

92. 462

91. 285

90. 481

89. 213

88. 102

87. 153

86. 239

85. 95

84. 224

83. 393

82. 338

81. 17

80. 358

79. 286

78. 223

77. 346

76. 326

75. 316

74. 105

73. 119

72. 33

71. 74

70. 302

69. 156

68. 34

67. 167

66. 118

65. 399

64. 498

63. 207

62. 336

61. 188

60. 66

59. 378

58. 208

57. 254

56. 12

55. 301

54. 398

53. 98

52. 488

51. 429

50. 132

49. 426

48. 85

47. 422

46. 260

45. 147

44. 244

43. 179

42. 189

41. 450

40. 158

39. 314

38. 409

37. 196

36. 202

35. 363

34. 457

33. 63

32. 64

31. 403

30. 463

29. 458

28. 451

27. 200

26. 211

25. 453

24. 164

23. 250

22. 194

21. 464

20. 13

19. 126

18. 449

17. 234

16. 335

15. 432

14. 121

13. 494

12. 493

11. 170

10. 441

9.   6

8. 428

7. 15

6. 469

5. 369

4. 269

3. 169

2. 420

1. 69

YouCubed is More Than Just Sloppy About Research

When I started writing about YouCubed in 2017, I said they were “sloppy” aboout research. Partly this was because they have a habit of talking about cognitive science and neuroscience in unscientific ways, e.g. talking about “brain sparks” or “blocked working memory.” It’s like if a doctor told you that Asprin would make your neurons vibrate, you might consider that a warning sign.

But the bigger sloppiness was their interpretation of a study by Jason S. Moser. The study found that people improved their performance on some task after they made mistakes, the reason being that they noticed the mistakes and improved. In the hands of YouCubed, this Moser study was cited as saying “the brain sparks and grows when we make a mistake, even if we are not aware of it,” i.e. precisely the opposite of what it says.

(Robert Kaplinsky also looked into this, by the way, and never heard back from Moser.)

I don’t want to write the same thing over and over again, because that’s boring, but I do want to add that it’s not just about their interpretation of existing research. The research YouCubed itself releases is often extremely misleading, or so shoddy that it’s hard to even talk about it as research.

There is a particularly clear example in a recent release, “Raising Expectations and Achievement: The Impact of Two Wide Scale De-Tracking Mathematics Reforms.” The authors, Jo Boaler and David Foster, want to show that eliminating tracking leads to better outcomes for kids.

The state of California used to allow 8th Graders to take a variety of courses: Pre-Algebra, 8th Grade Math, Algebra 1, Geometry, or Algebra 2. Students would take a test at the end of the year in whatever course they were in. Which is rather chaotic, sure, but this is California we’re talking about, my understanding is if 100 people sign a piece of paper they have to pick a name out of a hat and then that person has to be governor. California is a bit chaotic.

Anyway, California wanted to do something about this, so they made everybody take a class called “8th Grade Math,” and then they all took the same test.

Well, first of all, are you surprised that scores were better on average after they made everyone take the same test? Would you have predicted it?

I’m not surprised, though I wouldn’t have predicted it. That’s because the tests are totally different. I don’t have any clue what the Geometry test in California is like! Why would I have an opinion about whether students would do better on that or this 8th Grade test? And even if they did, what would that even show? Maybe the 8th Grade test is easier and kids who used to take Algebra 2 aced it.

Anyway, as it turns out, scores went up after this “de-tracking” and Boaler and Foster call this “Study 2” in support of de-tracking.

Image
I couldn’t fit this into the post but remember the time they just sorta took math activities from around the web without attribution and called it a curriculum?
Image
By the way, YouCubed has an entire Data Science initiative.

There are other publications that are less funny, but no less flawed. In particular, there is a MOOC paper where 100s of people dropped out of YouCubed’s intervention, and the study just ignores it. But when hundreds of people are dropping out of your intervention…if hundreds of people stop taking a drug in the middle of the trial, you need to ask some questions, questions like “why did they stop taking it?” and “did their hair grow back after they stopped?” and so on.

So, what can I say, YouCubed and Boaler are not producing legitimate research.

Which, as I often point out, wouldn’t be so concerning on its own. This is the world we live in. People misuse research, literally all the time.

But YouCubed’s entire shtick is that they are research-based. They have risen to influence by beating the drum of research with particular enthusiasm. Their mission mentions research not once, but three times: “Our main goal is to inspire, educate and empower teachers of mathematics, transforming the latest research on maths into accessible and practical forms. We know from research how to teach mathematics well and how to bring about high levels of student engagement and achievement but research has not previously been made accessible to teachers.” Research, research, research.

All this while Jo Boaler has become the closest math education in the US has to a celebrity. (You’ve seen #JoOnAStick, perhaps?) And of course, part of what makes her an important voice in education is that she is a researcher, who does research, and can speak for research.

So, what is going on?

I don’t know. It’s clearly all connected to the first big controversy in Boaler’s career surrounding the Railside study, but I don’t really understand the full trajectory. Clearly, Boaler now sees the world of math education in terms of conflict, frequently highlighting the fact that her agenda has opponents, signing every YouCubed email with “Viva La Revolution [sic].” Was this tendency towards conflict a result of the wars of Railside, or one of their causes?

Either way, it’s been four years since I started writing about YouCubed and I’m feeling ready to go further than “sloppy.” For whatever reason, YouCubed as an organization frequently produces or cites research in ways that don’t show what they claim to show. As a result, I wouldn’t trust the organization (or Boaler) to make any research-based claim.

And the issues will continue to grow until the mathematics education community decides that it isn’t OK to mislead people about research.