The Problem of Ill-Defined Ideals

How much do we need the median high-schooler to learn?

As noted in a previous post, I have a very long history of involvement in education, which means I’m generally interested in the subject. This gets picked up by our algorithmic overlords, and leads to me getting a whole bunch of material on the general subject pushed my way. Like this thread from ex-Twitter (screencap because our overlords are also petty), which links to this one as the source of the graph.

The graph shows scores on the NWEA math test (administered to a very broad swathe of American students) as a function of grade level, with the different lines representing various percentile levels. They also do an NWEA test of reading, which looks basically the same: the higher-scoring students increase their score year over year all the way through high school, while lower-scoring students plateau somewhere in middle school. You can also do this as a set of colorful bell-curve distributions (from the second thread linked above) of score distributions for different grade levels:

Again, there are graphs of reading scores that look essentially the same: narrow distributions at low grades that shift upward and broaden over time, with the mean not moving all that much over the last few grades.

These are eye-catching graphs, used to justify a sweeping claim: that since scores aren’t monotonically increasing, those last several years of school are just a waste of the students’ time and society’s resources. The students in the percentile groups that aren’t increasing ought to be shifted out of formal schooling and into… something else.

A someone who spent his public-school years on the track of the top line in the first graph, I am not completely without sympathy to at least the part of this that says that those upper-decile students could benefit from more attention than they’re getting. And as a child from a family of teachers, I absolutely agree that it would dramatically improve the quality of life of both other students and teachers if students who consistently disrupt classes were sent elsewhere, and those students do tend to be concentrated in the lower lines of that graph.

At the same time, there’s an off-putting vibe to a lot of these threads, that seems to regard a double-digit percentage of the population as not worthy of formal education. There’s a vagueness to the answers to “What would they be doing instead?” that I’m not wild about— I’d like to see that fleshed out a little more, because otherwise it seems a bit like you’re turfing 14-year-olds out to do manual labor. I might also quibble a bit with some of the presentation— the most ostentatiously futile of the lines in the top graph represents the first percentile of student scores on the NWEA, which is both a really tiny slice of the population and a subset that almost certainly includes a good number of students who are going through some shit, to put it mildly. Take that one out as skewed by extenuating circumstances, and the rest don’t look nearly as bad.

In the end, though, I think my problem with this is that, as with many public-policy debates involving population-level distributions of things, I’m not sure what the ideal result is supposed to be. I mean, if we’re talking about the prevalence of some unpleasant condition— guinea worm, say— I know what the goal is: we want all the lines to go exactly to zero. When the topic is something like the demographic distribution of a particular profession, the answer is less clear— “[high percentage] of college faculty have parents who are also college faculty” is a perennial favorite on social media, but I can never decide what to make of that. It makes a lot of sense that children raised by academics would be more likely to pursue academic careers just from exposure to the environment. So what, exactly, should we want that percentage to be?

And that’s kind of where I am with this. Those figures look pretty much like a classic plane-with-red-dots.jpg data set: students who score poorly on one test tend to continue to score poorly on future tests. The bell curves are also more or less what I would expect as you move up the educational ladder and course work starts to become both more challenging and more specialized. The uneven distribution of interests in particular subjects starts to come into play to a greater degree, pushing the top end up and leaving the low end where it is.

The specialization point is also an important one, because at the end of the day, the NWEA tests are looking at fairly basic skills in a few particular areas. The fact that a given student’s math score hits a plateau doesn’t necessarily mean that they’re not learning anything from being in school. There are other subjects in which they might be gaining knowledge that will be useful to them later on, depending on their career— you can learn a whole lot about history and politics without getting dramatically better at reading, for example. (See, for example, any number of successful professional politicians….) To say nothing of the non-academic stuff that comes with schooling— making friends, navigating various social situations, pursuing athletic or artistic interests, etc.

So at the end of the day, I’m just not sure what to make of this. Would it be nice if the median high-school graduate knew enough math to get a high score on the NWEA? As a person who regularly makes math references that go over people’s heads, I think it would be lovely. Is it essential, though? Enh. Honestly, as a practical matter, most people will probably be just fine without getting much beyond percentages, and that’s fine.

(Admittedly, far too many people suck at understanding percentages, so we could definitely stand to shorten the left-hand tail of those score distributions. I’m not convinced the peak needs to move all that far to the right, though.)

And this is what’s ultimately so frustrating about most arguments, especially online ones, around educational policy. It’s never clear what end distribution we’re supposed to be shooting for, and as a result, everything just defaults to ideological baselines, which I find pretty creepy at both extremes: on the right, a desire to just slough off a double-digit percentage at the low end of score, and on the left an exceedingly narrow focus on only the low end, with a complete disregard for whether that holds back the better students.

Everybody has a simple and elegant solution for whichever slice of the population they care most about (which, in keeping with the nature of simple and elegant solutions, is probably wrong), but I wish there were more clarity about what the goal is for the distribution as a whole. I’m not sure “Test score line goes up, always and forever” is realistic, or even what we ought to be after in the first place.

---

So, that’s some academic wibbling, in a manner that no doubt feels familiar to folks who’ve been reading for a while. If you’d like to experience that feeling in the future, here’s a button:

And if you have a simple and elegant answer to the question of what the overall goal should be, the comments will be open: