Rankings and Randomness

The not-so-secret obsessions of academia

Photo by Zhanhui Li on Unsplash

The other academic topic I’ve had in mind to write about, but wasn’t annoyed enough to displace other work for (as I did for the previous post), was that perennial favorite topic, the US News rankings of colleges and universities, which came out a week or so ago. In keeping with long-established practices of American academia, I will now do the ritual celebration of how they ranked places important to me (Williams continues its long-running hold on the top spot in the liberal arts colleges category, as is right and proper, and Union remains in the Top 50), before launching into a longer and more ambivalent discussion of the rankings and their uses.

As usual, the standard for this sort of thing is set by Timothy Burke (a professor at #4-ranked Swarthmore), who has a long and thoughtful post about what “should” matter for the process of rank-ordering colleges when it comes time to choose between them:

Academia: Rankings and Nonsense

This is generally a really good piece, and I particularly like his sorting of types of information into categories by degree of difficulty.

At the same time, though, I put scare quotes around “should” above for a reason: this is very much a list of data and criteria that a middle-aged academic would like prospective students to prioritize, rather than a list of things that actual existing 17-year-olds necessarily care about. (The dead giveaway is the inclusion of several (sub)items relating to faculty governance, which almost nobody who isn’t a professor or a dean gives a damn about.) This is fairly typical of almost all faculty discussions of college rankings and admissions, and student decision-making more generally: when you dig down, a lot of it ends up in the neighborhood of “Why don’t the students want the things we think they ought to want?” And that always gives me pause.

(This isn’t necessarily an illegitimate question, but it’s a very different question than what many faculty seem to think that they’re asking, and not one that’s answered by simply providing the students with more or different information. To the extent that it’s a problem to be solved, the solution has to start with a thorough and open investigation of what students actually want and why, at which point you can start to talk about how to encourage them to want something better.)

I used to more regularly talk with students about where else they were looking before they came to Union (though I’ve kind of fallen out of the habit), and there were always some head-scratchers. One that particularly sticks in my head, for some reason, was the student who had only applied two places: Union, and SUNY-Stony Brook. Those are institutions with next to nothing in common, but somehow, that pairing made sense to him.

Reflecting back on my own history with the process, my final choice came down to Williams vs. Swarthmore (I was also accepted by Cornell and MIT, but having grown up in a small town I found them too big and impersonal) which are at least similar types of institution. The ultimate decision was partly down to money— I got an incredibly generous scholarship from Williams that meant I didn’t need to take out any loans— but also a matter of vibes, for lack of a better word. I remember taking a bus from Boston to Williamstown between an overnight stay at MIT and one at Williams, and seeing the view from the world-famous Hairpin Turn, and thinking “Oh, yeah, this is awesome.” And I just found the atomsphere more congenial in my visit to Williams, where I spent the visit hanging out with my student host and his suitemates, bullshitting about movies. (Which is a little ironic, since what I did on my visit to Swarthmore— getting drunk at a frat party on a Wednesday night— was much closer to how I spent my time when I was actually a student at Williams.)

That vibe-based aspect of the decision introduces an element of randomness makes me think that a lot of what we talk about when we talk about ranking colleges really just doesn’t matter very much. The US News rankings and their various competitors are useful mostly for sorting a giant list of schools that can be overwhelming in its size into some rough categories— the exact positioning within the top 100-ish schools behind Williams in the liberal arts college rankings is less important than the fact of being in that general company. If I skim down the list, I have a vaguely positive impression of the first hundred-odd schools (with a few exceptions), but somewhere in the low triple digits we get to places I’ve never heard of, and I’ve spent thirty-odd years immersed in academia. There’s a somewhat similar dynamic in the National Universities list; the top of the list are mostly “Oh, yeah, that place” and somewhere in the 100-ish range there starts to be a lot of “Really? That place?”

That’s not to say there aren’t good schools on the low end of those rankings, but that’s around where I’d start questioning whether the $50K sticker price could be justified, absent some really compelling personal reason (a unique program or setting, a family connection, etc.). At some point, students would be better served by spending a year or two at a community college for cheap, generating a record that will enable them to transfer into a stronger school to finish up.

But, again, that’s the opinion of a middle-aged professor, looking back from a very different stage of life, with different priorities than the students who actually have to make these decisions.

In the end, I’m relatively unconcerned about students’ use of rankings and the somewhat random vibe-based decision-making they bring to the final choice of school, because at the end of the day what really matters is the students themselves. As I’ve written here and elsewhere, and said at the prospective-student events that Admissions keeps inviting me to, at the most fundamental level education is not something that faculty do to the students, it’s something that we give students the resources to do for themselves. The most important factor in terms of educational outcomes is not the size of the endowment or the publication record of the faculty or the amenities in the residence halls, it’s the level of interest and engagement and effort put in by the student.

If you’re engaged in the process, and care about the results, you can get an excellent education at almost any college or university, by maximizing the resources they make available to you (including resources that aren’t strictly academic— fellow students, alumni, student life programs, etc.). And, on the flip side, if you’re aimless and disengaged, you can make yourself miserable almost anywhere. A particular institution can make those final outcomes a little easier or a little harder to achieve, but at the end of the day, it’s much more down to the student than anything you could hope to capture at an institutional level.

---

It’s a little jarring to my self-image as a Science Guy that whenever I write about this stuff I end up in this wibbling “Enh, numbers don’t matter” place. Not sure what that says. Anyway, if you would like more of this in your inbox, here’s a button:

And if you want to make a case for greater quantitative rigor, the comments will be open: