Everybody Is Wrong About the Structure of Science (Part 2)

Historical inaccuracy and excessively glib models

Yesterday I wrote a big thing about people being Internet Wrong about how science works:

This was originally supposed to include some additional examples, but when I started writing the first of those expanded until it ate up all of the time I could allocate to writing. So it become a Part 1, punting other cases to Parts 2 through N, (the first of) which you get now.

The specific example I want to talk about today isn’t strictly new, but was new to me, courtesy of

bjkeefe

doing a rabbit-hole dive on Adam Mastroianni’s Experimental History over on Bluesky:

I generally like the blog— the big run of links actually prompted me to subscribe— but in a “I don’t always agree, but it’s at least thought-provoking” kind of way. His series on academic publishing (part one, part two) is very firmly in that category, and his alternative-model research report is a fun read. (I might come back to my disagreements with his posts at a later time…)

But the specific post from which that quote was taken (the link is in the first reply if you click through, but nobody clicks through) is Internet Wrong in a similar mode to what I was writing about yesterday. It’s got the overblown title “Ideas aren’t getting harder to find and anyone who tells you otherwise is a coward and I will fight them”, and is pretty much what it says on the label: a lengthy and vehement rejection of the idea that it’s harder than ever to do productive science. That’s something that I’d very much like to be true, but there are a few specific aspects of the argument that bug me.

The first problem point is hitting a pet peeve of mine, regarding the history of physics. In the course of arguing that the End Of Science has been predicted in the past, Mastroianni quotes a passage (from this 2013 piece in Nautilus) about how 19th century physicists were overly confident about the completeness of their discipline. This includes several reminiscences about the mood of the time, and the famous Albert Michelson “quote” about how “Our future discoveries must be looked for in the sixth place of decimals.”

I put scare quotes around “quote,” though, because it’s not actually an accurate citation of Michelson. The original lectures it’s taken from are available online, and the “sixth place of decimals” line appears on page 24, where Michelson puts it in quotes because he’s attributing it to unspecified others. More importantly, though, the context of the work makes clear that he’s not saying that the represents the end of physics, he’s making a pitch for precision measurement as the future of physics. The sentences immediately preceding it offer a list of discoveries that were made because of tiny anomalies— maybe not six decimal places out, but not factor-of-two errors by any means. Michelson was in the business of making ultra-precise measurements, and was selling this as the place where new developments would be found.

And Michelson was absolutely correct in the direction of the sentiment. The revolutionary new theories that re-shaped physics in the first half of the 20th century generally involve really small effects. The triumphs of General Relativity were a calculation of a tiny perturbation in the orbit of Mercury, and the measurement of a deflection of starlight that’s so small that even a hundred years on you can still find people arguing that Eddington fudged the data. One of the most dramatic successes of QED was the prediction of the Lamb shift, an effect that shows up in the seventh decimal place of the energy of the states involved.

Even the quantum effects that represent dramatic qualitative departures from classical predictions are generally very subtle. Einstein’s Nobel Prize citation mentions the photoelectric effect, which took a huge amount of fiddly experimental work by Millikan to nail down. The discovery of electron spin was driven by things like the anomalous Zeeman effect, which you need precision spectroscopy to see, and the Stern-Gerlach experiment, which was an impressive experimental feat that they could only barely pull off. The only quantum phenomenon that can be considered a glaringly obvious problem is the existence of spectral lines, and even that is really sorted out by precision measurement.

So, Michelson wasn’t misguided and arrogant, he was prescient. There were revolutionary discoveries coming, and they were, in fact, made through ultra-precise measurements.

(I’ve written about this before, both on the blog and in books. The other quotes cited in the passage Mastroianni quotes don’t demonstrate the claim, either, because they’re all retrospective. They’re from people who lived through the revolutionary developments of relativity and quantum mechanics looking back on the previous era, in the context of discussing how much has changed. There’s a natural and obvious bias there to inflate the degree of the change through recollection of what in modern parlance we’d call vibes. You get something similar with Einstein’s late-in-life insistence that all his discoveries were purely theoretical (when he was pretty clearly aware of experimental work at the time he was making his big breakthroughs), or the Newton-under-the-apple-tree story (which doesn’t show up until decades later when it provides some color in his priority disputes about the invention of calculus). It’s actually very difficult to find any firm citations of working physicists at the end of the 19th century expressing the attitudes that are generally imputed to them; the handful I’ve been able to check are mostly, like Michelson’s, being misinterpreted.)

This is kind of a lot about one minor example in one early point, I know— but then, I did say at the start that this is a personal peeve. I think it’s relevant to the larger thrust of Mastroianni’s piece, though, in that it illustrates an important point about physics, specifically, namely that it really has gotten harder to make discoveries. All the anomalies in the sixth decimal place were measured a century ago— modern physicists are looking for changes in the 14th decimal place. You can still come up with new ideas about physics, sure, but they’re incredibly tightly constrained by prior experiments, and testing those theories is vastly more complicated an expensive than it was in the golden era— you used to be able to discover new particles by leaving some photographic plates out on top of a mountain, but now you need millions to billions of dollars of sophisticated electronic detectors.

The basic error here is very much of a piece with what I wrote in part one of this: excessive generalization. Mastroianni is a psychologist, and writes as if all of science is operating in the same mode as psychology— where experiments are relatively cheap and easy, so the limiting factor is the quality of the ideas. (And, you know, the willingness of some people in the field to resort to outright fraud, but I digress…) That’s not remotely true across all of science— some subdisciplines are very much resource-limited on the experimental side. We haven’t had a great breakthrough in fundamental particle physics in a long time not because there aren’t enough people thinking about it, but because it’s so damn difficult to do the experiments.

I’ve also heard arguments that some life-science fields are more theory-limited— that there isn’t material progress being made because concepts like “consciousness” are still too nebulous, and without a good theoretical framework to guide them, experiments are just fumbling around blindly. As a physicist, I can’t really evaluate that— it feels suspiciously congenial to me, but then again I’ve seen versions of this shared by people who are serious scientists within those fields.

My main point is that I think it’s a mistake to think that there’s a single explanation for a perceived lack of progress across science, because the various disciplines are too different. There’s even a ton of diversity within disciplines— fundamental particle physics is in a thirty-year rut, as people in the field will bang on about endlessly, but there’s been enormous progress in other areas. Particularly in the experimental realm, we routinely do things now that were dismissed as likely impossible when I was an undergrad. Looking for an explanation for “low productivity” that attempts to explain everything from “let’s pay randos on the Internet a dollar apiece to fill out a questionnaire” to the Large Hadron Collider is a fool’s errand.

Generated from “lots of people doing incremental science”

Which brings us around to the other thing that kind of bugs me about Mastroianni’s piece, and the broader genre of which it is a part, which is the idea that it’s all about academic incentive structures. The general idea is that back in the golden era, scientists were more likely to be exercise their creativity in exploring widely, but these days they’re all a bunch of careerist plodders churning out incremental advances that produce citable publications but don’t amount to much. This illustrated by a cartoon that he reproduces from another argument, showing a bunch of stick figures shifting places:

I’m not sure the axis labels here make any actual sense, but that’s economists for you

I hear this a lot, and it always strikes me as a little too glib and self-flattering to be trusted. For one thing, it’s a little difficult to square with the narrative about past eras that I was previously objecting to— if physicists at the end of the 19th century really thought the had everything nailed down, then more or less by definition they had to be mostly working on incremental advances. It’s not especially coherent to push both of those narratives at the same time.

More than that, though, I think there’s a ton of survivorship bias involved in this (you can mentally insert the plane-with-red-dots image here if you like). I’ve done a bit of chasing down old journals in the course of various projects, and man, there’s a lot of deservedly forgotten boring incremental stuff padding those out. It looks like there was a lot more really exciting science going on Back In The Day for the same reason it looks like pop music of the past was better than the noise the kids are listening to nowadays— we’ve forgotten the bulk of the mediocre stuff, to say nothing of the actually bad stuff.

I think this also fails to account for the fact that institutional science is a recent invention, and has changed rather substantially over the time span of interest. Standards for things like “What’s worth publishing in a journal?” and “Who gets listed as an author on this paper?” have changed dramatically— there are tons of single-author papers from the early 20th century that contain asides or footnotes describing the contributions of people who would absolutely be credited as co-authors today. A lot of the people doing “incremental” science were always there and doing that work, they just weren’t getting credit for it (for reasons of class, gender, race, etc.) in ways that are readily countable from a century removed. I also suspect there’s a lot of work that now falls within academic physics that would’ve been considered too grubbby and technical in an era when science was more of a hobby pursued by the well-to-do. Old experimental papers contain a lot of asides that skip lightly past mixing up some material or building some bit of apparatus that would absolutely be a separate publication today— again, that’s work that was always happening, but has (rightly) been made more visible over the last hundred-odd years.

Put all that together with the way the conclusions of the argument are just a little too congenial for modern scientists— the key to unlocking progress is to relieve us of all the duties we find tedious and paid just for the fun parts of the job— and I just don’t buy it. That’s not to say that there aren’t massive problems with citation-counting as a metric, or careerism within academia, but I just don’t think it works as a Grand Unified Theory of why science as a whole isn’t living up to some imaginary standard of progress.

---

So there’s a second installment of this particular gripe. Will there be a Part 3? Click this button to find out:

And if you would like to make your own Take known in a way that can readily be credited to you, the comments will be open: