What Makes an Ugly Experiment?
The flip side of science history
Over on Twitter, Philip Ball asked a question about beautiful experiments from the history of science, which prompted me to dust off my Forbes blog for a post on that topic. I offer a few examples from the history of science, and a criterion that, for me, really sets some of them apart from others.
Ball also got a follow-up tweet that asked the opposite question:
This didn’t draw nearly the same level of response, for reasons that I think are kind of interesting in their own right.
One obvious explanation for the lack of response is that the ugliest sort of experiment is one that simply doesn’t work, and most of those are justly forgotten. The history of science mostly weaves around the thousands upon thousands of experiments that were intended to show something or another, but never panned out. Most of these are just not interesting at all, and so hardly anybody today knows that they even happened.
There’s an important subset of failed experiments, though, that shouldn’t be forgotten but are because of the way that publishing works. These are the ones where you spend six months trying and failing to do something that seems like it should work, only to bump into a colleague at a conference and have them say “Oh, yeah, we spent a year trying to do that. It doesn’t work because [good but subtle reason].” There’s really no place to publish that kind of thing, though we have occasionally joked about needing a Journal of I Swear I Thought That Would Work that you could check your ideas against, like the “Bad Circuit Ideas” section of the Horowitz and Hill textbook. It would save a lot of time and effort.
A slightly different category of experiment that you might count as “ugly” is the high-profile blunder. These are debacles like the OPERA experiment that seemed to show neutrinos moving faster than light, or the BICEP2 experiment that thought they had found a signal in the polarization of the cosmic microwave background radiation. Both of these made a big splash in the media, only to unravel over the following months when they turned out to be due to errors— a bad fiber-optic connection in the OPERA case, and an incorrect value for the contribution of dust to the BICEP2 signal.
These are ugly in the sense of being bad for careers and reputations— though Brian Keating has rebounded nicely— if not in the conception and execution. I would argue that these high-profile failures are still useful, though, in that they show the self-correcting nature of good science, and provide an illustrative contrast with pseudoscience. I’ve argued this at length in a chapter for an academic book, part of which is available as a free excerpt. (I should probably revisit putting the whole thing on the arXiv…)
The final category of “ugly” experiments double as the other major reason for the lack of responses to the tweet asking for examples: they’re the ones that are either unethical or fraudulent. One of the few replies to that tweet cites the infamous Stanford Prison Experiment, staple of intro psych courses, which has been the subject of numerous re-examinations and takedowns, both informal and academic. This experiment was kind of dodgy even by the loose standards of its day and now would be considered wildly unethical.
And then there are the outright frauds, like the case of Jan Hendrik Schön, who consciously faked the data leading to a whole slew of papers that were touted as potentially Nobel Prize material. In addition to the news story linked from his name, there’s a book by Eugenie Reich that’s pretty well-regarded.
This brand of ugliness is the sort of thing that scientists prefer not to talk about, for fear that it reflects poorly on the profession as a whole. We’d rather not have people in our fields who behave in this manner, or even have to cop to missing the frauds who walk among us. (Some of these are shockingly blatant— I had dinner once with someone who worked at Bell Labs around the time Schön was doing his thing, who told a story about a higher-up complaining about their budget saying “Why can’t you be more like Hendrik? He produces all these results, and doesn’t even buy liquid helium!”) Beautiful experiments are fun to talk about, ugly ones make us uncomfortable.
It’s probably a conversation worth having, though. Both the beautiful experiments of history and the ugly ones have lessons for scientists and students of today about how best to conduct research in the future.
I waffled a bit about whether to put this one here or over at Forbes; I’ve ended up leaving it here as an experiment of dubious quality as to who reads what. If you like this, here are some buttons for you to consider clicking:
And if you’d like to express a preference for one venue vs. the other, the comments will be open.
You mentioned a lot of ugly experiments in the sense that they failed or aren't even real, but are there any ugly experiments that succeeded?