Thoughts About Thinking About Thinking
Reactions to a philosophy talk
A little while ago, I wrote a bit about my reactions to a book by David Chalmers on the “simulation hypothesis,” the idea that the universe we experience might be some sort of simulation being run by beings at a deeper level of reality. The proximate cause of that was a “book club” organized in advance of a big public lecture by Chalmers (though his talk was not about the book). That talk took place Wednesday night, in the Nott Memorial (our signature domed building, which houses an art gallery and event space):
The title for the talk was “Can ChatGPT Think?”, so largely a talk about philosophy of mind. As I was wandering the art gallery before settling into one of the nice chairs in the study area on the third floor (I had played some really intense pick-up hoops at lunchtime, and was not having any part of the little folding chairs they set up for these things), it occurred to me that this is the second time I’ve gone to a public talk by a major figure in this area, the first being a talk by the late Daniel Dennett way back in 2003. I don’t really remember many details other than that the underlying attitude of the talk was really off-putting (as I wrote at the time on a very early iteration of my blog, whose archive is broken so I have to use the Wayback Machine). In retrospect, that was a good bit of foreshadowing for the whole “New Atheist” era, but in 2024 it made me a little nervous about what I was going to see…
Happily, I found Chalmers’s approach a little more congenial than Dennett’s; still a lot of self-citation, but I’ve sort of become resigned to that as just a professional tic of philosophers. Being a philosophy talk, it was mostly concerned with trying to define questions that are usually vague, specifically “What does it mean to think?” and “What counts as evidence of thinking?” Chalmers laid out a bunch of criteria for both, starting with the Turing test and various refinements thereof, and suggested that it’s at least arguable that state-of-the-art Large Language Model systems meet at least some of those criteria.
Most of this was concerned with parsing the output of these models, which as a person with a technical background and a side hustle slinging words I find inherently a little suspect. Maybe the most accidentally insightful bit of the whole thing was in the Q&A when a student asked whether there was a Turing test analogue for consciousness, and Chalmers said that some seemingly plausible attempts at that had had to be abandoned in the LLM era since they rely on getting the entity whose consciousness you’re trying to test to describe their experiences of consciousness. Which is problematic because even relatively crude LLMs excel at producing text that seems to describe conscious experience. Unsurprisingly, since that’s a thing that human writers bang on about endlessly, so it’s all through the training corpus.
That is, I agree with Chalmers, a pretty good argument against the idea of parsing LLM output texts for signs of consciousness. I’m not really sure why it isn’t also an argument against parsing that same output for indications of “thinking,” though. Which would kind of undermine the whole thrust of the talk…
The bit that piqued my interest most had to do with looking for (arguably) more objective evidence of “thinking” in the internal workings of the LLM’s. These are (in)famously complicated and opaque, but he referred to some work on using sparse matrix techniques to sift through the neural network and identify clusters of nodes that are more active when the system is prompted with particular concepts— the example he referred to involved the Golden Gate Bridge, which seemed to be associated with a particular pattern in a particular LLM. This set of nodes would get more active when prompted with text about the Golden Gate Bridge or images of that specific bridge, and was more weakly activated by discussion of bridges in general. Cranking up the weights in the network associated with that set of nodes made a model that would drag every conversation around to the Golden Gate Bridge, and so on.
How meaningful this is, it’s tough to say, but this seems at least loosely analogous to the sorts of things people do with fMRI techniques— sticking people in an MRI scanner and identifying clusters of neurons that light up when they think of an elephant. If that’s a real parallel, that seems intriguing, or at least more intriguing than the notion of merely parsing the text output of these models.
Unfortunately, this is pretty far from my own areas of expertise, so it’s hard for me to assess, and Chalmers skimmed over it at a really superficial level (as appropriate for a public lecture). So I don’t know if it’s actually evidence from the perspective of people who know more about these topics and start from a skeptical position.
So, on the whole, a thoroughly inconclusive evening. But at least Chalmers seemed to embrace that inconclusivity (it’s a word now, spellcheck): he didn’t make any sweeping assertions one way or the other, just suggested it as an interesting problem to consider, and one that’s likely to become more interesting if progress in these models continues.
So, yeah, that’s my quick take on the Chalmers talk. If you like this, here’s a button:
And if you have thoughts on the subject, or at least simulacra of thoughts about the subject, and would like to share them, the comments will be open:
In his eulogy for Professor Dennett, Professor Chalmers says it was Dennett's (and Hofstader's) 1981 book "The Mind's I" that inspired him to do Philosophy.
I haven't read Chalmers so I'll refrain from commenting on his approach, though I'll say that I share your suspicions. The undeniable success of LLMs is often allowed to mask how good they are at proving the ineffectiveness of the Turing test (which was always intended more as a thought experiment than as a rigorous criteria, anyway). The best recent work I've read on consciousness has been _The World Behind the World_ by Erik Hoel, which is at its best as an exploration of what criteria for consciousness might look like, I think.