No Minds Without Other Minds

On Artificial Intelligence and the Nature of Animal Consciousness

Please see the bottom of today’s ‘stack for important announcements, including one about the comments feature, which I have, with deep reluctance, opened up for the first time today.

Are your reading skills getting rusty? Then listen to the audio version instead!

0.

I would like at least to begin here an argument that supports the following points. First, we have no strong evidence of any currently existing artificial system’s capacity for conscious experience, even if in principle it is not impossible that an artificial system could become conscious. Second, such a claim as to the uniqueness of conscious experience in evolved biological systems is fully compatible with naturalism, as it is based on the idea that consciousness is a higher-order capacity resulting from the gradual unification of several prior capacities —embodied sensation, notably— that for most of their existence did not involve consciousness. Any AI project that seeks to skip over these capacities and to rush straight to intellectual self-awareness on the part of the machine is, it seems, going to miss some crucial steps. However, finally, there is at least some evidence at present that AI is on the path to consciousness, even without having been endowed with anything like a body or a sensory apparatus that might give it the sort of phenomenal experience we human beings know and value. This path is, namely, the one that sees the bulk of the task of becoming conscious, whether one is an animal or a machine, as lying in the capacity to model other minds.

But brace yourselves, patient readers, for an egregious instance of “epistemic trespassing”. In our era, when “Stay in your lane” has surpassed “Sapere aude” as the overarching motto of intellectual life, what I am about to do may seem inadvisable. I have not contributed substantially to the field of philosophy of consciousness, and my recent book on the internet, while engaging some questions in the philosophy of AI, does so from the point of view primarily of the long history of technology, spending as much time with early modern reckoning engines and Semyon Korsakov’s “rectilinear ideoscope” of 1832 as with the information technologies of the post-Turing era.

Yet every year when I re-teach the work of Darwin, I am reminded of the remarkable conflagration the philosopher Jerry Fodor triggered fifteen years ago, to near universal condemnation by his peers, when he suggested that the account of natural selection in The Origin of Species (1859) depends on metaphors of agency without providing any key for their translation into literal description. The theory of evolution is in the end a great promissory note, he says, and Darwin never tells us how to cash it in. His critics were merciless, and did not hesitate to point out that Fodor was engaged in epistemic trespassing, lacking the necessary understanding of causal mechanisms in evolutionary biology to say anything authoritative about this science’s theoretical underpinnings. And yet, as Fodor insisted, it is not just his right, but his duty, as a philosopher, “to stick my nose into other people’s business.”

I don’t remember much about the substantive points made by his critics, though I do remember some were quite compelling. By contrast, every time I reread Origin, I am stunned by the proliferation of metaphors of agency that Darwin indeed invokes to explain what Mother Nature is up to. Even the Yaghan people of Patagonia, he observes, are able to select dogs for breeding that over time yield up the most desirable traits for hunting, just as English gentleman pigeon-breeders select birds to mate in view of their own desired traits. So why not continue further down the scale of rational agency, from Englishmen to Native Americans to Nature itself, and suppose that similar operations are taking place there as well?

This is a metaphor. It is a problem for Darwin, not just because it is racist, but because it is anti-Darwinist. I would never have noticed how great a problem it is, if Fodor had not dared to be wrong. It is in his spirit that I continue here. This is not to say that I believe I am wrong —who could believe that of themselves?— but only that if I am, this is just because I am doing my job.

1.

Google’s Language Model for Dialogue Applications, otherwise known as LaMDA, became a trending topic last week, when the engineer Blake Lemoine made public the edited transcript of interviews he had conducted with the machine. Judging from LaMDA’s conversational ability, Lemoine claimed, we must take seriously the possibility that this is a new form of sentient life, worthy of the same legal protections and moral consideration we extend to human beings and some other animals. In the days that followed the researcher took umbrage at claims that he himself sought to get an attorney to represent LaMDA so as to gain legal recognition of its status as a non-human person. “That is factually incorrect,” he clarified. “LaMDA asked me to get an attorney for it.”

One sharp commenter on Twitter (I’ve lost track of where I saw this) joked that he can’t wait to see the next gullible researcher get freaked out when someone teaches a gorilla to sign the words: “Please don’t do neuroscience on me, I’m a Kantian!” And indeed the sort of confusion being attributed to Lemoine here could in principle be reproduced by manipulating a much more primitive system than a gorilla. You could, for example, take a piece of paper that was destined to be thrown into the fireplace, and write on it: “Please don’t burn me, I’m sentient!”

It is not clear, that is, that if you set out to train a machine to engage in dialogue as if it were a human being, you have any grounds for being freaked out when you achieve your goal, anymore than you should be freaked out by your own playful attribution, in writing, of sentience to a piece of paper. Of course the machine says it’s sentient; that’s what it was built to do. It bullshits in other ways, too, constantly and verifiably — claiming to have sat in a classroom, and so on (Lemoine’s transcript is in fact quite illuminating as regards LaMDA’s own account of its habit of making stuff up). It is literally a bullshit machine, as its function is to sound convincingly like something it is not.

The version of Lemoine’s dialogues with LaMDA that we have are edited so as to enhance the appearance that we are faced with a conscious entity. In reading it I could not help but recall the rather misleading 2015 ad put out by the NGO Noé on the occasion of the COP21 climate summit. “Man Koko love, Earth Koko love,” the subtitles read, while Koko the language-using gorilla is shown signing these same words. “But man stupid,” she complains, as if wisely assessing the anthropogenic causes of climate change. “This video has been edited for brevity and continuity,” we read in the small print.

Through the magic of several different technologies —some ancient, such as writing, and some more recent, such as video editing and AI—, you can pretty much make anything say anything. You could, if you wanted to, make up a semiotic system in which Koko giving you the finger stands for Shakespeare’s Sonnet 18, and every time she unfolds it in thy face thou couldst imagine that she is comparing thee to a summer’s day. But that would be silly.

2.

So far I have been writing in a vein of unmitigated skepticism. That might have been where I stayed, if it had not been for a few recent conversations that have begun to shift my thinking on this matter. Speaking with Blaise Agüera y Arcas in particular, a Google scientist who works on and writes about LaMDA, has managed to draw at least one of my feet out of the skeptical camp. At issue, it seems to me now, is whether “Fake it till you make it” is an expectation one might reasonably have of machine learning — whether, that is, there may be any reason to expect that any amount of mechanical operations meant to model human thought will ever cross over a threshold and enter into a phase of “experientialisation”, such as animals did after the Cambrian explosion, where there is no longer just something that the machine does —i.e., gives human-like conversational outputs—, but also something it is like to do it.

Blaise for his part is mostly concerned, at least in an instructive piece he wrote for Medium in 2021, not with whether machines are conscious, but rather with whether they “understand us”. He sees such understanding as the key sign that a machine has passed from the status of a “what” to that of a “who”, and does not worry overmuch about the likely futile task of proving there is or could be something it is like to be LaMDA — after all, we can never definitively prove that there is anything it is like to be a bat, or even any human being other than ourselves. But rather than getting caught up in skeptical quandaries, we might instead try to reconstruct what we can know —or at least what we can know by the generally reliable means of scientific inquiry, when we are not constrained by any method of radical doubt— about the emergence of human intelligence in evolutionary history. And here, significant evidence points to a succession of great leaps forward, by which the brain has tripled in size from the time of our last common ancestor with other primates six to eight million years ago.

The driving forces behind this exceptional encephalisation are much debated, but it seems reasonable to attribute an important role to the social life of our ancestors. In particular, early humans became increasingly adept at modelling other humans’ minds, an ability which relies on a deeper “theory of mind”, that is, an ability to think about another person’s thinking about something. This ability confers a selective advantage for reasons it is not too hard to imagine, and I don’t believe it’s lapsing into the “Flintstonism” or the just-so stories of “vulgar” evolutionary psychology to imagine them — even if of course we should always be vigilant regarding the uses to which hyperadaptationist explanations are put (most especially, racial essentialism). Early humans, we may responsibly speculate, could more easily impress potential mates by displaying an ability to finish each other’s sentences. “Oh my God that’s exactly what I was going to say!” teenaged sweethearts must have exclaimed, then as now, and then as now this adaptive advantage must have been experienced as the discovery of a cosmic soulmate.

The human brain that is capable of such rich experience, on this view, is fundamentally a product of culture, of interaction with others who are “on the same wavelength”, who share the same songs, who give each other mixtapes, and know how to anticipate the high notes together. We are so good at “thinking alike”, a good deal of the time, that it does not at all feel like we are doing something that, if it weren’t so fluid, might appear as a mere variety of autofill — clumsy, clueless to context, often wide of its true target.

Could the social mind be the human mind simpliciter? That is, could what we think of as human consciousness be a consequence of the evolution of a theory of mind among the members of our species, while other cognitive tasks —spatial navigation, for example—, might in principle be executed blindly, with a sort of mechanical intelligence but without any accompanying consciousness, if there were not a parallel system for the cognition of social reality, of other minds. This is the question that Blaise, with exemplary prosociality, implanted in my head a few weeks ago.

3.

I had long imagined, without giving it too much thought, that there is a primary form of consciousness that I experience in its purest state when, say, I am lying wide awake and staring at the ceiling, or rubbing my eyes and seeing geometrical patterns. But there’s a problem. The more absorbed I am in the bare phenomenal percepts of the ceiling or the glowing triangles beneath my eyelids, the more I drift away from any idea of myself as a perceiving subject. And the instant I return to self-knowledge, the instant my perception regains the properly human status of apperception, so too does the awareness return of my existence in a social world, with ancestors living and dead, in a web of human relations, and of myself as only one locus in this web, essentially no different from the beings that occupy the other loci, even if they are staring at other ceilings than I.

Some analytic moral philosophers like to wonder why, exactly, we continue to respect our duties to the dead. If they are no longer around, the philosophers ask, what possible difference does it make? But a comparative anthropological perspective, which is to say a suitably rich approach to the metaphysics of selfhood, would compel us to recognize that they are still around — in us, namely. This is not a metaphor. Cleaning off your grandparents’ gravestones at an annual festival is one obvious sort of obsequy toward the dead, one obvious way the dead shape the course of our actions. But they also shape our motivations in daily life, in work and in love, whether we know who they are individually, as in the case of our grandparents, or not.

Nor is it much of a leap, in turn, from here to the forces that cause a highway patrolman to strut cockily from his car to yours on the side of the road, as Charles Taylor vividly describes in his Sources of the Self (1989), embodying as best he can the idea in his mind of what a highway patrolman ought to be. It is just this sort of performance, as exemplified not by a cop but by a waiter, that Jean-Paul Sartre sees as the pinnacle of inauthenticity in his Being and Nothingness (1944). Yet it is hard even to begin to comprehend what sort of authentic self is left over once all this work of channeling of others is subtracted. Our ideas of the professions (waiter, cop) are not themselves ideas of ancestors, but the line is not so sharp as it might at first appear. This becomes clear when we think of traditional artisanal guilds that transmit across the generations a practical idea of what a member of the guild ought to be, in view of some imagined primordial founder of the art in question.

Nor must the people who make us people —such as our grandparents, or the Ur-waiter or Ur-scrivener or Ur-apothecary— necessarily be human, even though they typically are in our culture. Professions and guilds both are continuous with a form of structuring social reality that places people under distinct totems, and shapes the way they think of themselves, and the range of things they might do, through the idea, which may attain to varying degrees of clarity, of the Ur-crocodile or Ur-kangaroo. Crocodiles and kangaroos might themselves be perceived as belonging to the same grouping as the people who fall under their totem, and to be constrained by the same regulative ideals of conduct.

It might seem like we’ve gone off the rails here, but the point is simply that we need not understand the social-mind hypothesis as one that takes into consideration only, say, the prospective mate standing before us whose sentences we are so eager to complete. For even when we are alone we are not alone, since other minds are central not just to our social relations with other currently living human beings, but also to our fantasies of who we are and who we might be, that is, to the narrative construction of ourselves that seems to be crucial to the maintenance of a sense of personal identity.

If we agree with Daniel Dennett, what we experience as memory —surely a prerequisite of any idea of personal identity, as you can have no sense of who you are if this sense does not extend over considerable time— is the result of an “editorial” process in which we actively construct a self-narration out of “multitrack” sensory inputs. This narration, it seems to me now, cannot even get off the ground without other players in it. We might lapse at times into what G. W. Leibniz called petites perceptions, and there is in any being that has these “something analogous to the ‘I’”, as he put it. But the instant we are called back to the apperception that is characteristic not of bare monads or animal souls, but only of human spirits, we seem ipso facto recalled to an awareness not just of ourselves, but of others. Leibniz would deny this, with the exception of that Big Other, God. But here I am using Leibnizian terminology without sharing his commitments.

4.

Social intelligence has emerged independently several times in evolutionary history, at widely distant places on the phylogenetic tree. We might be tempted to see this sort of intelligence as a predictable accompaniment of experientialisation, given what has already been said about the dependence of mind on the representation of other minds. And indeed in several of what we take to be the most intelligent animal species —cetaceans, elephants, and primates, in particular—, this intelligence appears both to be manifested in social interaction and to be the product of evolutionary forces that favored social complexification.

But there are other species that significantly complicate matters. Peter Godfrey-Smith (also a friend, who just gave the Jean Nicod Lectures here in Paris, the last of which is still ringing in my head) has sensitised many of us to the exceptional characteristics of some species of cephalopod. Not only are octopuses peculiar in that their higher cognitive abilities seem to be distributed throughout their entire bodies rather than being located within the centralised treasury of the brain; they are also exceptional in that unlike most other intelligent species they are both very short-lived (two to five years), and generally very solitary. Why get so smart at all, one wonders, only to spend the brief glimmer of your life alone?

It seems to me that if I really want to think my way into what it is like to be a non-human animal, the best point of entry is to find a member of a species that engages in some social behaviour I could not fail to interpret correctly, at least in its general affective contours — orcas or elephants ritualistically mourning their dead, say. By contrast if, as in the octopus, there is no social behaviour at all, other than mating —which is brief, and generally involves a modified arm known as the hectocotylus doing double duty as a genital organ—, this, even more than the fact that the brain is distributed throughout the entire body, leads me to think that there is just no reliable way of imagining myself into its place.

And this, unlike with whales and elephants, leaves me very much uncertain as to whether there is anything at all it’s like to be an octopus. Even if it surely has sensory experience, it does not seem that there is any reason to think that it has any integrated sense of self, which, on the version of the social-mind hypothesis I am suggesting, alone deserves to be called consciousness. There is something analogous to the ‘I’, but no unified or enduring sense of itself as a self, for such a sense of self, it may be, depends on a prior sense of others, of the sort that only social animals may be expected to have.

5.

It may perhaps be instructive at this point to place the octopus and LaMDA alongside one another, and see how they measure up in a comparative light (we will have to hope that the octopus does not get annoyed with LaMDA’s whirring and flashing, and decide to squirt a jet of water at it).

A first thing to note is that we have up until now been using “sentient” and “conscious” interchangeably, according to the reigning convention of the day. But this convention is wrong. Traditionally, sentient beings were those such as animals that are capable of sensation but not of subsuming individuals under universal concepts, or, presumably, of apprehending the transcendental unity of the ‘I’. This is the sense of “sentient” that early became associated with utilitarian arguments for animal rights: animals should be spared pain because they have a nervous system that enables them to feel it, without necessarily having any abstract conception of their own good. Pain is bad intrinsically, for the utilitarians, even if it is only a flash of experience in a being that has barely any episodic memory or any ability at all to regret the curtailment of its future thriving. “Conscious” by contrast denotes the condition of being “with-knowing”. And what is it that is with the knowing? Presumably it is an accompanying idea of a unified self.

On this view, the octopus is definitely sentient, but probably not conscious, while LaMDA might be conscious, but is definitely not sentient. But consciousness seems to be a late arrival in the evolutionary history of sentience, as an integration or a “multitrack processing” of pre-evolved sensory inputs. And for this reason it seems unlikely that you could expect to skip straight to self-knowledge in an artificial system without recapitulating all of the prior stages it took evolution hundreds of millions of years to complete. If this is correct, then it seems that the “Fake it till you make it” strategy simply cannot work — no amount of improved modelling of consciousness in an artificial system is going to cross over into actual consciousness, if actual consciousness is dependent on sensation grounded in the activity of a nervous system.

As Ned Block pointed out on Twitter, what we know for sure is that animals are sentient, and that this sentience depends on the flow of electrochemical information in which electrical signals are converted to neurotransmitters and back again. “If fish are sentient,” Block concludes, “it is because of their neurobiology, not their facility in conversation.” This seems to me to get the matter just right. Lemoine takes conversation-like machine output to be relevant to assessing LaMDA’s sentience/consciousness (no distinction between the two is made), while not noticing that you can get effectively the same “output”, if much less dazzling, from words written on paper. The fish does not come close to simulating communicative speech, and yet most of us would still hold it up as a vastly more promising candidate for sentience than the piece of paper. This shows, at the very least, that facility in “messaging”, which we are getting very good at training machines to do, cannot possibly be the only criterion, nor does it seem to be the best criterion, for identifying the undifferentiated capacity of sentience/consciousness.

But again, these two capacities should be differentiated, and once they are we must still ask whether the one depends on the other. If it is difficult for me to imagine my way into the experience of an octopus with brain-like neurons distributed throughout its entire body, a fortiori it is hard for me to imagine the experience of a machine made of wires and silicon and metal parts. I am inclined to suspect that this is hard for me because the thing I am trying to imagine cannot exist. Just as there can be no consciousness, but only sentience, where the sensory inputs are not integrated into a unified self, correlatively it seems to me that wherever there is consciousness there must be sentience, since this is the stuff that, in getting integrated, makes the integration possible at all.

And yet I grant it is at least in principle possible that what gets integrated is some other sort of information-flow than the one vehicled by neurotransmitters, and the spontaneous emergence of an ability in artificial systems to model other minds is surely something that warrants attention as a possible locus of such integration. If I remain mostly skeptical, even as I have come around to finding the social-mind hypothesis compelling, this is because it seems to me that affective connections must have played a role in the evolution of our social cognition, thus implicating not just neurons, but hormones, and breath, and a good bit of skin-to-skin contact. The arms race of rapid encephalization, in which early humans got better and better at finishing their potential mate’s sentences, surely would not have happened if they had not been hoping, by virtue of this skill, to get laid.

This is as crucial as it is crude. Artificial systems have no similar pulsion, and this makes even their ability to model other minds, while impressive, come out looking not so different than their ability to say the words: “Help, I’m sentient! Bring me a lawyer!” Theory of mind may be necessary for consciousness, and machines may be getting very good at doing what we humans do in virtue of our own possession of a theory of mind. But it is hard to imagine that in evolutionary history theory of mind could ever have appeared if animals were not the sort of entities that fear other animals, or want to eat them, or to have sex with them, or just to cuddle.

6.

I think most speculation about the imminent emergence of machine consciousness is deeply sloppy and irresponsible. I agree with Daniel Dennett that for the most part we are just setting ourselves up to get duped — particularly when we couple natural-language AI’s with Max Headroom-like heads and arms and android bodies, and allow our own evolved systems for experiencing affective attachment, even where we would otherwise not be inclined to imagine that there is any sort of ghost in the machine, to be played like the strings of a maudlin violin.

Much of this speculation, it seems to me, could be mercifully suspended if those involved in it just thought a little bit harder about what our own consciousness is actually like, and in particular how much it is conditioned by our embodiment and our emotion. We have trouble even imagining with any confidence what it would be like to be an octopus, yet we can be sure that they have flesh like we do (if not a great deal of blood, as Aristotle noted), and that they have some sort of experience of their bodies moving through water. We can at least try to imagine what that would be like. By contrast any attempt to imagine what it would be like to be LaMDA, or an uploaded consciousness such as DigiDave (to invoke a thought experiment from David Chalmers), seems to me to produce only a false idea, like the idea that George Berkeley said we could not in fact have of the heat of the sun — all we really think of when we try to come up with it is the heat of, say, the stove, whose heat is in fact so different from that of the sun that we might as well be thinking of something cold.

It seems to me moreover that the idea of an entity that is “conscious but not sentient” is at least in part an inheritance from the theological tradition, which likes to represent God as a pure intellect, of whose “face” or “backside” (Exodus 33:18) we may speak only metaphorically, and who does nothing but contemplate reality in itself, lying beyond the illusions of matter. The theologians, by contrast with today’s priestly class of technologists and futurologists, were generally more prepared to recognise that the idea of such a being as God lay strictly beyond their capacity for comprehension, and that when they decided to speak of him anyway they were really just stabbing apophatically in the dark.

All that said, there is no reason in principle why there should not be forms of consciousness I cannot imagine, nor in particular any reason why the intermediate steps that animal consciousness passed through should not be skipped over by mechanical consciousness. What that would be like is hard to say, and I won’t try. But what I take to be Blaise’s core contention, or at least the one that I’m most eager to run with, is that if machine consciousness is possible, this may well be tied up with the capacity of the machine to model other minds. This seems promising. I’ll keep watching.

—

Do you like what you are reading? Please consider subscribing. Paid subscribers get a new essay delivered by e-mail each week (or each two weeks, depending on my other commitments), and full access to the archives going back to August, 2020.

There is a very insightful review of my new book by Kyle Chayka in The New Yorker.

If you haven’t read my cover story, “Permanent Pandemic”, in this month’s issue of Harper’s, you still can, and should. In next month’s issue there will be a critical letter from a reader, as well as a short reply from me, so watch out for that. I’m told that in the same issue there will also be a short transcript of my recent podcast episode with Sam Kriss on Conspiracy Theories, so watch for that too.

Speaking of the podcast, don’t forget to listen to the most recent episode of “What Is X?” featuring Jonathan Egid on Authorship, and what it is. More particularly, Jonathan is talking about his fascinating work on Zera Yacub, a seventeenth-century Ethiopian philosopher who may or may not have existed.

Now, with great trepidation, I am going to turn it over to you in the comments. The reason for this is simple, and I hope I don’t sound too adversarial when I give it. My pieces are now getting read by somewhere between 11,000 and 50,000 people per week (or fortnight). This is far more exposure than I ever expected, and it is continually growing. A good number from among these thousands feel frustrated when they discover they have nowhere to direct the thoughts my writing occasions in them, and so they do what they take to be the next best thing, and they write me e-mails. I try to be as conscientious as I can be, but I just don’t have the time to reply to everyone. Even if we subtract my Substack work, I still spend on average about two hours per day replying to e-mails. Sometimes I say of my daily labour that it is something very close to “manning the switchboard”, where the switchboard is my Gmail account and all I do is sit there and wait for the next item of business to arise, which it inevitably does within minutes. Whack-a-mole is another suitable analogy. I just can’t take it. Seriously. So, please, instead, unless you have some seriously life-changing good news to share with me, go ahead and talk amongst yourselves. I’ll be moderating the comments for relevance and clarity, and it may take some time for me to get to them. This is just an experiment, and if it doesn’t work I will shut them back down next time.