The Inaugural Hinternet Foundation Summit is a little more than a week away! Register now and join us at La Gaîté Lyrique in Paris on November 23!
In just the past two years or so I have noticed something at least close to a motte-and-bailey retreat among the more fervid of A.I. “enthusiasts”. I use this label in the same sense in which Kant applied it to Emanuel Swedenborg, when the latter started “seeing spirits”, as designating the Schwärmer, which we may loosely translate as “people who get worked up into a tumult.” Among the A.I. Schwärmer we may certainly include Eliezer Yudkowsky and Nate Soares, co-authors of If Anyone Builds It, Everyone Dies: The Case Against Superintelligent AI (Penguin Random House, 2025). As the title clearly tells us, these two are obviously not “enthusiastic” in the ordinary sense about the direction things appear to them to be going. They are enthusiasts, or at least one in particular is (Yudkowsky namely, whose work I have been following off and on for much longer than Soares’s), in the sense that, however much I strain when reading their urgent intervention, I find I cannot forget that it comes from the same community of nerds that brought us Roko’s Basilisk, and other such fever-dreams of our own era’s equivalents of the Geisteseher.
The retreat in question concerns precisely whether “ghosts” may be seen on the horizon or not: from a prediction, namely, that A.I. systems are likely to develop malign intentions towards human beings, of the sort that only entities endowed with, well, intentions can meaningfully be said to have; to an insistence that it is irrelevant whether they will do so or not. Yudkowsky has never been a hype-man for the imminent awakening of the machines, but, as I’ll show in what follows, much of his and Soares’s argument only makes sense on the presumption of such an awakening.
Unable to see this, the authors, having brought up the question whether Stockfish, a state-of-the-art chess-playing A.I., “wants” to win at the game or not, seek quickly to despatch the problem by declaring: “That’s between you and your dictionary.” The problem however is that it matters predictively whether A.I. becomes conscious or not, whether it wants or not, whether it turns out, to speak with Descartes, to be the sort of thing that “doubts, understands, affirms, denies, wills, refuses, and also imagines and senses.”
Here’s why.