The AI doomers are crying wolf and they’ll never admit it
The following is the transcript of my troop deployment (i.e. ending rant) from episode 105 of my news podcast The Mind Killer. If you like it, please consider subscribing to the podcast
AI doom is everywhere these days - sometimes even on this podcast! Read any rationalist-adjacent AI take and it’s all about how we’re on an inevitable march to becoming paperclips, and how everyone developing frontier AI models is being profoundly irresponsible. Earlier this month, everyone was all pissed off at Anthropic because “THEY WERE SUPPOSED TO BE THE SAFETY COMPANY” and they released an LLM that wasn’t noticeably worse than the other top models. So unsafe, they say.
But what exactly is the unsafe part here? Nobody thinks Claude is going to kill us all. But implicit in the anti-progress argument is that we’re close enough to a model that will kill us all that it’s irresponsible to push the frontier any further. We’re playing Russian Roulette with the future of humanity unless we impose strict government regulation designed to prevent AI from advancing far enough to provide the massive value that it’s clearly capable of. Far before it’s capable of taking over the world, it will be capable of helping people to do amazing things that will improve our lives immeasurably. But noooo, the doomers say. We need to stop progress now before we get to any of that and “solve alignment,” whatever that means.
The problem with this argument, though, is that you can’t pin down anyone to any specifics. Arguing with an AI doomer reminds me of nothing more than arguing with a theist, in the sense that they have an unshakeable faith but also refuse to make any testable predictions. All they’ll tell you is that AI is likely to kill us someday, maybe soon but maybe in the far future, and every day gets us closer to it, and there’s absolutely nothing that could happen that will prove them wrong. If AI kills us all? Told you so. If AI doesn’t kill us all? It’ll happen someday. If we solve alignment? Well that’s what they were saying to do all along. If we don’t solve alignment? Well they said we weren’t serious enough about it. If you spread flour on the floor to see the invisible dragon’s footprints? Well, you see, it floats.
It’s perfectly reasonable to think that AI might someday be a threat to humanity but also recognize that current models are nowhere near advanced enough to be worrying about that. The problem with the Discourse is that these two ideas keep being smushed together.
To the AI doomers, I propose a bet: in the next ten years, no AI will independently murder anyone. The only fatalities from AI will be situations in which humans intentionally used it to kill people, accidents similar to car accidents where the AI clearly just made a mistake, or trolley problem or triage type situations where someone was going to die either way. You win if an AI kills a person in circumstances suggesting it exercised agency and chose to kill without any human operator telling it to. The only stakes are that whoever is wrong just has to admit it and approach the issue with more humility in the future. Who wants in?