The Argument

The Argument

The Opening Argument

What happens if we build it and nobody dies?

The world after superintelligence

Kelsey Piper's avatar
Kelsey Piper
Nov 17, 2026
AI comes up more at the playground these days. (Illustration by The Argument using ChatGPT; Photo by Bettmann/Getty Images)

My online and offline lives used to be very different. Online I corresponded with people whose charts and diagrams suggested that the Singularity would happen in our lifetimes, and that in five years or so our world would be unrecognizable. If it doesn’t get us killed, I’ve been told more than once that it’ll be like stepping through a time machine — awakening one day in 2200 in a world where almost nothing remains the same.

Offline, the people I talked to largely assumed that things would mostly stay the same. Parents like me worried about which middle and high schools we’d send our kids to and commiserated about college admissions. We figured we would need to work our own jobs until retirement and prepare the next generation to succeed at theirs.

But AI comes up more at the playground these days. I don’t know, people will confess with a nervous giggle, what kind of economy my kids will graduate into. I don’t know what we’re trying to do here in the first place.

If you care about preparing your children to lead good lives, it is disconcerting to realize you have absolutely no idea what it will mean to lead a good life in 15 years — and that you’re just as clueless about it even if the next 15 years go well.

Let’s set aside, for a moment, the fear that companies like OpenAI and Anthropic will put their AIs to work building more powerful AIs, unsupervised, until we build a machine god that has no interest in working for us. Let’s imagine that through whatever combination of policy, luck, and technical breakthroughs, the bad outcomes are avoided: no war with China, no sovereign AI succeeding in coups against the world’s governments, no sudden plague wiping out AI’s human competition. In other words, we achieve superintelligence — a scenario in which general-purpose AI can do virtually every task better than a human being — and we find a way to make it benign.

What do we do with it then?

“What is striking to me is how little reassurance even some of the happiest scenarios offer,” biology researcher and occasional Silicon Valley observer Ruxandra Teslo writes in a recent essay. “One common and ostensibly optimistic vision is known as ‘handover’: humanity cedes control of its future to a godlike AI and we become the comfortably maintained inhabitants of a civilisation whose direction is no longer ours to decide.”

Some people are excited to pursue this very outcome. “If we do build a godlike AI, and determine to our own satisfaction that it’s fully-aligned, I’d rather let it choose its own direction for itself and for humanity than have it be under the control of the US president,” Scott Alexander writes in his defense of handoverism. Others are more wary but regard some kind of handover as inevitable — Teslo quotes one of the researchers at the leading AI labs as saying, “I cannot see any other option for where humanity will go.”

But it does seem, to me, that this would suck.

Share

More importantly, I think we could do so much better. If AI becomes as powerful as researchers believe it will become, we ought to be able to use it to build a world that we are actively eager to live in, rather than one we are unhappily resigned to. The world that futurists are envisioning is one where GDP rapidly rises a hundredfold or even a thousandfold — and yet where there is not all that much that most of us can look forward to.

Most people find the futurist projections where we are uploaded onto and immortalized by computers to be somewhere between disturbing and horrifying; even those of us who can take that one in stride just have to walk a few steps farther to find something we find disturbing and horrifying. To me the thing that is most painful to contemplate is a world in which none of the virtues that I have strived to cultivate matter, a world in which I cannot offer my children any meaningful guidance because none of the challenges or choices they face are comprehensible to me.

Subscribe for free to continue reading

Enter your email and get every post from The Argument.
Already have an account? Sign in
© 2026 Jerusalem Demsas · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for writing, podcasting, and media founders.