With more technology, we’d likely make even fewer copies of our genes.

Humans jockey for prestigious promotions and Ivy League admissions far more than they jockey for opportunities to donate to sperm or egg banks.

Sperm and egg banks pay the donors for their trouble, rather than the other way around.

Most tyrants throughout history didn’t even attempt to use their power to have thousands of children. And actual birth rates in the world today are falling.

Many humans prize having children, but many others don’t, and it’s extremely rare for anyone to try to maximize their number of offspring (e.g., by interfacing with sperm banks as much as possible). Instead, humans largely jockey for things like sex, fame, and power — things that are at best messy proxies for reproductive fitness.

Nevertheless, one might look at this picture and say: Well, humans wound up caring a little about having children, even if that care is less than maximal. Maybe AIs will have a little care about us, and toss us some sort of bone, rather than killing us all.

One trouble with this hope is that the proxies we care about have recently (on evolutionary timescales) become untethered from actual reproductive fitness, and they’ll likely grow ever further apart in the future, as humans continue to find new technological avenues to satisfying their desires.

For instance: Our drive for children is not quite a drive for genetic propagation. Suppose that in the future, a technology is created that replaces all of the DNA in a person’s cells with different molecular machinery that makes the person immune to all illnesses and extends their healthy lifespan.

(Suppose as well that this technology doesn’t change the person’s personality or cause other harmful side effects, such that the many people who have reasonable hesitations about the safety of new technology are mollified.)

We expect that lots of parents would be thrilled to hear that their children got the treatment. And maybe there would be a number of holdouts at first, but we expect that if the technology were proven to work, and if it became cheap and reliable, that it would eventually become ubiquitous. Which reveals us for what we are: people who like having children, having families, having fun — not people who like propagating our DNA.

It seems to us that most humans simply don’t care about genetic fitness at all, in the deep sense. We care about proxies, like friendship and love and family and children. We maybe even care about passing on some of our traits to the next generation. But genes, specifically?

Every time that humanity has unlocked a technology that let us get more of what we like — such as tasty foods, or sex without reproduction — humanity has taken the bargain. We aren’t technologically advanced enough that we’re able to trade away genomes for longer and healthier lives. But that sort of thing looks possible in physical principle,^* and so it doesn’t look good for natural selection in the long run.

If AIs wind up caring about goodness and kindness and friendliness in anything like the way humanity cares about genetic fitness, then we expect that AIs will eventually invent things that are to “friendliness” what birth control and DNA-less children are to genetic fitness — namely, that they’ll pursue things that are only a pointless shadow of what any human would wish or intend.

AIs caring about humans a little would not be good.

For all that most humans seem to care about children and family more than they care about genetic propagation per se, there are undoubtedly some humans who insist that they care about their genes at least a little bit. We are a bit skeptical about some of these claims — e.g., perhaps some people in the modern world who try to pass on their genes as much as possible are doing it for a sense of beating out the competition, and perhaps that sort of person would instead compete over how many DNA-free children they could have, if DNA-free children ever became ubiquitous. But perhaps other such claims are true. Perhaps there really are a handful of humans who care deeply about propagating their genes, in a robust way, at least a little bit. Humans have all sorts of preferences, after all!

Might not the same be true of AI? If there are lots of weird and diverse AIs, might not at least some of them turn out to care about humans at least a little bit?

They might. Unfortunately, we expect that this would mostly not go well for humanity, either. This is a topic we’ll take up in earnest after Chapter 5, with the primary discussion being whether AIs might end up caring about us at least a little.

But before we get there, let’s take a step back for a moment. Imagine that the situation with AI is that modern methods can’t make AIs care about us very much, but we’re hoping that if we make a lot of AIs, then some tiny fraction will care about us some tiny amount, if only by chance. The idea being that if we build AIs today, then their most preferred outcome is that they grab almost all of the resources in the universe and spend them on something pointless, while perhaps keeping a few humans around on a small human preserve.

If humanity rushes ahead and rolls the dice with superintelligence, then we expect a far, far worse outcome. But this still seems like a very bad plan to us, even if we had reason to think AIs would care about us some tiny amount. So this line of speculation seems not only mistaken but moot.