This is the online resource for Chapter 4 of If Anyone Builds It, Everyone Dies. Some of the questions we cover implicitly in that chapter (and therefore skip over below) include:

• What will AIs want?
• Why would an AI trained to be helpful turn out to want the “wrong” things? Wouldn’t that be a disadvantage that would be tuned out of it during training?
• How does gradient descent differ from natural selection? What does that say about how AI wants will turn out?
• What’s so bad about AIs turning out to have strange preferences?

Below, we cover a number of topics related to “Why isn’t it easy to make AIs nice?”