Building AIs that can do sufficiently impressive things will tend to cause the AIs to want things.

When we say that an AI “wants” something, we don’t mean that the AI will necessarily have human-style desires or feelings. Maybe it will, or maybe it won’t. The thing we instead mean is that the AI will behave as though it wants things. It will reliably steer the world toward certain sorts of outcomes — anticipating obstacles, adapting to changing circumstances, and staying focused, targeted, and driven.

This is the central thread for the content below. Topics we don’t cover below, because they’re in the book, include:

• How could a machine acquire the ability to “want” things, in the relevant sense?
• Is there any evidence that AIs can want things?
• Must more advanced AIs want things?