When you only have one example of how something works, it’s easy to imagine that it must only work that way.

If you have seen birds but not airplanes, you might imagine that all flying devices must flap their wings.

If you have seen human arms but not robotic arms, you might expect robotic arms to bleed when cut.

If you have seen brains but not computers, you might imagine that all computation has to have similar characteristics to a brain, which runs a lot of slow neurons in an extraordinarily parallel way, using relatively low power consumption.

You might see that neurons tire after spiking and need to reset by transferring millions of potassium ions through the cell membrane, a process that takes about a millisecond. You might implicitly infer from this that any small computing element is likely to tire for a millisecond (arguing, perhaps, that if it was possible to make neurons that could reset in less than a millisecond, then evolution would have built them already).

But if you reason that way, you’ll be shocked by transistors, which can operate at a speed of 800 GHz, or about eight hundred million times faster.

Once you study the details of the transistors, you can see all sorts of reasons why the biological comparison just isn’t very informative. Neurons have to not only fire, but also be cells, building the firing mechanism out of cellular organelles. They’re large, and powered by blood-borne nutrients. Transistors can be mere atoms wide, and are powered by electricity. Once you know some of the details, it seems a bit ridiculous to imagine that much about a transistor’s potential firing speed can be inferred from the firing speed of a neuron.

When you learn the details of how planes fly (using lift and speed), you see that the details render irrelevant most facts about birds (such as lightweight bones and flapping wings). When you learn the details of how robotic arms are built (using steel and pneumatics and electricity), you see that the details render unimportant most facts about arms (like blood and muscle and bone). When you learn the details of how transistors trigger (using electricity and only a few atoms), you see that the details render most facts about neurons irrelevant.^*

When you don’t know the details of how an AI works, it’s easy to imagine that it will possess lots of aspects of biological minds — that it will work like your brain does. But if you did know the details, then lots of those inferences would start to feel ridiculous. They’d start to feel like expecting a robot arm to bleed when cut. The AI would turn out to work in a completely different way.

But that’s harder to see if you know very little about how modern AIs work. In Chapter 2, we’ll describe the process by which modern AIs are created, and we’ll discuss how nobody knows how they work inside. Which explains why it’s so easy for people to make the mistake of expecting them to act like other people or technology that they have experience with, rather than seeing how strange they already are, and how strange they will become as the technology progresses.