In 1942, Chicago Pile-1 was built under the direction of Enrico Fermi. It comprised 45,000 graphite blocks weighing 330 tonnes, 4.9 tonnes of uranium metal, and 41 tonnes of uranium dioxide, placed underneath the stands at the Stagg Field rackets court at the University of Chicago. Depending on how you define terms, you could call it the first nuclear reactor; it wasn’t meant to produce power for industrial use, but it was the first engine for a sustained critical reaction.

By modern standards, some safety corners were cut. For example, the part where it was built underneath the stands in a rackets court at a university inside a major city.

General Groves, director of the overall Manhattan Project, had tried to have the experiment happen near rather than directly in Chicago, and had ordered a building constructed for that purpose, but construction had fallen behind. Arthur Compton, the Nobel laureate physics professor at the University of Chicago who hosted CP-1, had avoided asking the university president for permission since, Compton later explained, the president would have had to say no, and that would have been the wrong answer.

The labor of stacking up the bricks was done by high school dropouts looking to earn some extra money while awaiting their military draft.

The uranium was enclosed in a seven-meter rubber cube, rather than a metal reactor vessel. There was, of course, no giant concrete containment building.

Upon later learning these facts, James Conant, chair of the National Defense Research Committee, is said to have turned white. Even for the 1940s, this wasn’t considered totally normal scientific behavior.

If you were reading about all this in a history book without knowing where it was heading, you might expect that you were reading the prelude to some great safety failure. So much is missing that the culture of 2025 thinks of as standard safety measures. Where are the inspectors and clipboards? The huge, weighty rulebooks of operating regulations? The soberly debating committees? The impact statements? The regulations saying that only Very Credentialed People are allowed to stack up the uranium bricks? Where’s the paperwork?

But the pile of uranium bricks and graphite bricks did not melt down.

And the reason is that Fermi knew what he was doing; he had predicted the rules in advance.

Fermi was not just stacking up mysterious bricks that generated more heat when they were brought into close proximity. He knew that some uranium atoms would spontaneously decay and fission. He knew that when this happened, the fission would generate neutrons. He knew those neutrons would sometimes knock into other uranium atoms, and that this would sometimes trigger another fission.

Fermi understood in advance; he did not have to find out the hard way, that he was dealing with an exponential process. Not in the sense that today’s media overuses the word “exponential” to just mean “large” or “fast,” but a process whose rate of increase is proportional to its current level: mathematical exponentiation.

Fermi knew that by stacking up more uranium bricks and graphite bricks, he was increasing the factor multiplied within an exponential process. As discussed in the book, there is a world of difference between a neutron multiplication factor below 100 percent and a neutron multiplication factor above 100 percent.^* Below 100 percent, you just have a warm pile of bricks. But past 100 percent, the radioactivity level of the pile goes up. And up. And up.

It does not behave like all of the previous, smaller heaps of uranium bricks that you may have tested. If you didn’t understand what you were doing well enough to under-moderate the reactor (so that the chain reaction would slow down if the reactor started overheating), then the reactor would not have stabilized itself like the smaller piles did. If you let it keep running overnight, you wouldn’t get a new, industrially useful level of power output the next day.

The heap would just get more and more radioactive until the graphite caught fire or the uranium melted into slag.

The firefighters would come then, and they would find a confusing fire that did not stop putting out heat when they poured water on it.

1942 would not have been a great year to attend the University of Chicago.

But Fermi already knew about all of that! So it was fine. When Fermi ordered a control rod (a wood plank with a cadmium sheet nailed to it) to be pulled out twelve more inches on December 2, 1942, he called in advance that this would be the withdrawal that made the measured radioactivity levels “climb and continue to climb…it will not level off.”

Then the radioactivity doubled over the next two minutes, and doubled again, until they’d let the reaction run and double every two minutes for a total of twenty-eight minutes, going up by a factor of around 16,000.

A 16,000-fold increase of radioactivity was the pile’s expected behavior, predicted correctly, understood in detail in advance. It wasn’t a surprise gotcha, run into by somebody ordered to pile up ten times as many uranium bricks as last time to see if anything interesting and profitable happened.

As discussed in the book, there is a very narrow margin between a nuclear reactor and a nuclear explosion. A margin of slightly more than half a percent, to be exact. That is the difference between a reactor that puts out an industrially useful amount of power and a reactor that explodes.

Which is to say: You have to make a nuclear reaction more and more powerful, before it really starts working at all. And then, a bare moment after it gets that powerful, if it gets a hair more powerful than that, if you go 0.65 percent further, it explodes.

That is a kind of problem that reality is allowed to hand you. It happens.

But Fermi and Szilard and their team had predicted all of these rules in advance of finding out the hard way. They knew about delayed neutrons and prompt neutrons. (See Chapter 10 for more about this part of the story.) So once Fermi got the neutron multiplication factor up to 100.06 percent, Fermi didn’t order the control rod pulled out further, to see what happened with an even more powerful heap. He went only up to criticality, not 0.65 percent further to prompt criticality. Fermi got the result he had predicted, and he knew what would happen if he went any further. So he went no further.

Twenty-eight minutes later, with radioactivity doubling every two minutes to a 16,000-fold increase, Fermi shut down the world’s first nuclear reactor — the piled uranium bricks under the stands of a university stadium inside a major city.

To be clear, we wouldn’t claim that Fermi was being completely responsible just because he had an apparently self-consistent model of low-energy reactor physics. Fermi could have been wrong. Humanity has run into some surprises over the course of nuclear engineering.

The Castle Bravo test of the first thermonuclear weapon had three times its anticipated yield because it contained mixed lithium-6 and lithium-7 as nuclear fuel for a fusion reaction. The people making the weapon knew about some potent nuclear products from fusing lithium-6 but none from fusing lithium-7, and it turned out that lithium-7 was not actually inert.

Fermi, in running his reaction at a low intensity and not at a level where it was putting out industrially useful levels of power, avoided many complications that appear in nuclear reactors powerful enough to be profitable. If there had been any reaction-rate-dependent neutron-factor-increasers that Fermi did not anticipate — any previously unknown phenomena, of the sort that showed up in the Castle Bravo test — any surprises that manifested once the neutron flux went up by a factor of 16,000 and bumped up the multiplication factor from 1.0006 to 1.02 faster than the reaction time for a human to dump in emergency cadmium — then today, America would have a Chicago Exclusion Zone.

Even so, we’re not saying that Fermi was necessarily wrong to run that experiment. It wasn’t the sort of experiment that could have destroyed the human species. There were arguably stakes worth wagering a Chicago Exclusion Zone as the non-default outcome of encountering a hidden new phenomenon that upset a hopefully precise understanding. In reality, Nazi Germany wouldn’t end up close to obtaining nuclear weapons by 1945, but nobody in 1942 knew that would be true. Predictions like that are hard calls. Piling up the uranium bricks outside a major city would have been inconvenient, and inconveniences have real costs in war.

Our goal in recounting this event isn’t to pass a moral judgment one way or the other. To start with, we would need to spend more time looking at the historical details of what happened to understand how those exact people’s exact options looked and whether they passed up a better option.

The lesson we’d draw is more about the difference between stereotypical “safety” and what it actually takes to have reality not kill you.

Chicago Pile-1 had a great absence of stereotypical, visible, ostentatious safety measures of the kind that bureaucrats know how to demand. Disaster was avoided by understanding, not by safety theater. Fermi’s understanding proved sufficient; it imaginably might not have, but in reality it was. And that level of understanding was what reality demanded, not any amount of pretense.

If nobody had understood at a deep level what was going on inside a pile of weird metal bricks…then it would not have helped much for lots of inspectors in sober-looking suits to peer at the bricks of inscrutable metal, or print a well-bound official-looking Safety Handbook saying that only Certified Operators are allowed to stack the weird metal bricks.

We can imagine a world where Chicago Pile-1 was built without an Enrico Fermi. Without anyone, indeed, who understood the true laws governing the mysterious self-warming bricks.

In such a world, perhaps another scientist still could have seen the lethal danger coming before it was too late. We can imagine an exchange like the following:

Salviati: The way that the bricks jump in power when brought together is an obvious signature of a self-reinforcing process, the sort of process that can make itself stronger. If you look for mathematical models that can describe a process like that at all, they tend to have a mode where, if you push them far enough, they explode.

Simplicio: What nonsense! In real life, it’s scientific to believe that every kind of process like that eventually runs into a limit. They can’t go on forever to infinity! So stacking up bricks of uranium and graphite ought to be perfectly safe, because it’ll hit a limit, see, and be harmless.

Salviati: That’s like arguing that a supernova can’t be dangerous because it can’t get infinitely hot, or arguing that an artificial superintelligence would be harmless because it wouldn’t be infinitely intelligent. Or like arguing that a bullet must have some limit to its speed and therefore won’t pierce skin. Just because there’s a limit somewhere doesn’t mean the limit is low. All the mathematical models we have of why the bricks are self-warming suggest that there’s a critical threshold somewhere, such that going past that threshold will make the pile explode and kill everyone nearby.

Simplicio: But scientists can’t even agree on where that threshold is! If there were a scientific consensus that adding a few more bricks was dangerous, I’d stop. But when scientists can’t even agree where exactly the danger lies, why worry?

Salviati: When many of the leading scientists warn that there’s a serious possibility of a lethal explosion, the fact that they can’t calculate exactly when the explosion starts should make you more worried, not less. Maybe if we knew precisely how the bricks worked, we’d see that there was some narrow band where we can safely extract energy, below which the bricks are useless and above which the bricks are lethal. But the fact that the scientists are still bickering means that we don’t know what we’re doing yet! Which means that it’s not the time to be playing around with whatever chain reaction is making those bricks warm today, lest it make them explode and kill us tomorrow! Figure out the science first.

We are very, very far from being able to model AI even a fraction as well as Fermi understood nuclear chain reactions.

At some unknown point, if we continue down this path, we will run at breakneck speed into an outcome far more serious than irradiating Chicago.

Notes