Resources › Chapter 2Why does gradient descent matter?It’s important for understanding how engineers can and cannot shape modern AIs. 2 min readDo experts understand what’s going on inside AIs?No. 12 min readIs intelligence understandable in principle?Probably. 1 min readBut some AIs partly think in English — doesn’t that help?Not as much as you might hope; we already see signs of infidelity. 5 min readAren’t AIs “just math”?Saying AIs are “just math” is like saying humans are “just biochemistry.” 3 min readAren’t AIs just predicting the next token?Predicting tokens requires understanding the world. 3 min readAren’t AIs only able to parrot back what humans say?AIs can already surpass their training data, or forego human data. 6 min readWon’t AIs inevitably be cold and logical, or otherwise missing some crucial spark?No. 2 min readWon’t LLMs be like the humans in the data they’re trained on?There’s a difference between the machinery it takes to be one person and the machinery it takes to predict many individuals. 8 min readHow could an AI trained only on human data surpass humans?Perhaps by learning general skills and implementing them better. 5 min readWhat makes you think people can build superhuman AI when they don’t even understand intelligence?Past AI progress hasn’t required much understanding of intelligence. 1 min readDon’t hallucinations show that modern AIs are weak?Hallucinations reveal both a limitation and a misalignment. 6 min readBut won’t we run out of data before AI goes all the way? Or electrical power? Or funding?Probably not. 2 min readCould LLMs advance all the way to superintelligence?Other approaches may achieve superintelligence soon, even if LLMs don’t. 1 min readYour question not answered here?Submit a Question.Extended DiscussionWhat Good Does Knowledge of LLMs Do?“Obvious” Insights Take TimeA Full Description of an LLMLoad more