#2 Why ChatGPT isn't a step towards AGI
December Edition of Critical Rationalism News. On ChatGPT, Degrowth, Wokeism, & Knowledge Generation in AI.
Last month, OpenAI introduced ChatGPT to the world. The chatbot immediately took the Internet by storm, crossing a million users in less than 5 days of its research launch.
There's a ton of excitement around its impressive capacities. Scientists, journalists, writers, programmers, teachers, students, people-working-jobs-that-it-threatens, and of course, AI researchers—everyone's been talking about the new chatbot.
But there's a fundamental difference between a tool like ChatGPT and the much-maligned artificial general intelligence (AGI) that fearmongers fail to appreciate. Something like ChatGPT does not get us any closer to attaining general intelligence.
Though the number of transistors, memory, and speed of our computers have seen exponential growth over the years, AGI won’t emerge as a product of this ever-increasing complexity.
As David Deutsch wrote in his 2012 article for Aeon magazine,
“An AGI is qualitatively, not quantitatively, different from all other computer programs…Expecting to create an AGI without first understanding in detail how it works is like expecting skyscrapers to learn to fly if we build them tall enough.”
The actual barrier to developing artificially intelligent entities is not a breakthrough in computer science but in philosophy.
The problem is that we don’t understand how creativity works. We humans have it. The ability to think and create new explanations about what is out there and how it works is the fundamental functionality that separates us from animals. But we haven’t yet been able to explain how creativity functions.
None of our programs have ever created a new explanation—something entirely different from what they were coded to do. They can create new content by interpolating between a dense manifold of training data. But that will always be constrained to existing knowledge.
There’s the crux of it. Until we find a way to explain creativity in people, we’re only going to be able to create programs that obey their programmers. People admire ChatGPT as an AI, but they should really admire the programmers who wrote it.
An AGI will be able to disobey and create novel theories in its mind. It will be able to rationally and creatively criticize ideas.
“But,” say the naysayers, “certain AI programs (including ChatGPT) can learn, can’t they?”
Not so. In spite of educational institutions upholding this belief, learning isn’t a product of instruction. Knowledge creation in the minds of people happens through an active process of conjecture and criticism. We don’t absorb ideas, nor do we copy them inside our brains. Rather, we recreate knowledge using the method of trial and the elimination of error, as the epistemology of Karl Popper—that of critical rationalism—explains more deeply than any other philosophy of learning.
If a system has general intelligence, it will be able to take control over its own learning.
This obviates the specific concern for rogue AGI. AGIs could indeed be very dangerous. But humans are too. Since knowledge—scientific and moral—is objective, and an AGI would have the power to critically think about moral ideas and question their decisions, they’ll be able to converge upon moral truth with us and won’t be any more dangerous—and wonderful—than we humans are.
Credit for the ideas in this piece goes to David Deutsch with any errors of my own. In particular two of his essays “Creative Blocks” and “Beyond Reward and Punishment” were a heavy source of inspiration for this piece.
In addition to that, Brett Hall’s excellent critique of Nick Bostrom’s Superintelligence was of great help and also deserves a mention.
1. The prestigious science journal “Nature” publishes an article on “Degrowth”. Brett Hall superbly critiques it and explains how the plan of degrowth is actually one for civilizational decline and disaster, not greater well-being.
2. Can two people of different backgrounds (and hence conflicting moral frameworks) ever have a productive conversation? Yes. Here’s why the woke meme is anti-rational.
3. The latest twitter chaos is fascinating. Anders Sandberg shares some practical notes on free speech and the openness of platforms.
4. Why Popper was right about the link between certainty and extremism.
“Absolute certainty is the foundational component of totalitarianism: if one is sure that one’s political philosophy will lead to the best possible future for humankind, all manner of terrible acts become justifiable in service of the greater good.”
5. Jeremy Shearmur is organizing the Fifth International Zoom-Based Karl Popper Conference.
6. Will ChatGPT Kill the Student Essay?
From the Archives
1. The only way to make progress is through conjectured solutions, not arriving at definite truths:
“...the discovery of speculative truths is the sole means of advancing the human race”— Condorcet, Condorcet: Selected Writings, p.111
2. Dwarkesh Patel interviews David Deutsch about AI, creativity, fun, Bayesianism and more.
3. Cam Peters neatly ties up the conflicts with critical rationalism and “rationalism”.
4. The scientific method seeks truth while Machine Learning seeks patterns. The Problem of Induction and Machine Learning by Vaden Masrani.
Written by Arjun Khemani.
Edited by Logan Chipkin,, and suggestions by Aaron Stupple.
Thanks for reading Critical Rationalism News! Subscribe to receive new posts by email.
This is an oversimplification for the case of readability. Transformers can create new content by interpolating between a dense manifold of training data. But arguably it will always only be a "mix-mash" of the existing data. You can read more about this here.
Interesting thoughts! Although I'd push back on some of the arguments here.
We can create AGI without first understanding creativity. Understanding the theory is not a prerequisite to creating a practical implementation. Humans can and have created transformational technology without first understanding how it worked. The Wright Brothers created airplanes without understanding the principles of aerodynamics. Fleming created Penicillin, and only decades later did we understand how the underlying mechanics and properties worked.
I'd also argue that the "mix-mashing" of data you see in current models is just an artifact of the models not being large enough to fully represent the subset of reality they're trained on. Fundamentally, LLMs attempt to represent the world in their matrices of numbers. Information Theory shows us how to represent any arbitrary amount of information in bits -- which is exactly what LLMs do.
DALL-E showed us that LLMs are capable of representing Van Gogh's artistic style in their massive matrices. ChatGPT showed us that human language and computer code can be represented accurately enough as vectors for the LLM to produce *and explain* (!!) correct output. In the limit, a large enough model is theoretically capable of representing *and explaining* the nature of reality in its massive arrays of vectors; including language, emotions, mathematics, physics, etc... at which point I'd posit that it's capable of producing novel explanations about reality.