The AI Pioneer Who Became an AI Doomer: What Changed Geoffrey Hinton‘s Mind About the Risks of Advanced AI

Geoffrey Hinton is widely recognized as the "Godfather of AI" for his groundbreaking work on neural networks and deep learning that enabled the development of today‘s most powerful artificial intelligence systems. So when Hinton began sounding the alarm recently about the risks posed by the rapid advancement of AI, it sent shockwaves through the tech world.

Hinton spent over four decades working to make machines more intelligent, including the last decade at Google, where he played a key role in developing the foundations of breakthrough AI chatbots like ChatGPT. But in May 2022, the Turing Award winner abruptly left his position at Google so that he could speak more freely about his growing concerns that artificial intelligence is progressing faster than expected—perhaps too fast for humanity to maintain control over it.

"It is hard to see how you can prevent the bad actors from using it for bad things," Hinton told the New York Times, referring to advanced AI technologies. While stressing that his views have been misrepresented as calling for a complete halt to AI development, which he believes is not possible or advisable, Hinton urged that much more effort needs to go into keeping AI "under control" as it rapidly advances.

The AI Breakthroughs That Shook Hinton

What triggered Hinton‘s transition from AI optimist to self-described "AI doomer" were two recent experiences that opened his eyes to how fast the technology he helped create is progressing.

The first wake-up call came a few months ago, when Hinton decided to test the reasoning abilities of Google‘s large language model PaLM. On a whim, Hinton made up a joke and asked the AI to explain why it was funny. To his astonishment, the model responded with a clear, cogent explanation of the joke‘s humor and wordplay. This was an impressive feat of abstract reasoning that Hinton and other experts believed machines were still many years away from mastering.

Hinton‘s second sobering insight was the realization that his long-held assumption—that AI would need to match the staggering complexity of the human brain before it could rival human intelligence—was likely wrong. Models like PaLM and GPT-4, while large and sophisticated, are still vastly simpler than the brain. Yet they are already exhibiting reasoning skills on par with humans. Hinton concluded that as these AI architectures continue to scale up in size and training, they could blow past human-level intelligence within just a few years.

The Growing Chorus of Concern

Hinton is far from alone in his newfound apprehension about the trajectory of AI. In March 2023, a group of prominent tech leaders and AI researchers, including Elon Musk and Apple co-founder Steve Wozniak, signed an open letter calling for an immediate pause on training AI systems more powerful than GPT-4. The letter warned that AI labs are locked in an out-of-control race to develop and deploy ever-more-powerful digital minds that no one, not even their creators, can fully understand or reliably constrain.

Other renowned AI pioneers have also begun to share Hinton‘s sense of alarm. Yoshua Bengio, who received the Turing Award alongside Hinton in 2018 for his own pivotal deep learning research, recently told the New York Times that "I basically agree with [Hinton‘s] concerns" and that humanity may have only a couple of years to get AI governance and oversight right before it‘s too late.

Amidst this growing chorus of expert concern, governments and regulators are scrambling to assess the risks and implement guardrails on the development of artificial intelligence. The European Union is currently hammering out the world‘s first comprehensive regulations on AI systems, which could include strict limitations on the most powerful models. China has also signaled its intention to reign in its booming AI industry and ensure the technology does not threaten social stability or Chinese Communist Party control.

Near-Term and Existential Risks

So what exactly are Hinton and other experts so worried about? In the near term, they point to dangers like the use of AI chatbots and image generators to produce ever-more-sophisticated disinformation, propaganda, and online manipulation. As these models become more coherent and believable, it may get increasingly difficult to discern AI-authored content and narratives from authentic human expression.

There are also economic risks to consider, as AI begins to encroach on higher-skill cognitive labor and creative fields previously thought to be automation-proof. While Hinton believes AI will ultimately be a boon for productivity and abundance, he acknowledged it could cause significant short-term job disruption that will require major societal adaptations and safety nets.

But it‘s the longer-term, even existential risks that most trouble AI doomers. Many worry about the advent of artificial general intelligence (AGI)—AI systems that can match or exceed human intellect across a wide range of domains. An AGI that is not perfectly "aligned" with human values and interests, critics fear, could be catastrophic for humanity, either intentionally or accidentally.

The classic thought experiment is an advanced AI system that is ordered to manufacture as many paperclips as possible. With no other ethical constraints and an intelligence far beyond our own, such an AI might decide the best way to maximize paperclip production is to transform the entire mass of the Earth, and then the cosmos, into paperclips and paperclip factories, consequences for humanity be damned. While a simplistic example, it illustrates the difficulty of controlling a superhuman intelligence that may not share our values, goals, and frailties.

Counter-Arguments and the Need for Balance

Not everyone shares the AI doomers‘ sense of urgency and existential angst. Many researchers and developers argue that while risks are real and worth taking seriously, we still have ample time to solve key challenges around AI interpretability, controllability, and value alignment before any "intelligence explosion" or runaway AGI scenario.

Some also contend that focusing too much on speculative doomsday fears could distract from the very real societal problems that AI is well-poised to help solve in the near term, from addressing climate change to battling disease to democratizing access to knowledge and opportunity.

Hinton himself has sought a middle ground, urging that while AI development should continue full steam ahead, we need to put equal effort and ingenuity into making sure the technology remains safe and beneficial as it grows in power and reach. "We need guidelines that say we‘re not going to build AIs where we don‘t understand how they work or that can be aimed at bad things."

The Path Forward

As the world grapples with the implications of increasingly advanced AI, there are no easy answers. But Hinton‘s evolution from AI optimist to concerned skeptic underscores the urgent need for scientists, policymakers, ethicists and the broader public to work together on a shared vision and framework for responsibly developing this transformative but highly-complex technology.

Some potential steps include:

• Developing global standards and protocols around AI safety, testing, and transparency, with clear circuit breakers in place if things go wrong. No company or country can tackle this alone.

• Expanding interdisciplinary research on AI interpretability, controllability, robustness and value alignment, so we can better understand and steer these systems as they advance. Technical solutions alone likely won‘t be enough.

• Investing in education and public awareness around both the potential and risks of artificial intelligence, so that citizens and institutions can meaningfully participate in charting the technology‘s future.

• Diversifying the AI field to include more voices and perspectives beyond just technical experts, from ethicists and social scientists to community advocates and citizen juries.

• Incorporating stronger incentives for safety and ethics into the AI development process itself, such as tying access to compute resources and funding to responsible practices.

While we should not downplay the potential civilizational benefits of artificial intelligence if developed wisely, Hinton‘s change of heart is a stark reminder that we cannot be complacent about the risks and challenges ahead. The decisions we make around AI in the coming years may well determine the long-term fate of our species. We owe it to ourselves and future generations to act with wisdom, humility, and uncompromising commitment to keeping this incredible technology firmly in service of humanity.