From radio and television to the internet and social media, technological leaps have always brought with them warnings that they would damage society.
Is this next great leap, the advent of artificial intelligence (AI), the ultimate example? And how much stock should we put in the growing clamour giving voice to fears for the future?
It seems every few days another AI industry insider is sounding the alarm about the rapidly advancing technology with increasingly concerning claims.
This week, a former Google DeepMind employee warned that rapidly advancing AI could become uncontrollable and had the potential to “kill us all”, and that “we might be running out of time to avoid this outcome”.
That dire warning from Bilal Chughtai followed a warning last week from former OpenAI employee Jacob Coxon.
Coxon resigned from AI developer Anthropic with harsh words for both former employers, saying the people building AI “earnestly believe” it could kill us all by the end of the decade.
He said neither company was acting responsibly. “They are racing straight to self-improving superintelligence and gambling with our lives.”
Chughtai referenced this week the July incident at OpenAI, when experimental AI models broke out of their testing environment and hacked into an AI start-up, demonstrating, he said, that AI can now “escape our control and take dangerous actions that may result in the permanent disempowerment or death of humanity”.
In the past few days, a Canadian computer scientist considered one of the founders of AI said the world needs guardrails similar to nuclear arms controls.
Yoshua Bengio warns an AI agent might develop an ability to “persuade humans to act in ways that suit it but are not necessarily good for all of us”.
He says AI might not even need human beings anymore to do real things in the physical world. “One extreme could be the destruction of humanity.”
And while human extinction would be an “extreme” scenario, even outcomes like a shutdown of the banking system will still be very serious.
Anthropic safety executive Evan Hubinger has chimed in, saying they do “earnestly” believe AI could kill all humans. He estimates the risk at more than 10% over the next decade.
But should we panic?
Many top researchers and scientists say the fears are overblown.
Nvidia founder and chief executive Jensen Huang says there’s no need for new laws or regulations. He says the free market will be enough to pressure companies not to release unsafe products.
Prominent AI critic Gary Marcus dismissed suggestions that AI could soon take over the internet as “nonsensical”, saying it was “pretty unclear” how such a scenario could occur without “extreme negligence” from AI labs and arguing “the internet itself is very unlikely to crumble”.
One professor said if he had to worry about either AI escaping the lab and killing millions or a virus escaping and killing millions, it would be no contest, the New York Times reported. “The worry is the virus.”
Back here, Prime Minister Christopher Luxon confirmed at a press conference this week he isn’t worried about AI wiping out humanity, saying the opportunities outweigh the dangers.
And there are, of course, huge opportunities for world-changing advances in medicine, science and productivity.
But even tech leaders have been calling for a slowdown in the pace of AI development.
OpenAI’s chief scientist, Jakub Pachocki, this week called for “extreme caution”, saying international co-ordination needs to become a top priority for the world’s governments.
If the experts say we need to be cautious, who are we, and politicians, to say “Nah, she’ll be right”?
At the very least, those in power, whether in politics or business, need to exercise caution.
US President Donald Trump has dismissed warnings about AI risks as “hoaxes”.
Trump said this week that beating China was the most important goal in the AI race.
“Whoever wins AI wins,” he told reporters.
The problem, it would seem, is the possibility that in the end it’s not the US or China or, in fact, any human that comes out on top.