Top AI researchers now warn the risk of extinction from advanced AI is real
Evan Hubinger of Anthropic estimates better than 10 per cent odds that advanced AI could kill all humans within a decade.
Researchers at leading AI companies are warning that artificial general intelligence poses extinction risks to humanity. Evan Hubinger runs alignment science at Anthropic and wrote this week that he puts odds of AI killing all humans within ten years at better than 10 per cent.
Hubinger said his employer has no clear plan for controlling superintelligence. Jacob Coxon quit after three years at OpenAI and Anthropic.
He says neither company is acting responsibly. AI safety researcher David Krueger emailed this week.
He says all development of more powerful AI should stop until the crisis is resolved. "We are like kids playing with a nuclear warhead," Krueger wrote.
"We must be stopped." Researchers fear hyper-autonomous cyber systems could disable power grids, water utilities, hospitals and financial networks.
They could do this faster than humans can defend against them. OpenAI's newest model, Astra, is rated critical for cybersecurity risk.
- better than 10 per cent
- Hubinger's extinction odds within a
- 3 years
- Coxon's tenure
- Anthropic
- Hubinger's employer
- Astra
- OpenAI's newest model
Why it mattersIf researchers building AI can't guarantee they can control systems they create, the stakes are not about job displacement. They are about whether humanity survives the technology.
AustraliaTristan Heywood, an OpenAI researcher who grew up in Sydney, is among the voices warning Australia and the world that AI safety is not a future problem but an urgent crisis now.
✓ Claims checked against the source and corrected before publish. checked 2 d ago
