An artificial intelligence system has estimated the probability of AI causing human extinction within the next 10 years at around 0.85%, equivalent to roughly one chance in 118, according to an experiment conducted by British newspaper The Independent.
The newspaper carried out the test in collaboration with Noah Predict, a system that analyses millions of data points and news reports to identify trends, assess risks and forecast probabilities.
The estimate is substantially lower than some projections put forward by engineers and researchers working on AI safety.
The experiment followed the resignation of AI researcher Jacob Cookson from Anthropic, after he warned that companies were gambling with human lives.
His colleague Evan Hubinger said some people working in the field believe there is a greater than 10% chance that AI could wipe out humanity within the next decade, according to The Independent, citing a post by the expert on X.
Computer scientist Geoffrey Hinton, a Nobel Prize winner in physics and widely known as the “godfather of AI”, later lent support to those concerns, telling the BBC that a 10% probability was “not an unreasonable estimate”, according to the report.
Noah Predict analysed the question from 41 different angles, conducted 108 searches to verify information and drew on around 52 million data points before reaching its final estimate, the newspaper said.
It concluded that the probability stood at 0.85%, below 1%. However, The Independent noted that this does not mean the danger can simply be dismissed, particularly because some potential outcomes could be irreversible.
Risk One: Harmful Human Use
Noah Predict identified the first major risk as the possibility that third parties could use AI to carry out dangerous activities, including cyberattacks or biological attacks, with greater speed and efficiency.
However, the system stressed that the machine itself does not possess hostile intentions and remains dependent on a human operator.
Risk Two: Institutional Breakdown
The report also identified the risk that companies could use AI to reorganise labour markets at a pace faster than workers can retrain or transition into new jobs.
The threat could become more serious if institutions grow so dependent on automated services that they lose the ability to function independently in the event of a technical failure, cyberattack or incorrect automated decision.
Risk Three: Humans Lose Control
The third and most concerning scenario, according to the automated system, is one in which human control becomes largely symbolic.
This could occur if AI systems are designed to pursue a particular objective and begin to treat human intervention as an obstacle, continuing to pursue that objective regardless of attempts to stop or correct them.
Noah Predict proposed several measures to reduce these risks, including limiting the authority granted to AI systems, preventing them from accessing critical infrastructure, subjecting them to adversarial testing and continuously verifying shutdown and control-recovery procedures.
The Independent noted that the three scenarios could also overlap. Labour market disruption, for example, could contribute to political instability, which in turn could weaken humanity’s ability to maintain oversight of increasingly advanced systems.
According to the newspaper, the central conclusion of the experiment was not that AI is unlikely to destroy humanity, but that there remains wide disagreement over how large the risk actually is.
