Leaders of major artificial intelligence companies told the UN Security Council that governments must establish safeguards for the rapidly advancing technology, warning that poor management could threaten humanity and leave people unable to control its consequences.
Dario Amodei, chief executive of Anthropic, said AI could become a risk to humanity as a whole if it is managed badly. OpenAI chief executive Sam Altman warned that people could lose control of the future to AI.
The executives argued that countries should act through international institutions to prevent AI from becoming too powerful to restrain. They also urged governments to stop control of the technology from gathering in a single company or country, while stressing its potential to improve people's lives and give individuals more power over their own decisions.
British Foreign Secretary Ed Miliband said the warnings showed that AI safety could not be left solely to the companies developing the systems. He said the United Kingdom would place AI control at the centre of discussions when it leads the G20 next year, with the aim of finding common global standards for safe development.
A dispute over global rules
The United States took a different position. White House Science Adviser Michael Kratsios said concerns about AI control should not lead to a pause in development or the creation of a new global governance structure. He said the US wanted allies and partners to share in the technology's benefits.
The disagreement comes as the United States and China compete for leadership in AI. Chinese UN Ambassador Fu Cong said countries should be free to choose the technology they consider appropriate. He warned against forming a technology bloc that pressures nations to take sides or enables unnamed major powers to pursue digital hegemony.
Fu called for greater use of open-source models, an approach associated with China's position on AI development. Across Europe, Asia and other regions, governments and regional organisations are separately working on AI policies. US states are also developing their own rules.
Weapons and human responsibility
The UN debate also focused on whether humans or machines should make life-and-death decisions. UN Secretary General Antonio Guterres had warned the previous day about “killer robots”, a concern that experts and European leaders said was approaching reality.
Amodei urged governments to agree that AI should not be used to build biological weapons. He also called for ways to verify what AI models can do, along with shared standards for testing, misuse and situations in which humans lose control.
French Foreign Minister Jean-Noel Barrot, whose country convened the meeting, compared AI's current development with both the fictional computer HAL9000 in 2001: A Space Odyssey and the beginning of the atomic age. He said international regulation was needed to capture AI's benefits while avoiding its worst consequences.
The issue has an immediate military dimension. Ukrainian President Volodymyr Zelensky said human control over machines was needed soon, warning that AI could begin deciding what happens on battlefields. He said Ukraine had been at war for 4½ years and needed peace before that point was reached.
Kentaro Toyama, an information technology professor at the University of Michigan, said a Russian drone had seemingly used AI to make an autonomous targeting decision that killed 3 Ukrainians last month. Rumman Chowdhury, co-founder of Humane Intelligence, said the central accountability question was who authorised a machine to choose and attack a target, under what limits, and who would answer for the outcome.
US-China discussions
The UN meeting took place as Chinese President Xi Jinping travelled to Washington for a state visit expected to draw attention from the AI industry. US Treasury Secretary Scott Bessent said the 2 countries had discussed a possible notification mechanism for AI incidents affecting national security.
US President Donald Trump had described AI as “super intelligence” and said the United States would reject efforts to control the technology while competing with China. Kratsios was among the few speakers at the Security Council to use that expression.
Conclusion
The UN discussion exposed a sharp divide over AI governance. Company leaders and several governments want shared safeguards and human control, while the United States opposes a new global framework that could constrain development and China calls for open access and national choice.
Frequently Asked Questions
Q. What did AI company leaders ask the UN to do?
They asked countries to establish safeguards, common testing standards and protections against misuse and loss of human control.
Q. Who warned that AI could threaten humanity?
Anthropic chief executive Dario Amodei said poorly managed AI could pose a risk to humanity as a whole. OpenAI chief executive Sam Altman also warned that people could lose control of the future to AI.
Q. What is the US position on global AI rules?
White House Science Adviser Michael Kratsios said concerns about AI control were not a reason to create a new global governance structure or pause further development.
Q. What did China propose?
Chinese UN Ambassador Fu Cong called for countries to choose AI technology freely and promoted open-source models.
Q. Why are autonomous weapons part of the debate?
The discussion includes concerns that AI could make battlefield targeting decisions without sufficient human control. A reported Russian drone decision was linked to the deaths of 3 Ukrainians.
Q. What did the UK say it would do next?
British Foreign Secretary Ed Miliband said the UK would make AI control a central issue when it leads the G20 next year and would seek common global standards.
