AI pioneer urges global safeguards as Trump rejects control efforts
Professor Yoshua Bengio, co-chair of the UN's Scientific Panel on AI and one of the field's most respected pioneers, has called for urgent international agreement on artificial intelligence safeguards, warning that big tech companies are taking enormous risks with the technology's development.
Speaking to the ABC's 7.30 program from New York, where he is set to address the UN Security Council, Professor Bengio said the future of humanity depends on coordinated global action, not the decisions of a small number of corporate leaders or governments.
His remarks follow US President Donald Trump's speech to the UN General Assembly, in which he labelled concerns about AI a hoax and rejected a joint statement from 22 countries, including Australia, calling for humans to remain in control of the technology.
The US totally rejects any attempt to construct a globalist scheme to control the artificial intelligence being spoken of so much now, Mr Trump said.
Why international cooperation on AI matters
Professor Bengio, who won the prestigious Turing Award for his pioneering work in the field, said the coalition of 22 nations represents a significant step forward.
What happened with this group of countries is extraordinary and it is important, he said. We have to continue in that direction. We need more countries. We need the world to wake up so that we can have influence on our future.
He emphasised that countries like Australia can still make a meaningful difference in shaping AI's trajectory.
The reason I am able to sleep at night and the reason why a country like Australia can make a difference is that we have agency right now, Professor Bengio said. When we form coalitions, when we put our resources together, our minds, our energy, our talent, we can be in the game and we can bring the world into a place that's better aligned with our values, with democracy, and the idea that we should keep our sovereignty in a multilateral world.
US-China dynamics and the risks of competition
The comments come ahead of a meeting in New York between President Trump and Chinese President Xi Jinping, where AI is expected to feature prominently on the agenda.
Professor Bengio acknowledged that American AI companies currently hold roughly a six-month lead over their Chinese counterparts, but argued that competitive pressures should not prevent cooperation between Washington and Beijing.
It is reason enough to sit at a table with the Chinese because, you know, they're humans too. They want to have a life. They want their children to have a life, he said.
What is recursive self-improvement and why does it matter?
Professor Bengio identified recursive self-improvement as one of the greatest dangers posed by AI. This refers to systems where artificial intelligence trains the next generation of AI, potentially leading to machines that surpass human intelligence and become increasingly difficult to control.
Imagine we have an AI that is smarter than us, so it is able to do that research and engineering all by itself, he said. It could design an AI that will be even more powerful than itself and even less controllable and even more malicious.
What will Professor Bengio tell the UN Security Council?
Professor Bengio is expected to urge world leaders to act decisively, warning that the current trajectory is unsustainable.
The forces of competition between companies and countries is not healthy, he said.
His address to the UN Security Council comes at a critical juncture, as governments worldwide grapple with how to regulate a technology that is evolving faster than policy frameworks can keep pace.
This article is based on reporting from the Australian Broadcasting Corporation's 7.30 program.