by Michael Clements, Executive Director, Business and Human Rights Centre

Last week, executives from some of the world’s largest AI companies told the UN Security Council that their industry urgently needs global regulation to avert dangers that could threaten humanity. We’ve seen this before. Every few weeks, more proof emerges that these systems are dangerous. But the repeated return to accelerated “business as usual” by AI companies after every Code Red tells us we need far more than voluntary slowdowns and discussion of future regulation if we are to avoid catastrophe.

Instead, we need a global moratorium on the development and sale of advanced and autonomous AI models, so-called “frontier AI”, until an international agreement is in place to direct AI companies and their capital away from human harm and towards clear public good.

This might sound dramatic, but this is the only path forward to address the urgency of the moment. Over 60 civil society organisations and experts have endorsed this call for a moratorium.

An expert chorus warns that machine intelligence could wipe out humanity by 2030. But AI’s human rights harms are already upon us. It is already enabling mass fraud, accelerating surveillance and weapons systems, amplifying misinformation, eliminating entry-level jobs and concentrating power and wealth.

We should be sceptical of tech giants’ motives to call for regulation (as they simultaneously pursue US action to prevent regulation and fair taxation in other markets like Europe). Perhaps they know that their incessant hype is unlikely to deliver the promise of untold wealth to investors. Perhaps they want to pull up the ladder and prevent new entrants that might damage their future profits. They certainly want to control the design of the regulation that may emerge after the US mid-term elections.

But none of these ulterior motives weakens the case for urgent and decisive action by the public, governments and investors to institute a moratorium on advanced and autonomous AI systems models, until binding safeguards are in place. This should be grounded in five mutually reinforcing conditions with robust enforcement mechanisms:

1. Independent oversight, assessment and certification of all advanced and autonomous models before release: Independent teams, responding to a plurilateral governmental body, should be embedded within all AI development teams and training programmes. Certification should be approved through models meeting internationally agreed standards, including a ‘Kill Switch’, not the company’s own definitions of safety, and reported publicly to the UN. This mechanism should be modelled on the successful Nuclear Weapons Non-proliferation Treaty with 191 members and the Chemical Weapons Convention that has 193 members, all accepting regular inspections. Both have held up despite many moments of enormous geopolitical tension.
2. Transparency on safety: Public corporate accounts that demonstrate a substantial rebalancing of funds from advanced model development to safety design and assessment. Currently this is massively out of kilter, with estimates that around ten per cent go to safety and more than 90 per cent to acceleration of model development.
3. Human rights and environmental due diligence laws as proposed by UN Human Rights High Commissioner, Volker Türk: This should be coupled with both civil and criminal liability for harms, including for negligence regarding safety for people and the environment. These last months make clear that AI bosses have no claim to innocence through ignorance of harms created for profit maximisation, just as Meta has been shown in court to have been actively pursuing the algorithmic addiction of children to their apps.
4. A ban on development, deployment or sale of dangerous advanced or autonomous models with powerful abilities such as recursive self-improvement, which Amodei warns could outrun our ability to control them.
5. Exclusion of AI models from critical sectors without extensive impact assessment and extra oversight – especially in high-risk sectors such as health, finance, public service and education - alongside weapons and surveillance systems.

In these dark days of narrow nationalisms, this proposal will meet resistance from vested interests. Yet an immediate and indefinite moratorium is the only commensurate response to the threat before us. It reflects no more than is expected from other sectors with the potential for major human rights harm, such as pharmaceuticals, aviation and ‘controversial weapons’. The fact that the tech giants have been able to resist these essential regulatory approaches up to now reflects the extraordinary wealth and power they have accumulated. Their impunity must now be tackled and governments must act on their obligation to protect society’s best interests from a handful of private interests.

Any rapid, international negotiation on AI will be fraught. It will take clear ethics and spine from our most senior political leaders. The world cannot wait for the USA and China to act. Governments across regions, including European states, Brazil, South Africa, South Korea, Canada and members of the G77, should collaborate now to build a robust and inclusive plurilateral agreement and use diplomatic pressure to secure its wider adoption. In parallel, they should create coherent national and regional regulation to protect their publics and create momentum towards global standards.

Too many investors still appear enthralled by the promises of fantastic wealth, though many are beginning to worry about the bubble and existential threat. We need many more investors to institute their own moratorium; and speak out collectively to ask governments to institute robust regulations.

As AI bosses continue to publish open letters warning of these risks, whistleblowers continue to emerge, and civil society works overtime to mobilise public awareness, every alarm bell is ringing.  It’s time to wake up.