The power of AI is worrying senior Labour figures – as is the fear of ‘falling behind the public’

Westminster appears to be waking up to warnings over AI safety, with concerns over its use by extremist groups and bad actors

“I have this sense that, within a few months, AI is going to be all anyone is talking about,” said one cabinet minister, with an ominous stare.

It was Labour conference, with everyone from ministers to activists jubilant at Andy Burnham’s takeover and a bounce in the polls after the change of leader.

But a number of senior Labour figures were privately starting to name the power of AI as the thing worrying them most, alongside the immediate pressures on the cost of living and fragile economy.

One cabinet minister – with a brief unrelated to technology – told the Guardian that dangerous, out-of-control AI was the issue keeping them up at night, fretting about the threat to humanity. It was “not killer robots” causing the worry, they explained, but the idea of a huge amount of concentrated, super-intelligent power potentially ending up in the hands of bad actors.

Another expressed concern that the government was “falling behind the public” in terms of dealing with worries about AI safety – and that it needed to catch up fast.

In his conference speech, Ed Miliband was the most strident about the “daunting” dangers of AI, making the case that politicians could not and would not allow the profits of tech companies “to come at the expense of our children or indeed humanity”.

And it has not taken long for these alarm bells to ring even louder, with the last seven days looking like the week Westminster fully woke up to the warnings.

A flurry of activity started on Monday with Kanishka Narayan, the AI minister, stating it was “likely, if not inevitable” that rogue agents would hack the British state, and saying the UK was looking to strengthen its forthcoming cybersecurity legislation.

It was the main topic of discussion at cabinet on Tuesday, and that was followed by the MI5 chief Ken McCallum warning AI was “supercharging the cybersecurity contest between attackers and defenders”. He said the biggest threat was extremist groups using the technology to unleash bioweapons – toxins used to cause death and disease.

Liam Byrne, the chair of the business, innovation, science and trade committee, said the change of government from Keir Starmer to Burnham was partly behind the increased focus on protecting the UK from AI threats but more still had to be done. “The reality is that the Starmer administration was much too close to the big tech companies,” Byrne said. “And Burnham is not, frankly, so there has been a change of posture. Those things, plus the security services issuing their warning, that is what has changed the weather on this.”

One senior Labour government figure agreed that Burnham’s government was taking AI safety warnings more seriously. “I do feel there has been a shift. Under the last lot, too many of them had drunk the tech-bro Kool-Aid,” he said.

Byrne’s committee held an evidence session this week with civil service leaders in charge of AI security and executives from the key companies – Anthropic, OpenAI, DeepMind and Meta – with a split between them about whether government testing of new models should be mandatory.

He said his committee was “lighting up” the issue, particularly in terms of working with the US to regulate and monitor new models. It had been reported that Donald Trump’s administration had asked AI companies not to share their latest models with the UK’s cutting-edge AI Security Institute for testing, though Narayan confirmed this week that they had managed to retain cooperation.

But Byrne said the government now needed to seize the moment and take the lead on new regulation, particularly if Trump’s Republicans lost control of Congress in the midterm elections to the Democrats.

“I think the government is still caught in a dilemma between how to keep model access, given a capricious US president, and the safeguards that the industry itself now says are essential. It’s make-your-mind-up time, I would say,” Byrne said.

“The companies want to cooperate … It’s not in their commercial interest to issue models which are lethal. Now they too have got to navigate the White House. So I just think this is a moment when the government has actually got to do what is right for the country because Congress is about to change the dynamics on this in the US.”

The UK’s upcoming G20 presidency will be the key moment to push for international action to rein in AI. Geoffrey Irving, the former chief scientist at AISI who spoke at this week’s parliamentary hearing, believes Whitehall concerns are justified. OpenAI and Anthropic’s technology has already shown its hacking capabilities but their models are released with safety guardrails in place that at least mitigate against the threat of malicious behaviour.

However, so-called open source AI models are freely available and can be adapted. Typically, they caught up with the paid-for models such as OpenAI’s within less than a year, said Irving, although they could cause disruption already.

“It’s very likely that open source models are already powerful enough to do serious damage,” Irving told the Guardian, adding that he expected open source technology to reach the capabilities of OpenAI and Anthropic products “soon”. Safety guardrails “either don’t exist or can be trivially removed” from open source models, he added.

The business committee heard this week that AI-enabled bioweapons could become the next serious threat to manifest itself, in the wake of cybersecurity, as models become more powerful and therefore capable of designing novel bio-harms – although experts suggest there is a gap between designing dangerous pathogens and weaponising them.

Concerns over cybersecurity and bioweapons can escalate rapidly to fears about AI wiping out humanity in a variety of scenarios – a concern that is widespread within the AI community, and is shared by Irving.

However, some experts argue that the gove…