When the People Building AI, Start Warning Us About AI

The resignation of an Anthropic researcher is another reminder that artificial intelligence belongs in a much wider conversation about systemic risk, human judgement and the technologies we are creating faster than we can govern them.

For years, discussion about artificial intelligence has tended to divide into two camps. On one side are those promising extraordinary gains in productivity, science, medicine and economic growth. On the other are those warning that increasingly capable systems may introduce risks that are difficult to predict and harder still to control.

What is changing is the identity of some of the people sounding the alarm.

Jacob Coxon, a researcher who says he spent the past three years working on pretraining research at both OpenAI and Anthropic, announced his resignation from Anthropic with an unusually stark assessment.

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”

That is not the language of an outside campaigner speculating about a technology they do not understand. It comes from someone who says he has worked inside two of the organisations at the centre of the modern AI race. His argument is not simply that artificial intelligence could become dangerous, but that competitive pressure may be pushing companies to continue increasing capabilities even while some of the people involved believe those capabilities could eventually become an existential threat.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote. “This is not a marketing stunt.”

The race nobody wants to lose

The immediate question is obvious. If researchers genuinely believe artificial intelligence could threaten civilisation, why continue building it?

Coxon addresses that contradiction directly. His description suggests that the problem is no longer purely technological. It is strategic, commercial and geopolitical.

“At Anthropic, the stakes are well-understood, but they are locked in a race to get there first. They believe no one else will act responsibly, so they must do it themselves, despite the risk.”

This is one of the most uncomfortable dynamics in technology policy. Every participant may privately prefer a slower race while simultaneously concluding that slowing down alone would be more dangerous than continuing. The result begins to resemble an arms race in which nobody necessarily wants the destination, yet everyone fears allowing somebody else to reach it first.

Coxon calls this “a hubristic gamble”.

His criticism becomes even more significant because the decisions involved are largely being made within private organisations. Those companies may contain extraordinarily capable and conscientious people, but commercial competition is not the same thing as democratic consent over the level of existential risk society is prepared to accept.

“Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack.”

Superintelligence is not simply a better chatbot

Most people currently encounter AI through tools that write emails, answer questions, generate images or help with programming. That makes it tempting to imagine future AI as essentially the same technology, only more capable.

The concern around superintelligence is considerably broader.

Researchers are contemplating systems that could outperform the best humans across a wide range of intellectual tasks and potentially contribute to the development of even more capable systems. If AI becomes substantially better at AI research itself, the speed of technological progress could change dramatically.

Coxon asks researchers to think about what that moment might actually mean.

“Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?”

The danger does not require an AI system to become angry, conscious or malevolent. A sufficiently capable system could create serious harm through cyber operations, manipulation, autonomous research, infrastructure disruption or simply by pursuing objectives in ways humans did not anticipate.

Capability and control are separate problems.

AI is only one part of the risk landscape

This is where the debate connects directly with the work we have been developing through Lomond Logic.

Artificial intelligence matters enormously, but it sits within a much wider landscape of systemic risks. Human civilisation depends on tightly connected technological, economic, environmental and geopolitical systems, which means risks that appear unrelated can quickly reinforce one another.

A geopolitical crisis can disrupt energy and food supplies. Climate pressure can contribute to migration and political instability. A pandemic can expose fragile supply chains. A major cyberattack can affect banking, transport, healthcare and communications simultaneously.

Artificial intelligence could become another multiplier within that system.

Lomond Logic is being developed around the idea that these risks are better understood when viewed together rather than as isolated headlines. Our geopolitical dashboard helps monitor changing geopolitical pressures, while other dashboards are being developed around earthquakes, volcanoes, asteroid threats, climate change, pandemics, tsunamis and flooding.

We are also looking at slower-moving structural risks including population decline and the growing energy demands of AI data centres. The Security Dashboard adds another layer by examining digital privacy, tracking and online infrastructure risk.

Individually, each dashboard monitors a particular category. Together, they begin to provide something far more useful, a changing picture of humanity’s overall risk environment.

AI could become both solution and threat

Artificial intelligence is unusual because it has the potential to interact with almost every other risk category.

Advanced AI could improve climate modelling, accelerate drug discovery, analyse satellite imagery, identify emerging disease patterns and help scientists understand natural hazards. It may become one of the most powerful tools humanity has ever developed for recognising and managing systemic risk.

The same capability could also accelerate cyber attacks, weapons development, misinformation and political manipulation. It could concentrate power in a small number of organisations while placing enormous additional demands on global electricity infrastructure.

That makes simplistic descriptions of AI as either salvation or apocalypse increasingly inadequate. The more important question is how much capability humanity should create before it understands how to govern it.

Monitoring is not prediction

The purpose of a risk dashboard is not to announce that catastrophe is imminent. It is to make uncertainty visible enough that decisions can be made before a crisis arrives.

An asteroid impact may be extremely unlikely but extraordinarily consequential. A pandemic may be impossible to predict precisely while still producing observable signals. Geopolitical conflicts often develop through months or years of accumulating pressure before suddenly dominating the news.

Artificial intelligence presents a similar problem. Nobody can responsibly say that superintelligence will destroy humanity, but nobody can credibly prove that sufficiently advanced systems could never become dangerous either.

The sensible debate lies somewhere between panic and complacency.

Coxon remains hopeful that coordination is possible. He argues that governments and laboratories may eventually need agreements powerful enough to slow or temporarily restrict increases in model capability.

“I am optimistic about the potential for coordination.”

Whether that coordination arrives before the technology forces the issue remains uncertain.

Seeing the whole picture

The greatest risks facing civilisation do not sit neatly inside separate boxes marked climate, technology, geopolitics, biology or natural disaster. They overlap, amplify one another and evolve at very different speeds.

That is the thinking behind Lomond Logic.

Artificial intelligence may become one of the most beneficial technologies humanity has ever created. It may also become one of the most consequential risks we have ever chosen to take.

Both possibilities deserve to be monitored.

Because before we can manage risk, we first have to see it.