The Price of Slowing Down

A few days ago we were asking why the people building artificial intelligence were continuing to race towards something some of them apparently believe has a meaningful chance of killing us. Apparently Dario Amodei had been wondering much the same thing. The Anthropic chief executive has now called for frontier AI development to slow down, beginning with independent evaluators, greater coordination and, ultimately, international agreements capable of constraining the development of particularly dangerous capabilities.

He is not alone. Altman, Musk and Google DeepMind's Demis Hassabis have supported Amodei's call, while OpenAI and Anthropic have committed to independent evaluation. Zuckerberg, on the other hand, doesn't think coordinated slowing is necessary, while Nvidia's Huang has also pushed back against additional constraints that might weaken American competitiveness. President Trump certainly doesn't appear inclined to sacrifice America's technological advantage over China, while China is understandably suspicious that an American-led attempt to slow the global AI race might have something to do with preserving America's lead in it.

All of which is useful, because we no longer need to construct a hypothetical Prisoner's Dilemma around artificial intelligence. One is beginning to play out in front of us, and the interesting question is rapidly becoming what it will cost anybody to do something about it.

What Are We Actually Prepared to Pay?

Almost nobody is arguing for unsafe artificial intelligence, which would be a difficult position to market. The disagreement begins when we ask how much effort anybody is actually prepared to devote to making it safer and, ultimately, what cost they are prepared to incur.

That is surprisingly difficult to see. We have extraordinary visibility over the money flowing into greater AI capability: hundreds of billions of dollars being invested in chips, data centres, energy, models and the infrastructure surrounding them, together with the revenues, valuations and strategic advantages expected to follow. We have nothing like the same public visibility over the resources devoted specifically to ensuring that increasing capability remains safe. That doesn't mean the effort isn't substantial, and it would be foolish to invent some ratio between capability and safety spending that we cannot observe. But the absence of that visibility is interesting when some of the people closest to the technology are simultaneously attaching extraordinary probabilities to what happens if safety fails.

Amodei's proposed slowdown therefore gives us another way of looking at the question. Instead of asking companies and governments how seriously they take AI risk, we can ask what they are actually prepared to give up to reduce it. For a company that might mean delaying the next model, postponing revenue, accepting lower returns or risking market share. For a country it could mean surrendering some of the economic, scientific, military and strategic value associated with technological leadership.

The seriousness afforded to a risk is revealed not simply by what people say about it, but by the effort they devote to reducing it and the cost they are prepared to incur. So what is the price of slowing down?

The Visible Price

Artificial intelligence is now driving one of the largest investment cycles in economic history. BofA Global Research estimates that five hyperscalers will spend around $795 billion in total capital expenditure this year and nearly $1.08 trillion in 2027, with much of it directed towards AI infrastructure. The Bank for International Settlements separately estimates that the five largest technology companies will invest more than $1 trillion in AI-related capital expenditure across 2025 and 2026, while industry forecasts cited by the BIS suggest global AI investment could eventually reach as much as $4 trillion a year by 2030. Whichever perimeter we choose, we are no longer talking about an interesting research programme. Around the frontier laboratories now sits an enormous industrial system of semiconductor manufacturers, cloud providers, data-centre developers, electricity companies, software businesses and investors whose expectations increasingly depend upon the AI boom continuing.

Slowing frontier development does not mean stopping that machine. Amodei himself is explicit that pacing does not mean halting model training or technical progress. Existing models remain extraordinarily capable, applications continue to be developed, data centres continue operating and some resources could simply move from pushing the capability frontier towards extracting more value from what already exists, improving efficiency and doing rather more of the safety work everybody says is necessary. Cutting the rate at which we push the frontier is not the same thing as stopping artificial intelligence.

There is nevertheless a genuine price. More capable products arrive later, some productivity gains and scientific benefits are delayed, revenues move into the future and investors wait longer for returns. But the price becomes considerably larger when restraint is unilateral. If Anthropic slows while OpenAI, Google, Meta and the Chinese laboratories continue at full speed, Anthropic has not simply postponed some future revenue. It risks surrendering competitive position while producing a safety benefit shared by everybody, including the companies that carried on.

Exactly the same logic applies to countries. Amodei acknowledges that the amount by which democratic countries can slow is constrained by whatever technological lead they possess over China. The interesting measure is therefore not simply whether America would surrender six or twelve months of development, because time itself isn't the cost. The question is what those months are worth in economic output, scientific capability, military power and strategic advantage.

And that is where cooperation changes the economics.

The Cheapest Way to Buy Safety

Suppose Anthropic slows and nobody else does. Anthropic carries the commercial cost while whatever safety benefit results from its restraint is shared by everybody. If all the major laboratories accept the same constraint, more capable AI still arrives later and there is still an opportunity cost, but much of the competitive penalty disappears. Anthropic no longer loses ground to OpenAI if OpenAI is operating under the same rules.

The same applies internationally. If the United States slows while China continues, the cost includes whatever strategic advantage America might surrender. If both countries accept the same credible constraint, they still incur the opportunity cost of reaching the next frontier later, but neither necessarily loses relative position. Indeed, this is close to the economic logic underlying Amodei's proposal: coordinated pacing could create additional time for alignment and evaluation without forcing an individual company or country to sacrifice the competitive position it would lose by acting alone.

Cooperating on safety makes the pursuit of safety cheaper.

That seems embarrassingly obvious, but it is the heart of the problem. Unilateral restraint asks the responsible participant to carry a private cost while sharing the benefit with everybody, including those who refused to pay it. Common restraint spreads the cost and removes much of the competitive advantage of refusing to participate.

Unfortunately, it also creates a very large incentive to cheat. A company that secretly pushes ahead while its competitors observe the constraint could acquire an enormous commercial advantage, while a country doing the same thing could acquire something considerably more consequential. So cooperation requires more than everybody agreeing that safety would be nice. It requires common rules, independent evaluation, monitoring and some credible means of identifying defection. That is why verification sits at the centre of Amodei's proposal, beginning with independent evaluators inside the frontier laboratories and eventually extending to international coordination.

Trump and Xi are due to meet later this month, with AI expected to feature in the discussions. Perhaps they will agree to slow the race together. I wouldn't bet my last pound on that outcome - and I only have one. But the economics of why cooperation matters go well beyond whether those two gentlemen happen to get along.

The Price We Don't See

Markets reacted quickly when the calls for slower AI development emerged, with AI-linked technology and semiconductor stocks falling as investors reconsidered the investment outlook. There is nothing particularly surprising about that. Markets can see the price of slowing down rather well: delayed revenues, deferred expenditure, lower or later expected returns and the possibility that a company loses competitive position. What they cannot see nearly as clearly is the price of going faster.

Artificial intelligence could create enormous social benefits, and I think it probably will. But increasing capability also creates risks extending far beyond the companies developing the technology and the customers buying it. Amodei himself identifies cyberattacks, biological misuse, economic disruption and loss of control among the principal concerns. Other risks arise from the way increasingly autonomous systems interact with financial markets, information systems and shared infrastructure, while at the far end of the distribution sit the catastrophic and existential possibilities that started this discussion in the first place.

The economic asymmetry is fairly straightforward. The company developing the additional capability receives the revenue, its shareholders receive the increase in value and its employees receive salaries and options, while the country in which it operates receives investment, productivity, tax revenues and strategic capability. If the same increase in capability also increases systemic risk, however, much of that risk is carried by everybody else, including people who never bought the product, never owned the shares, may not live in the country and certainly weren't consulted about how quickly the technology should develop.

That's the externality, and it changes the question from whether faster AI creates economic benefits - it plainly does - to whether the people deciding how fast to go are actually facing the full economic and social consequences of that decision.

Who Is Paying for the Speed?

Zuckerberg has provided a useful counterargument. His position is essentially that competition and liability already give AI companies sufficient incentives to develop safely, pointing among other things to Meta delaying the release of its Muse agent for additional security work. If he is right, much of the externality problem is already being internalised because companies know that unsafe behaviour can damage their reputation, competitive position and ultimately expose them to liability.

Liability can certainly help. If a company knows that it will have to compensate people for the damage it causes, some of the external cost becomes a private cost. Insurance premiums rise, unsafe behaviour becomes more expensive and the price signal begins to improve. The problem is that this works best when we can identify the damage, identify who caused it and compensate whoever suffered it. The more distributed and systemic the harm becomes, the harder each of those things gets.

If thousands of autonomous agents collectively destabilise a financial market, who caused the loss? If an AI-generated vulnerability propagates through shared infrastructure, how should responsibility be divided between the developer, deployer, user and the systems with which it interacted? If the effects spread between countries and across markets, who pays whom? And if the ultimate concern really is human extinction, liability becomes a particularly unsatisfactory mechanism for pricing it. There would, after all, be nobody left to sue.

That sounds flippant, but the economic point is serious. The larger, more distributed and more irreversible an externality becomes, the less capable conventional liability is of forcing the organisation creating the risk to carry its full expected cost. The resulting price signal is therefore incomplete, and people and companies respond to the prices they actually face rather than the ones missing from their accounts.

We Have Seen This Before

Which is why none of this is really a new economic problem. We spent decades arguing about the cost of reducing carbon emissions because the cost of changing the energy system was highly visible. Power stations cost money, cleaner technologies cost money, industrial processes had to change and some assets became stranded. The cost of changing the atmosphere largely sat somewhere else.

For much of the industrial era we therefore compared the visible financial cost of avoiding environmental damage with an economic baseline in which a significant part of that environmental damage carried little or no price. Unsurprisingly, avoiding it frequently appeared expensive. AI risks reproducing exactly the same mistake. If the accounts recognise the cost of restraint but omit a material part of the cost of acceleration, the price signal will favour acceleration whether or not it creates the greatest overall value.

That difference matters because when the accounts omit material consequences, the resulting price signal can reward behaviour that destroys value somewhere else. And it is here that cooperation becomes economically important for a second reason.

If one participant slows alone, it carries the private cost, potentially hands an advantage to its competitors and then shares the resulting safety benefit with everybody, including those competitors. If everybody accepts the same constraint, the underlying opportunity cost remains but much of that competitive-transfer cost disappears. And if the constraint is sufficiently broad, something else happens: part of the external cost of acceleration is finally brought back onto the people benefiting from it.

Cooperation therefore does more than divide the bill. It changes the economics of behaving responsibly. Instead of asking companies or countries to choose between responsibility and competitiveness, a common floor makes responsibility part of the conditions of competition itself.

The Old Problem, Accelerated

This is where I think artificial intelligence becomes more interesting than artificial intelligence. The underlying problem appears whenever individual actors can benefit from behaviour whose wider costs are carried by the system around them. Climate change has it, biodiversity loss has it, fisheries have it, financial markets have it, tax competition has it and arms races certainly have it. Everybody may benefit from a common constraint while every individual participant retains an incentive to escape it.

Artificial intelligence has simply brought that problem into unusually sharp focus. The potential benefits of acceleration are enormous, the rewards for getting ahead are immediate, the competition is global and some of the people closest to the technology are simultaneously telling us that the downside could be catastrophic. If ever there were a useful laboratory for testing whether humanity can solve a collective-action problem before experiencing the consequences of failing to solve it, this may be it.

The answer is not to decide centrally which company should win, which country should lead or what artificial intelligence should ultimately be used for. Nor is it to remove competition. The objective should be a sufficiently strong common floor within which companies, countries and individuals remain as free as possible to compete, experiment and create. Above that common floor they should be encouraged to build better models, find better applications, create companies, make money, cure diseases, increase productivity and explore whatever this extraordinary technology makes possible. The purpose of the common constraint is not to suppress that freedom, but to ensure that behaving responsibly does not become a competitive disadvantage.

A level and sufficiently safe playing field does not reduce freedom. It establishes the conditions within which freedom and competition can coexist.

That was really the lesson from the coral reef too. The objective wasn't to determine what every organism should do, but to preserve the viability of the system within which they remained free to do it. AI presents the same problem at extraordinary speed, and perhaps that is why the implications extend well beyond AI. If we can establish common boundaries here, not because everybody suddenly becomes altruistic but because cooperation changes the economics of protecting the common system, the same logic should bleed into many of the other problems we have spent decades failing to solve.

We already know that individually sensible behaviour can collectively destroy value, that external costs do not disappear because our accounts fail to record them, and that common rules can remove the competitive disadvantage of doing the right thing. The difficult bit has always been getting everybody inside the same boundary.

Artificial intelligence hasn't invented that problem. It has simply compressed it into a race moving quickly enough, involving enough money and carrying sufficiently serious potential consequences that we may finally be forced to confront what the economics has been telling us all along: cooperation isn't simply the nicer option. Sometimes it is the cheapest one.

Disclosure: This essay was developed with the assistance of artificial intelligence. The author remains responsible for its arguments, sources and conclusions.