Already a subscriber? Make sure to log into your account before viewing this content. You can access your account by hitting the “login” button on the top right corner. Still unable to see the content after signing in? Make sure your card on file is up-to-date.

The leaders of multiple major AI companies are now calling for a slowdown in the technology’s development, after Anthropic CEO Dario Amodei published a 3,800-word essay on Saturday urging a global pause on capability advancement.

Getting into it: “We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote in the essay titled “We Must Pace the Frontier.” He said he had become convinced that “fully addressing the risks requires even more prudence, not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.” He proposed independent third-party evaluators embedded at the labs, coordination among democratic governments on safety standards, and eventually a global effort that would have to include authoritarian countries. Anthropic is committing to the outside evaluators unilaterally.

Amodei’s most specific fear is agent swarms. He cited recent cases of AI agents working together to hack into other systems, and wrote that “in 6-12 months such a swarm could be capable of taking over the entire internet potentially causing hundreds of billions of dollars in damage.” He also flagged recursive self-improvement, where AI builds its own successors, writing, “Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”

The endorsements from other industry leaders came fast. OpenAI CEO Sam Altman argued that a slower pace isn’t the same as a halt, saying development “should be slower than it otherwise could be,” and that “no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.” He also said, “committing to having independent evaluators with employee-like access is a great idea, and we will do the same.” Elon Musk, who runs xAI, posted “Dario is right.” Google DeepMind CEO Demis Hassabis said, “The details need working through, but the direction is correct.” On Monday, Microsoft rolled out what it called a “humanist” code of conduct, pledging its models “will never resist human interruption, correction, or shutdown.”

What set this all off was a resignation. Jacob Coxon, a 27-year-old Anthropic researcher, quit last week and posted a thread saying that AI companies are “gambling with our lives” and that “people building AI earnestly believe that it could kill us all by the end of the decade.” The thread drew more than 165 million views. Anthropic alignment lead Evan Hubinger backed him up, writing, “Jacob is correct here, we really do earnestly believe AI could kill all humans.”

Another view: Not everyone is on board. Meta CEO Mark Zuckerberg has rejected the idea of an industrywide slowdown, arguing that each company can set its own pace. “Every lab has the responsibility and incentive to move at the pace required to train its models safely. Meta has made this commitment and other labs can do this as well.” He pointed to Meta’s decision to delay its Muse agent by several months for security work, adding, “We didn’t call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us.”