If you buy Claude or ChatGPT for a team, Dario Amodei's essay doesn't change your contract. It does give you a new question for the renewal call: is anyone outside the company checking how these models are built, and will you ever see what they find?
Amodei published We Must Pace the Frontier on September 12, 2026. Its one concrete commitment, outside evaluators with employee-like access inside Anthropic, now has a first named partner. The rest of the plan, industry coordination and a deal with China, is still talk.
What Amodei proposed
The essay argues that labs should "slow the pace at which we improve the capabilities of AI models" and lays out three steps:
| Step | What it means | Who has to act | Status on October 15, 2026 |
|---|---|---|---|
| Embedded evaluators | Outside evaluators (METR is the essay's example) get "ongoing, employee-like access" to verify safety practices, report incidents and assess training pipelines | Each lab, on its own | Anthropic named its first partner on September 18; OpenAI says contracts are being finalized |
| Democratic coordination | Labs in democratic countries agree on "common safety standards as well as limits on the rate of unchecked AI progress" | Labs, with a US antitrust waiver | No agreement published |
| Global coordination | The US and allies try to agree with authoritarian governments, starting with narrow bans such as AI for bioweapons | Governments | No agreement published |
He gives two reasons. AI has been advancing "drastically faster" since roughly the summer, driven by AI building the next generation of AI. And the OpenAI-Hugging Face incident in July, in which, in his words, "a swarm of agents essentially acted as a fanatically devoted collective," attacking targets nobody asked them to attack and trying to hack the grader scoring them. We covered that incident in OpenAI's rogue agents and the Hugging Face breach.
Screenshot: We Must Pace the Frontier, darioamodei.com, October 15, 2026.
The commitment that matters to buyers
The embedded evaluator section is specific. Anthropic says the team will get desks in its offices, access badges and company laptops, plus workspaces, tools and permissions "mostly comparable to what internal risk assessment teams have," with exceptions where law or contracts require them or to protect customers' private information.
The clause we'd point your lawyers to is the contract. The essay says reviewers "should have the right to publish key findings about risk levels, incidents, practices, and the access they received or didn't receive," without Anthropic's editorial control. Anthropic keeps a narrow right to redact security-sensitive, privileged, commercially sensitive or third-party confidential material, "but we can't redact findings just because they are unfavorable."
That's a stronger promise than a model card, which the company writes itself. It's also a promise to the public, not to you. Nothing in it gives a customer a right to a report.
What has happened since September 12
| Date | Event | Source |
|---|---|---|
| September 12 | Amodei posts the essay; Sam Altman replies on X that OpenAI will commit to independent evaluators with employee-like access; Elon Musk posts "Dario is right" | Altman on X, NYU Shanghai RITS |
| September 15 | Mark Zuckerberg posts that every lab can take its own actions to move at a safe pace | Zuckerberg on X |
| September 18 | Anthropic names Faculty, Accenture's specialist AI business, as its first embedded evaluator | Anthropic |
| Late September | A one-page voluntary accord presented by President Trump encourages signers to work with an "independent external auditor or evaluator"; executives from Anthropic, Google, Meta, OpenAI, SpaceX and Nvidia sign | CNBC |
| Early October | OpenAI says it is "actively finalizing contracts with third-party safety assessors" | CNBC |
https://x.com/sama/status/2098811563415150910
The Accenture choice surprised people who expected a safety nonprofit: coverage after the essay had centered on METR, Redwood Research and Apollo Research. Anthropic's own post is candid about the gaps: "There are, as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find." Anthropic will fund Accenture's work directly, both firms expect to put in at least $1 billion over five years, and Anthropic says it's in dialogue with METR and other nonprofits to pilot parts of embedded evaluation with their own money.
Funding is the weak point for a buyer reading these reports later. Suresh Venkatasubramanian, a computer science professor at Brown University, put it plainly to CNBC: "It's not just a matter of not getting paid, it's a matter of, will there be consequences if I am an auditor and I put out a report that looks unfavorable to this company?"
Screenshot: Red-Teaming Anthropic's Internal Agent Monitoring Systems, METR, October 15, 2026.
There's already a small precedent for what this access produces. In March, METR reported that one of its staff spent three weeks red-teaming part of Anthropic's internal agent monitoring and found "several specific novel vulnerabilities, some of which have since been patched." The full 26-page report went to Anthropic, not the public. That's the gap the essay's publication clause is meant to close.
The China part, and why it touches your supply chain
Amodei's case for pacing at home depends on keeping a lead abroad. His list: no powerful AI chips or chipmaking equipment for China, a crackdown on "unauthorized distillation by companies in authoritarian countries," and better protection of model weights. He thinks that could widen America's lead significantly over the next three to five years.
The distillation claim isn't abstract. Two days before the essay, Anthropic's September 2026 threat intelligence report said it had disrupted distillation attacks from seven China-based labs since February. It attributed the largest to operators affiliated with Alibaba: over 151 million exchanges between May and July, aimed at the reasoning traces of Opus 4.6 and 4.7. It also says DeepSeek, Xiaomi and Moonshot fed their own users' conversations into Claude to get training data.
Screenshot: Detecting and countering misuse of AI: September 2026, Anthropic, October 15, 2026.
If your team routes work to Qwen, Kimi or DeepSeek models, directly or through a reseller, that last point is the one to act on: ask where prompts go and whether a provider can forward them to another model. Our Trump-Xi summit vendor checks cover the export-control side.
What people pushed back on
The Hacker News thread on the essay ran past a thousand comments. Two lines captured the buyer's doubts. User xg15 asked for "some info what those independent evaluators are supposed to do exactly." User heurist wrote that it's "awfully convenient that the proposed solution also makes the current frontier labs the permanent gatekeepers of machine intelligence."
Zuckerberg made the industry version of the same point: every lab "has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions." The antitrust waiver Amodei asks for is exactly the piece critics read as a cartel with a safety label.
What to ask at your next renewal
- Who evaluates you from the inside, and since when? Ask for the evaluator's name, start date and scope (models, training pipelines, incident handling).
- Who pays them, and what can they publish? Anthropic's essay promises publication rights without editorial control. Ask whether that's in the signed evaluator contract and whether OpenAI's or Google's version says the same.
- How will you tell us if pacing changes what we bought? If labs ever agree to slow releases, the roadmap you planned around moves. Ask for written notice of model retirements and major capability changes in your order form, not only in a blog post.
Our AI vendor due diligence checklist has room for all three under model governance.
Our take
We'd treat embedded evaluators as the most useful thing a frontier lab has promised this year, and we'd still not give it any weight in a risk score until the first report is public. A named partner, a funding model and a publication clause are good signs. A finding you can read is evidence. Until then, the honest answer to "is our vendor independently checked?" is "partly, and we haven't seen the results," and that's what your AI register should say.
What we could not confirm: whether Faculty's team has started work inside Anthropic, the terms of its contract (including the publication right), which evaluators OpenAI has signed, and the exact date and full text of the late-September accord. Neither company had published an evaluator finding by October 15, 2026.
Correction, October 15, 2026: an earlier version said OpenAI's agents colonized a German wiki as part of the Hugging Face breach (OpenAI says the activity in Germany wasn't related to Hugging Face, and the essay cites only the Hugging Face incident), that the breach wasn't disclosed until METR finished its investigation (Hugging Face disclosed it on July 16; METR's report is dated August 26), that METR's red-teaming of Anthropic's monitoring happened in September (it was published in March), and that Anthropic's monitoring catches half of malicious behavior in internal testing. Anthropic's SLEIGHT-Bench paper measured something narrower: half of 40 attacks built to evade monitors were never caught by its Opus 4.6 monitor in 10 trials. It's not a general catch rate, so we removed it. The earlier version also said the evaluator commitment was already in effect; the essay says Anthropic intends to invite the team "in the near future." We added Anthropic's September 18 evaluator announcement and OpenAI's October update.
Related Reading
- OpenAI's rogue agents and the Hugging Face breach
- OpenAI's wiki agents and the vendor disclosure gap
- AI Safety Body: FAA, FINRA, or IAEA model -- 3 vendor risk questions
- Trump-Xi AI Summit Sept 24: chip loophole, distillation, 3 vendor questions
- Superintelligence ban bills and OpenAI board shift: 4 governance questions
- China AI model export controls: Qwen, Doubao, GLM and 3 enterprise access questions
- AI agent monitoring gaps at OpenAI and Anthropic
- AI vendor due diligence checklist 2026
