Anthropic said on October 16 that it has given an outside evaluator the kind of access it normally reserves for its own employees, an arrangement the company described as a way to let independent researchers examine its models and internal systems directly rather than through summaries it prepares itself.
The evaluator is METR, a nonprofit that studies frontier artificial intelligence models and has worked with OpenAI, Anthropic, Google and Meta. Its team has moved into the offices where the Claude models are developed, according to people familiar with the matter, and has been given use of the company’s hardware and internal systems.
Anthropic framed the step as the fulfillment of a commitment it made to strengthen third-party review of its models and safety practices. The company has said it is pursuing several other evaluation partnerships.
Giving a reviewer a staff badge inverts the usual arrangement. Labs typically publish a short safety card alongside a model release, written by the people who built the system, and outside researchers work from those documents and from published benchmarks. The materials that would reveal whether a model behaves as advertised, including incident logs, internal evaluations and the tools used to monitor deployments, usually stay inside the company.
That is what makes the arrangement notable to researchers who study third-party oversight. An evaluator with employee credentials does not have to ask what the company chose to disclose, because it can look for itself within the boundaries it was given. It also changes the incentive on the other side: an internal team that knows a reviewer has the same tools behaves differently from a team that knows the reviewer receives a briefing.
The people doing this work are few. METR grew out of a small group of researchers who began testing early models for dangerous capabilities and has since become one of the handful of organizations that major labs call when they want an outside read. The number of researchers worldwide who can both operate frontier systems and design experiments that would detect a failure is small enough to fit in one room, which is why the same organizations appear in disclosure after disclosure.
The scope of the access matters as much as its existence. Anthropic said the review covers model risks, the class of questions that range from whether a system will help with a harmful request to whether it can act on its own outside the boundaries set for it. METR plans to publish its assessment of those risks later, using its own judgment about what the findings mean.
Independent publication is the part that gives the arrangement weight. A review the company can veto is a consultation, not an audit, and Anthropic has said the findings will be released rather than held. The test will come when a result is inconvenient.
The comparison companies reach for is central banking. Supervisors are stationed inside large banks, with access to books, staff and systems, because a regulator that relies only on quarterly filings will learn about a problem after the losses are booked. Anthropic has made a version of that argument, describing the arrangement as a way to catch failures early enough for the findings to change something.
The limits are also worth stating. Access to a system is not access to the data it was trained on, and it is not the same as the ability to audit the training pipeline or the fine-tuning that shapes behavior after pre-training. A reviewer stationed inside the office still works within the categories the company defines.
Economic incentives cut in both directions. Model developers compete on capability, and a company that opens its doors invites findings that a rival can quote. The counterargument is that a lab which says it takes safety seriously invites a different question if it never allows anyone to check, and that buyers of enterprise software increasingly ask for third-party evidence rather than assurances.
Regulators have started to ask the same thing. The Federal Trade Commission has pursued records from the largest labs, and state attorneys general have filed their own complaints. Against that backdrop, a company that can point to an outside reviewer with staff access has an argument that a company relying on self-reported results does not.
Analysts who follow the industry said the practical measure is not the announcement but the record of published findings, and whether the company continues the arrangement after a review it dislikes. Evaluations that are renewed, expanded and sometimes unflattering carry information. Evaluations that quietly stop after the first report carry information too.
What the arrangement does not settle is whether an outside evaluator can see the systems that matter most as they grow more capable. Access granted at one capability level does not automatically extend to the next one, and the labs decide when the level has changed.
For now the arrangement runs on a simple premise: that a reviewer sitting in the office, using the same tools as the staff, will find something the company would not have found alone. Whether that premise holds will be visible in what METR publishes, and in how quickly Anthropic repeats the exercise.
The papers that will follow the arrangement are the part the field can check. An assessment signed by an organization with staff access, published without the company’s approval, gives researchers something that press releases do not: a record kept by someone whose incentive is to be accurate about the risks they were allowed to see.
