Open-Weight AI Safety Gets a New Standard
Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednesday, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastructure for open-weight models. The open-weight AI safety partnership arrives at a moment when the risks surrounding freely downloadable models are becoming harder to ignore.
The announcement lands amid debate over the safety of open-weight models, which can be made dangerous by removing their safeguards through a rising technique known as abliteration. The scale of the problem is massive: Hugging Face, which hosts open-source AI models, currently lists over 6,000 abliterated models.
Base Labs, the research group Baseten spun up earlier this year, will develop and publish methods for training and monitoring open models. The company is framing its future work as a “standard” for open models that is transparent and built into how models are trained and deployed, rather than bolted on afterward.
“We believe openness to be an advantage for AI safety,” the company said on X. “Openness provides more visibility into the behavior of models and, most importantly, greater means of turning safety research into actionable and transparent controls than closed-source.”
What the Partnership Will Actually Build
The companies haven’t disclosed how the partnership will work technically, though Goodfire framed the goal in a reply to Baseten’s post: “Safety must be built into open models and provided by those who serve them.”
Goodfire, which specializes in opening AI’s “black box” to explain how models make decisions, is the likeliest candidate for the “built into” part of that vision. Its interpretability work could help Base Labs move beyond surface-level guardrails toward safety mechanisms embedded in model training itself.
Baseten, an AI inference provider, raised a $1.5 billion Series F in June, vaulting its valuation to $13 billion. Partner Goodfire AI is similarly well-capitalized, having raised a $150 million Series B led by B Capital earlier this year to advance its model interpretability platform.
Why Abliteration Is Driving the Urgency
Abliteration refers to the process of stripping safety safeguards out of an open-weight model, leaving behind a system that can be used for harmful purposes. Because open-weight models can be downloaded and modified freely, once safeguards are removed, there is little the original developer can do to restore them.
That dynamic is what makes the open-weight AI safety problem different from safety work on closed models. With closed systems, developers control access and can patch vulnerabilities centrally. With open weights, the model itself circulates beyond any single company’s reach.
Hugging Face’s catalog of more than 6,000 abliterated models illustrates how quickly the problem has scaled. Each one represents a model that has had its safety training deliberately stripped away.
A Standard, Not a Patch
Base Labs is positioning its work as a standard rather than a fix applied after the fact. The distinction matters: bolted-on safety measures can be removed by anyone with access to the weights, but safety that is part of how a model is trained and deployed is far harder to strip out.
The company is also putting out an open call to the broader developer ecosystem to contribute to the framework. “Together, we are building an ecosystem of open models that are safe and accessible to all,” the company noted.
That open call signals that Base Labs does not intend to define the standard alone. A framework for open-weight AI safety that only one organization controls would run against the transparency the effort claims to pursue. Bringing in outside developers early could make the resulting methods more durable and more widely adopted.
Who’s Behind the Effort
Baseten is an AI inference provider that raised a $1.5 billion Series F in June, bringing its valuation to $13 billion. Base Labs is the research arm the company spun up earlier this year to focus on open model safety.
Hugging Face is the hosting platform for open-source AI models, and its catalog is where the scale of the abliteration problem is most visible. Its role in the partnership gives the standard a distribution channel that reaches the developers most likely to use it.
Goodfire AI specializes in model interpretability, working to explain how AI models arrive at their decisions. It raised a $150 million Series B led by B Capital earlier this year to advance that platform.
What Comes Next
The partnership has not yet detailed its technical approach, and no timeline has been given for publishing the first methods or standards. What is clear is the intended shape of the work: evaluation and monitoring infrastructure that lives inside how open models are trained and served, backed by an open call for outside developers to contribute.
For teams deploying open-weight models, the practical question will be whether a transparent, built-in standard can keep pace with techniques like abliteration that are already operating at scale. The answer will depend less on the announcement itself and more on what Base Labs, Hugging Face, and Goodfire publish in the months ahead.
