AI Agent Safety Gets a $40 Million Bet
AI agent safety is now a funded category. A day after Anthropic researcher Jacob Coxon quit over concerns that AI could kill us all by the end of the decade, two industry veterans sat down to talk about a more immediate problem: keeping AI agents from going rogue inside enterprises. Rune Kvist, an early Anthropic employee, and Rajiv Dattani, former COO of the AI safety research organization METR, believe they have a solution.
The pair launched the Artificial Intelligence Underwriting Company (AIUC), a startup focused on bringing AI agent safety to enterprises and to the companies building AI models and agents. Cursor, Lovable, Harvey, and ElevenLabs are already named as customers.
On Tuesday, AIUC announced a $40 million Series A led by Ribbit Capital, with participation from First Harmonic. The company previously closed a $15 million seed round from Nat Friedman through his fund NFDG, along with Emergence, Terrain, and Anthropic co-founder Ben Mann, among others. That brings total funding to $55 million.
Why Enterprises Hold Back on AI
Kvist and Dattani are brothers-in-law — Kvist is married to Dattani’s sister. Their pitch rests on a problem that goes beyond raw model capability.
“AI is getting smarter at an increasingly rapid rate. The surprising thing about AI is that it becomes harder to adopt and harder to control as AI gets smarter, not easier,” Kvist said.
That counterintuitive dynamic is what stalls deployment in high-stakes settings. As Kvist put it, banks, hospitals, governments, and militaries no longer decline to deploy AI because a model isn’t smart enough. They decline because they’ve made commitments to their own customers about what a system will and won’t do, and nobody can currently guarantee that.
How AIUC Approaches AI Agent Safety
AIUC’s answer is to apply a familiar cybersecurity model to a new set of AI risks. The company has built a third-party audit and certification layer for AI agents, modeled on the widely adopted cybersecurity standard SOC 2.
Using that standard as its muse, AIUC developed a standard called AIUC-1, along with a testing service to validate agents against it. The goal is straightforward: give enterprises an independent assessment of how safe their AI agents are before they buy.
Built With 250 Security and Risk Leaders
To build the standard, AIUC assembled a consortium of about 250 security and risk leaders — the people who actually buy agents. Their input shapes the tests directly.
“These are the people who we meet with on a monthly basis, and the question we ask them is: When you’re buying agents from someone, what would you look for? What are the questions you’d want to ask, and what would you want to see addressed?” Dattani told TechCrunch.
5,000 Tests, a 100-Page Report
The startup then runs an agent through a suite of roughly 5,000 tests to see how it behaves in scenarios involving jailbreaks, hallucinations, and data leaks.
The results produce a roughly 100-page report detailing where an agent performs safely and reliably — and where it doesn’t. Notably, AIUC uses AI agents to run the tests and AI to analyze the data. Humans verify the final audit, Kvist said.
METR, Anthropic, and the Push for Third-Party Evaluators
If the approach sounds familiar, there’s a reason. Dattani’s former employer METR, where he was COO from 2024 to 2025 and remains a board member, does similar testing for the frontier labs. Until recently, that work focused mostly on performance — whether agents can reliably complete tasks. METR was one of the independent research orgs OpenAI used to investigate its Hugging Face incident.
Anthropic CEO Dario Amodei has also recently called for the AI industry to pace frontier development, citing a rapid increase in bad-behavior incidents. In his post, Amodei floated the idea of requiring frontier labs to use embedded third-party evaluators to observe and verify safety, and named METR as one possibility.
AIUC isn’t proposing to embed itself at customer sites, but the overall idea is similar: give enterprises an independent assessment of how safe their AI agents are.
“Here’s where it passes and where you can trust it. And here’s where there’s concerns. You should be aware of those references before you make the decision to buy,” Dattani said.
What AI Agent Safety Means for Buyers
For enterprises evaluating agents, AIUC’s model reframes the buying decision. Instead of trusting a vendor’s own claims, buyers get a third-party report that maps where an agent is dependable and where risks remain.
That matters most in regulated industries, where a commitment to customers about system behavior carries legal and reputational weight. The certification layer is designed to give those buyers a reference point before signing off on deployment.
The Funding Signal
The A-list investor group behind AIUC suggests the market is taking AI agent safety seriously. With $55 million total raised and customers including Cursor, Lovable, Harvey, and ElevenLabs, the startup is positioning itself between the companies building agents and the enterprises being asked to trust them.
Whether that model becomes as standard as SOC 2 for traditional software remains to be seen. But for now, AIUC is betting that the harder AI gets to control, the more enterprises will pay for proof that it behaves.
