The data shows a C+ for Anthropic, a C for OpenAI. The AI safety index, released by an undisclosed agency, claims to measure the industry's commitment to safe deployment. But the numbers tell a story of mediocrity, not leadership. The entire sector is scoring below par, and the market is treating this as a signal of impending doom. I see something else: a methodology that confuses governance theater with technical security.
Context: The Hype of the Safety Index
The AI safety index has become a talking point in both tech and crypto circles. It purports to rank companies on their 'safety commitments'—including transparency, red teaming, and external audits. Anthropic, the darling of 'constitutional AI,' scored a C+. OpenAI, the mass-market leader, scored a C. The narrative is clear: the industry is failing at safety. But as an on-chain detective who has spent years dissecting protocol audits, I know that a rating is only as good as its underlying data. The index does not reveal its scoring methodology, weightings, or sample window. It does not differentiate between 'safety promises' and 'safety outcomes.' In my experience auditing the 0x protocol v2, I learned that a commitment to security is not the same as a secure codebase. The same applies here.
Core: A Systematic Teardown of the Index
Let me apply the same forensic logic I used during the Terra/Luna collapse. The index's core flaw is its reliance on self-reported governance metrics. An AI company can publish a red-teaming policy, but that does not mean the red team actually found critical vulnerabilities. It can promise external audits, but the scope of those audits may be limited to marketing materials. I have seen this pattern in DeFi protocols: high audit scores from unknown firms, followed by a catastrophic exploit. The index is a classic case of 'security theater'—it measures what is easy to measure, not what matters.
First, the lack of transparency. The index does not publish the raw data, the evaluator criteria, or the individual scores for each dimension. Compare this to a blockchain security audit, where every finding is documented with a transaction hash. Here, we have a black box. The rating agency could be a small group of researchers with a specific agenda, or a for-profit firm selling consulting services. Without verifiable data, the index is a narrative, not a fact.
Second, the confusion between governance and capability. The index measures safety policies, not actual safety incidents. A company can have a high governance score while still suffering from high jailbreak rates, data breaches, or hallucination cascades. In my work on the DeFi Summer liquidity stress test, I found that protocols with the highest APY promises often had the worst tokenomics. The same principle applies here: the index rewards marketing, not engineering.
Third, the missing denominator. The index does not compare scores against the actual risk profile of the models. A C+ for Anthropic might be acceptable if its model is inherently low-risk, but a C for OpenAI might be catastrophic if its model is deployed in critical infrastructure. The index treats all models as equal, ignoring the context of use. This is like giving a stablecoin the same security rating as a DeFi leveraged yield farm—it makes no sense.
Contrarian: What the Bulls Got Right
To be fair, the index is not entirely useless. It does highlight a genuine industry problem: AI companies are not transparent enough about their safety practices. The fact that both Anthropic and OpenAI score below an A suggests that even the best players have room for improvement. The index also raises a valid concern about the increasing ties between AI companies and the military. As I noted in my post-mortem of the 2022 crash, trust must be replaced by verifiable code. The index is a step toward forcing companies to disclose their governance, even if the disclosure is flawed.
Moreover, the index has sparked a conversation about the need for third-party auditing of AI systems. This is similar to the early days of smart contract auditing, where the market was flooded with untrustworthy firms. Over time, the industry consolidated around a few reputable auditors. The same could happen for AI safety, driving demand for rigorous, on-chain verifiable audits. In that sense, the index is a catalyst, not a conclusion.
Takeaway: Accountability Requires Verifiable Data
Logic outlives the hype cycle. The AI safety index is a product of the bull market in AI hype, but its methodology is as fragile as a governance token without a treasury. If the industry wants to build trust, it must move beyond self-reported governance scores and adopt the same forensic standards that define blockchain security. Code speaks louder than promises—and until the index publishes its underlying data, I treat it as noise, not signal.
Trust is verified, not given. The next time you see a safety rating, ask for the raw data. Follow the gas, not the narrative. The real story is not that Anthropic scored higher than OpenAI—it's that the entire industry is still hiding behind a curtain of unverifiable claims. The market will eventually learn to discount such ratings, just as it learned to discount protocol audits that could not be replicated on-chain. Until then, I remain skeptical, and I recommend you do the same.