Silence in the logs is louder than the crash.
On August 25, 2024, OpenAI Codex users logged in to find their usage limits fully reset. No warning. No explanation. Just a clean slate. The official post-mortem blamed three technical issues: image compression inefficiency, auto-title generation overhead, and an unoptimized Computer History feature. But the data tells a deeper story. This was not a random bug. It was a systematic failure in context management—a flaw that will define the next generation of AI tools.
Context: The Black Box of Usage Limits
Codex is OpenAI's flagship coding assistant, priced on a tiered subscription model. Each user gets a monthly token allowance—a virtual budget for model inference. The system automatically deducts tokens for every interaction: code generation, file analysis, even auto-generated conversation titles. The problem? The deduction logic is opaque. Users cannot see which actions cost what. They cannot audit their own consumption. When the system started burning tokens faster than expected, the only signal was a limit reached notification. No diagnostic. No recourse.
OpenAI's engineering lead, Tibo, confirmed three root causes: (1) multiple image compressions in long conversations creating “extra waste,” (2) auto-title generation consuming more than anticipated, and (3) the new Computer History feature—which streams Mac operations into the context—overrunning its budget. The company reset all paid users and promised a “new optimization plan” to boost efficiency. They did not disclose the underlying algorithms.
Core: The Engineering Anatomy of the Bleed
Let me apply the same forensic approach I used in 2018 when auditing a DeFi contract that nearly lost $2.5 million due to a reentrancy bug. The code never lies—it just hides. Here, the three causes are symptoms of a deeper architectural disease.
First, image compression waste. The report states that “when images are many and compressed multiple times, the process generates extra waste.” This is not a minor inefficiency. It suggests a non-linear expansion of visual tokens under repeated compression. Standard compression algorithms (like JPEG or WebP) reduce size linearly. But when you compress a compressed image, artifacts accumulate. In a transformer-based model, each compression step may re-encode the entire image into the latent space, doubling the token count. If Codex uses a “full recompression” strategy—reprocessing all previous images every time a new one is added—the waste scales quadratically with conversation length. A five-image conversation could burn 10x the tokens of a single-image one. Precision is the only currency that never inflates. But here, precision was sacrificed for convenience.
Second, caching failure. Tibo admitted that “some users experienced degraded cache hit rates yesterday.” Caching is the backbone of cost-efficient inference. When a user repeats a query—like asking for a function definition—the system should reuse the precomputed key-value (KV) cache. If the cache misses, it must recompute the full attention, consuming far more tokens. A degraded cache hit rate means more full recomputations. Why did it degrade? The likely culprit is context compression itself. If the compressed representation of a conversation is not deterministic—e.g., it includes a timestamp or random seed—then the cache key changes every time, making the cache useless. The system was effectively working against itself.
Third, auto-title generation. This seems trivial—a small model call to summarize each conversation. But in a high-volume environment with short conversations, that fixed cost adds up. Every new chat triggers a separate inference. If the user starts 10 quick chats, they lose 10 title-generation tokens. The real cost is not the title itself, but the lost opportunity to reuse the context. The system should have batched this into a lightweight, asynchronous call, or reused the existing context. It didn't.
The Computer History feature is the wildcard. It streams screenshots, app states, and web content from the user's Mac into the conversation. This is a continuous data stream. If the tokenization is naive—storing full-resolution screenshots rather than diff-based summaries—the consumption skyrockets. The feature likely lacked a pre-allocated token budget, allowing it to consume without bound. The floor is an illusion; the floor is a trap. Users thought they had a predictable limit, but the floor was actually a trapdoor.
Contrarian: What the Bulls Got Right
Despite the engineering failure, OpenAI's response was strategically sound. A full reset of all paid users—not just the affected ones—is a high-cost signal. It says: “We value your trust over short-term profit.” This is the same logic I saw in 2022 when Terra/Luna collapsed. The team that moved fast to acknowledge and compensate fared better in the long run. OpenAI's reset is a textbook “cost-for-trust” trade.
Moreover, the underlying model—GPT-4o—remains the most capable coding engine on the market. The bugs are in the surrounding infrastructure, not the core inference. Competitors like GitHub Copilot or Cursor may have more transparent pricing, but they lack the raw reasoning power. The “new optimization plan” hints at a shift: perhaps a more efficient attention mechanism, better KV cache design, or a dedicated compression model. If executed well, Codex could emerge more efficient than before.
Takeaway: The Accountability Call
This event is a stress test—not for the model, but for the business model. AI pricing is moving from “pay per token” to “pay per value.” But value is meaningless without transparency. Users need to see where their tokens go. They need dashboards, alerts, and refunds for anomalies. The silence in the logs—the absence of monitoring—is what caused the crash, not the code itself.
OpenAI will fix the bugs. That is easy. The hard part is rebuilding trust in a system where users cannot audit their own consumption. Until that changes, every reset is a bandage, not a cure. And the next bleed will be louder.