Failure PatternDecision layer

The Coral Bricks Cache Blind Spot: Why Agencies Fail With Coral Bricks on Agent Retainers

Symptom: Client invoices show GLM 5.3 input tokens billed at $1.12 per 1M on every agent run, even though the same system prompt and repo context repeat on each call, which means cached reads are not being hit at all. Root cause: Coral Bricks charges per token with free cached reads, but caching only pays off when the request prefix is stable and ordered correctly. Agencies that rebuild the system prompt or shuffle tool definitions on every call pay full input price and never touch the cache.

By InnovaAI ResearchPublished Updated

AI Infrastructure
How do you recognize it?
  • •Client invoices show GLM 5.3 input tokens billed at $1.12 per 1M on every agent run, even though the same system prompt and repo context repeat on each call, which means cached reads are not being hit at all.
  • •Agent loops that pass a 200K-token codebase on every step crawl past the client's latency budget, and the agency blames the model rather than the request shape.
  • •A single client agent quietly burns through the monthly usage estimate because nobody separated cache write tokens at $1.68 per 1M from fresh input tokens.
  • •The delivery team points OpenCode, Codex CLI, and Cursor at different model slugs (coralbricks/glm-5.3-fp4 versus coralbricks/deepseek-v4.1-flash-fast-fp4) with no shared prompt template, so cost per task varies wildly between clients.
  • •A retainer client asks for a fixed monthly price and the agency cannot answer, because it never instrumented token volume per agent session.
Why does it happen?
  • •Coral Bricks charges per token with free cached reads, but caching only pays off when the request prefix is stable and ordered correctly. Agencies that rebuild the system prompt or shuffle tool definitions on every call pay full input price and never touch the cache.
  • •The platform is an inference API, not a managed agent product. Nothing in Coral Bricks enforces a spend ceiling, model routing policy, or per-client budget, so cost control is entirely the agency's responsibility.
  • •Long-context work up to 1M tokens is a capability, not a default. Teams that send the whole repository on every step instead of trimming to the relevant files pay for context they do not need.
  • •Model slugs differ in price and speed, and the agency rarely documents which slug belongs to which client workload, so a cheap research task can end up running on an expensive coding model.
How do you fix it?
  • •Pull the usage breakdown in the Coral Bricks dashboard and split each client's spend into cached reads, cache writes, and fresh input tokens. Anything with near-zero cached reads is a request-shape problem, not a model problem.
  • •Freeze the system prompt and tool schema per client agent, place them at the front of every request, and stop injecting timestamps or session IDs ahead of them. That single change is what turns cached reads from zero into the majority of tokens.
  • •Standardize one model slug per workload type across all client agents (for example coralbricks/glm-5.3-fp4 for coding loops, a flash slug for short research calls) and write it into the delivery runbook.
  • •Before quoting a fixed retainer, run the client's real agent loop for one week and multiply measured token volume by the published per-1M rates, then add a margin line for cache write tokens at $1.68 per 1M.
Open the Coral Bricks courseCoral Bricks decision recordBrowse the Academy

More on Coral Bricks

More for AI Infrastructure