Portkey

Every AI-inclusive retainer an agency signs is a bet on third-party compute pricing it does not control. As frontier model access commoditizes, the agencies that win are the ones that treat model routing, caching, and self-hosted inference as delivery infrastructure rather than as line items on a client invoice. The strategic question is no longer which model to use; it is how much of the inference stack you own before a provider repricing event eats your margin.

Inference Cost Pass-Through Ceiling is the point at which an agency can no longer absorb a model provider's price or latency change inside a fixed retainer, so the cost has to move to the client or the work has to shrink. The framework asks three questions per client engagement: what share of delivery cost is metered inference, how fast can that share be re-routed to a cheaper model, and what contract language lets you reprice. Forrester's 2027 predictions flag AI growth colliding with energy and infrastructure limits, which converts compute scarcity into API price movement on agency tools. A concrete case: an agency running document analysis on a frontier API can shift bulk classification to a smaller open-weight model served through Ollama or a gateway like Helicone, keeping the frontier model only for reasoning steps. That split is the ceiling defense.

Provider Substitution Window is the interval during which an agency can move a client workload from one model provider to another without rewriting prompts, evals, or integration code. The window is widest at the orchestration layer and narrowest at the fine-tuned weights layer: a gateway swap takes hours, a retrained model takes a quarter. Agencies that measure this window per client account know exactly when they hold pricing leverage and when a vendor holds it. Forrester's 2027 predictions flag compute and energy constraints pushing API pricing upward, which turns a wide substitution window into a margin defense rather than an engineering nicety. A concrete case: an agency routing Claude and GPT traffic through a gateway such as Helicone or Portkey can shift a client's summarization workload in an afternoon when one provider raises rates, while a competitor with hardcoded SDK calls absorbs the increase on a fixed retainer.

Margin Defense Stack treats AI infrastructure as a layered cost structure rather than a single line item. The bottom layer is raw compute and API tokens, the middle layer is routing and caching, and the top layer is the client-facing retainer price. Agencies that only negotiate the top layer absorb every shock from the layers beneath. Forrester's 2027 predictions flag that AI expansion is colliding with energy and infrastructure limits, which translates into API price increases for agency tools and compresses margins on AI-inclusive retainers. A concrete defense: route repeat prompts through a gateway such as Helicone or Portkey so cached responses cut token spend before it reaches the client invoice, and keep a local fallback like Ollama for privacy-sensitive work. When a client asks why the AI retainer costs what it does, the stack shows exactly which layer each dollar covers.

Map every AI dependency in the delivery stack to a named provider, a fallback route, and a pass-through cost clause before quoting fixed-fee client work.

Put a routing or gateway layer between your application and every model provider before any client deliverable depends on one vendor's endpoint.

IF client work spans more than one model family, more than one pricing tier, or more than one data-residency requirement, THEN route every request through an orchestration layer so a provider price change or capability shift becomes a routing edit rather than a rebuild. IF a single provider's model is the product itself and switching cost is already sunk into fine-tunes and evals, THEN a direct integration is cheaper and simpler than adding a gateway. The frame is not which vendor wins; it is whether the agency owns the routing decision or rents it.

The routing layer, not the model, is where agency margin is decided: a gateway that logs spend and swaps providers turns a pricing change from a renegotiation into a config edit. Observability-first tools suit teams that need to see cost before they can control it, while routing-first and hosting-first platforms suit teams already committing client builds to production. Match the layer to how many client systems you operate, not to which model currently benchmarks highest.

A delivery pattern for agencies that stand up a provider-agnostic routing and observability layer between client applications and frontier model APIs, so pricing changes, deprecations, or safety-policy shifts at any single lab become a config edit rather than a rebuild.