MyIP

यह पृष्ठ अभी तक हिन्दी में अनूदित नहीं है। अंग्रेज़ी मूल दिखाया जा रहा है।

Prompt caching

Not paying twice for the same prefix

Some inference engines can reuse the KV-cache computed for a previous request's prompt, so a follow-up request that repeats the same opening tokens does not pay full price to recompute them. When that happens, the reused portion shows up as cached tokens, billed at a lower rate.

How it's billed

prompt_tokens already includes the cached portion; it is not a separate count. We subtract it back out and price it separately:

billable_prompt = prompt_tokens - cached_tokens

cost = billable_prompt × prompt_price
     + cached_tokens    × cache_read_price
     + completion_tokens × completion_price

If a model's catalog entry has no published cache-read price, cache hits are priced at 10% of the prompt price automatically — see How costs are calculated. You never need to look this rate up yourself; it is already baked into the sale price you can query on GET /models:

bash
curl -s "https://openrouter.myip.co.kr/api/v1/models" | jq '.data[] | select(.id=="lgai/exaone-4.0-32b") | .pricing'
json
{
  "prompt": "0.000030000000",
  "completion": "0.000150000000",
  "input_cache_read": "0.000003000000",
  "input_cache_write": null,
  "currency": "KRW"
}

Cache write tokens (the ones spent establishing a new cache entry) are recorded in usage_records for visibility but are never billed — upstream engines don't report cache-write counts consistently enough for us to charge for them safely.

Reading how much a request saved

The usage object includes a prompt_tokens_details breakdown:

json
{
  "usage": {
    "prompt_tokens": 10339,
    "completion_tokens": 60,
    "total_tokens": 10399,
    "prompt_tokens_details": {
      "cached_tokens": 10318,
      "cache_write_tokens": 0
    },
    "cost": 0.003410,
    "cost_details": { "upstream_inference_cost": 0.002598 }
  }
}

cached_tokens greater than zero means part of the prompt was served from cache and billed at the discounted rate. After the request settles, the same numbers come back from GET /generation as native_tokens_cached.

What we do not have

MyIP OpenRouter serves a small, fixed catalog rather than dozens of external providers, so several things openrouter.ai documents for prompt caching do not apply here and are worth ruling out explicitly:

  • No cache_control field. Anthropic-style explicit cache breakpoints, and OpenAI-style prompt_cache_options / prompt_cache_breakpoint, are provider-specific mechanisms for models we do not serve. If you send them, they are forwarded upstream unchanged like any other non-routing field, and our two catalog models simply ignore fields they don't recognize.
  • No session_id sticky routing. openrouter uses session_id to keep repeat requests pinned to the same provider so a cache stays warm across many candidate endpoints. We don't need that mechanism: a model we serve locally always answers from the same slot, and a model we forward externally is not multi-homed across competing providers the way openrouter's marketplace is. There is no session pinning to configure.
  • No per-provider cache pricing tables. Whatever the serving engine reports as cached_tokens is priced by the single formula above — there's one price list, not one per upstream vendor.

None of this requires action on your side. If your existing client code sets cache_control or session_id out of habit (for example because it also talks to openrouter.ai), it is harmless here — those fields are simply not routing keys, so they pass through and are ignored by models that don't understand them.

What this means day to day

  • You don't opt in or out of caching per request.
  • Keeping the start of your messages array stable across calls (same system prompt, same few-shot examples up front, variable content pushed toward the end) gives a caching-capable engine the best chance to find a reusable prefix, the same principle as with any prefix-cache.
  • Check prompt_tokens_details.cached_tokens after the fact to see whether it helped, rather than assuming it did.

Related

अंतिम अद्यतन 5 सित॰ 2026