Streaming

Add "stream": true to the request body and the response arrives as Server-Sent Events. Both /chat/completions and /completions support it.

content-type: text/event-stream; charset=utf-8
cache-control: no-cache, no-transform
x-accel-buffering: no
X-MyIP-Currency: KRW
X-MyIP-Generation-Id: gen-…
X-MyIP-Request-Id: req-…
X-MyIP-Model: google/gemma-4-26b-a4b
X-MyIP-Provider: MyIP Local GPU

Frame format

Every event begins with data: and ends with a blank line (\n\n). The last one is always data: [DONE].

data: {"id":"gen-7kq…","object":"chat.completion.chunk","created":1788452301,"model":"google/gemma-4-26b-a4b","provider":"MyIP Local GPU","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"gen-7kq…","object":"chat.completion.chunk","created":1788452301,"model":"google/gemma-4-26b-a4b","provider":"MyIP Local GPU","choices":[{"index":0,"delta":{"content":"The"},"finish_reason":null}]}

data: {"id":"gen-7kq…","object":"chat.completion.chunk","created":1788452302,"model":"google/gemma-4-26b-a4b","provider":"MyIP Local GPU","choices":[{"index":0,"delta":{"content":" sea"},"finish_reason":null}]}

data: {"id":"gen-7kq…","object":"chat.completion.chunk","created":1788452302,"model":"google/gemma-4-26b-a4b","provider":"MyIP Local GPU","choices":[{"index":0,"delta":{},"finish_reason":"stop","native_finish_reason":"stop"}]}

data: {"id":"gen-7kq…","object":"chat.completion.chunk","created":1788452302,"model":"google/gemma-4-26b-a4b","provider":"MyIP Local GPU","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":9,"total_tokens":27,"cost":0.001890,"cost_details":{"upstream_inference_cost":0.000383}}}

data: [DONE]

Three fields are rewritten by us on every chunk:

• id — the upstream id is replaced with our generation id. It is identical across the whole stream and matches the X-MyIP-Generation-Id header.
• model — the model id of the candidate that actually answered.
• provider — that candidate's provider display name.

When a chunk has finish_reason but no native_finish_reason, we mirror the value into it.

The final usage chunk

Streaming responses carry no X-MyIP-Cost-KRW header. Headers go out before the first byte, and at that moment neither the token counts nor the cost exist. So the cost rides on the final usage chunk instead.

This number is exactly the total_cost you will later get from GET /generation.

Reading a stream

curl -N https://openrouter.myip.co.kr/api/v1/chat/completions \
  -H "Authorization: Bearer $MYIP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-26b-a4b",
    "messages": [{"role": "user", "content": "Describe the sea in three sentences."}],
    "stream": true
  }'

Failure mid-stream

Once the first byte has been sent, we never retry on another candidate. Tokens the client already has cannot be unsent. Instead we report the error as an SSE event and close the stream cleanly.

data: {"id":"gen-7kq…","object":"chat.completion.chunk","created":1788452310,"model":"google/gemma-4-26b-a4b","provider":"MyIP Local GPU","error":{"code":502,"message":"upstream disconnected","metadata":{"error_type":"provider_error"}},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}

data: [DONE]

The HTTP status was already sent as 200, so you must check for the error field inside chunks. Looking only at the status code, this reads as a success.

Retrying on another candidate happens only before the first byte. If the whole chain fails at that stage you get a normal JSON error response instead of a stream.

Disconnecting early

If the client closes the connection we abort the upstream call too — and we settle on the tokens generated up to that point. Cutting a stream must not be a way to use the service for free. The request is recorded as cancelled, and GET /generation reports cancelled: true.

Streaming on /completions

The legacy /completions endpoint streams too. Chunks carry object: "text_completion" and the text arrives in choices[].text rather than delta.content. Everything else — the usage chunk, [DONE], the error event — is the same.

When settlement happens

For a non-streaming request, settlement finishes before the response is built, so /generation has the record the moment you get your answer. Streaming is different: settlement runs after the stream closes, detached from the response lifetime. Calling /generation immediately after [DONE] may still return 404, so retry briefly.