Streaming

Every model behind /api/v1/chat/completions and /api/v1/completions can stream. Set "stream": true and tokens arrive as Server-Sent Events as they are generated, instead of waiting for the whole reply.

When it's worth it

• Chat UIs, obviously — users see the answer forming instead of staring at a spinner.
• Long completions, where time-to-first-token matters more than total time.
• Anything you might cancel, since a streamed request lets you stop paying for tokens the moment you close the connection (see Cancellation below).

It costs nothing extra: streaming and non-streaming requests to the same model are billed by the identical formula in How costs are calculated. The only practical difference is where the cost appears — a non-streaming response carries it in the X-MyIP-Cost-KRW header; a streaming response cannot, because headers go out before the first byte and the cost isn't known yet. It rides on the final usage chunk instead.

The minimal loop

const response = await fetch('https://openrouter.myip.co.kr/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.MYIP_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'google/gemma-4-26b-a4b',
    messages: [{ role: 'user', content: 'Write a haiku about Busan harbor.' }],
    stream: true,
  }),
});

if (!response.ok) {
  const { error } = await response.json();
  throw new Error(error.message);
}

const reader = response.body!.getReader();
const decoder = new TextDecoder();
let buffer = '';

for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  let index: number;
  while ((index = buffer.indexOf('\n\n')) >= 0) {
    const frame = buffer.slice(0, index);
    buffer = buffer.slice(index + 2);
    if (!frame.startsWith('data:')) continue;

    const payload = frame.slice(5).trim();
    if (payload === '[DONE]') break;

    const chunk = JSON.parse(payload);
    if (chunk.error) throw new Error(chunk.error.message);

    process.stdout.write(chunk.choices?.[0]?.delta?.content ?? '');
    if (chunk.usage) console.log('\ncost (KRW):', chunk.usage.cost);
  }
}

Two details that matter and are easy to get wrong the first time:

• Split on the blank line (\n\n), never on \n alone. One SSE event can span more than one network read; buffer the trailing partial frame and prepend it to the next chunk, exactly as above.
• Check chunk.error before you look at choices[0].delta. A mid-stream failure is a normal data: event with HTTP 200 already sent — see Handling errors during streaming.

An OpenAI SDK does this buffering for you:

import os
from openai import OpenAI

client = OpenAI(base_url="https://openrouter.myip.co.kr/api/v1", api_key=os.environ["MYIP_API_KEY"])

stream = client.chat.completions.create(
    model="google/gemma-4-26b-a4b",
    messages=[{"role": "user", "content": "Write a haiku about Busan harbor."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print(f"\n\ncost (KRW): {chunk.usage.cost}")

Where the model and provider actually served the request show up

Because models[] can fall back across candidates, the id you asked for is not necessarily the one that answered. Every chunk carries the winner:

"model": "google/gemma-4-26b-a4b",
"provider": "MyIP Local GPU"

The same values are in the X-MyIP-Model / X-MyIP-Provider response headers, set as soon as the winning candidate starts responding. See Model fallbacks.

The final usage chunk

The last data: event before [DONE] carries usage, and we inject the cost into it:

{"choices":[],"usage":{"prompt_tokens":18,"completion_tokens":9,"total_tokens":27,"cost":0.001890,"cost_details":{"upstream_inference_cost":0.000383}}}

usage.cost is the exact amount deducted from your balance, in KRW, rounded to six decimal places — the same number GET /generation will later report for this request's total_cost. We always request usage from the upstream ourselves (stream_options: {"include_usage": true}), so you do not need to set it, and a value you send there is discarded.

Errors before vs. during the stream

The full shape of the mid-stream error event, and why HTTP 200 does not mean success once you're inside a stream, is in the API reference.

Cancellation

Aborting the client request (closing the connection, AbortController.abort()) stops the upstream call and ends billing at that point — you are charged for the tokens generated up to the cancellation, not for a full response. The usage record for a cancelled request has status: "cancelled", and GET /generation reports cancelled: true.

const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 5000); // give up after 5s

try {
  const response = await fetch('https://openrouter.myip.co.kr/api/v1/chat/completions', {
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.MYIP_API_KEY}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ model: 'google/gemma-4-26b-a4b', messages: [...], stream: true }),
    signal: controller.signal,
  });
  // ...consume response.body as above
} finally {
  clearTimeout(timer);
}

We do not bill for zero tokens generated before a cancellation, but we do bill for whatever was already streamed — closing the connection is not a way to get free partial output.

/completions streaming

The legacy /completions endpoint streams too. Chunks have object: "text_completion" and the text is in choices[].text rather than delta.content; the usage chunk, [DONE], and error events are identical.

Related

• Streaming — API reference — exact frame format, settlement timing, and full error/cancellation reference
• Model fallbacks — how the candidate chain is built and when it stops retrying
• Tool calling — streaming tool_calls deltas specifically