Streaming

Receive generated output incrementally with Server-Sent Events.

Open a stream

Set stream to true on a chat completion. SDK iterators handle SSE framing; cURL prints the raw stream with buffering disabled.

curl

curl --request POST --no-buffer \
  --url 'https://api.tokenlane.de/v1/chat/completions' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "deepseek/deepseek-v4-flash-0731",
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ],
  "max_tokens": 400,
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}'

node

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: "https://api.tokenlane.de/v1", apiKey: process.env.TOKENLANE_API_KEY });
const response = await client.chat.completions.create({
  "model": "deepseek/deepseek-v4-flash-0731",
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ],
  "max_tokens": 400,
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}, {});
for await (const chunk of response) {
  if (chunk.error) throw new Error('Stream failed');
  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? '');
}

python

import os
import json
from openai import OpenAI

client = OpenAI(base_url="https://api.tokenlane.de/v1", api_key=os.environ["TOKENLANE_API_KEY"])
body = json.loads(r'''{
  "model": "deepseek/deepseek-v4-flash-0731",
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ],
  "max_tokens": 400,
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}''')
response = client.chat.completions.create(
    model=body["model"],
    messages=body["messages"],
    max_tokens=body["max_tokens"],
    stream=body["stream"],
    stream_options=body["stream_options"]
)
for chunk in response:
    if getattr(chunk, 'error', None):
        raise RuntimeError('Stream failed')
    print(chunk.choices[0].delta.content or '' if chunk.choices else '', end='', flush=True)

vercel

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { streamText } from 'ai';

const tokenlane = createOpenAICompatible({ name: 'tokenlane', baseURL: "https://api.tokenlane.de/v1", apiKey: process.env.TOKENLANE_API_KEY, includeUsage: true });
const result = streamText({
  model: tokenlane("deepseek/deepseek-v4-flash-0731"),
  messages: [
  {
    "role": "user",
    "content": "Hello"
  }
],
  maxOutputTokens: 400,
});
for await (const part of result.fullStream) {
  if (part.type === 'error') throw new Error('Stream failed');
  if (part.type === 'text-delta') process.stdout.write(part.text);
}

Read events safely

Transport chunks are not event boundaries. A custom parser must buffer incomplete events, join multiple data: lines, ignore comment lines beginning with :, and recognize data: [DONE]. Usage frames can contain no generated text. Check for an error field before reading content. An HTTP 200 alone does not establish successful generation.

Handle cancellation and failures

Abort the SDK request or close the HTTP connection to cancel locally; the gateway drops its upstream request. HTTP errors before streaming and error events after streaming need separate handling. An already started stream is never automatically moved to another provider. Retrying after partial output can duplicate output and costs. See Errors and retries.