流式传输

ARouter 支持所有模型的流式响应。启用流式传输后，token 会在生成时实时传递。要启用流式传输，请在请求体中设置 stream: true。

Python (OpenAI)
Node.js (OpenAI)
Go
cURL
fetch (raw)

from openai import OpenAI

client = OpenAI(
    base_url="https://api.arouter.ai/v1",
    api_key="lr_live_xxxx",
)

stream = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "How would you build the tallest building ever?"}],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

# Final chunk includes usage stats
# Access via: stream.get_final_completion().usage

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.arouter.ai/v1",
  apiKey: "lr_live_xxxx",
});

const stream = await client.chat.completions.create({
  model: "openai/gpt-5.4",
  messages: [{ role: "user", content: "How would you build the tallest building ever?" }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);

  // Final chunk includes usage stats
  if (chunk.usage) {
    console.log("\nUsage:", chunk.usage);
  }
}

stream, err := client.ChatCompletionStream(ctx, arouter.ChatCompletionRequest{
    Model: "openai/gpt-5.4",
    Messages: []arouter.Message{
        {Role: "user", Content: "How would you build the tallest building ever?"},
    },
})
if err != nil {
    log.Fatal(err)
}
defer stream.Close()

for {
    chunk, err := stream.Recv()
    if err == arouter.ErrStreamDone {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    fmt.Print(chunk.Choices[0].Delta.Content)
}

curl -N https://api.arouter.ai/v1/chat/completions \
  -H "Authorization: Bearer lr_live_xxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "messages": [{"role": "user", "content": "How would you build the tallest building ever?"}],
    "stream": true
  }'

const response = await fetch('https://api.arouter.ai/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer lr_live_xxxx',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'openai/gpt-5.4',
    messages: [{ role: 'user', content: 'How would you build the tallest building ever?' }],
    stream: true,
  }),
});

const reader = response.body?.getReader();
if (!reader) throw new Error('No response body');

const decoder = new TextDecoder();
let buffer = '';

try {
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });

    while (true) {
      const lineEnd = buffer.indexOf('\n');
      if (lineEnd === -1) break;

      const line = buffer.slice(0, lineEnd).trim();
      buffer = buffer.slice(lineEnd + 1);

      if (line.startsWith('data: ')) {
        const data = line.slice(6);
        if (data === '[DONE]') break;

        try {
          const parsed = JSON.parse(data);
          const content = parsed.choices[0]?.delta?.content;
          if (content) process.stdout.write(content);
        } catch (e) {
          // ignore invalid JSON
        }
      }
    }
  }
} finally {
  reader.cancel();
}

Anthropic 流式传输

Anthropic SDK 使用其自有的流式传输格式：

import anthropic

client = anthropic.Anthropic(
    base_url="https://api.arouter.ai",
    api_key="lr_live_xxxx",
)

with client.messages.stream(
    model="claude-sonnet-4.6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "How would you build the tallest building ever?"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Gemini 流式传输

Gemini 使用 streamGenerateContent 而非 generateContent：

import google.generativeai as genai

genai.configure(
    api_key="lr_live_xxxx",
    transport="rest",
    client_options={"api_endpoint": "https://api.arouter.ai"},
)

model = genai.GenerativeModel("gemini-2.5-flash")
response = model.generate_content("How would you build the tallest building ever?", stream=True)

for chunk in response:
    print(chunk.text, end="", flush=True)

SSE 格式

底层流式传输使用 Server-Sent Events。每个内容事件的格式如下：

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

[DONE] 之前的最后一个数据块包含用量数据，且 choices 数组为空：

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[],"usage":{"prompt_tokens":10,"completion_tokens":20,"total_tokens":30,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}

data: [DONE]

ARouter 可能偶尔发送 SSE 注释（以 : 开头的行）以防止连接超时。根据 SSE 规范，这些注释可以安全忽略。

取消流式请求

流式请求可通过中断连接来取消。对于支持的提供商，这将立即停止模型处理。

Node.js (AbortController)
Python
fetch (AbortController)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.arouter.ai/v1",
  apiKey: "lr_live_xxxx",
});

const controller = new AbortController();

try {
  const stream = await client.chat.completions.create(
    {
      model: "openai/gpt-5.4",
      messages: [{ role: "user", content: "Write a long story" }],
      stream: true,
    },
    { signal: controller.signal },
  );

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) process.stdout.write(content);
  }
} catch (error) {
  if (error.name === "AbortError") {
    console.log("Stream cancelled");
  } else {
    throw error;
  }
}

// To cancel:
controller.abort();

import requests
from threading import Event, Thread

def stream_with_cancellation(prompt: str, cancel_event: Event):
    with requests.Session() as session:
        response = session.post(
            "https://api.arouter.ai/v1/chat/completions",
            headers={"Authorization": "Bearer lr_live_xxxx"},
            json={
                "model": "openai/gpt-5.4",
                "messages": [{"role": "user", "content": prompt}],
                "stream": True,
            },
            stream=True,
        )
        try:
            for line in response.iter_lines():
                if cancel_event.is_set():
                    response.close()
                    return
                if line:
                    print(line.decode(), end="", flush=True)
        finally:
            response.close()

cancel_event = Event()
t = Thread(target=lambda: stream_with_cancellation("Write a long story", cancel_event))
t.start()

# To cancel:
cancel_event.set()

const controller = new AbortController();

try {
  const response = await fetch('https://api.arouter.ai/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: 'Bearer lr_live_xxxx',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5.4',
      messages: [{ role: 'user', content: 'Write a long story' }],
      stream: true,
    }),
    signal: controller.signal,
  });

  // process stream...
} catch (error) {
  if (error.name === 'AbortError') {
    console.log('Stream cancelled');
  } else {
    throw error;
  }
}

// To cancel:
controller.abort();

流式传输中的错误处理

ARouter 根据错误发生的时间，以不同方式处理流式传输中的错误。

发送任何 Token 之前出现的错误

如果在开始流式传输之前发生错误，ARouter 会返回带有适当 HTTP 状态码的标准 JSON 错误响应：

{
  "error": {
    "code": 400,
    "message": "Invalid model specified"
  }
}

常见 HTTP 状态码：

代码	含义
400	Bad Request — 参数无效
401	Unauthorized — API key 无效
402	Payment Required — 额度不足
429	Too Many Requests — 已被限速
502	Bad Gateway — 提供商错误
503	Service Unavailable — 无可用提供商

已发送部分 Token 后出现的错误（流中途）

如果在已传输部分 token 后发生错误，ARouter 无法更改 HTTP 状态码（此时已为 200 OK）。错误将以 SSE 事件的形式发送：

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","error":{"code":"server_error","message":"Provider disconnected unexpectedly"},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}

关键特征：

错误出现在顶层，与标准响应字段并列
choices 数组包含 finish_reason: "error" 以终止流
由于响应头已发送，HTTP 状态保持 200 OK

错误处理代码示例

Python (OpenAI)
Node.js (OpenAI)
fetch (raw)

from openai import OpenAI, APIStatusError

client = OpenAI(
    base_url="https://api.arouter.ai/v1",
    api_key="lr_live_xxxx",
)

try:
    stream = client.chat.completions.create(
        model="openai/gpt-5.4",
        messages=[{"role": "user", "content": "Write a story"}],
        stream=True,
    )
    for chunk in stream:
        content = chunk.choices[0].delta.content
        if content:
            print(content, end="", flush=True)
except APIStatusError as e:
    print(f"\nError {e.status_code}: {e.message}")

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.arouter.ai/v1",
  apiKey: "lr_live_xxxx",
});

try {
  const stream = await client.chat.completions.create({
    model: "openai/gpt-5.4",
    messages: [{ role: "user", content: "Write a story" }],
    stream: true,
  });

  for await (const chunk of stream) {
    // Check for mid-stream errors
    if ("error" in chunk) {
      console.error(`Stream error: ${(chunk as any).error.message}`);
      if (chunk.choices?.[0]?.finish_reason === "error") {
        console.log("Stream terminated due to error");
      }
      break;
    }
    const content = chunk.choices[0]?.delta?.content;
    if (content) process.stdout.write(content);
  }
} catch (error) {
  if (error instanceof OpenAI.APIError) {
    console.error(`Error ${error.status}: ${error.message}`);
  } else {
    throw error;
  }
}

async function streamWithErrorHandling(prompt: string) {
  const response = await fetch('https://api.arouter.ai/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: 'Bearer lr_live_xxxx',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5.4',
      messages: [{ role: 'user', content: prompt }],
      stream: true,
    }),
  });

  // Check initial HTTP status for pre-stream errors
  if (!response.ok) {
    const error = await response.json();
    console.error(`Error: ${error.error.message}`);
    return;
  }

  const reader = response.body?.getReader();
  if (!reader) throw new Error('No response body');
  const decoder = new TextDecoder();
  let buffer = '';

  try {
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      buffer += decoder.decode(value, { stream: true });

      while (true) {
        const lineEnd = buffer.indexOf('\n');
        if (lineEnd === -1) break;
        const line = buffer.slice(0, lineEnd).trim();
        buffer = buffer.slice(lineEnd + 1);

        if (line.startsWith('data: ')) {
          const data = line.slice(6);
          if (data === '[DONE]') return;

          try {
            const parsed = JSON.parse(data);

            // Check for mid-stream error
            if (parsed.error) {
              console.error(`Stream error: ${parsed.error.message}`);
              return;
            }

            const content = parsed.choices[0]?.delta?.content;
            if (content) process.stdout.write(content);
          } catch (e) {
            // ignore parse errors
          }
        }
      }
    }
  } finally {
    reader.cancel();
  }
}

快速开始

核心概念

路由

功能特性

使用指南

隐私

管理

最佳实践

框架与集成

供应商接入

帮助

Anthropic 流式传输

Gemini 流式传输

SSE 格式

推荐的 SSE 客户端库

取消流式请求

流式传输中的错误处理

发送任何 Token 之前出现的错误

已发送部分 Token 后出现的错误（流中途）

错误处理代码示例

快速开始

核心概念

路由

功能特性

使用指南

隐私

管理

最佳实践

框架与集成

供应商接入

帮助

​Anthropic 流式传输

​Gemini 流式传输

​SSE 格式

​推荐的 SSE 客户端库

​取消流式请求

​流式传输中的错误处理

​发送任何 Token 之前出现的错误

​已发送部分 Token 后出现的错误（流中途）

​错误处理代码示例

Anthropic 流式传输

Gemini 流式传输

SSE 格式

推荐的 SSE 客户端库

取消流式请求

流式传输中的错误处理

发送任何 Token 之前出现的错误

已发送部分 Token 后出现的错误（流中途）

错误处理代码示例