流式传输

ARouter 支持所有模型的流式响应。启用流式传输后，token 会在生成时实时传递。要启用流式传输，请在请求体中设置 stream: true。

Python (OpenAI)
Node.js (OpenAI)
Go
cURL
fetch (raw)

from openai import OpenAI

client = OpenAI(
    base_url="https://api.arouter.ai/v1",
    api_key="lr_live_xxxx",
)

stream = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "How would you build the tallest building ever?"}],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

# Final chunk includes usage stats
# Access via: stream.get_final_completion().usage

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.arouter.ai/v1",
  apiKey: "lr_live_xxxx",
});

const stream = await client.chat.completions.create({
  model: "openai/gpt-5.4",
  messages: [{ role: "user", content: "How would you build the tallest building ever?" }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);

  // Final chunk includes usage stats
  if (chunk.usage) {
    console.log("\nUsage:", chunk.usage);
  }
}

stream, err := client.ChatCompletionStream(ctx, arouter.ChatCompletionRequest{
    Model: "openai/gpt-5.4",
    Messages: []arouter.Message{
        {Role: "user", Content: "How would you build the tallest building ever?"},
    },
})
if err != nil {
    log.Fatal(err)
}
defer stream.Close()

for {
    chunk, err := stream.Recv()
    if err == arouter.ErrStreamDone {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    fmt.Print(chunk.Choices[0].Delta.Content)
}

curl -N https://api.arouter.ai/v1/chat/completions \
  -H "Authorization: Bearer lr_live_xxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "messages": [{"role": "user", "content": "How would you build the tallest building ever?"}],
    "stream": true
  }'

const response = await fetch('https://api.arouter.ai/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer lr_live_xxxx',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'openai/gpt-5.4',
    messages: [{ role: 'user', content: 'How would you build the tallest building ever?' }],
    stream: true,
  }),
});

const reader = response.body?.getReader();
if (!reader) throw new Error('No response body');

const decoder = new TextDecoder();
let buffer = '';

try {
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });

    while (true) {
      const lineEnd = buffer.indexOf('\n');
      if (lineEnd === -1) break;

      const line = buffer.slice(0, lineEnd).trim();
      buffer = buffer.slice(lineEnd + 1);

      if (line.startsWith('data: ')) {
        const data = line.slice(6);
        if (data === '[DONE]') break;

        try {
          const parsed = JSON.parse(data);
          const content = parsed.choices[0]?.delta?.content;
          if (content) process.stdout.write(content);
        } catch (e) {
          // ignore invalid JSON
        }
      }
    }
  }
} finally {
  reader.cancel();
}

Anthropic 流式传输

Anthropic SDK 使用其自有的流式传输格式：

import anthropic

client = anthropic.Anthropic(
    base_url="https://api.arouter.ai",
    api_key="lr_live_xxxx",
)

with client.messages.stream(
    model="claude-sonnet-4.6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "How would you build the tallest building ever?"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Gemini 流式传输

Gemini 使用 streamGenerateContent 而非 generateContent：

import google.generativeai as genai

genai.configure(
    api_key="lr_live_xxxx",
    transport="rest",
    client_options={"api_endpoint": "https://api.arouter.ai"},
)

model = genai.GenerativeModel("gemini-2.5-flash")
response = model.generate_content("How would you build the tallest building ever?", stream=True)

for chunk in response:
    print(chunk.text, end="", flush=True)

SSE 格式

底层流式传输使用 Server-Sent Events。每个内容事件的格式如下：

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

[DONE] 之前的最后一个数据块包含用量数据，且 choices 数组为空：

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","choices":[],"usage":{"prompt_tokens":10,"completion_tokens":20,"total_tokens":30,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}

data: [DONE]

ARouter 可能偶尔发送 SSE 注释（以 : 开头的行）以防止连接超时。根据 SSE 规范，这些注释可以安全忽略。

取消流式请求

流式请求可通过中断连接来取消。对于支持的提供商，这将立即停止模型处理。

Node.js (AbortController)
Python
fetch (AbortController)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.arouter.ai/v1",
  apiKey: "lr_live_xxxx",
});

const controller = new AbortController();

try {
  const stream = await client.chat.completions.create(
    {
      model: "openai/gpt-5.4",
      messages: [{ role: "user", content: "Write a long story" }],
      stream: true,
    },
    { signal: controller.signal },
  );

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) process.stdout.write(content);
  }
} catch (error) {
  if (error.name === "AbortError") {
    console.log("Stream cancelled");
  } else {
    throw error;
  }
}

// To cancel:
controller.abort();

import requests
from threading import Event, Thread

def stream_with_cancellation(prompt: str, cancel_event: Event):
    with requests.Session() as session:
        response = session.post(
            "https://api.arouter.ai/v1/chat/completions",
            headers={"Authorization": "Bearer lr_live_xxxx"},
            json={
                "model": "openai/gpt-5.4",
                "messages": [{"role": "user", "content": prompt}],
                "stream": True,
            },
            stream=True,
        )
        try:
            for line in response.iter_lines():
                if cancel_event.is_set():
                    response.close()
                    return
                if line:
                    print(line.decode(), end="", flush=True)
        finally:
            response.close()

cancel_event = Event()
t = Thread(target=lambda: stream_with_cancellation("Write a long story", cancel_event))
t.start()

# To cancel:
cancel_event.set()

const controller = new AbortController();

try {
  const response = await fetch('https://api.arouter.ai/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: 'Bearer lr_live_xxxx',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5.4',
      messages: [{ role: 'user', content: 'Write a long story' }],
      stream: true,
    }),
    signal: controller.signal,
  });

  // process stream...
} catch (error) {
  if (error.name === 'AbortError') {
    console.log('Stream cancelled');
  } else {
    throw error;
  }
}

// To cancel:
controller.abort();

流式传输中的错误处理

ARouter 根据错误发生的时间，以不同方式处理流式传输中的错误。

发送任何 Token 之前出现的错误

如果在开始流式传输之前发生错误，ARouter 会返回带有适当 HTTP 状态码的标准 JSON 错误响应：

{
  "error": {
    "code": 400,
    "message": "Invalid model specified"
  }
}

常见 HTTP 状态码：

代码	含义
400	Bad Request — 参数无效
401	Unauthorized — API key 无效
402	Payment Required — 额度不足
429	Too Many Requests — 已被限速
502	Bad Gateway — 提供商错误
503	Service Unavailable — 无可用提供商

已发送部分 Token 后出现的错误（流中途）

如果在已传输部分 token 后发生错误，ARouter 无法更改 HTTP 状态码（此时已为 200 OK）。错误将以 SSE 事件的形式发送：

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","model":"openai/gpt-5.4","error":{"code":"server_error","message":"Provider disconnected unexpectedly"},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}

关键特征：

错误出现在顶层，与标准响应字段并列
choices 数组包含 finish_reason: "error" 以终止流
由于响应头已发送，HTTP 状态保持 200 OK

错误处理代码示例

Python (OpenAI)
Node.js (OpenAI)
fetch (raw)

from openai import OpenAI, APIStatusError

client = OpenAI(
    base_url="https://api.arouter.ai/v1",
    api_key="lr_live_xxxx",
)

try:
    stream = client.chat.completions.create(
        model="openai/gpt-5.4",
        messages=[{"role": "user", "content": "Write a story"}],
        stream=True,
    )
    for chunk in stream:
        content = chunk.choices[0].delta.content
        if content:
            print(content, end="", flush=True)
except APIStatusError as e:
    print(f"\nError {e.status_code}: {e.message}")

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.arouter.ai/v1",
  apiKey: "lr_live_xxxx",
});

try {
  const stream = await client.chat.completions.create({
    model: "openai/gpt-5.4",
    messages: [{ role: "user", content: "Write a story" }],
    stream: true,
  });

  for await (const chunk of stream) {
    // Check for mid-stream errors
    if ("error" in chunk) {
      console.error(`Stream error: ${(chunk as any).error.message}`);
      if (chunk.choices?.[0]?.finish_reason === "error") {
        console.log("Stream terminated due to error");
      }
      break;
    }
    const content = chunk.choices[0]?.delta?.content;
    if (content) process.stdout.write(content);
  }
} catch (error) {
  if (error instanceof OpenAI.APIError) {
    console.error(`Error ${error.status}: ${error.message}`);
  } else {
    throw error;
  }
}

async function streamWithErrorHandling(prompt: string) {
  const response = await fetch('https://api.arouter.ai/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: 'Bearer lr_live_xxxx',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'openai/gpt-5.4',
      messages: [{ role: 'user', content: prompt }],
      stream: true,
    }),
  });

  // Check initial HTTP status for pre-stream errors
  if (!response.ok) {
    const error = await response.json();
    console.error(`Error: ${error.error.message}`);
    return;
  }

  const reader = response.body?.getReader();
  if (!reader) throw new Error('No response body');
  const decoder = new TextDecoder();
  let buffer = '';

  try {
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      buffer += decoder.decode(value, { stream: true });

      while (true) {
        const lineEnd = buffer.indexOf('\n');
        if (lineEnd === -1) break;
        const line = buffer.slice(0, lineEnd).trim();
        buffer = buffer.slice(lineEnd + 1);

        if (line.startsWith('data: ')) {
          const data = line.slice(6);
          if (data === '[DONE]') return;

          try {
            const parsed = JSON.parse(data);

            // Check for mid-stream error
            if (parsed.error) {
              console.error(`Stream error: ${parsed.error.message}`);
              return;
            }

            const content = parsed.choices[0]?.delta?.content;
            if (content) process.stdout.write(content);
          } catch (e) {
            // ignore parse errors
          }
        }
      }
    }
  } finally {
    reader.cancel();
  }
}

快速开始

核心概念

路由

功能特性

使用指南

隐私

管理

最佳实践

框架与集成

供应商接入

帮助

Anthropic 流式传输

Gemini 流式传输

SSE 格式

推荐的 SSE 客户端库

取消流式请求

流式传输中的错误处理

发送任何 Token 之前出现的错误

已发送部分 Token 后出现的错误（流中途）

错误处理代码示例

​Anthropic 流式传输

​Gemini 流式传输

​SSE 格式

​推荐的 SSE 客户端库

​取消流式请求

​流式传输中的错误处理

​发送任何 Token 之前出现的错误

​已发送部分 Token 后出现的错误（流中途）

​错误处理代码示例

Anthropic 流式传输

Gemini 流式传输

SSE 格式

推荐的 SSE 客户端库

取消流式请求

流式传输中的错误处理

发送任何 Token 之前出现的错误

已发送部分 Token 后出现的错误（流中途）

错误处理代码示例