Skip to content
Browse documentation
typescript sdk

Streaming and cancellation

Consume incremental model events while preserving terminal validation and cancellation ownership.

Consume the model stream

Use stream on the same configured inference instance. Keep model-specific wire events behind the adapter.

const stream = await inference.stream({
  model: "openai/gpt-5.4-mini",
  messages: [{ role: "user", content: "Summarize this request." }],
});
for await (const chunk of stream) {
  // Narrow chunk.type before reading the matching payload.
  console.log(chunk.type);
}

Treat visible output as committed

Once any chunk has been exposed, later failures do not trigger automatic retry or fallback. Starting another response after visible output can duplicate or contradict what the user already saw.

Pass an AbortSignal through the invocation when the caller owns cancellation. Dispose stream observation when leaving a screen, but decide separately whether application work should continue.

Provider cancellation cannot prove that no remote computation or charge occurred. Validate the terminal structured result before treating streamed content as a saved application outcome.