Back to blog

OpenTracy SDK v2: Fallbacks, Streaming, and Cost Tracking

OpenTracy Team//
announcementsdk

Announcing OpenTracy SDK v2 with automatic fallbacks, streaming support, built-in cost tracking, and async clients for Python and TypeScript.

Today we're releasing OpenTracy SDK v2, a major update focused on reliability, real-time responses, and cost visibility.

What's New

Automatic Fallbacks

Configure backup models that activate when your primary model fails or is unavailable:

from opentracy import OpenTracy

client = OpenTracy()
response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    fallbacks=["anthropic/claude-3-haiku", "openai/gpt-4o-mini"]
)

If the primary model fails, OpenTracy automatically routes to the next available fallback — no retry logic needed.

Streaming Support

Get real-time token-by-token responses:

stream = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)
for chunk in stream:
    print(chunk.choices[0].delta.content, end="")

Built-in Cost Tracking

Every response includes detailed cost and latency metrics:

print(f"Input: ${response.usage.input_cost_usd}")
print(f"Output: ${response.usage.output_cost_usd}")
print(f"Total: ${response.usage.total_cost_usd}")
print(f"Latency: {response.usage.latency_ms}ms")

Async Client

Full async support for high-throughput applications:

from opentracy import AsyncOpenTracy

client = AsyncOpenTracy()
response = await client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)

Migration Guide

Upgrading from v1 is simple:

pip install --upgrade opentracy

The API is backwards-compatible — your existing code will continue to work.

What's Next

We're working on:

  • More provider integrations
  • Advanced routing strategies
  • Enhanced evaluation tools

    • Try v2 today and let us know what you think!