Ideas sobre infraestructura de LLMs, optimización de costos y desarrollo con IA.
Most teams track token costs. Almost nobody tracks the full cost of routing decisions. We analyzed 2.3 million requests across 47 production deployments and found that routing overhead — not token pricing — is the dominant cost driver at scale.
Today we're launching OpenTracy, a platform that automatically creates Small Language Models from your production traces, cutting inference costs by up to 57%.
A technical deep-dive into knowledge distillation and how we use it to create production-ready Small Language Models.
Practical strategies for reducing your LLM inference costs, from simple optimizations to advanced techniques like distillation.
A comprehensive guide to evaluating SLMs, including metrics, test sets, and common pitfalls to avoid.
Everything you need to know about deploying SLMs to your own infrastructure, from hardware requirements to serving frameworks.
Announcing OpenTracy SDK v2 with automatic fallbacks, streaming support, built-in cost tracking, and async clients for Python and TypeScript.