Research paperTandemn: Self-Calibrating Planner for LLM Inference Fleets
papers
2X the Capacity From the GPUs you already have
Before you buy more GPUs, get more capacity from the ones you already have. Tandemn is an open-source, algorithmic planner for self-hosted AI inference that works with your existing orchestration to place and configure workloads across heterogeneous hardware, clouds, and regions, giving you room to do more without expanding your GPU fleet.
Improve All Your Key Metrics. Simultaneously
GPU optimization usually forces tradeoffs between latency, throughput, utilization, SLOs, and cost. Tandemn continuously adapts GPU selection, parallelism strategies, replica counts, and workload placement to actual demand. Across heterogeneous fleet simulations, it admitted more online and batch jobs while improving SLO-honoring goodput, TTFT attainment, and cost per token.
More Jobs admitted
Higher Goodput
Lower TTFT
Lower Cost / token
One fleet, allocated in real time.
Tandemn treats fragmented, heterogeneous GPUs as one adaptive capacity pool. As inference demand shifts, it reallocates workloads so more jobs fit on the fleet you already run.
ChatbotEmbeddingsRerankingVision
Shared capacity map99/128 GPU in use
Proven by Research. Validated on Real Hardware.
Tandemn does more than improve GPU utilization.
We've tested it across diverse workloads, GPU configurations, time horizons, and performance objectives, comparing Tandemn with alternative scheduling approaches.
The result: more usable GPU capacity while simultaneously improving latency, goodput, SLO attainment, and cost efficiency.
Up to 80%More capacity
Up to 45%Lower cost
All your workloads. All your hardware. One shared pool.
Different GPU types, clusters, clouds, and regions shouldn't mean separate islands of capacity. Tandemn connects your infrastructure into one shared resource pool for online inference, batch jobs, embeddings, vision, and more.
With a unified view of every workload and available resource, Tandemn configures and places deployments across the fleet. Capacity can serve demand wherever it arises, without reserving a separate hardware silo for each model or team.
Streamline Model Deployment
Getting a model into production means untangling a tightly coupled stack. Model architecture, serving engine and software versions, GPU type and memory, quantization, and parallelism all affect which configurations work—and how well they perform.
Tandemn covers the full deployment lifecycle, from configuration and placement to scaling and ongoing optimization. Deploy on autopilot from day 1, without stitching every layer together by hand.
+ Any Open-Weight LLM Model
Research & Writing
Ideas, research, and notes from the Tandemn team.
