Algorithm

The planner needs to know which configuration changes are worth exploring and how well their expected effects are supported. Causal confidence provides that foundation: it connects deployment decisions to measurable runtime behavior, then to serving outcomes, focusing search on relationships relevant to each workload’s targets.

For example, a change in parallelism can alter memory pressure and communication overhead, which can affect throughput and latency. The graph makes that chain of effects explicit and testable through serving telemetry. Decisions X connect to runtime behavior V and outcomes Y.

Observed deployments update each relationship’s belief with supporting or contradictory evidence. Confidence measures that support; uncertainty tracks how much remains to learn. Two relationships can have the same confidence but different amounts of evidence, so the planner uses both signals to guide proposals and identify informative deployments to test.

Tandemn’s distinctive approach combines this persistent causal evidence with language-model-guided search and deterministic evaluation. Related edges form hypotheses scoped to particular models, hardware, and workloads, allowing relevant evidence to inform future proposals. Only deployment evidence updates causal confidence; deterministic tools predict, score, and validate candidates before a fleet-wide plan is selected. Each deployment therefore informs both the next allocation and the planner’s understanding of serving behavior.

A deployment ladder $L_i$ specifies job i’s configurations, environments, and replica groups. Each predicted outcome is compared with the job’s reference target, then weighted by its importance to that workload.

Cost and latency are minimized; throughput and serving headroom are maximized. The reference $z_{ij}^{\star}$and scale $\sigma_{ij}$ normalize these different units into comparable gaps.

The augmented Tchebycheff score prioritizes the largest weighted gap, so strength on one metric cannot mask weakness on another. The sum term distinguishes ties. A higher score means closer alignment with the workload’s objectives.

Exploration rewards testing uncertain edges and scoped mechanisms. Activation indicators count each relationship once per plan, avoiding duplicate credit when several deployments test the same hypothesis.

SLO violation risk estimates use prediction-error bands calibrated against observed residuals. The per-objective breach probabilities are combined using an independence approximation over the job’s constrained objectives.

Reconfiguration cost accounts for loading and warm-up, concurrent execution, draining and termination, and transient underperformance. Retaining the existing deployment incurs no reconfiguration cost.

A plan $P=\{L_i\}$ assigns a ladder to each active and pending job. The fleet objective weights SLO violation risk with $\gamma$, reconfiguration cost with $\lambda_t$, and learning value with $\beta_t$. Learning and reconfiguration weights adapt across planning cycles.

The feasible set $\mathcal F_t$ enforces resource capacity, physical validity, SLO violation risk, reconfiguration limits, quotas, and priorities. Search is bounded: Tandemn commits the highest-scoring feasible plan found within its budget.

Tandemn evaluates each candidate plan with a single cluster-wide score. It adds up workload performance scores, subtracts weighted penalties for SLO violation risk and reconfiguration cost, and rewards useful learning across the fleet.

The weights control the balance between SLO violation risk, reconfiguration, and exploration. Learning is rewarded once across the plan, so multiple workloads testing the same relationship do not inflate its value. Tandemn selects the highest-scoring feasible plan found within its search budget, subject to capacity, serving targets, quotas, and priorities.