Against production’s own placements.
We are testing a GPU placement engine against the decisions a hyperscale production scheduler actually made, pod by pod and hour by hour, on its own published trace. This note sets out the protocol before the results: the data, the rules that keep the comparison fair, how a decision is made, what every run has to report, and the conditions for publishing results.
Stranded GPUs
A GPU cluster can run out of room while a large share of its GPUs sit idle. The idle capacity is in the wrong shape: a sliver of a device no job can use, two free GPUs on a server when the next job wants four, or a free GPU on a server with no CPU left to pair with it. That stranded capacity is paid for and earns nothing, and how much of it there is depends on where the scheduler put the jobs that are already running.
Placement is a packing problem with many dimensions — GPU fractions, whole devices, CPU, and rules about how many replicas of a service may share a server or a rack switch — and it has to be decided continuously as work arrives. The question: given the exact placements a production scheduler made, could a better placement engine strand less capacity, without moving running work around, and fast enough to sit in the scheduling loop?
The one trace that says where
Several public cluster traces record what jobs asked for. Only one we know of records where a production scheduler put them: Alibaba’s ASI trace, released with an OSDI ’26 paper. It covers six months of Alibaba Serverless Infrastructure, with tens of thousands of GPU servers, and for every pod in every hour it gives the server it ran on, its GPU and CPU request, the GPU model, the cluster and its priority class. That makes production’s own decisions the baseline — not a research policy, and not a simulator’s default.
The pod table is published as a single very large archive whose host serves no per-file downloads. It does serve byte ranges, so we read the archive’s index off its tail and fetch only the hours an experiment needs.
The score is the trace authors’ own fragmentation metric, transcribed term for term from their release: for each of eight reference job shapes, idle GPU is split into fractional slivers, whole GPUs too few for the shape, and GPUs with no CPU to pair with. We call the total unallocatable idle GPU; lower is better. One substitution, disclosed: the authors count measured GPU use, while a scheduler can only move requests, so we score requested GPU.
Keeping the comparison fair
Each modelling choice below is the conservative one, and each was checked against the data rather than assumed.
- GPU model and cluster are hard constraints. Every pod’s GPU model and cluster match its host server’s throughout the trace, so the fleet splits exactly into independent pools, and no pod may leave its pool.
- Replica spread is production’s own. The real anti-affinity rules are not published, and the obvious guess — one replica per server — is wrong: production does co-locate replicas of a service. So each service may never exceed the most of its own pods production ever put on one server, or behind one rack switch. Collapsing replicas to win the metric is ruled out.
- Spot work is preemptible. Pods freed and reused within an hour make the hourly snapshot oversubscribed. Low-priority pods are shed from those servers, largest first, until the recorded load fits. Both sides then hold the identical pod set, and production’s own layout must pass our independent checker with zero violations — the first test that the model is right.
- Running pods stay put. We start from production’s layout. Every pod already running is pinned where it is; each hour, only the pods that arrived, plus any whose server stopped fitting them, are placed — against the state our own earlier decisions produced. Every move of a running pod is counted and reported.
- A layout that breaks a rule is discarded, not reported. Every pool, every hour, passes an independent verifier before it counts.
What one decision is
One decision places every free pod in every pool of the fleet for one hour. Three design choices are what let it run inside the scheduling loop.
Search only what can move. Pinned pods are not search variables at all; they enter as a fixed load on each server and device. Handing the search every pod and masking most of them out wastes nearly all of its effort on pods that cannot move.
Batch the fleet into a few shapes. The pools are grouped once, at startup, into a handful of shape classes by size and server width. Each hour is then one launch per class on a single GPU, every pool in a class searched in parallel, with nothing recompiled after the warm-up.
Construct, then search. A best-fit construction places every arrival first; parallel search chains then improve on it, with moves scored by the exact change in the metric, including two pods exchanging slots. The construction’s own layout is always a candidate, so the search can never return anything worse than it, and nothing is accepted without passing the verifier.
What every run reports
No single number can carry a placement result. Every arm — production, our construction alone, and the engine — is reported on the same pods, the same pools and the same hours, with all of the following side by side:
Quality
Unallocatable idle GPU, per hour and summed
The trace authors’ metric, for every arm. The engine is always also compared with its own construction, so the search’s contribution is never confused with what any sensible greedy rule would already achieve.
Disruption
Running pods moved, against production’s own rate
A packing can always be improved by moving running work. Every relocation is counted, and production’s own rate of server changes in the trace is the reference line, not zero.
Room for the next hour
Admission of the next hour’s real arrivals
Less stranded GPU should mean more room. We test it directly: each committed layout is offered the arrivals the trace records for the following hour, counted two ways — up to the first refusal, and every pod that fits.
Completeness
Pods left unplaced
A layout that leaves work out can score well by omission. Unplaced pods are reported for every hour, and a pool that cannot be fully placed is scored at production’s value, not ours.
Speed
Wall-clock for the whole fleet’s decision
Measured end to end with nothing excluded, over consecutive hours after a single warm-up — never on one hand-warmed hour — and with the one-time warm-up reported separately.
Distance to optimal
Proven optima, where they can be had
An exact model of one pool-hour, scoring layouts identically to the metric code, solved with the open-source CP-SAT solver. Where it proves an optimum, we report how much of the gap between construction and optimum the search closes — and state which pools are too large to prove.
Measurement pitfalls
Each of these produced a misleading result in early runs. The protocol above is designed to rule them out.
Inconsistent handling of returning pods.
Pods regularly vanish from one hourly snapshot and reappear later. One of our pipelines remembered where they had been; another treated them as new arrivals. The two disagreed, and the first explanation offered for the gap was wrong. Every comparison now states which reading of returning pods it uses.
Quality gains that depended on migrations.
One configuration matched a slower pipeline’s quality inside the time budget — by relocating running pods on a scale no operator would accept. Migrations are now reported beside every score, and a configuration that needs that many is not counted as a pass.
Search results worse than the starting construction.
Reported together with the construction, the search looked fine. Measured apart, it was making layouts worse: the construction was not kept as a candidate, moves were steered by a proxy rather than the metric, and translating the seed into the search’s slot model unseated multi-GPU pods. That is why the construction is now scored separately in every run.
A metric gain that reduced admission.
On one pool, a search tuned hard on the metric scored far better while admitting almost none of the next hour’s real arrivals: it had filled the wholly free servers that real work needs. That is why admission is a standing gate rather than an optional extra.
Conditions for results
Our runs so far cover a short window of the trace. Before any figure goes on this page, the result has to hold under the conditions below:
- More of the trace. Several days and different hours of the day, not one window.
- The strict reading of returning pods. Results under both readings — a returning pod keeps its old slot, or it is a fresh arrival — and the stricter one quoted.
- Admission against production itself, not only against our own earlier configurations.
- Construction and search reported apart, so it is clear how much of any gain is the engine and how much a good greedy rule.
- Migrations next to quality, against production’s own relocation rate.
- Timing on a quiet machine, with the warm-up stated, over consecutive hours.
- The limits stated with the numbers: hourly decisions are an optimistic stand-in for a truly online scheduler; host memory is not in the trace; the largest pools are too big to prove optimal.
Beyond that, we are building a head-to-head against open-source Kubernetes GPU schedulers on simulated live clusters, under ground rules fixed before any run. The results will be published here, with the protocol above, when they pass it.