Your Homegrown System Was Right in 2018. It's a Liability Now.
Millions of batch jobs, migrated in 4 weeks, and nobody noticed
TLDR
Netflix wrote Compute Managed Batch (CMB) in 2018 because nothing in the Kubernetes ecosystem did what they needed. Hierarchical tenants. Fair sharing. Capacity reservations. Priority queues. They built all of it themselves.
Seven years later, Kueue does all of that, plus the features CMB never shipped. So Netflix ripped out CMB’s queuing and scheduling logic and replaced it with Kueue. Millions of batch jobs moved over. The production migration took 4 weeks. Users did nothing and noticed nothing.
The interesting part is not the swap. It is how they decided when to stop maintaining their own thing, and how they migrated without ever exposing a new API to customers.
What CMB Actually Was
CMB is a managed batch layer sitting on top of Titus, Netflix’s container platform. Users submit jobs that run to completion. CMB queues them, orders them by priority, and enforces capacity per tenant.
The tenant model is a tree. Internal tenants are branches, they exist to group things and hold capacity for their subtree. Leaf tenants are where work actually lands, and only leaves have queues. An org might run one tenant for everything, or mirror its entire team structure in the tree.
Capacity came in two flavors:
Reserved. A leaf tenant with a reservation partitions that capacity out of the hierarchy. Nobody else can take it. That guarantees throughput.
Shared. The Compute team runs a global burst pool. Any tenant can pull from it. It is fair-shared across tenants.
Here is the flaw, and it is the flaw that eventually forced the rewrite. CMB fair-shared the pool at admission only. There was no preemption. Once a job was admitted, it ran to completion no matter how the demand picture changed underneath it. A tenant that got in early held its slice until it finished. Fair sharing that only applies at the door is not fair sharing. It is a queue with extra vocabulary.
The other cost was structural. CMB sat far away from the actual Kubernetes clusters. Titus federates jobs across cells, so CMB talked to one endpoint and stayed blissfully unaware of cluster topology. Great for simplicity. Terrible when you want to build preemption, because preemption is a scheduler-level concern and CMB was several abstraction layers above the scheduler. Every new feature meant re-implementing something Kubernetes already had a place for.
Why Kueue and Not the Others
Netflix looked at YuniKorn and Volcano too. Both replace the pod scheduler. That was the disqualifier.
Titus has its own scheduling profiles. Swapping out kube-scheduler means fragmenting job placement, and fragmented placement means worse bin-packing, which means burning money on idle capacity. If you have ever watched a fleet run at 40% utilization because two schedulers disagreed about where things go, you know exactly what this costs. Netflix has written before about how much compute efficiency is worth at this scale.
Kueue does not touch pod scheduling. It sits above it and decides whether a job gets admitted, then hands the pods to the existing scheduler. That single design choice made it the only option that composed with what Netflix already ran.
The rest of the case was straightforward:
Multi-tenant quota over heterogeneous hardware, which is table stakes when your fleet is a zoo of instance types
Works on
v1.Podandbatch/v1.Job, and also onRayJob/RayClusterfor the ML workloads coming down the pipeNative preemption, all-or-nothing scheduling, topology-aware scheduling. All the things CMB wanted and never got
Real adoption momentum, which matters more than feature checklists over a five year horizon
The Migration: Change Nothing the User Can See
They called it Netflix Batch. Three tenets, and the first one is the one worth stealing:
Zero lift for CMB users. Completely transparent.
No regression in container launch rate or max throughput.
Replace CMB queuing and scheduling with Kueue.
