Teams training AI agents can buy expensive GPUs and still spend most of their time waiting, according to Di Jin, VP of AI Training at Nebius, who told The New Stack that the systems running agent tasks often become the main constraint on training speed and cost. Agents must try tasks and generate results before models can learn from them, and a handful of slow attempts can delay the next training update. Engineers coming from pretraining or supervised fine-tuning tend to underestimate the demands these systems place on infrastructure, Jin says, and "if you staff and design it as a training project, inference becomes the bottleneck nobody owns."

In one experiment Nebius conducted on a Terminal-Bench-based workload, adjustments to routing, sandbox execution, and batch assembly cut the time to collect a 256-trajectory batch by 66.9%. That measurement covered batch collection with fixed weights, excluding the trainer's forward pass, backward pass, and optimizer operations, and did not reflect end-to-end training speed. Rollout generation frequently consumes most of a training step's wall-clock time, making batching, cache management, and scheduling critical to how quickly the model can improve. Agent trajectories vary in length—many finish fast, while others run long, leaving training GPUs idle while they wait for stragglers to complete. Jin recommends engineers track time to a trainable batch alongside throughput, since average tokens per second can mask the long tail that holds up synchronous batches.

According to Jin, data and environments consume most of the effort in successful reinforcement learning post-training, with algorithm design representing the smallest portion while infrastructure determines iteration speed. "Building environments is the new data curation," he says. Supervised fine-tuning relies on a collected dataset, but agentic RL has the model generate trajectories by interacting with an environment, so reproducible containers, codebases, websites, and APIs with realistic states help learning transfer to real use. Jin warns that staleness—using data generated by an older model version—isn't inherently harmful, but unmeasured staleness is. The useful metric is divergence between the policy that produced a trajectory and the policy being trained, not a fixed limit on how many steps old the data can be.

Jin argues that the mismatch between rollout engine and trainer probabilities can introduce systematic bias into training updates, even when both start from identical weights, due to quantization, kernels, batching, and expert routing in mixture-of-experts models. He recommends aligning numerics first, then measuring and correcting remaining differences. Engineers should also distinguish infrastructure failures from model failures and exclude the former from policy-gradient updates, while failures caused by the model's own actions—like triggering a timeout—should still count as model failures so errors don't become a way to dodge penalties. "Your most expensive GPU is often waiting on your cheapest container," Jin says, and he advises scaling CPU and sandbox capacity independently of GPUs and provisioning it generously, since idle containers cost relatively little. Jin tells smaller teams to begin with a simple setup and add asynchronous complexity only when measured bottlenecks justify it, checking batch readiness, policy freshness, numerical alignment, environment reliability, weight publication, and evaluation precision before scaling. Organizations that treat the inference engine as part of the reinforcement learning optimizer—rather than a separate training concern—stand a better chance of keeping their most expensive resources productive instead of idle. The real competitive edge may lie less in model architecture than in the unsexy work of scheduling sandboxes and managing stale data.