A widely used technique for improving AI chatbot accuracy is inadvertently discarding nearly one-third of the questions it was designed to answer, according to a new technical study published on The New Stack. The research examined production pipelines that break complex user messages into separate sub-questions, retrieve evidence for each, then merge and rerank the results before generating a response. While these systems successfully located the correct documentation for every question, 31.1% of sub-intents received zero space in the final context window sent to the language model—a phenomenon the author terms "context starvation." The study argues that allocation policy, not retrieval quality, has become the dominant bottleneck in multi-intent question systems.
The experiments used GitLab's public documentation—10,000 chunks spanning 2.2 million tokens—and tested 100 multi-intent queries composed from 61 labeled single-intent questions. Each query combined between two and seven separate questions, randomized by position and topic. At a 2,000-token context budget, the standard production pipeline starved 31.1% of sub-intents whose evidence had already been retrieved, beating no decomposition by nine percentage points. A simple flat split—dividing the budget equally by the number of questions—beat the production default by fourteen points. Reserving one passage per sub-intent before greedy packing delivered only seven points of improvement when selections were scored against the original query, but reached 89.4% coverage when each reservation was scored against its own sub-question instead. That change cut allocation starvation from 30.6% to 10.1%. Questions appearing in the middle of a seven-part message starved at rates reaching 48.0%, while first and last questions starved at 1.3% and 13.3% respectively. Sub-intents spanning distinct topics starved twice as often as questions drawn from a single topic—38.1% versus 18.3% in seven-part queries.
The research included a head-to-head comparison between retriever quality and allocation policy. Upgrading from a quantized small embedding model to a stronger base model with reranker improved coverage by 12.7 percentage points on the production-default pipeline. Changing only the allocation policy on the same stronger retriever—switching to fragment-scored reservations—delivered a 20.5-point gain. The weaker retriever paired with the fragment-scored floor reached 84.5% coverage, beating the stronger retriever with greedy packing at 68.9% by sixteen points. "A worse retriever with a better allocator wins," the report states. The author tested a real LLM decomposer blind to the number of sub-questions and found it disagreed with ground truth on 41% of queries, but on inspection was correct every time—it identified five supposedly single-intent questions that actually contained two distinct information needs.
The failure mode matters because starved sub-intents don't produce silence—they produce unsupported answers. When correct evidence reached the packed context, the generated reply addressed that question 100% of the time across 261 cases. When a sub-intent was starved, the system answered it anyway 48.1% of the time based on whatever else happened to be in the window, explicitly flagged the gap 45.6% of the time, and went silent only 6.3% of the time. The report attributes the problem to greedy packing after reranking: merging all retrieved passages, scoring them against the original multi-part query, then filling the context window top-to-bottom with no fairness constraint. That scheduler protects high-scoring passages—often multiple chunks answering the same question—while squeezing out mid-scoring evidence for other sub-intents. Below roughly 1,000 tokens per sub-intent, allocation policy dominates performance; above that threshold, everything fits and the allocator becomes irrelevant.
The report recommends that production teams reserve one passage per sub-intent and select each reservation by relevance to its own fragment rather than the combined query, rerank against fragments instead of the original message, and log per-sub-intent coverage as a leading indicator of unsupported generation. It advises computing context budget divided by average distinct questions per message: if that number falls below 1,000 tokens, allocation policy costs more than retriever quality, and the reranker upgrade in the backlog will deliver less than reserving one slot per question. The author calls for teams to log how many sub-intents end up with zero passages in the packed context for every multi-intent request, warning that "a support system that cannot tell you which question it dropped will keep answering that question anyway, about half the time, out of whatever else was in the window." The cheapest intervention—fragment-scored floors—takes an afternoon to implement and outperformed a full retriever upgrade by eight points in the tests conducted. Systems architects have spent months optimizing the wrong chokepoint: the ranker models that score passages matter less than the four-line policy that decides which ones survive into the window the model actually reads. For organizations running customer support or internal knowledge bots on multi-turn conversations, this suggests an immediate audit of packing logic may yield faster returns than the embedding model fine-tuning currently consuming engineering cycles.