Large language models require roughly 100,000 times more words than children to achieve fluency in human language, according to an August 2026 article in MIT Technology Review examining the data efficiency gap between human and machine learning. While a preteen raised in a linguistically rich environment may have encountered around 100 million words, and perhaps 300 million by age 20 when including reading, modern LLMs like Meta's Llama 3.1 consume approximately 15 trillion tokens during pretraining. The staggering difference in scale—which cognitive scientists call the data efficiency gap—raises fundamental questions about how children manage to master language from such limited exposure, and whether reverse-engineering their learning process could produce more efficient AI systems.
The contrast becomes vivid through analogy: if all the words used to train a modern LLM were printed on paper, the stack would reach past the International Space Station, while a human preteen's 100 million words would pile just 20 meters high. Toddlers typically begin producing grammatically correct sentences after hearing somewhere between 10 million and 30 million words. Frontier AI models could be pretraining on 10 times more data than Llama 3.1's 15 trillion tokens, yet easily available training data from the internet may run dry as soon as the 2030s. One recent model trained on about 100 million words—the 2024 winner of the BabyLM competition, called GPT-BERT—managed to outperform Meta's Llama 2 70B on one benchmark, despite the latter being pretrained on roughly 15,000 times more data.
Michael C. Frank, a cognitive scientist at Stanford University, describes the paradox this way: "We still have to burn down a forest and scrape the entire sum of all human knowledge to re-create this milestone that happens in our living rooms over the course of a year." He adds that training GPT-2 on 30 million words produces "a nonsense generator; you don't get a kid." The article notes that developmental psychologist Alison Gopnik of UC Berkeley was surprised that LLMs could learn syntax at all through pure statistical pattern matching, stating she "didn't think that was going to turn out to be true." According to the piece, researchers are now training models on headcam footage from babies' daily lives—including a Princeton project that recorded 12 hours daily across the first 1,000 days of 17 children's lives—though these multimodal models remain far from achieving childlike fluency.
The data efficiency gap exists because children possess several learning advantages that current AI models lack. Kids actively explore their environment and choose their own data, constantly experimenting in ways that maximize their ability to make predictable impacts on the world, research shows. They interpret information differently when they recognize an adult is deliberately teaching them, reasoning not just about the evidence but about the teacher's knowledge and intentions. Children are also aware of their own knowledge gaps and driven to fill them, unlike models that learn passively and in isolation from static datasets. Attempts to incorporate these features—such as curriculum learning that starts with simple data and progresses to complexity, or allowing models to interact with other models socially—have so far failed to match the performance of standard approaches in competitions like BabyLM.
Closing the data efficiency gap could enable universities and smaller organizations to train competitive models without massive computing resources, and make it possible to build capable language models for minority languages that have only tens of millions of tokens available—roughly a toddler's exposure. The article suggests that while current frontier labs aren't racing to borrow tricks from children, the next generation of AI architecture beyond the transformer may incorporate lessons from developmental psychology. Yet the most compelling reason to pursue this research may be scientific rather than practical: as Alex Warstadt, a linguist at UC San Diego who co-founded the BabyLM competition, explains, for 100,000 years humans were the only language users in the universe, but now there's another linguistic entity, creating the first opportunity for comparative studies of how language can be learned. Business leaders betting on continued AI scaling should recognize that efficiency gains modeled on human learning may reshape competitive advantage before the data well runs dry, while the failure of models to match children despite vast resources suggests that raw scale alone won't solve intelligence.