DeepSeek has released the inference engine it uses to run its models in production, reversing a decision from last year when the company said such a move wasn't yet feasible. The framework, called DeepInfer, lets developers operate DeepSeek's V4.1 Flash model on their own systems with many of the same performance improvements the company applies behind its API, according to a report published Tuesday on The New Stack. The company said it's sharing the technology to help cut the cost of running large AI models.
Released under the Apache 2.0 license, DeepInfer includes optimized computing kernels and communication libraries—among them DeepSeek's previously open-sourced FlashMLA, DeepGEMM, and DeepEP—alongside tools for spreading inference work across multiple chips. The framework supports speculative decoding, a method that accelerates response generation by predicting and checking multiple tokens simultaneously, and asynchronous scheduling to cut the time processors wait for new tasks. It also allows structured outputs, enabling developers to dictate how model responses should be formatted. The publicly available software currently supports offline inference and includes core features needed for large-scale serving, but several components required to run a complete production service remain absent, including a KV cache management system, infrastructure for separating inference stages across machines, and load balancing across replicas.
Taking to X on Tuesday, DeepSeek said it was making its internal inference technology available to outside developers in an effort to bring down the expense of running large AI models. "We're sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable," the company writes. Alongside DeepInfer, the company is also open-sourcing deepseek-recipe, a companion library that handles how API requests are converted into model inputs and how the resulting output is processed into responses. Available for Rust and Python, it's designed to help developers reproduce the behavior of DeepSeek's official API when running their own deployments. The announcement comes just days after DeepSeek released an updated version of V4.1 Flash, called DeepSeek-V4.1-Flash-1017, touting improvements in coding performance and efficiency.
The release goes further than DeepSeek's previous plans to share its inference technology. Back in April 2025, the company published a document outlining its plans for its inference engine, explaining that it had considered releasing the entire system before deciding to contribute individual components and improvements to existing projects instead. At the time, DeepSeek identified several obstacles: its inference software had evolved from an early version of vLLM into a highly customized system, and the company questioned whether its relatively small research team could maintain another major open-source project. There were also practical difficulties in making the software usable outside DeepSeek's own infrastructure—"The engine is tightly coupled with our internal infrastructure, including cluster management tools, making it impractical for public deployment without significant modifications," the company wrote at the time. DeepSeek hasn't explained how it overcame those earlier difficulties, though the narrower scope of the new release—which opens up much of the engine responsible for running its models while leaving developers to supply some surrounding infrastructure required to operate a full-fledged service—may help explain how the company has made its technology available despite the challenges it identified last year.
The hardware supported from day one carries particular significance. Nvidia is included, as expected, though support is restricted to its SM100 architecture, used in Blackwell data center GPUs. Notably, the other supported platform is Huawei's Ascend 950 NPU series, putting the Chinese chipmaker alongside Nvidia in DeepSeek's list of compatible hardware. Older Nvidia architectures, including Hopper, are conspicuously absent. This ties back to DeepSeek's published open-source plans from 2025, when the company committed to working with hardware manufacturers ahead of future model launches, with the aim of making its models available across different chip architectures from day one. US export restrictions have complicated Chinese companies' access to Nvidia's most advanced chips, adding another dimension to DeepSeek's growing ties with Huawei. In September, Bloomberg reported that DeepSeek was planning to deploy at least 160,000 Huawei Ascend 950DT chips at a new data center in Inner Mongolia, specifically to run its models. The company has largely relied on Nvidia for training, although The Information reported in September that DeepSeek also plans to increase its use of Huawei chips for training, with deliveries expected as early as the fourth quarter.
For developers without access to either chip family, the immediate value may lie in studying the newly available code and adapting its optimizations to other inference engines. Feature support differs between the Nvidia and Huawei implementations—DeepInfer's batch-invariant inference and deterministic sampling features, which help ensure reproducible outputs under specified conditions, are available only on Nvidia hardware. DeepSeek's published performance measurements also use Nvidia GPUs, with four SM100 GPUs used for prefill and 64 for decoding, leaving open the question of how the framework performs on Huawei's chips by comparison. For now, DeepSeek has given the wider open-source community considerably more of its inference technology to experiment with. The dual-platform approach signals a strategic bet on hardware diversity that may reshape how AI companies plan infrastructure investments, particularly as geopolitical constraints force technical compromises that would have seemed unthinkable just a few years ago.