DeepInfer
DeepSeek open-sourced DeepInfer under Apache 2.0, the inference framework that powers its production services, with launch support for DeepSeek-V4.1-Flash on NVIDIA SM100 and Huawei Ascend 950.
DeepInfer packages DeepSeek's serving stack, including FlashMLA, DeepGEMM and DeepEP kernels, attention data parallelism, expert parallelism with load balancing, draft-token decoding, batch-invariant inference, deterministic sampling and structured outputs. It was originally built on vLLM and has since been heavily reworked. This release covers offline inference only. The KV cache system, prefill and decode disaggregation and load balancing across replicas are not included.
- Date
- Tuesday 20 October 2026
- Lab
- DeepSeek
- Kind
- infra
- Access
- open weights
This is an inference engine under Apache 2.0, not model weights. The repo shows a benchmark section, but no concrete figures were readable, and coverage says no Huawei benchmarks are published.
Sources
- github.com/deepseek-ai/DeepInfer
- trendingtopics.eu/deepseek-deepinfer-open-source-inference-engine/
- gizmodo.com/deepseek-dials-up-the-pressure-on-openai-and-anthropic-with-open-source-infere
This record was checked and corrected against its sources on 20 October 2026. How we check