DeepSeek built a speculative decoding module that speeds up its production models’ text generation by more than 50 percent without sacrificing accuracy, then made its technique open source.
What’s new: Xin Cheng and colleagues at Peking University and DeepSeek introduced DSpark, a speculative decoding method in which a small model (called a draft module) generates tokens for a large language model to verify in a single pass. The team applied DSpark to their DeepSeek-V4 models and later released the checkpoints DeepSeek-V4-Pro-DSpark and DeepSeek-V4-Flash-DSpark, which add the draft modules to unchanged preview weights of DeepSeek-V4-Pro and DeepSeek-V4-Flash. The modified DeepSeek models are free to download from Hugging Face under a commercially permissive MIT license.
Key insight: In speculative decoding — the technique DSpark builds on — a small draft module proposes a block of tokens and the large model it serves checks the whole block in one pass. That pass computes the large model’s own next-token choice at every drafted position simultaneously, so it keeps the longest run of drafted tokens that match those choices. This speeds up text generation while the verification step maintains text quality. The authors identify three components that affect speed: the cost of drafting tokens, how many drafted tokens survive the check, and how much checking the large model does. Earlier drafters traded the first two against each other and left the third fixed regardless of server load. DSpark works on all three, but its main contribution is adjusting the third dynamically, verifying more under light server loads and less under heavy server loads.
How it works: A DSpark draft module attaches to the larger target model, which stays frozen. DeepSeek trained only three parts of the module: a drafting backbone, a small sequential component, and a confidence head, since the module borrows the target’s embedding layer and output head. In offline experiments, the team fed prompts from Open-PerfectBlend, an open dataset, to the target model and trained the drafter on the target model’s responses. Training pushed the drafter to match the target model’s token probability distribution and taught the confidence head to estimate each token’s odds of acceptance.
- The authors adopted DSpark’s backbone from DFlash, an earlier drafter from another team. Like DFlash, DSpark proposes tokens for every position in a block in a single pass. One pass costs the same, however long the block, so a parallel drafter can afford more layers than a token-by-token drafter, making its early guesses stronger. But because the drafter predicts each position independently, it can mix and match across valid continuations; in the authors’ example, where the context could continue “of course” or “no problem,” it may splice together “of problem”. Accuracy falls sharply toward the end of a block, wasting computation.
- To address this, the authors added a sequential component, a compact lookup the authors call a Markov head, which adjusts each position’s token probabilities based only on the token drafted just before it. For example, after the drafter picks “of,” the odds shift toward “course” and away from “problem.” The step runs token by token but is so small that lengthening drafts from 4 tokens to 16 tokens only adds 0.2 to 1.3 percent to each round’s latency compared to the unmodified DFlash backbone.
- For each drafted token, a confidence head estimates the probability that the token will survive verification given that all earlier tokens in the block survived. Such estimates run overconfident, and choosing how many tokens to verify requires their true magnitudes rather than a mere ranking of stronger and weaker tokens. The authors added a calibration step that rescales the estimates, position by position, until chains of them match acceptance rates observed on held-out data.
- At serving time, a scheduler multiplies the per-token confidence estimates together, since a draft survives to a given length only if every token before it survives, yielding a survival probability for each possible draft length. The scheduler then sets each request’s verification length to maximize expected total output across users, consulting a profile of the system’s speed at each load level, measured once at startup. Under light traffic, it verifies long drafts to cut waiting times; under heavy traffic, it drops drafted tokens with low odds of acceptance to free capacity for other users.
Results: DeepSeek evaluated DSpark offline against its own retrained versions of earlier drafters on open-weights models and in production against its previous serving setup. In all cases, DSpark outperformed competing draft modules.
- DSpark raised the average number of tokens accepted per verification round, a measure of draft quality. Compared to the sequential drafter EAGLE-3, it gained 30.9 percent, 26.7 percent, and 30.0 percent on Qwen3-4B, Qwen3-8B, and Qwen3-14B, respectively. Compared to the parallel drafter DFlash, it gained 16.3 percent, 18.4 percent, and 18.3 percent on the same models. The gains held on Gemma4-12B, a model family separate from Qwen3, which suggests the advantage isn’t specific to one lineage.
- In production, DSpark made DeepSeek-V4-Flash generate tokens for each user 60 percent to 85 percent faster and DeepSeek-V4-Pro 57 percent to 78 percent faster than they did with DeepSeek’s previous production drafter, MTP-1, which proposed one token per cycle.
- DeepSeek compared DSpark to its existing draft module at different hardware speeds. At 80 tokens per second per user for DeepSeek-V4-Flash and 35 tokens per second per user for DeepSeek-V4-Pro, DSpark raised the total tokens generated per second across all users by 51 percent and 52 percent, respectively. At 120 tokens guaranteed per second per user and 50 tokens per second per user for the respective models, gains reached 661 percent and 406 percent. Practically, this amounts to 60 to 85 percent boost in per user generation speeds. DeepSeek’s old drafter nearly failed at the higher speeds, so the authors read the figures as marking new feasible operating points rather than just speedups.
Behind the news: Authors at Google Research first described speculative decoding in 2022. The technique has since become common in production serving, and designs for token-drafting modules have multiplied. Early drafters were sequential, generating draft tokens one at a time: EAGLE-3, the drafter DSpark was compared against, works this way, using features from several layers of the target model to predict each new token. Parallel drafters then broke the sequential bottleneck by drafting an entire block of tokens at once: DFlash, which supplies DSpark’s backbone, does so with a small diffusion model. Its authors reported up to 2.5 times the speedup of EAGLE-3, and Nvidia said it boosted inference up to 15 times on the company’s Blackwell GPUs. DSpark keeps the parallel speed of drafters and the coherence of sequential drafters. DeepSeek’s used a simple sequential drafter since DeepSeek-V3 until DSpark, combining the advantages of sequential and parallel drafters, replaced it two weeks after the DeepSeek-V4 preview launched.
Why it matters: Every token a deployed model outputs costs its provider money and makes the user wait, both of which limit what developers can build. Common remedies like using smaller or quantized models sacrifice accuracy. DSpark cuts cost and time without touching model weights or degrading output: On net, the gain converts into cheaper tokens and faster responses.
We’re thinking: DeepSeek made its name by training strong models inexpensively and sharing techniques like reinforcement-learning-based reasoning models through open papers. We’re glad that it’s also optimizing model serving and sharing the results and code for anyone to use.