LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond

Today, we release an experimental DSpark draft model for our vision-language model (VLM) LFM2.5-VL-3B. As with our recently released LFM2.5-DSpark drafter models for our text-based Liquid Foundation Models (LFMs), it trades a minimal increase in memory footprint for substantially faster decoding without changing output quality. We achieve decoding throughput improvements of up to 2.66× on GPUs and 3.13× on edge devices, with end-to-end throughput gains of up to 2.27× and 2.62×, respectively.

This post covers how we trained the vision-language drafter, the speedups it delivers, and the limits of speculative decoding for vision workloads on edge hardware.

The vision DSpark model is available on Hugging Face, with support in llama.cpp, SGLang, and MLX-VLM.

How speculative decoding works for vision language models

The vision drafter builds on largely the same concept as our previously released LFM2.5-DSpark drafter models [1]. We use the model's hidden states at different layers to speculate about the next k tokens with a lightweight drafter model.

From the drafter's perspective, the input modality is irrelevant because by the time tokens reach the hidden layers, text and image patches are represented as multi-dimensional tensors. Whether those states encode a paragraph or an image makes no difference to the drafting computation. This allows us to apply exactly the same inference algorithm for our vision-language models as for our text-based ones.

Training

We ran ablations to select the training data mixture for our drafter models. The final mixture combines supervised fine-tuning (SFT) data including common vision-language tasks, weighted toward the workloads we expect the model to serve.

For the drafter architecture, we followed the recipe from our text DSpark release: simplified attention-only draft models, with the number of layers and block size selected via ablations on a subset of the training data. We train on the corpus for 10 epochs and report the acceptance rate across the target vision benchmarks. Based on the experiments on the target hardware, we settled on a drafter with 4 layers and block size 9. At inference time, we recommend a block size of 8 or 9, depending on the hardware (see Inference section).

The resulting drafter has approximately 280M parameters (Table 1) and increases the deployed model’s parameter count by just 8.9%. All ablations and training runs were executed exclusively on AMD hardware using Liquid AI's training framework.

Inference

The DSpark draft model for LFM2.5-VL-3B ships with day-one support across the inference ecosystem:

• llama.cpp — GGUF checkpoints for efficient edge inference
• MLX-VLM — Optimized inference for Apple Silicon
• SGLang — a high-performance GPU serving framework

All inference numbers presented use 16-bit processing for both the encoder and the language backbone. Acceleration of quantized models remains outside the scope of this release. All runs were collected on Pipette, the same benchmarking infrastructure behind Liquid AI's public device performance data.

Both configurations are evaluated on six diverse vision-based tasks, following the MMSpec benchmark (General VQA, Text VQA, Image Captioning, Chart VQA, Complex Reasoning, Multi-turn Conversation) [2].

On-device inference. We measure on-device throughput with MLX-VLM on an M5 Max MacBook Pro and llama.cpp on an M3 Ultra using FP16 weights at batch size 1, temperature 0, block size 8, and up to 2,048 output tokens (median answer length 90 tokens).

Speculation improves throughput across all six task categories on both stacks. With MLX on an M5 Max, decoding runs 2.30x to 3.13x faster by task, and end-to-end latency improves by 1.56x to 2.62x. With llama.cpp on an M3 Ultra, decoding improves by 1.57x to 2.14x and end-to-end by 1.30x to 1.77x. Acceptance falls in a similar range on both stacks, roughly 3.2 to 4.5 tokens per verification pass, reflecting that acceptance depends on the drafter and the workload rather than hardware or runtime.

GPU inference. We measure GPU throughput with SGLang on a single H100 80GB in BF16 at batch size 1 and temperature 0 with a block size of 9.

The same drafter delivers 2.04x to 2.66x faster decoding on the H100, with end-to-end improvements of 1.64x to 2.27x. Acceptance ranges from 3.46 to 4.57 tokens per verification pass.

Interactivity

DSpark’s gains extend beyond batch size 1 and persist at higher concurrency. We evaluate the throughput-interactivity frontier, which captures the trade-off between aggregate system throughput and the generation rate each user experiences as concurrency changes. As we increase concurrency, higher arithmetic intensity gradually transitions the decode phase from a memory-bound to a compute-bound regime. Since DSpark verifies multiple draft tokens in each target-model pass, arithmetic intensity is increased proportionately to the block-size.

As Figure 5 shows, DSpark maintains a throughput advantage across all measured concurrency levels, although the gap narrows as concurrency increases. All tests were run in SGLang on a single H100 using a fixed verification window.

Impact of sampling on speed and quality

At non-zero temperatures, the drafter samples a token from its distribution, and the target either accepts it or draws a corrected replacement. Under matched sampling settings, speculative decoding is equivalent in distribution to directly sampling from the target model [3], and thus, the produced output is lossless.

As we increase the temperature, what is affected is the acceptance rate and, therefore, throughput. At lower temperatures, the drafter and target tend to concentrate on the same top tokens. As temperature increases, probability mass spreads to lower-ranked candidate tokens where the models are more likely to disagree. In our experiments, an increase in temperature resulted in a reduced acceptance rate and, as a result, negatively impacted throughput.

Limitations of speculation for vision workloads on edge

In Large Language Model (LLM) inference, prefill is largely compute-bound, and its cost grows (sub)quadratically with prompt length. VLM inference compounds prefill cost: the image must first pass through a vision encoder, after which the language backbone must process the hundreds of visual tokens it produces alongside the text prompt.

This gets more problematic on edge devices, where the compute throughput is substantially lower than on datacenter GPUs. As a result, prefill accounts for a more substantial share of the end-to-end latency. This is most clearly visible through time-to-first-token and decode measurements across Apple silicon and H100. Newer Apple silicon partially narrows this gap by adding a neural accelerator to each M5 GPU core [4].

Speculative decoding only accelerates the decode phase of LLM inference. Vision encoding and prefill remain unchanged. When these stages already account for a substantial share of wall time, even a large decode speedup translates into only a modest end-to-end latency improvement. This is a classic example of Amdahl's law, where the overall speedup is bounded by the portion of the workload that remains unaccelerated. 

Get Started

Our vision DSpark draft model is available on Hugging Face in Safetensors and GGUF formats. 

With LFM2.5, we're delivering on our vision of AI that runs anywhere. These models are:

• Open-weight — Download, fine-tune, and deploy without restrictions.
• Fast from day one — Day-one support for llama.cpp, MLX, and SGLang.
• A complete family — From base models for customization to specialized audio and vision variants, one architecture covers diverse use cases

We can’t wait to see what you build.

Download on Hugging Face

Curious how small models perform on-device? Check out Pipette.

Citation

For citations, please use the following reference or BibTeX:

@article{liquidAI2026vldspark,
  author = {Liquid AI},
  title = {LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/lfm2-5-vl-dspark},
}

References

1. [1]Cheng et al. (2026). DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation. https://arxiv.org/abs/2607.05147
2. [2]Shen et al. (2026). MMSpec: Benchmarking Speculative Decoding for Vision-Language Models. https://arxiv.org/abs/2603.14989
3. [3]Leviathan et al. (2023). Fast Inference from Transformers via Speculative Decoding. The equivalence proof appears in Appendix A.1, “Correctness of Speculative Sampling.”  https://arxiv.org/abs/2211.17192
4. [4]Apple Machine Learning Research (2025). Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU.