Today, we release updated 4-bit checkpoints for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B trained with Quantization-Aware Distillation (QAD). QAD is a technique to distill a high-precision teacher model into a quantized student model. These checkpoints keep the low memory footprint and high throughput of Q4_0 GGUFs while recovering most of the accuracy lost to quantization: all four land at roughly 97% of their BF16 averages.

The QAD GGUFs are available today on Hugging Face: LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B.

Benchmarks

For all four models, we compare their released GGUFs produced with post-training quantization (PTQ) against the trained QAD Q4_0 checkpoints on a benchmark suite spanning reasoning, instruction-following, tool use, and agentic capabilities: GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. The BF16 GGUF serves as the in-format ceiling. We also add one scale-appropriate math evaluation: GSM8K for LFM2.5-230M and LFM2.5-350M, and AIME25 for LFM2.5-1.2B-Instruct and LFM2.5-2.6B. We report the mean across five repeats.

Across model sizes, QAD substantially improves the Q4_0 checkpoint, with the strongest recovery generally occurring in the smaller models where PTQ Q4_0 incurs a larger relative quality loss. We quantify recovery as the fraction of the BF16-to-Q4_0 quality gap that QAD closes. QAD closes 70.6% of the gap for LFM2.5-230M, 73.4% for LFM2.5-350M, 65.5% for LFM2.5-1.2B, and 48.4% for LFM2.5-2.6B. The resulting checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of their respective BF16 baseline performance.

Deployment tradeoffs

To place the recovery in deployment terms, we plot the seven-task average benchmark score against llama.cpp decode throughput, measured in tokens per second across four hardware backends. QAD Q4_0 uses the native Q4_0 throughput because the two checkpoints have the same GGUF format, tensor layout, and runtime path.

• MacBook Pro: GPU inference on an Apple M5 Max; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown.
• NucBox EVO-X2: GPU inference on an AMD Ryzen AI Max+ 395; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown.
• Samsung Galaxy S26 Ultra: Arm CPU inference on a Qualcomm Snapdragon 8 Elite Gen 5; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown, with F16 serving as the practical full-precision reference.
• Raspberry Pi 5: Arm CPU inference on a Broadcom BCM2712. For the 230M, 350M, and 1.2B models, throughput is measured using a shorter decode-only run of 32 generated tokens with no prefill, with F16 as the full-precision reference.

LFM2.5-230MLFM2.5-350MLFM2.5-1.2BLFM2.5-2.6B

Natively, Q4_0 is the weakest 4-bit format in terms of quality and the fastest across our backends. This is precisely the tradeoff QAD targets. QAD Q4_0 matches Q5_K_M quality within run-to-run variance at 230M and 350M and comes within half a point at 1.2B, while staying smaller and faster. 

It also beats Q4_K_M, the standard 4-bit option, and matches Unsloth’s UD-Q4_K_XL, a strong external post-training quantization checkpoint.

Get Started

The QAD GGUFs are available today on Hugging Face: LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B.

Citation

For citations, please use the following reference or BibTeX:

@article{liquidAI2026Q40,
  author = {Liquid AI},
  title = {LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/qad},
}