---
title: "LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment"
description: "LFM2.5 Q4_0 GGUF checkpoints trained with Quantization Aware Distillation (QAD), recovering up to 97% of BF16 accuracy for faster edge deployment."
canonical: "https://www.liquid.ai/blog/qad"
locale: "en"
datePublished: "2026-08-19"
dateModified: "2026-10-27"
---

# LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment

Today, we release updated 4-bit checkpoints for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B trained with **Quantization-Aware Distillation (QAD)**. QAD is a technique to distill a high-precision teacher model into a quantized student model. These checkpoints keep the low memory footprint and high throughput of Q4_0 GGUFs while recovering most of the accuracy lost to quantization: all four land at roughly 97% of their BF16 averages. 

The QAD GGUFs are available today on Hugging Face: [LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M-GGUF), [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M-GGUF), [LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF), and [LFM2.5-2.6B](https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF).

## Benchmarks

For all four models, we compare their released GGUFs produced with post-training quantization (PTQ) against the trained QAD Q4_0 checkpoints on a benchmark suite spanning reasoning, instruction-following, tool use, and agentic capabilities: GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. The BF16 GGUF serves as the in-format ceiling. We also add one scale-appropriate math evaluation: GSM8K for LFM2.5-230M and LFM2.5-350M, and AIME25 for LFM2.5-1.2B-Instruct and LFM2.5-2.6B. We report the mean across five repeats. 

![LFM2.5 QAD GGUFs Benchmark plots](https://aypchzzf9pftwuto.public.blob.vercel-storage.com/lfm25_gguf_eval_scores-S4rIwH7v6jM1p9qjMy4B9lwXscNk8V.png)

Across model sizes, **QAD substantially improves the Q4_0 checkpoint**, with the strongest recovery generally occurring in the smaller models where PTQ Q4_0 incurs a larger relative quality loss. We quantify recovery as the fraction of the BF16-to-Q4_0 quality gap that QAD closes. QAD closes 70.6% of the gap for LFM2.5-230M, 73.4% for LFM2.5-350M, 65.5% for LFM2.5-1.2B, and 48.4% for LFM2.5-2.6B. The resulting checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of their respective BF16 baseline performance.

## Deployment tradeoffs

To place the recovery in deployment terms, we plot the seven-task average benchmark score against llama.cpp decode throughput, measured in tokens per second across four hardware backends. QAD Q4_0 uses the native Q4_0 throughput because the two checkpoints have the same GGUF format, tensor layout, and runtime path.

- **MacBook Pro**: GPU inference on an Apple M5 Max; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown.
- **NucBox EVO-X2**: GPU inference on an AMD Ryzen AI Max+ 395; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown.
- **Samsung Galaxy S26 Ultra**: Arm CPU inference on a Qualcomm Snapdragon 8 Elite Gen 5; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown, with F16 serving as the practical full-precision reference.
- **Raspberry Pi 5**: Arm CPU inference on a Broadcom BCM2712. For the 230M, 350M, and 1.2B models, throughput is measured using a shorter decode-only run of 32 generated tokens with no prefill, with F16 as the full-precision reference. 

**LFM2.5-230M**

![LFM2.5-230M Decode Throughput QAD GGUF](https://aypchzzf9pftwuto.public.blob.vercel-storage.com/lfm2.5-230M-decode-throughput-qad-gguf-vnaGPH57CRUIVWvFQIL5xBdWSfXjQm.png)

**LFM2.5-350M**

![LFM2.5-350M Decode Throughput QAD GGUF](https://aypchzzf9pftwuto.public.blob.vercel-storage.com/lfm2.5-350M-decode-throughput-qad-gguf-0IiaRdktfzhPnTahUx4K32vsckexa6.png)

**LFM2.5-1.2B**

![LFM2.5-1.2B Decode Throughput QAD GGUF](https://aypchzzf9pftwuto.public.blob.vercel-storage.com/lfm2.5-1.2B-decode-throughput-qad-gguf-4k6OHRWNY9dwlUNVogjqWFPsyeXZPt.png)

**LFM2.5-2.6B**

![LFM2.5-2.6B Decode Throughput QAD GGUF](https://aypchzzf9pftwuto.public.blob.vercel-storage.com/lfm2.5-2.6B-decode-throughput-qad-gguf-OfG5eH26dyPIMwHZRECbIB89ZbMWtQ.png)

Natively, Q4_0 is the weakest 4-bit format in terms of quality and the fastest across our backends. This is precisely the tradeoff QAD targets. QAD Q4_0 matches Q5_K_M quality within run-to-run variance at 230M and 350M and comes within half a point at 1.2B, while staying smaller and faster. 

It also beats Q4_K_M, the standard 4-bit option, and **matches Unsloth’s UD-Q4_K_XL**, a strong external post-training quantization checkpoint.

## Get Started

The QAD GGUFs are available today on Hugging Face: [LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M-GGUF), [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M-GGUF), [LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF), and [LFM2.5-2.6B](https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF).

## Citation

For citations, please use the following reference or BibTeX:

Liquid AI, "LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment", Liquid AI Blog, Aug 2026.

```bibtex
@article{liquidAI2026Q40,
  author = {Liquid AI},
  title = {LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/qad},
}
```
