News and Updates

We introduce LongevityBench, an open suite of 17 tasks spanning five biodata domains, and use it to assess 18 frontier AI systems from six developer teams.

New recipe expands pre-trained LLM tokenizers in place, cutting non-English token counts up to 4x with no quality loss.

FPRM is a Transformer-based Fixed-Point Reasoning Model for looped architectures, using pre-norm layers, residual scaling, and fixed-point halting to improve signal propagation, adapt compute to task difficulty, and deliver strong results on Sudoku, Maze, state-tracking, and ARC-AGI.

NeuroLDS introduces the first machine learning framework for finite low-discrepancy sequences, generating point sequences with low prefix discrepancy and outperforming classical LDS methods across numerical integration, robot motion planning, scientific machine learning, and simulation.

SwitchCraft is a programmable framework for designing state-switching proteins using backpropagation through compositional constraints and structure prediction models, enabling allosteric regulation, ligand discrimination, and de novo fluorescent biosensor design for biotechnology.

MeMo (Memory as a Model) is a modular framework that adds timely, domain-specific knowledge to LLMs without changing model weights, capturing cross-document relationships, resisting retrieval noise, avoiding catastrophic forgetting, and supporting plug-and-play use with open and closed-source models.

Preconditioned delta-rule recurrences improve subquadratic long-context modeling by incorporating curvature from online least squares. The method links linear attention and the delta rule, introduces efficient variants of DeltaNet, GDN, and KDA, and boosts recall and language modeling performance.

Linear attention offers a computationally efficient yet expressive alternative to softmax attention, maintaining a recurrent state that functions as a linear associative memory. However, recent empirical results indicate that the associative memory of trained linear attention models often exhibits a low-rank structure, suggesting that these models underexploit their capacity in practice. To illuminate this phenomenon, we provide a theoretical analysis of the role of rank in linear attention, revealing that low effective rank can affect retrieval error by amplifying query noise, as well as poorly condition query gradients.

Flow Matching typically relies on white noise sources, a choice often misaligned with the power spectra of natural data, which tend to decay with frequency. To address this, we introduce , a variant of Flow Matching based on an operator-modulated interpolant. This formulation induces a time-varying spectral bias that transitions from the source spectrum to a frequency-decaying bias as the path approaches the data. We validate our method on unconditional image generation tasks, including the scientific Galaxy10 dataset. Empirically, we show that our method is particularly effective when paired with adaptive ODE solvers, where it improves or preserves sample quality while substantially reducing sampling cost compared to standard baselines.

Mixture-of-Experts (MoE) architectures scale computation via sparse expert activations, yet they remain memory-bound because all expert weights must reside in memory. Mixed-precision quantization can substantially reduce this footprint, but existing quantization methods estimate expert importance and assign bits based on calibration data. For frontier MoE LLMs, however, the original training data (and thus its true training distribution) is proprietary and inaccessible. Thus, any calibration set is at best a surrogate and may yield a biased or incomplete view of expert utilization, leading to suboptimal bit allocation. To address these problems, we propose AlphaQ, a novel calibration-free bit-allocation method for MoE quantization.

Large language models excel at reasoning but lack key aspects of introspection, including the ability to anticipate their own success and the computation required to achieve it. Humans use real-time introspection to decide how much effort to invest, when to make multiple attempts, when to stop, and when to signal success or failure. Without this ability, LLMs struggle to make intelligent meta-cognition decisions. Test-time scaling methods such as Best-of-N drive up cost and latency by using a fixed budget of samples regardless of the marginal benefit of each one at any point in generation, and the absence of confidence signals can mislead people, prevent appropriate escalation to better tools, and undermine trustworthiness.

Many methods have been developed to predict static protein structures, however understanding the dynamics of protein structure is essential for elucidating biological function. While molecular dynamics (MD) simulations remain the in silico gold standard, its high computational cost limits scalability. We present DynaProt, a lightweight, SE(3)-invariant framework that predicts rich descriptors of protein dynamics directly from static structures.

Antislop detects and removes repetitive LLM “slop” using inference-time suppression, profiling, and targeted fine-tuning.

State Space Models (SSMs), developed to tackle long sequence modeling tasks efficiently, offer both parallelizable training and fast inference. At their core are recurrent dynamical systems that maintain a hidden state, with update costs scaling with the state dimension. A key design challenge is striking the right balance between maximizing expressivity and limiting this computational burden. Control theory, and more specifically Hankel singular value analysis, provides a potent framework for the measure of energy for each state, as well as the balanced truncation of the original system down to a smaller representation with performance guarantees.

We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under edge latency and memory constraints, we obtain a compact hybrid backbone that combines gated short convolutions with a small number of grouped query attention blocks, delivering up to 2x faster prefill and decode on CPUs compared to similarly sized models.

Large language models (LLMs) are typically trained by reinforcement learning (RL) with verifiable rewards (RLVR) and supervised fine-tuning (SFT) on reasoning traces to improve their reasoning abilities. However, how these methods shape reasoning capabilities remains largely elusive. Going beyond an accuracy-based investigation of how these two components sculpt the reasoning process, this paper introduces a novel analysis framework that quantifies reasoning paths and captures their qualitative changes under each training process (with models of 1.5B, 7B, and 14B parameters on mathematical domains).

Grafting edits pretrained diffusion transformers into efficient hybrids using <2% compute, delivering strong quality and up to 1.43x speedups.

Introducing effective state-size (ESS), a metric that quantifies memory utilization in sequence models to improve initialization and distillation.

STAR uses evolutionary search over architecture genomes to automatically design hybrids that beat Transformers on quality, size, and cache.

Towards a theory of learning dynamics in deep state space models

The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof

Large Scale Dataset Distillation with Domain Shift

LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery

RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation

Position: Future Directions in the Theory of Graph Machine Learning

Mechanistic Design and Scaling of Hybrid Architectures

State-Free Inference of State-Space Models: The *Transfer Function* Approach

Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution

Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation

Graph Metanetworks for Processing Diverse Neural Architectures