Mamba-2 kernel tuning shows gains without claiming faster end-to-end training

Jane Street engineers and NVIDIA describe a Mamba-2 forward pass written with cuTile, a Python language for tiled GPU computation. They divide the work into cumulative sums, state propagation and output calculation, then tune memory layout and matrix operations. NVIDIA reports 1.14–1.82 times the speed of its Triton reference on a B300 GPU. These are forward-kernel results, not measured improvements to complete training runs.

NVIDIA Technical Blog · Oct 19, 2026

Why it matters

A concrete kernel walkthrough exposes where memory movement and hardware-friendly matrix operations determine performance.

More context

The implementation is available in TileGym. The reported setup uses cuTile 1.5.0, CUDA 13.3 and a single B300. Study the code and match its software and hardware assumptions before attempting a comparison; tuning searches input-dependent block sizes and occupancy choices.

Keep in mind

The backward pass is outside this comparison. Different sequence shapes, GPUs and training pipelines can change the benefit, and the vendor’s measurements were not independently reproduced.

Original read