Ai2's StateBench shows frontier models fail at tracking long variable chains

Ai2 released StateBench on Nov. 3, a benchmark of how models track changing values across long documents in four difficulty levels. Frontier models handle simple cases but fall to 13% or lower on the longest hardest problems. Small 275M Gated DeltaNet models trained on the task stayed near perfect on lengths about 250 times longer than training for easier levels.

HEAT

OUTLETS

0

FIRST SEEN

Nov 3, 2026, 12:15 ET

LAST SIGNAL

12h ago