Token-by-token AI is getting competition. Whole-sequence scoring, parallel drafting, cheaper memory models, and self-updating systems all push toward coherence over confident-sounding guesswork.
The story
Trend Analysis – *State-space models were a footnote in most ML classes two years ago.* Now they’re poking a bigger question in my brain: what replaces token-by-token storytelling as the core “how it thinks” model for AI?
Here’s what is shifting:
→ *Whole-sequence scoring is back on the menu.* Energy-style approaches score the entire output in one go, closer to asking, “Does this hang together?” NVIDIA’s diffusion-style language work and Eve Bodnia’s scalar scoring of reasoning traces hit the same human need: coherence you can inspect, where the nonsense becomes findable. Boltzmann-GPT’s “world model” vs “language mouth” split is the same move – separate plausibility from speaking, so confidence is earned, not sprayed.
→ *Parallel refinement replaces the single-track sentence.* Diffusion language models draft many tokens together, then clean them up in rounds. InclusionAI’s LLaDA 2.1 and Inception Labs’ Mercury 2 are chasing lower wait time and mid-generation editability. This feels less “next word” and more “revise the draft,” like changing a cluster of beliefs without breaking the whole page.
→ *Attention no longer gets to be the default.* Attention is the “look back” habit; it’s expensive at long lengths. State-space models like Mamba-2 push a cheaper memory shape, and NVIDIA and Tencent are blending these ideas into production. More memory doesn’t equal more wisdom, but the compute shape still matters.
→ *Systems that change their own rules while running are arriving.* Google’s HOPE keeps layered memory and updates its learning rules during inference. Growth, with drift risk. Guardrails become part of the architecture, not a policy memo.
Notice how these overlap? Holistic scoring, parallel refinement, efficient sequence memory, and adaptive memory all want the same thing: reliability without pretending certainty.
Recent Comments