Researchers at Mila have introduced Markovian Thinking, a novel technique that significantly enhances the efficiency of large language models (LLMs) in complex reasoning tasks. Through an environment called Delethink, the method breaks down reasoning into fixed-size chunks, addressing the computationally expensive quadratic scaling problem found in traditional long-chain reasoning approaches. This chunk-based system allows models to reason over millions of tokens with fixed memory, yielding a linear compute cost instead of quadratic. Testing on math problems and other complex tasks showed that models trained with Delethink outperformed or matched those using conventional reinforcement learning methods, often with far less computational cost. This efficiency extends to inference, enabling more cost-effective long-horizon AI applications. Importantly, even existing off-the-shelf models exhibit abilities consistent with Markovian Thinking principles. Delethink demonstrates promising scalability with large state-of-the-art models, opening avenues for advanced AI capabilities such as scientific discovery by allowing longer, more extensive reasoning periods.
Back
