Researchers from Shanghai Jiao Tong University and collaborators have introduced MemRL, a novel framework for large language model (LLM) agents that enables learning new skills without costly fine-tuning. MemRL equips agents with episodic memory, allowing them to retrieve and leverage past experiences to tackle previously unseen challenges, continuously refining strategies using environmental feedback. This approach contrasts with traditional fine-tuning, which risks catastrophic forgetting, and non-adaptive retrieval methods like RAG. Instead, MemRL freezes the LLM’s parameters and relies on an external, evolving memory organized as “intent-experience-utility” triplets, which include a Q-value score that reflects the success of past experiences. Agents use a two-phase retrieval, first selecting relevant memories semantically, then re-ranking by utility to prioritize effective solutions. Reinforcement learning is integrated, updating memory scores based on outcomes without adding significant computational load. Tested on industry benchmarks, MemRL consistently outperformed RAG and other models, especially in complex environments requiring exploration, highlighting its superior adaptability and generalization. Importantly, MemRL integrates easily into existing tech stacks and offers transparency and reversibility in its learning process. This technology represents a significant step toward self-evolving AI agents that can adjust dynamically in real-world applications without the need for repeated full model retraining.
Back
