AI agents are becoming more capable of autonomously handling business tasks, yet IT leaders remain cautious about granting them access to critical enterprise systems. One core issue is how AI reliability is assessed. Traditional metrics like EVAL scores provide only a limited snapshot of performance and often miss factors like consistency across varied prompts and environments. Bryan Silverthorn, director of Amazon’s AGI Autonomy research lab, explained that their team is shifting focus beyond performance benchmarks to a framework emphasizing consistency, robustness, predictability, and safety.
Amazon’s approach includes decoupled systems, such as sandboxed environments where AI agents suggest changes that humans review before implementation. This method helps build trust, especially in sensitive sectors like finance, where the risks are high. VentureBeat’s Q2 Pulse Research showed that only 4% of senior tech leaders fully trust model guardrails, with many concerned about unauthorized access and prompt manipulation.
At VB Transform 2026, Silverthorn will detail Amazon’s method of engineering reliable agentic AI, guiding companies from simple agent wrappers to advanced multi-tool architectures capable of self-correcting mid-task. The event also features sessions on safe AI deployment, including insights from Waymo on building efficient, safe AI for real-world applications.
If interested in attending, senior technology leaders can request complimentary passes to VB Transform 2026 or purchase tickets online.
