Xavi Amatriain, Expedia Group’s chief AI and data officer, shared insights at VB Transform 2026 about how evaluation metrics (‘evals’) are becoming the new product requirement documents (PRDs). He explained that evals, which include red teaming and security checks, are embedded into product design from the start, simplifying the development process especially with AI-assisted code. Drawing from his experience at Google, Amatriain emphasized the importance of specialized AI agents over monolithic models, advocating for a system composed of narrowly scoped components to improve security and functionality.
Amatriain also discussed Expedia’s risk-calibrated governance framework, which involves layered principles, tools, and automated checkpoints tailored to the level of risk each AI agent poses. He stressed minimizing guardrails to avoid biasing user feedback, while maintaining essential safeguards based on risk. Expedia’s AI agents blend real-time data and user context to provide timely, trustworthy recommendations without taking final actions like bookings, ensuring user control remains paramount.
On security, Amatriain highlighted the growing threat from other AI systems and the necessity of embedding security considerations early in design. He underscored the critical role of continuous monitoring and rapid incident response, noting that many companies have already faced AI agent security incidents. These insights illustrate the evolving challenges and strategies in deploying AI responsibly at scale.
