For AI systems to continually improve in knowledge-intensive tasks, they require either autonomous self-improvement or human evaluators to catch errors and provide quality feedback. Industry efforts have heavily focused on the former, overlooking the critical role of human evaluators. The problem is compounded as entry-level jobs training new experts—like document review, research, and code review—are increasingly automated. This limits the next generation’s ability to develop the deep judgment essential for evaluating AI effectively.
Unlike games like Go, where AI can learn through stable rules and clear outcomes, knowledge work is dynamic and evolving, requiring human insight to interpret changes and provide nuanced evaluation. Historical knowledge loss typically happens due to external crises, but here it results from rational economic decisions that phase out human expertise. Entire fields risk “hollowing out,” where models perform well superficially, but the human capacity to innovate and validate disappears.
Rubric-based evaluations help reduce dependence on human evaluators but fall short because they only measure explicit criteria, missing deeper intuitive judgment. While AI development should continue, the human evaluation gap needs urgent attention. Ignoring this risks degrading the foundations that underpin AI’s knowledge and progress.
Ahmad Al-Dahle, CTO of Airbnb, calls for treating the loss of expert evaluators as a serious challenge—one requiring research and investment equivalent to the focus on AI capability advancements.
