Preserving Human Judgment in the Age of Advanced AI: The Key Challenge for Responsible AI Design

For the fifth consecutive year, MIT Sloan Management Review and Boston Consulting Group (BCG) have worked with a global panel of AI experts to explore the state of responsible AI implementation. Recent research warns that excessive dependence on AI may weaken our cognitive and ethical judgment, leading to what experts call "epistemic de-skilling." This concern has even spurred measures such as New York City's ban on generative AI in schools to protect children’s development.

When asked if preserving trustworthy human judgment amidst advancing AI is the defining challenge of responsible AI, 65% of the experts agreed, emphasizing that human judgment is a critical skill at risk of erosion but essential to preserve. They argued that human judgment brings context, wisdom, and accountability that AI lacks. However, 19% disagreed, citing the inherent flaws and limitations of human judgment, while 15% remained neutral. The consensus is that the goal isn’t preserving human judgment for its own sake but designing AI systems that enable meaningful human judgment where it matters most.

Experts highlighted that human judgment remains vital for high-stakes decisions, ethical trade-offs, and situations where AI lacks context or encounters novel challenges. But human judgment itself has weaknesses, including fatigue and bias, and simply keeping a human "in the loop" does not guarantee trustworthiness. Overreliance on AI risks displacing human judgment and reducing the skills needed to verify and intervene effectively.

The path forward lies in human-machine collaboration designed to leverage strengths of both. Organizations should identify where human judgment is critical, deliberately build opportunities for humans to exercise judgment, and use training and feedback loops to maintain and improve judgment skills.

Recommendations include defining roles for human judgment at the earliest design stages, creating real opportunities for intervention, practicing judgment regularly, tailoring training by experience, and continuously assessing the effectiveness of human oversight. This careful balance can preserve human agency and accountability while maximizing AI’s value.

In essence, responsible AI means designing systems that keep human judgment strong, trusted, and active—when and where it truly counts.