Skip to main content

In a survey of 157 enterprises, many are giving AI agents increased autonomy without fully trusting the tests designed to ensure their reliability. Half of these companies have deployed AI features that passed internal evaluations but later failed in real-world customer interactions. Despite this, two-thirds allow AI agents to be deployed to production automatically, without human oversight. The biggest issue is the "evaluation gap"—a disconnect between the autonomy granted to AI agents and the trust in the evaluations meant to oversee them. Only 5% of organizations fully trust automated evaluations, citing poor alignment with real-world outcomes as the primary concern. Moreover, monitoring in production often checks only if the AI is functioning rather than verifying the correctness of its outputs. The market for evaluation tools remains fragmented, with many companies relying on provider-native tools or none at all, though investment in human review and observability is growing. While enterprises are moving towards greater automation, they continue to invest in oversight, highlighting a cautious approach amidst burgeoning autonomy. This points to a critical need for evaluations that better reflect actual performance to ensure AI agents can be trusted as their independence grows.

The Autonomy-Trust Divide in Enterprise AI: Why Organizations Push Ahead Despite Evaluation Shortfalls

Across 101 enterprises, the challenge isn’t accessing AI context—it’s trusting it. Retrieval-augmented generation (RAG) dominates as the leading method to feed AI agents relevant business information, yet many organizations experience their AI confidently providing incorrect answers due to inconsistent or missing context. Despite the rise of provider-native retrieval tools like OpenAI’s file search and Google’s Vertex AI Search overtaking traditional vector databases, a significant trust gap remains. Most enterprises are actively building or piloting a governed semantic layer to address this gap, aiming for hybrid retrieval models that combine accuracy with governance. However, widespread adoption is still in progress, with many caught between the convenience of bundled provider tools and a desire to maintain best-of-breed solutions. This tension highlights that the core issue is not retrieval volume, but the quality, governance, and consistency of AI context—elements critical to AI’s reliable performance in business environments.

Bridging the AI Trust Divide: Why Enterprise AI Struggles More With Trust Than Data Retrieval

A study of 107 enterprises reveals significant gaps in AI agent security as organizations grant these agents system access without adequate controls. Over half (54%) have experienced security incidents or close calls involving AI agents. Despite this, only about a third of enterprises assign each agent a unique scoped identity, while most still share credentials, increasing risk. Additionally, just 30% isolate their highest-risk agents in sandboxes to limit damage if compromised. Security measures primarily rely on native provider tools from OpenAI, Google, and Microsoft rather than specialized agent security vendors. Satisfaction with these controls remains high, though spending on AI agent security is generally minimal, accounting for less than 10% of overall security budgets. Enterprises are divided on whether their AI defenses surpass attackers, with only a third confident they are ahead. A majority plan to update their security tools soon, indicating recognized vulnerabilities amid growing agent use.

Security Challenges with AI Agents: Over Half of Enterprises Face Incidents Amid Credential Sharing and Limited Isolation

A study across 107 enterprises reveals that AI infrastructure spending is growing rapidly, yet most organizations struggle to understand the actual costs involved. Many companies rely on familiar hyperscalers and model-provider APIs, but their next investments are focused on specialized AI compute resources rarely used today. Over half of these enterprises plan to switch or add providers within the next year, placing value on integration and total cost of ownership rather than on token price. However, GPU utilization remains low, with 83% running at 50% capacity or less, and only 44% of companies rigorously track their AI compute expenses. This mismatch between fast spending and poor cost visibility creates a "compute gap"— heavy investment without sufficient control. The report also notes high churn intentions among vendors and an emerging but under-recognized shift from GPU compute to memory bandwidth as inference demands grow. The findings suggest enterprises are eager to expand AI capabilities but need better tools to measure and manage their investments effectively.

The AI Investment Challenge: Enterprises Outpacing Their Ability to Track Costs

Artificial intelligence is dominating conversations across organizations, influencing goals and workflows. However, many leaders overlook critical risks associated with AI adoption. From ensuring content relevance and data quality to preventing over-reliance on autonomous systems, these risks can undermine trust, creativity, and culture if ignored. Industry experts highlight issues such as diminished critical thinking, security vulnerabilities from unsecured AI use, the financial costs of AI deployment, and gaps in user adoption. Success with AI depends not just on technology but on human judgment, transparent governance, and aligning teams with clear goals.

22 Overlooked AI Risks Every Leader Should Acknowledge

Ndidi Oteh discusses how creativity goes beyond being a cultural ideal to become a key driver of innovation, transformation, and sustained business success.

The Impact of Creative Leadership on Business Growth

Are you noticing that many of your product's features go largely unused? Discover how to identify unnecessary feature bloat, remove the excess, and concentrate on the elements that genuinely enhance customer satisfaction and fuel growth.

Cutting Down on Feature Bloat: How to Maximize Your Product's Value

Marketing's influence on business decisions is growing, yet traditional tools to gauge this impact are losing accuracy. Consumer decision-making has evolved, and it's crucial that our methods of measuring marketing effectiveness evolve too. CEOs focusing solely on website traffic numbers could be missing the bigger picture of how marketing drives engagement and conversions in today's landscape.

Rethinking Website Traffic Metrics: Why CEOs Might Be Misjudging Marketing Impact

Your team didn’t overlook your message — the existing system created a safer alternative that took priority.

Why Your Team Isn't Responding to Your Clear Message — Uncovering the Hidden Barriers

When you treat customer experience solely as a cost to manage rather than a strategic growth driver, you miss out on substantial revenue opportunities. Shifting the perspective of customer experience from expense to value creation can unlock untapped business potential.

Why Viewing Customer Experience as a Cost Center Hurts Your Revenue Growth