Why Enterprise AI Deployment Keeps Stalling at the Integration Layer: the Architecture That Unblocks It

TLDR: Enterprise artificial intelligence programmes stall primarily because integration architecture lags model capability, and the organisations generating measurable returns have all made the connectivity layer their primary AI infrastructure investment.

The Bottleneck Lives in the Architecture, Separate From the Model

The enterprise artificial intelligence (AI) conversation has concentrated on model selection, prompt engineering, and use-case identification. A consistent body of research points to a different variable as the principal differentiator between AI programmes that generate returns and those that generate reports.

A March 2025 McKinsey survey of 1,491 executives across 101 nations found that more than 80% of respondents report generative AI delivering earnings before interest and taxes (EBIT) impact below the threshold of attribution, with only 17% attributing at least 5% of EBIT to AI initiatives. By November 2025, AI adoption had become near-universal: a follow-on McKinsey study found 88% of organisations using AI regularly in at least one function, up from 33% the prior year, while only approximately 6% qualified as high performers with meaningful EBIT attribution.

The gap between adoption and impact is the integration gap. McKinsey tested 25 deployment attributes against EBIT outcomes; workflow redesign ranked first. AI high performers are 2.8 times more likely to have fundamentally redesigned workflows, with 55% doing so versus 20% among the broader population. The remaining 79% layer AI on top of human-readable, process-oriented systems built for human operators rather than application programming interface (API) consumption, and the AI system inherits all of those process constraints alongside it.

957 Applications, 27% Connected: Quantifying the Integration Deficit

The MuleSoft 2026 Connectivity Benchmark Report, drawn from 1,050 information technology (IT) leaders, puts a precise number on the structural problem. The average enterprise manages 957 applications, with only 27% connected. Thirty-six percent of IT team time goes to designing, building, and testing custom integrations. Eighty-two percent of respondents cite data integration as their biggest AI challenge, and 86% agree that AI agents operating in disconnected environments add more complexity than value.

The trend line from MuleSoft’s prior year reinforces the concern: the 2025 Connectivity Benchmark Report recorded 897 average applications at 29% connectivity, with 95% of IT leaders reporting integration as an AI implementation hurdle. Application sprawl is accelerating while the connectivity rate stagnates. An IDC and Amazon Web Services study of over 900 organisations found that 46% name system integration as the top barrier to agentic AI, with 44% citing security limitations in existing data platforms. Half of all AI agents currently operate in silos, functioning as isolated point solutions rather than coordinated multi-agent systems.

The pattern is consistent across sources and across years: the AI model is ready; the enterprise environment surrounding it requires structural readiness before value compounds.

JPMorganChase Builds the Integration Layer First

JPMorganChase’s deployment of LLM Suite illustrates the architecture sequence that produces scale. A large language model (LLM) platform built in-house, LLM Suite integrates models from OpenAI and Anthropic with the bank’s internal databases, compliance architecture, and workflows. Rather than deploying models to employees and expecting adoption, the bank built the integration and governance layer first, constructing an internal AI hub that enforces data privacy compliance and internal governance before any employee-facing surface became active.

The results document what that sequence achieves. LLM Suite grew from zero to 200,000 users within eight months of its summer 2024 release, a scale that confirms the activation power of integration-first architecture. The critical architectural insight is the sequencing: integration and governance infrastructure preceded deployment, and the compounding value came from that order. Enterprises that prioritise deployment speed over integration readiness encounter the alternative pattern: adoption plateaus and EBIT attributability stays below the measurable threshold.

Data Readiness: The Condition AI Systems Require to Operate

The second structural constraint operates at the data layer. A Gartner February 2025 analysis of 1,203 data management leaders found that 63% either lack or remain unsure about adequate AI-ready data management practices; Gartner forecasts that 60% of AI projects with structurally deficient data will be abandoned through 2026.

AI systems, particularly those using retrieval-augmented generation (RAG) pipelines, require structured, governed, and accessible data products. Extract, transform, load (ETL) processes that move data in overnight batches to a warehouse deliver stale context; AI agents reasoning over customer accounts, inventory positions, or risk exposures require live data streams. Event-driven architectures built on platforms such as Apache Kafka or Azure Event Grid provide the streaming layer that batch ETL architectures are structurally limited in supplying.

The data governance dimension adds a further constraint. Domain-siloed repositories mean AI systems reason within departmental boundaries, producing answers that are locally coherent but globally constrained. The data mesh paradigm, theorised formally in arXiv:2402.04681, addresses this by reorganising data ownership along business domain lines, with each domain accountable for data products under defined service-level agreements.

Kroger’s 84.51° Makes AI-Ready Data a Domain Responsibility

Kroger’s data analytics subsidiary 84.51° deployed a data mesh architecture that redistributes data ownership to business domains. Supply chain, merchandising, and customer teams each own and govern their data products under defined quality commitments. A data fabric layer built on Databricks Unity Catalog and Alation provides the metadata governance and integration connectors that allow AI systems to reason across domains while each domain maintains accountability for its own data quality, as documented in the Alation vendor case study.

The architecture displaces the centralised siloed repository model, in which a central team governs all data and AI requests queue behind that bottleneck. In the domain-owned model, AI systems access governed data products through standardised interfaces, and quality accountability rests with the teams closest to the data. The data fabric layer acts as connective tissue, transforming domain-governed data products into an AI-accessible knowledge graph spanning enterprise boundaries. The Kroger implementation demonstrates that data mesh functions as an AI readiness strategy as much as a data architecture one.

The Emerging Enterprise AI Integration Stack

Across practitioner literature and architecture documentation, six integration challenges surface consistently, each with a corresponding architectural response. The exhibit below maps those layers against the solutions now in production use.

The LLM-to-enterprise translation layer warrants particular attention because it sits at the seam between AI reasoning and enterprise record systems. LLMs expect structured APIs and produce structured outputs; legacy enterprise systems communicate in proprietary protocols, remote function calls, and flat files. The Model Context Protocol (MCP) gateway approach, announced by SAP at TechEd 2025, exposes all SAP APIs and legacy connections including R/3 BAPI and RFC endpoints as MCP tools for AI agents, providing a standardised translation layer between the AI reasoning environment and the enterprise record system. The SAP Architecture Center documents the build pattern for AI agents on SAP Business Technology Platform. Microsoft’s Copilot Studio and SAP integration addresses the same seam from the Microsoft side.

For broader enterprise orchestration, Semantic Kernel provides the LLM orchestration layer managing sequential, concurrent, and handoff agent patterns with session-based state management and middleware telemetry. The Azure AI agent design patterns documentation maps the orchestration options for enterprise deployment. For systems operating entirely beyond the API surface, robotic process automation (RPA) bridges provide continuity: Microsoft Copilot Studio combined with Power Automate Desktop deploys automation agents on Azure virtual machines, enabling AI orchestration to interact with legacy interfaces through UI automation where a modern API surface is absent.

The integration platform as a service (iPaaS) layer anchors the connectivity architecture. Platforms such as MuleSoft Anypoint and Azure Integration Services provide API-first middleware normalising communication across the application estate, with the Azure Integration Services architecture guide providing the reference patterns. At the governance layer, Salesforce’s Einstein Trust Layer mediates between AI and customer relationship management data, masking personally identifiable information, enforcing toxicity checks, and maintaining a zero-persistence design across AI sessions. The Agentforce framework builds on that foundation to connect AI agents to enterprise data and workflows within the Salesforce estate.

Integration Investment as the Primary AI Budget Decision

Gartner’s June 2025 forecast that over 40% of agentic AI projects will be cancelled by the end of 2027 due to escalating costs and unclear business value aligns with the structural evidence across this body of research. A Gartner survey of 140 senior supply chain leaders conducted from October to November 2025 found that 56% identify legacy system integration as a major challenge, with only 17% pursuing transformational workflow redesign while 83% apply AI incrementally to existing process structures.

The reading across this evidence is structural. Enterprise AI underperformance is an integration problem, and the organisations generating durable EBIT from AI are those that treat connectivity architecture as the primary infrastructure investment, pursued before or in parallel with model deployment. Model capability has reached a level of sufficiency for the majority of enterprise use cases; the enterprise environment surrounding the model is the binding constraint on outcomes. Budget directed at model fine-tuning while integration debt accumulates is budget directed at the wrong layer of the stack.

The JPMorganChase and Kroger cases illustrate the same principle from different angles: one built the integration hub before activating the AI surface; the other reorganised data ownership so the AI layer encountered governed, standardised data products rather than siloed repositories. Both made the integration architecture decision first, and both achieved scale as a result.

Kainjoo’s enterprise AI practice identifies integration layer readiness as the first diagnostic in any AI programme assessment, on the basis that the fastest path to measurable returns begins with a connectivity audit rather than a model selection exercise.

---

References

1. McKinsey, “The state of AI: How organizations are rewiring to capture value,” March 12, 2025 — https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-how-organizations-are-rewiring-to-capture-value
2. McKinsey, “The state of AI: Agents, innovation, and transformation,” November 2025 — https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
3. Gartner, “Lack of AI-Ready Data Puts AI Projects at Risk,” February 26, 2025 — https://www.gartner.com/en/newsroom/press-releases/2025-02-26-lack-of-ai-ready-data-puts-ai-projects-at-risk
4. Gartner, “Survey Finds Technology Integration and Talent Perceived as Key Roadblocks to Scaling AI in Supply Chain,” April 29, 2026 — https://www.gartner.com/en/newsroom/press-releases/2026-04-29-gartner-survey-finds-technology-integration-and-talent-perceived-as-key-roadblocks-to-scaling-ai-in-supply-chain
5. Gartner, “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027,” June 25, 2025 — https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
6. MuleSoft/Salesforce, “2026 Connectivity Benchmark Report,” February 2026 — https://blogs.mulesoft.com/agentic-perspectives/connectivity-benchmark-report/
7. MuleSoft/Salesforce, “2025 Connectivity Benchmark Report” — https://www.salesforce.com/blog/mulesoft-connectivity-benchmark-2025/
8. IDC/AWS, “Agentic AI adoption: 5 key trends driving enterprise growth,” 2025 — https://aws.amazon.com/isv/resources/agentic-ai-idc-study/
9. JPMorganChase official technology blog, June 3, 2025 — https://www.jpmorganchase.com/about/technology/blog/llmsuite-ab-award
10. Alation, “How Kroger Leverages Data Mesh and Data Fabric for Data Value” — https://www.alation.com/blog/kroger-data-mesh-data-fabric-unlock-value/
11. SAP News Center, “New Agentic Capabilities on SAP BTP Supercharge Developers for What’s Next,” November 2025 — https://news.sap.com/2025/11/new-agentic-capabilities-sap-btp-supercharge-developers/
12. SAP Architecture Center, “Build AI Agents on SAP BTP” — https://architecture.learning.sap.com/docs/golden-path/ai-golden-path/build-and-deliver/build-ai-agents
13. Microsoft Azure Architecture Center, “Basic Enterprise Integration” — https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/enterprise-integration/basic-enterprise-integration
14. Microsoft Learn, “Integration architecture design” — https://learn.microsoft.com/en-us/azure/architecture/integration/integration-start-here
15. Microsoft Learn, “Introduction to Semantic Kernel” — https://learn.microsoft.com/en-us/semantic-kernel/overview/
16. Microsoft Learn, “AI Agent Orchestration Patterns” — https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns
17. Microsoft Learn, “Copilot Studio architecture guidance” — https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/architecture-overview
18. Microsoft Learn, “SAP with Microsoft AI: Copilot Studio and SAP overview” — https://learn.microsoft.com/en-us/azure/sap/microsoft-ai/copilot-studio/copilot-with-sap-overview
19. Salesforce Developer Blog, “Inside the Einstein Trust Layer,” October 2023 — https://developer.salesforce.com/blogs/2023/10/inside-the-einstein-trust-layer
20. Salesforce Developers, “Agentforce and Generative AI Overview” — https://developer.salesforce.com/docs/einstein/genai/overview
21. arXiv, “Architectural Design Decisions for Self-Serve Data Platforms in Data Meshes,” arXiv:2402.04681 — https://arxiv.org/pdf/2402.04681
22. MuleSoft, “Anypoint Platform — Enterprise Integration” — https://www.mulesoft.com/platform/enterprise-integration
23. Apache Kafka, “Documentation” — https://kafka.apache.org/documentation/