A study across 107 enterprises reveals that AI infrastructure spending is growing rapidly, yet most organizations struggle to understand the actual costs involved. Many companies rely on familiar hyperscalers and model-provider APIs, but their next investments are focused on specialized AI compute resources rarely used today. Over half of these enterprises plan to switch or add providers within the next year, placing value on integration and total cost of ownership rather than on token price. However, GPU utilization remains low, with 83% running at 50% capacity or less, and only 44% of companies rigorously track their AI compute expenses. This mismatch between fast spending and poor cost visibility creates a “compute gap”— heavy investment without sufficient control. The report also notes high churn intentions among vendors and an emerging but under-recognized shift from GPU compute to memory bandwidth as inference demands grow. The findings suggest enterprises are eager to expand AI capabilities but need better tools to measure and manage their investments effectively.
Back
