The rise of private clouds in AI workloads

The era of AI experimentation has evolved, positioning the private cloud as one of the primary environments for deploying enterprise AI workloads. According to the latest findings from the Private Cloud Outlook 2026 report by Broadcom enterprises are increasingly turning to private clouds for their AI needs.

While initially there was a balance between public and private clouds, 2026 is seeing a decisive shift towards private clouds for AI. This transition is influenced by three primary factors: costs, complexity, and control, which are becoming challenging to manage in public cloud environments.

Key Findings:

• Shift to Private Clouds: A striking 61% of enterprises now run AI inference on private clouds, marking a 19% decline in public cloud utilisation for these workloads.
• IT Challenges: Enterprises report that data protection, privacy, security, and control are the pressing demands brought by AI, with data protection taking precedence.
• Cost Concerns: The economics of public cloud use show that 99% of IT leaders feel there is wastage in their budgets, with over half estimating this to be a significant percentage.
• Geopolitical Influence: The evolving geopolitical landscape is influencing how IT strategies are formed, highlighting data sovereignty and residency as top concerns.
• Repatriation Trends: A growing number of enterprises are considering workload repatriation to private clouds, with cost predictability driving this movement.

These findings indicate a broader industry pattern where enterprises prefer private clouds for the core AI functions like inference, driven by the need for cost mitigation, enhanced governance, and robust security. While public clouds may still be viable for training and piloting, when scaling up, the benefits of private clouds become undeniable.

Numerous IT leaders have voiced concerns over the rising costs associated with using the public cloud at scale. The private cloud offers a more predictable cost structure, alongside enhanced data control and protection. Notably, 86% of IT leaders have highlighted the influence of geopolitics on their strategic infrastructure decisions. For sectors with stringent compliance requirements, such as finance, health, and the public sector, maintaining control of sensitive data remains critical.

The trend of rising costs in the public cloud domain is steering industries towards considering a private cloud-first approach. Almost 83% of enterprises are contemplating workload repatriation, with 44% having already made the transition.

Investment in private cloud infrastructure is climbing, driven by an urge for more predictable expenses and stricter data governance. This move towards private clouds is poised to shape the landscape of enterprise AI strategy significantly.