Moving Enterprise AI into Production Requires Infrastructure Planning

Last Updated 2 hours ago by Kenya Engineer

Artificial intelligence projects can demonstrate value in controlled trials but encounter a different set of challenges when organisations attempt to deploy them at scale. Beyond the software and computing resources, production AI requires adequate electrical power, cooling, connectivity and operational support.

A new infrastructure planning guide from Vertiv, developed with digital infrastructure research firm STL Partners, provides a framework for helping organisations assess these requirements before committing to an AI deployment. The guide, Designing Infrastructure for Enterprise AI, focuses on matching the needs of individual AI workloads to suitable deployment environments and physical infrastructure.

The underlying challenge is that AI workloads do not all place the same demands on infrastructure. A project designed to test a model may operate comfortably within an existing cloud environment, while a sustained production application may require dedicated computing capacity, tighter control over data, predictable response times or a more carefully engineered operating environment.

For engineering teams, this means AI deployment should not be treated solely as an IT procurement exercise. The physical infrastructure supporting the computing equipment can influence project cost, performance, reliability and the time required to move from experimentation to operational use.

Starting with the workload

The Vertiv guide proposes a four-step planning process. Organisations first define the intended business outcome and AI workload, assess its scale and operating requirements, determine the appropriate deployment location, and then match the workload to a suitable physical infrastructure model.

This sequence places the intended application ahead of the infrastructure specification. Rather than selecting equipment before understanding what it must support, planners begin by establishing the workload’s performance requirements and how it is expected to operate.

The distinction is important because AI development and production involve different patterns of computing demand. Model development and retraining can require substantial computing resources over extended periods, while production applications may place greater emphasis on consistent service availability, response times and the ability to handle changing demand.

The guide groups enterprise AI into six workload archetypes, covering model development, retraining and four categories of production use. These categories provide a basis for comparing workloads with similar performance, operating and infrastructure requirements.

The practical value of this approach is that infrastructure can be designed around the actual application rather than an assumed, universal requirement for AI computing.

Four infrastructure models

The guide identifies four physical infrastructure models: Edge AI Cluster, Small AI Cluster, Large AI Cluster and High-Density AI Cluster. Each provides a starting point for planning the computing environment and its supporting systems.

The appropriate model depends on the workload, its scale, the location in which it must operate and the infrastructure already available. The models are not interchangeable equipment packages; they are planning references that require further assessment and engineering before implementation.

For an edge deployment, for example, proximity to the point where data is generated or used may be an important consideration. Other workloads may be better suited to a larger centralised facility or a hybrid arrangement that combines enterprise infrastructure with external computing resources.

High-density AI deployments bring particular demands on power delivery and heat removal. As computing equipment becomes more power-intensive, cooling system capacity, electrical distribution and the physical arrangement of equipment must be considered together. A facility that can accommodate conventional IT equipment may require substantial modifications before it can support a higher-density AI installation.

The infrastructure assessment must also consider connectivity, controls, resilience, maintenance access and the operational expertise needed to support the installation throughout its service life.

Vertiv’s approach is to use the four models and their associated reference designs as a basis for further planning. Actual specifications would still need to be established through site assessments, equipment selection and detailed engineering.

Power and cooling can determine readiness

Available power is one of the first constraints to examine when assessing a proposed AI facility. The assessment should establish not only whether sufficient electrical capacity exists, but also whether the distribution system can deliver the required power reliably and accommodate anticipated growth.

Cooling presents a related challenge. AI computing equipment can concentrate substantial heat within a relatively small area, making cooling capacity and heat removal central considerations in facility design. The appropriate solution depends on equipment characteristics, rack power density, ambient conditions and the wider thermal management strategy.

Electrical and mechanical systems therefore need to be assessed as an integrated installation. Increasing computing capacity without providing the corresponding electrical and cooling infrastructure can create operational constraints, while an oversized installation can commit an organisation to unnecessary capital and operating expenditure.

Other considerations include network capacity, physical space, redundancy, monitoring, maintenance arrangements and the ability to expand without extensive disruption. These requirements should be evaluated against the workload’s operating profile and the consequences of service interruption.

Assessing facility readiness early can reveal whether existing infrastructure can accommodate a proposed deployment, requires modification or is unsuitable for the intended workload. It also gives engineering and project teams an opportunity to identify major cost and programme implications before equipment procurement begins.

Choosing where AI should run

The framework does not prescribe a single location for enterprise AI. Public cloud, colocation facilities, enterprise-owned computing capacity, hybrid environments and edge installations can each serve different requirements.

Cloud services can provide access to computing resources without requiring an organisation to build and operate its own facility. Colocation can offer access to purpose-built infrastructure, while enterprise-owned systems may be appropriate where organisations require greater control over their operating environment. Edge deployments can be useful when processing needs to take place close to data sources or users.

A hybrid approach may combine these arrangements, allowing different workloads to run where their technical, operational and business requirements can best be met.

The choice should account for more than the initial cost of computing resources. Power and cooling provision, connectivity, data handling, governance, operating responsibilities and future capacity requirements all influence the long-term suitability of a deployment model.

In this respect, infrastructure planning is closely connected to the economics of AI. Decisions about where workloads run affect the facilities and services an organisation must procure, operate and maintain.

From planning to implementation

Vertiv’s guide brings the workload categories and infrastructure models together in a phased roadmap for planning, deploying and evolving enterprise AI infrastructure. Its central proposition is that organisations should identify infrastructure constraints before they become obstacles to implementation.

The guide offers a structured starting point for business leaders, IT teams and facilities engineers assessing how to support AI applications. It does not replace detailed electrical, mechanical, network or facility design, nor does selecting an infrastructure model establish that a particular site is ready for deployment. Those decisions require workload-specific calculations, equipment data and site-level engineering validation.

As organisations move beyond AI trials, the challenge will increasingly involve integrating computing requirements with the buildings, power systems, cooling equipment and operational arrangements that sustain them. Establishing these requirements early can help reduce deployment risks and provide a clearer basis for investment decisions.

Vertiv’s Designing Infrastructure for Enterprise AI guide, which sets out the six workload archetypes, four infrastructure models and planning roadmap, is available at Vertiv’s website.