Cloud Infrastructure Considerations for Artificial Intelligence

The Value of Cloud Computing for AI

Generative AI deployments are becoming a top priority for many enterprises. Expanding infrastructure and resources to support those workloads may be the largest challenge IT leaders are facing. While modernizing existing on-premises data centers can be a great way to optimize space, power, and OPEX for new AI deployments, it isn’t the only option. Many organizations use cloud computing as their primary resource and need to determine the best way to leverage it for AI deployment. 

Deploying new workloads like AI in the cloud can minimize CAPEX but can also lead to a significant increase in OPEX. Because of this, choosing virtual machines (VMs) with the right processor for a given workload is as important as choosing the right server platform in the data center. Newer, higher performance VMs, though more expensive on a per-VM basis, can have performance-driven cost savings by doing more work-per-VM and lowering the total number needed to meet demand. Optimizing VM selection for existing workloads can streamline OPEX spending and ensure that resulting savings are directed toward AI deployment in the cloud.

Organizations looking to implement AI in their data centers can also benefit from cloud solutions. With the widespread demand for AI applications, getting physical hardware to scale on-premises AI resources can be difficult, expensive, and prone to supply-chain delays. Organizations that use cloud computing may have an easier time getting AI projects up and running quickly, even if they intend to run AI in the data center long term. This is especially true when working with the largest cloud providers who often get quicker access to new hardware at scale.

Also, organizations with a hybrid-cloud model, running compute-intensive workloads on-premises while running general purpose applications in the cloud, may alleviate the on-premises computing burden by shifting more work to the cloud.

Challenges in the Cloud

Deploying AI in the cloud can seem simple, but there are some important considerations to a successful deployment. Cloud providers offer hundreds of different VMs with varying CPUs and memory configurations, each with strengths and weaknesses. Finding the right VM for AI workloads is critical, but so is finding the best VM for pre-existing enterprise workloads, like data management for AI training/inference. This holistic approach allows cloud-based organizations to get the most from their cloud footprint across all aspects of their business.

Often, this means moving from older VM types, based on previous generations of CPU, to newer instances that maximize performance-driven cost efficiency. Beyond this, it also means identifying the volume of work for each key workload, AI and non-AI, and finding the right balance of different instances to optimally meet those needs. For example, teams may find that general purpose instances featuring AMD EPYC™ 9004 series processors are ideal for handling administrative workloads while minimizing the total number of VMs utilized. Meanwhile, GPU-accelerated instances can be better suited to training AI models, and memory-optimized instances perform best for running the high-performance databases that underpin any successful AI deployment. Without taking this approach, organizations can run into operational inefficiencies and spend more to do less.

Choosing the Right Cloud VM for AI

GCP C3D Standard-16 vs GCP N2 Standard-16

In many situations, choosing the lowest priced VMs that meet performance requirements can end up costing more than VMs running on the latest CPUs. Instead, the right cloud VM is often the best performance-per-dollar VM for the specific workload. In this regard, AI applications should be no different than general purpose, high-performance computing, or big data analytics workloads.  As an example for currently popular cloud workloads, customers running in Google Cloud can opt for the C3D VM series, built with AMD EPYC™ 9004 processors. These new VMs provide a 37% performance uplift on average across a selection of popular workloads when compared to the previous generation competitive offering.¹

When considering cloud computing OPEX, understanding the cost associated with each ‘unit of work’ is critical to properly deciding on VMs. With a performance uplift, more work can be done with each VM purchased. This can drive down costs, shrink the VM pool, and free up resources to expand AI projects in the cloud.

By choosing cloud VMs built with AMD EPYC™ processors, IT leaders are giving their teams fantastic value in the cloud and creating more opportunities for AI development and deployment in the cloud.

To learn about how enterprise organizations can approach AI in the cloud, read this article from the Wall Street Journal.

To learn more about how AMD is optimizing cloud computing, visit the Cloud Computing Page.