Pepperdata adds GPU-tracking tools to help IT teams maximize usage

Pepperdata today announced it has extended its portfolio of tools for monitoring and optimizing processors to now include graphics processing units (GPUs) that are widely employed to train AI models, among other applications.

GPUs are among the most expensive compute resources enterprise IT organizations consume today. They are also in short supply, thanks to issues with chip production that have arisen in the wake of the COVID-19 pandemic. That supply shortage is further exacerbated by competing demands for GPUs among providers of gaming systems, which have also seen high demand during the pandemic.

These issues make it imperative for enterprise IT organizations to maximize the number of workloads that can be run per GPU, Pepperdata CEO Ash Munshi told VentureBeat.

Pepperdata has historically provided tools that automatically scale traditional CPU system resources by analyzing application and infrastructure metrics in real time. Those capabilities are now being extended to provide visibility into GPU memory usage and waste, along with surfacing recommendations to fine-tune GPUs. The Pepperdata tools also enable IT teams to attribute usage and associated costs to specific end users.

Measurement methods

There are other approaches to measuring GPU performance, but Mushi said those tools lack application context. Pepperdata’s tools, however, enable IT teams to see how a specific Kubernetes cluster running on GPUs might be further optimized, he said.

That’s critical because different classes of GPUs offer different levels of performance at varying costs, Munshi added. Based on their performance requirements, some workloads might be shifted to lower-cost GPUs to reduce expenditures, he noted. GPUs also consume a lot of energy that could be reduced by moving workloads, Munshi said. “There are many kinds of GPUs,” he added. “It’s a big umbrella.”

Many IT organizations are now being tasked with reducing the number of carbon applications they generate as part of a larger effort to meet sustainability goals, adding more urgency to the problem of optimizing workload deployments.

Multiplicity

The types and classes of processors enterprise IT organizations employ have never been more diverse. The days when organizations standardized on a specific class of CPUs provided by a company like Intel are over. In addition to employing systems based on CPUs from multiple providers of x86 processors, organizations are employing GPUs provided by multiple vendors alongside field-programmable gate arrays (FPGAs). Applications are increasingly invoking a medley of processors to optimize various types of workloads that make up that application. Those workloads might be deployed on-premises or running on a cloud within the context of a single application.

Going forward, the bulk of enterprise applications will incorporate AI models. As a result, organizations that build their own applications should see a steady increase in the number of GPUs employed to train AI models. In some cases, GPUs will even be employed to run AI inference engines as an alternative to traditional CPUs.

Regardless of the type of processor employed, platform efficiency has become a larger economic issue. In the immediate aftermath of the pandemic and the rush to move applications to the cloud, many organizations did not stop to evaluate costs. Developers, especially, are inclined to select platforms based on how accessible they are rather than analyzing the cost of using another class of service or an on-premises IT alternative. As the overall economy continues to recover, companies are reevaluating many of the decisions about which workloads should run where. Those decisions, however, are not going to be easily made without visibility into how those platforms are being utilized.