What two criteria are essential for monitoring GPU performance in a large-scale AI infrastructure?

Monitoring GPU performance in a large-scale AI infrastructure requires a focus on metrics that directly reflect the GPUs' operational efficiency and capabilities during high-intensity computation. The first essential criterion is GPU utilization percentage, which indicates how effectively the GPU is being used during processing tasks. High utilization means the GPU is actively engaged in computations, leading to better resource usage and faster processing times. Conversely, low utilization suggests that the GPU is underutilized, which can result in wasted resources and less efficient performance.

The second crucial criterion is memory bandwidth usage on GPUs. Memory bandwidth refers to the speed at which data can be read from or written to the GPU's memory. In AI workloads, especially those dealing with large datasets and complex models, having high memory bandwidth is key to ensuring the GPU can handle data effectively without bottlenecks. Monitoring memory bandwidth helps identify if the GPU is limited by memory performance, which can severely hamper overall AI processing tasks.

Other options focus on aspects that do not directly impact GPU performance in the context needed for AI workloads. For instance, the number of active CPU threads provides insights into CPU performance rather than GPU. GPU fan noise levels are more indicative of hardware status rather than performance metrics, making them less relevant for monitoring the GPU's ability to execute