NVIDIA GPU Infrastructure Fundamentals: A structured guide to NVIDIA GPU infrastructure, from CUDA to production operations

Decode the NVIDIA GPU ecosystem in one structured guide. Compare technologies, understand how platform layers interact, and build the judgment to evaluate infrastructure choices and trade-offs.Key FeaturesUnderstand how GPUs, CUDA, networking, storage, and DPUs support AI workloadsLearn the roles of MIG, vGPU, DCGM, Kubernetes, Slurm, NGC, and TritonConnect infrastructure components across the AI development and deployment lifecycleBook DescriptionNVIDIA GPU infrastructure spans hardware, system software, networking, storage, orchestration, MLOps, and inference. Understanding how these components fit together, where their responsibilities overlap, and which distinctions matter requires a clear, structured path.This book provides that path through one coherent narrative of the NVIDIA GPU infrastructure stack. It covers accelerated computing, CUDA, and the NVIDIA software ecosystem before comparing data center GPUs against workload characteristics. You will examine MIG, vGPU, and DCGM for resource sharing and monitoring; Kubernetes and Slurm for GPU scheduling; and the networking and storage layer, including Ethernet, InfiniBand, RDMA, GPUDirect Storage, BlueField DPUs, and DOCA.Later chapters connect infrastructure to the AI lifecycle through Airflow, MLflow, and Kubeflow for MLOps, NGC for software delivery, and ONNX, TensorRT, and Triton for inference. You will also explore the Kubernetes components, monitoring technologies, scaling considerations, and diagnostic concepts that support production GPU clusters.By connecting these technologies instead of presenting them as isolated products, the book helps you compare platform choices, understand component boundaries, discuss trade-offs, and develop a durable mental model of NVIDIA GPU infrastructure.What you will learnDistinguish AI, machine learning, and deep learningExplain why GPUs accelerate modern AI workloadsMatch NVIDIA GPUs to training and inference requirementsSelect MIG or vGPU for common resource-sharing scenariosCompare Ethernet and InfiniBand for distributed AI workloadsMap MLOps tools to the right stage of the AI lifecycleDifferentiate ONNX, TensorRT, and Triton in inference workflowsTrace GPU cluster issues across platform layersWho this book is forThis book is for system administrators, cloud and DevOps professionals, data center and networking teams, solution architects, technical managers, presales professionals, and beginners who need a clear understanding of NVIDIA GPU infrastructure. It is especially relevant to professionals moving into AI infrastructure roles, evaluating GPU platform technologies, or collaborating across compute, networking, MLOps, and operations teams. Basic familiarity with IT, cloud, or data center concepts is helpful; programming, data science, and previous GPU experience are not required.Table of ContentsUnderstanding AI Workloads and Accelerated ComputingCUDA and the NVIDIA AI Software StackNVIDIA GPU Architecture and Platform SelectionSharing, Monitoring, and Scheduling GPU ResourcesBuilding High-Throughput Storage and Network PathsVirtualized GPU Infrastructure and DPU OffloadOperating the AI Lifecycle with MLOps and InferenceDelivering and Operating NVIDIA AI PlatformsPreparing for the NCA- AIIO Exam and the Next Step Read more