Data Parallelism: How to Train Deep Learning Models on Multiple GPUs Training

Course Overview

In this one-day, instructor-led CUDA course in Washington, DC Metro, Tysons Corner, VA, Columbia, MD or Live Online, students will learn techniques for data-parallel deep learning training on multiple GPUs to shorten the training time required for data-intensive applications. Working with deep learning tools, frameworks, and workflows to perform neural network training, they will learn how to decrease model training time by distributing data to multiple GPUs, while retaining the accuracy of training on a single GPU. After taking this course, learners will be able to:

• Understand how data parallel deep learning training is performed using multiple GPUs
• Achieve maximum throughput when training, for the best use of multiple GPUs
• Distribute training to multiple GPUs using Pytorch Distributed Data Parallel
• Understand and utilize algorithmic considerations specific to multi-GPU training performance and accuracy

Schedule

Program Level

Intermediate

Prerequisites

Students taking this course should have experience with deep learning training using Python.

Course Outline