×
AI Training

Data Parallelism: How to Train Deep Learning Models on Multiple GPUs Training

BONUS! Cyber Phoenix Subscription Included: All Phoenix TS students receive complimentary ninety (90) day access to the Cyber Phoenix learning platform, which hosts hundreds of expert asynchronous training courses in Cybersecurity, IT, Soft Skills, and Management and more!

Course Overview

In this one-day, instructor-led CUDA course in Washington, DC Metro, Tysons Corner, VA, Columbia, MD or Live Online, students will learn techniques for data-parallel deep learning training on multiple GPUs to shorten the training time required for data-intensive applications. Working with deep learning tools, frameworks, and workflows to perform neural network training, they will learn how to decrease model training time by distributing data to multiple GPUs, while retaining the accuracy of training on a single GPU. After taking this course, learners will be able to:

  • Understand how data parallel deep learning training is performed using multiple GPUs
  • Achieve maximum throughput when training, for the best use of multiple GPUs
  • Distribute training to multiple GPUs using Pytorch Distributed Data Parallel
  • Understand and utilize algorithmic considerations specific to multi-GPU training performance and accuracy

Schedule

Currently, there are no public classes scheduled. Please contact a Phoenix TS Training Consultant to discuss hosting a private class at 301-258-8200.

Program Level

Intermediate

Prerequisites

Students taking this course should have experience with deep learning training using Python.

Course Outline

Introduction

Stochastic Gradient Descent and the Effects of Batch Size

Training on Multiple GPUs with PyTorch Distributed Data Parallel (DDP)

Maintaining Model Accuracy when Scaling to Multiple GPUs

Workshop Assessment

Final Review

Next Steps

 

 

BONUS! Cyber Phoenix Subscription Included: All Phoenix TS students receive complimentary ninety (90) day access to the Cyber Phoenix learning platform, which hosts hundreds of expert asynchronous training courses in Cybersecurity, IT, Soft Skills, and Management and more!

Phoenix TS is registered with the National Association of State Boards of Accountancy (NASBA) as a sponsor of continuing professional education on the National Registry of CPE Sponsors. State boards of accountancy have final authority on the acceptance of individual courses for CPE credit. Complaints re-garding registered sponsors may be submitted to the National Registry of CPE Sponsors through its web site: www.nasbaregistry.org

Subscribe now

Get new class alerts, promotions, and blog posts

Phoenix TS needs the contact information you provide to us to contact you about our products and services. You may unsubscribe from these communications at anytime. For information on how to unsubscribe, as well as our privacy practices and commitment to protecting your privacy, check out our Privacy Policy.

Download Course Brochure

Enter your information below to download this brochure!

Name(Required)