Multi-GPU AI Train the Trainer Workshop
5-day workshop covering multi-GPU AI training, including PyTorch Distributed Data Parallel, model parallelism, PyTorch Lightning, fine-tuning neural networks, computer vision, MLOps on HPC, Ray, RAG, and hyperparameter tuning.
ai
hpc
pytorch
difficulty:
advanced
maturity:
beta
This 5-day train-the-trainer workshop covers:
- Introduction to GPU architectures and HPC infrastructure
- Introduction to deep learning
- PyTorch Distributed Data Parallel
- Model parallelism with PyTorch
- PyTorch Lightning
- LLM, Fine-tuning & HuggingFace Accelerate, DeepSpeed
- Computer Vision with CNN
- MLOps on HPC
- Ray + Retrieval Augmented Generation (RAG)
- Hyperparameter tuning
The course is designed for scientists, engineers, and PhD students who need to scale AI models efficiently or optimize distributed training workflows.
2026
Organized for CASTIEL2 by ENCCS