NCCL: High-Speed Inter-GPU Communication for Large-Scale Training - Sylvain Jeaugey, NVIDIA
Introduction to NCCL
NCCL, or NVIDIA Collective Communications Library, is an inter-GPU communication library optimized for deep learning frameworks. Develope...