Multi node PyTorch Distributed Training Guide For People In A Hurry

PyTorch is designed to be the framework that’s both easy to use and delivers performance at scale. Indeed it has become the most popular deep learning framework, by a mile among the research community. However, despite some lengthy official tutorials and a few helpful community blogs, it is not always clear what exactly has to be done to make your PyTorch training to work across multiple nodes.

Link