Link to You May Also Enjoy
Optimized large Message broadcast for deep learning workloads: mpi, mpi+nccl, or nccl2
Background There are new opportunities and challenges for the High-Performance Computing (HPC) community to rethink and enhance communication middleware like...
Nvidia modulus singularity run on gpu
Get the docker image from NVIDIA website. You need to register and login. Download file is modulus_image_v21.06.tar.gz (5.7G). Build a singularity image. ml ...
Limit gpu memory usage in tensorflow
```Python import tensorflow as tf
Mpi processes per gpu
Some guidlines on selecting the number of MPI processes per GPU When using the GPU package, you cannot assign more than one GPU to a single MPI task Mul...