Mpi processes per gpu
Some guidlines on selecting the number of MPI processes per GPU When using the GPU package, you cannot assign more than one GPU to a single MPI task Mul...
Recent Posts
Are transformers all we need
Impact of Transformers on NLP (and ML more broadly)
Why attention: seq2seq the bottleneck problem
A more general definition of attention: Given a set of vector values, and a vector query, attention is a technique to compute a weighted sum of the values, d...
Ai isn’t as smart as you think but it could be
What is AI, really? Jeff Dean, the head of Google’s AI efforts, explains the underlying technology that enables artificial intelligence to do all sorts of th...
Nvidia profilers
Profiling PyTorch (PyProf) PyProf is a tool that profiles and analyzes the GPU performance of PyTorch models. PyProf aggregates kernel performa...