24
Mar
2026
Jingchao Zhang, PhD
HPC/AI @Microsoft
HPC/AI @Microsoft
Introduction
TL;DR — I captured per-port, 10-millisecond resolution InfiniBand traffic during FSDP fine-tuning of Qwen-7B (dense) ...
If you deploy GPU VMs on Azure using VMSS Uniform mode, three settings can make the difference between a successful a...
We implement a ring allreduce algorithm from scratch in Python, run it on 16 NVIDIA H100 GPUs across 2 nodes with Inf...
Hands-on experiments on an NVIDIA H100 GPU reveal why KV cache — not model weights — dominates GPU memory during infe...