Jingchao Zhang, PhD

10 Mar 2026

Multi-Node Fine-Tuning: FSDP Sharding Strategy Matters

Introduction In a previous post, I ran Qwen2.5-72B inference on Azure H100 nodes and showed how NVLink’s 900 GB/s ba...

09 Mar 2026

LLM Inference Across NVLink and InfiniBand

Introduction When serving a large language model across multiple GPUs, the choice of parallelism strategy directly d...

28 Feb 2026

Debugging InfiniBand Performance Degradation Caused by Background GPU Stress Tests

The Problem

03 Oct 2023

DLProf TF2 Profiling

Profiling AI/ML models on single/multi-GPUs using AzureHPC images

24 Sep 2023

AMLFS in AzHOP

Integrating Azure Managed Lustre File System (AMLFS) with Blob storage into AzHOP provides a seamless environment for...

14 Sep 2023

HPC Network Technologies

In the rapidly evolving landscape of High-Performance Computing (HPC) and Artificial Intelligence (AI), understanding...