Training a 360M Parameter Model with Performance Discipline
Pretraining SmolLM-360M on a single A100 GPU within a 30-hour window, focusing on feasibility analysis, throughput measurement, and hardware efficiency optimization.
Training a 360M Parameter Model with Performance Discipline
Pretraining SmolLM-360M on a single A100 GPU within a 30-hour window, focusing on feasibility analysis, throughput measurement, and hardware efficiency optimization.
Featured
Roofline Analysis of Dot Product on an H100
Analyzing the dot product operation through the roofline model on NVIDIA H100 GPU hardware.
A Mental Model of TPUs for Performance Engineering
A visual mental model for understanding TPU architecture and how it relates to ML workloads.
Recent