Training a 360M Parameter Model with Performance Discipline
Pretraining SmolLM-360M on a single A100 GPU within a 30-hour window, focusing on feasibility analysis, throughput measurement, and hardware efficiency optimization.
Posts tagged "GPUs"
AllAIAI Systems PerformanceAI systems performanceAI-agentsArithmetic IntensityGPUsLLMsTPUscompilerscomputationhardwarelox-interpretermathmodel-evaluationspre-trainingprogramming-languagespythonsetssoftware-engineering
Roofline Analysis of Dot Product on an H100
Analyzing the dot product operation through the roofline model on NVIDIA H100 GPU hardware.