Feb 08, 202612 min read
Training a 360M Parameter Model with Performance Discipline
Pretraining SmolLM-360M on a single A100 GPU within a 30-hour window, focusing on feasibility analysis, throughput measurement, and hardware efficiency optimization.
Featured deep-dive
Recent writing
↳ The archive, latest first
Everything is One Token Stream
An LLM sees one stream of tokens. All the different components—system prompt, user messages, tool results, model responses—serve different purposes, have different trustworthiness, and carry different authority, yet they all appear in one token stream.
Nov 10, 202610 minMachine Baseline for CPU Performance Engineering on an M4 Pro
Establishing single-core FP32 compute, DRAM bandwidth, and cache hierarchy ceilings on Apple M4 Pro as denominators for kernel optimization.
Apr 19, 20267 minExtending Lox Language with Custom Features!
Extending Lox beyond the original spec with len(), explicit initialization checks, and break statements.
Dec 13, 20256 minWriting my own interpreter for Lox, Part 9 - Inheritance
Adding single inheritance with method overriding and super keyword support to complete the Lox class system.
Oct 25, 20255 minWriting my own interpreter for Lox, Part 8 - Classes
Implementing object-oriented programming in Lox with classes, instances, methods, and the this keyword.
Oct 18, 20255 minWriting my own interpreter for Lox, Part 7 - Resolving and Binding
Fixing closure scoping bugs with a static resolver pass that computes variable binding distances.
Oct 05, 20257 minWriting my own interpreter for Lox, Part 6 - Functions
Adding functions as first-class values, closures, and native functions to the Lox interpreter.
Sep 03, 20255 minWriting my own interpreter for Lox, Part 5 - Control Flow
Adding control flow structures that make Lox Turing complete.
Aug 17, 20257 minWriting my own interpreter for Lox, Part 4 - Statements
Extending the Lox interpreter with statements, variables, environments, and lexical scoping.
Aug 16, 20253 min