CPU Performance Engineering
Optimizing SGEMM and prefix sum kernels from naive implementations to within 50% of hardware peak on an Apple M4 Pro.
All writing
A Mental Model of TPUs for Performance Engineering
A visual mental model for understanding TPU architecture and how it relates to ML workloads.
Scaling LLM Pretraining
A systems performance worklog on scaling LLM pretraining — starting on a single GPU and progressively exploring data, tensor, and pipeline parallelism.
Writing an Interpreter for Lox
A worklog series building a tree-walking interpreter for the Lox language in Python, following Crafting Interpreters.
- 01Writing my own interpreter for Lox, Part 1 - Scanner
- 02Writing my own interpreter for Lox, Part 2 - Parsing Expressions
- 03Writing my own interpreter for Lox, Part 3 - Evaluating Expressions
- 04Writing my own interpreter for Lox, Part 4 - Statements
- 05Writing my own interpreter for Lox, Part 5 - Control Flow
- 06Writing my own interpreter for Lox, Part 6 - Functions
- 07Writing my own interpreter for Lox, Part 7 - Resolving and Binding
- 08Writing my own interpreter for Lox, Part 8 - Classes
- 09Writing my own interpreter for Lox, Part 9 - Inheritance
- 10Extending Lox Language with Custom Features!
An Interactive Guide to Rooflines
A walk through the roofline model: compute and memory bounds, arithmetic intensity, where common neural-network kernels land, and a calculator for your own.
Writing a HTTP Server from scratch in less than 200 lines
Building a functional HTTP server using Python, from basic TCP connections through file handling and gzip compression.
Can LLM agents improve other LLM agents?
Exploring whether language model agents can enhance the performance of other LLM agents through a meta-benchmark approach.
On Enumerability
Exploring what it means for a set to be countable, with proofs and examples from set theory.