MilaOpen LLMs, at the metal
A C++23 library for open LLMs — inference and training, built from explicit neural-network components you can read and understand. Run Gemma 4 12B and Llama 3.x locally on a 12 GB consumer GPU.
Fast
1,817 tok/s prefill and 49.1 tok/s decode at a 32K context, on an RTX 4070 12GB.
Explicit
No autograd engine, no hidden execution engine. Call forward() and you know exactly which kernel runs.
Validated
Gemma 4, Llama 3.x and GPT-2 each reproduce a HuggingFace reference token-for-token.
Get started
Build and run it
Toolchain, a clone-to-
ctest build, model weights, and a
first local inference run.
Start here →
Blog
Building Mila
Hand-written CUDA, profiled and rewritten — flash attention, FP4
kernels, and the bugs that passed every test.
9 posts →
Documentation
API reference
Generated reference for every component, operation and tensor type.
Browse →
Source
GitHub
MIT licensed, public beta. Build it, read it, push back on the
design.
ToddThomson/Mila →Open models
Each one raised from the metal and held to parity — a short, curated set rather than a model zoo.
Sponsoring
Contact
One developer. Design argument is welcome and often changes things.
Discussions
Ask anything
Questions, ideas, and pushback on the design. The main venue, and
where the writeups get discussed.
Open discussions →
Issues
Report a bug
Build failures, wrong output, and feature requests. Two issues are
tagged
good first issue.
Open issues →
Email
Get in touch
Sponsorship, collaboration, or anything that does not belong in
public.
todd.thomson@me.com →