MilaLocal LLMs at the metal
Gemma 4, Llama 3.x and GPT-2 — built from explicit C++23 components and hand-written CUDA kernels you can read and understand, on the consumer hardware you already own.
No execution engine. Your application owns main() and the loop; Mila makes the model an ordinary C++ object inside them.
Gemma 4, Llama 3.x and GPT-2 each reproduce a HuggingFace reference token-for-token, at their target precisions.
Within 1.03x of llama.cpp on decode and 1.14x on prefill at long context, measured on a consumer RTX 4070.
Every component is templated on device and precision, with concept-constrained policies — weight quantization, KV-cache compression — on the layers that use them.
// The type is the configuration - no config object, no runtime dispatch.
using QuantizedProjection =
Linear<DeviceType::Cuda, TensorDataType::BF16, PerGroupFp4<128>>;ctest build, model weights, and a
first local inference run.
Start here →
Source
GitHub
MIT licensed, public beta. Build it, read it, push back on the
design.
ToddThomson/Mila →Model architectures implemented
Three families, each raised from the metal and validated token-for-token against a HuggingFace reference — a short, curated set rather than a model zoo.
Contact
One developer. Design argument is welcome and often changes things.
good first issue.
Open issues →
Email
Get in touch
Sponsorship, collaboration, or anything that does not belong in
public.
todd.thomson@me.com →