Achilles SoftwareMila C++23 · CUDA · open LLMs
← Home

Building Mila

Hand-written CUDA, profiled and rewritten — flash attention, FP4 kernels, and the bugs that passed every test.

July 2026 Running Gemma 4 12B in Docker Behind an OpenAI-Compatible API One container and one command puts Gemma 4 12B behind an OpenAI/Anthropic-compatible endpoint, so Codex CLI and Claude Code can drive a local C++ … Read → July 2026 Flash-Decoding for MQA in CUDA: One KV Head, Split-K, and an FP4 Matvec at 90% of DRAM Peak Hand-writing a fused flash-decoding kernel for Gemma 4's single-KV-head global layers, plus an FP4 matvec taken from 63% to 90% of DRAM peak. 38.65 to … Read → July 2026 Writing a FlashAttention Prefill Kernel for Gemma 4: Head Dim 512, Tensor Cores and an FP8 GEMM Hand-writing a tensor-core flash-attention prefill kernel for Gemma 4's 512-wide heads, plus an FP8 activation GEMM. Ten sessions from 1.95x behind … Read → June 2026 Bringing Up Gemma 4 in C++: Sliding-Window Attention, Dual RoPE and Token-for-Token Parity Porting a C++ inference runtime from Llama 3 to Gemma 4: per-layer local/global sliding-window attention, dual RoPE, GeGLU, logit softcap, and the … Read → May 2026 Closing a Local Agentic Loop: Mila Inference Server and Codex CLI A full agentic round trip driven entirely on local hardware - tool calls, results, and resume, with no cloud model in the loop. Read → May 2026 Driving a Local C++ Inference Server from Claude Code and Codex CLI Point a foreign agentic harness at your own OpenAI-compatible endpoint: wiring the Mila Inference Server to Claude Code and Codex CLI, end to end. Read → April 2026 The Eight-Headed Bug That Hid Behind a System Prompt Adding system prompt support to the chat CLI turned working prompts into fluent, confident nonsense. The cause was eight heads and an offset. Read → April 2026 From GPT-2 to Llama 3.2 in C++: RMSNorm, SwiGLU, Grouped Query Attention and RoPE What it actually takes to build a modern transformer from scratch in C++23 and CUDA - swapping LayerNorm for RMSNorm, GELU for SwiGLU, MHA for grouped … Read → February 2026 The Lobotomized Attention Head Bug: One Head Does All the Work, the Others Stare Into the Void A transformer debugging rite of passage: the fast prefill-to-decode path looked correct, but only one attention head was doing anything. Read →