Gemma.Protocol.ixx File Reference

Canonical Gemma 4 native token grammar: the turn template, the tool-declaration renderer, the tool-call grammar and answer extraction. More...

#include <string>
#include <string_view>
#include <optional>
#include <span>
#include <stdexcept>
#include <vector>
#include <cstdint>
#include <cstdlib>
#include <memory>
import nlohmann.json;
import Dnn.Models.Conversation;

Namespaces
namespace	Mila
	Mila main API namespace.

Typedefs
using	Mila::Dnn::Gemma::GemmaToolCall = Conversation::ToolCall
	A tool call parsed out of the model's native <|tool_call> emission.

Functions
std::string	Mila::Dnn::Gemma::extractAnswer (std::string_view text)
	Reduce a channel-structured response to just the user-facing answer.
std::string	Mila::Dnn::Gemma::formatPrompt (std::span< const Conversation::Turn > history, const std::string &tool_declarations={}, bool continue_open=false)
	Render a conversation into Gemma's native prompt, primed for generation.
std::string	Mila::Dnn::Gemma::formatToolCall (std::string_view name, std::string_view arguments_json)
	Render an assistant tool call back into Gemma's native call grammar.
std::string	Mila::Dnn::Gemma::formatToolResponse (std::string_view name, std::string_view result_json)
	Render a client-executed tool result into Gemma's <|tool_response> grammar.
std::string	Mila::Dnn::Gemma::formatTurn (const Conversation::Turn &message)
	One closed turn: <|turn>{role} {content}<turn|> .
std::optional< GemmaToolCall >	Mila::Dnn::Gemma::parseToolCall (std::string_view text)
	Parse the most recent native Gemma tool call out of accumulated model output, once generation has stopped right after <tool_call|>.
std::string_view	Mila::Dnn::Gemma::roleSpelling (Conversation::Role role)
	How the template spells a role.
std::string	Mila::Dnn::Gemma::serializeToolDeclarations (const std::string &tools_json)
	Render tool schemas into Gemma's trained <|tool>declaration:...<tool|> grammar.
std::string	Mila::Dnn::Gemma::stripControlTokens (std::string_view text)
	Remove every enumerated control token, then scrub whatever the set missed.

Variables
constexpr std::string_view	Mila::Dnn::Gemma::kBos = "<bos>"
constexpr std::string_view	Mila::Dnn::Gemma::kChannelClose = "<channel|>"
constexpr std::string_view	Mila::Dnn::Gemma::kChannelLabels []
	The labels a reasoning channel is opened with, for answer extraction.
constexpr std::string_view	Mila::Dnn::Gemma::kChannelOpen = "<|channel>"
constexpr std::string_view	Mila::Dnn::Gemma::kEos = "<eos>"
constexpr std::string_view	Mila::Dnn::Gemma::kOutputKeys [] = { "output", "result", "content", "stdout", "text" }
constexpr std::string_view	Mila::Dnn::Gemma::kPad = "<pad>"
constexpr std::string_view	Mila::Dnn::Gemma::kStringDelimiter = "<|\"|>"
constexpr std::string_view	Mila::Dnn::Gemma::kThink = "<|think|>"
constexpr std::string_view	Mila::Dnn::Gemma::kThoughtPrime = "<|channel>thought\n<channel|>"
	The empty reasoning channel that opens a fresh model turn.
constexpr std::string_view	Mila::Dnn::Gemma::kToolCallClose = "<tool_call|>"
constexpr std::string_view	Mila::Dnn::Gemma::kToolCallOpen = "<|tool_call>"
constexpr std::string_view	Mila::Dnn::Gemma::kToolClose = "<tool|>"
constexpr std::string_view	Mila::Dnn::Gemma::kToolOpen = "<|tool>"
constexpr std::string_view	Mila::Dnn::Gemma::kToolResponseClose = "<tool_response|>"
constexpr std::string_view	Mila::Dnn::Gemma::kToolResponseOpen = "<|tool_response>"
constexpr std::string_view	Mila::Dnn::Gemma::kTurnClose = "<turn|>"
constexpr std::string_view	Mila::Dnn::Gemma::kTurnOpen = "<|turn>"

Detailed Description

Canonical Gemma 4 native token grammar: the turn template, the tool-declaration renderer, the tool-call grammar and answer extraction.

The Gemma 4 grammar is a property of the model, not of any single adaptor, so this runtime module is the one source of truth for it – the same rule Qwen's module states (see Qwen.Protocol.ixx). The Chat harness consumes it directly and the inference server reaches it through the binding, so neither carries a copy that can drift.

It was seeded from the union of the two prior implementations – the Python gemma_protocol.py, which carried the spec-verified behaviors (the <|"|> string delimiter, tool-response output-field distillation, failed-tool error surfacing), and the C++ GemmaToolCallParser it replaces. The template, the <|tool> declaration renderer and answer extraction came down later, from the same Python module, which then retired: until they did, Gemma's grammar was written THREE times (this module, gemma_protocol.py, and a second template in the server's prompt.py) and the adaptors disagreed – Chat advertised tools as a JSON array while the server rendered the trained declaration form the model was tuned on. See GemmaChatProtocol.md.

String-level parse/format only. Token-level splice into the live KV cache is the decided direction but is post-release (see MilaProductFamily.md).

Typedef Documentation

GemmaToolCall

using Mila::Dnn::Gemma::GemmaToolCall = Conversation::ToolCall

export

A tool call parsed out of the model's native <|tool_call> emission.

The family-neutral type, so a host describes a conversation once rather than once per family: arguments is a JSON object string, which is what the model wrote and what a dispatcher takes without re-parsing the grammar.

Function Documentation

extractAnswer()

std::string Mila::Dnn::Gemma::extractAnswer ( std::string_view text )

inlineexport

Reduce a channel-structured response to just the user-facing answer.

Reasoning channels appear more than once and INTERLEAVED with answer text – the 12B emits mid-answer thought channels on the agentic path despite the empty-thought prime – so every channel span is removed, not just a leading run. Removing only the markers would leave the label and its reasoning body behind as literal text.

formatPrompt()

std::string Mila::Dnn::Gemma::formatPrompt ( std::span< const Conversation::Turn > history, const std::string & tool_declarations = {}, bool continue_open = false )

inlineexport

Render a conversation into Gemma's native prompt, primed for generation.

The system turn is assembled here rather than taken from history, because the tool declarations are a SUFFIX of it and a caller that had already concatenated them could not put them in that order.

Parameters

history	The conversation so far, oldest turn first. Any system turn in it is ignored: this function assembles that turn itself, for the reason above.
tool_declarations	What serializeToolDeclarations returns. Empty advertises none.
continue_open	Emit the final turn OPEN, with no closing marker and no thought prime, so the next token continues it. That is the shape after a tool response: the turn already carries its thought channel from before the call, and priming a SECOND empty one mid-turn is off-distribution – the model parrots it back. Otherwise a fresh model turn is opened and primed, which is where the empty thought channel belongs and the only place it does.

Exceptions

std::invalid_argument

if continue_open is set on an empty history, since there would be no turn to resume.

formatToolCall()

std::string Mila::Dnn::Gemma::formatToolCall ( std::string_view name, std::string_view arguments_json )

export

Render an assistant tool call back into Gemma's native call grammar.

Non-object / unparseable arguments render as an empty body rather than throwing, so a malformed history entry degrades to a bare call.

formatToolResponse()

std::string Mila::Dnn::Gemma::formatToolResponse ( std::string_view name, std::string_view result_json )

export

Render a client-executed tool result into Gemma's <|tool_response> grammar.

When the result is a JSON envelope only its primary output field is surfaced (the first non-empty of output/result/content/stdout/text); metadata siblings are dropped. A failed tool ({"content": "", "error": "..."}) has no usable output field, so its error is surfaced explicitly – without it the model sees an empty result and blind-retries. A non-JSON result is passed through under the value: key, which is what the canonical template emits for a non-mapping response.

formatTurn()

std::string Mila::Dnn::Gemma::formatTurn ( const Conversation::Turn & message )

inlineexport

One closed turn: <|turn>{role}
{content}<turn|>
.

An Assistant turn appends each of its tool calls in the native call grammar. A Tool turn carries its result ALREADY rendered – formatToolResponse is the renderer, and it needs the tool's name, which a Conversation::Turn does not carry.

parseToolCall()

std::optional< GemmaToolCall > Mila::Dnn::Gemma::parseToolCall ( std::string_view text )

export

Parse the most recent native Gemma tool call out of accumulated model output, once generation has stopped right after <tool_call|>.

Returns nullopt when no <|tool_call> ... call:name{...} block is present or the block is malformed (so the caller surfaces the text as-is rather than looping on a call it cannot dispatch).

roleSpelling()

std::string_view Mila::Dnn::Gemma::roleSpelling ( Conversation::Role role )

inlineexport

How the template spells a role.

The assistant is "model" in Gemma's vocabulary, and a tool result is a USER turn – rendering Tool by its own name would open a role the model was never trained to read.

serializeToolDeclarations()

std::string Mila::Dnn::Gemma::serializeToolDeclarations ( const std::string & tools_json )

inlineexport

Render tool schemas into Gemma's trained <|tool>declaration:...<tool|> grammar.

There is ONE declaration form because Gemma has one. A plain-text JSON list is not a Gemma format at all – it is prose plus a JSON dump – and when it was measured against this the 12B invented tools that did not exist.

Deliberately NO call-syntax instructions: the model emits calls through its trained <|tool_call> protocol, so teaching a foreign call format only confuses it.

Declarations concatenate with no separator and no leading blank line, which is how the template appends them to the system turn – <|tool> is an atomic special token, so it needs no whitespace to separate it from the system prose.

ONE deliberate deviation from the template: it closes the parameters block from inside its type: branch, so a schema with no type leaves the block unclosed and a comma dangling. This always closes it. Malformed either way; this stays parseable.

Parameters

tools_json

A JSON array of tool schemas, each either an OpenAI function envelope ({type, function:{name, description, parameters}}) or a bare declaration. Filtering which tools to advertise is the host's business, not the model's – a harness with UI-only tools excludes them before calling.

Returns

Empty when the array is empty, does not parse, or names nothing, which omits the declarations entirely – and that absence is what tells the model there are none.

Variable Documentation

kChannelLabels

std::string_view Mila::Dnn::Gemma::kChannelLabels[]

inlineconstexprexport

Initial value:

                                             = {
"thought", "thinking", "analysis", "reasoning" }

The labels a reasoning channel is opened with, for answer extraction.

kThoughtPrime

std::string_view Mila::Dnn::Gemma::kThoughtPrime = "<|channel>thought\n<channel|>"

inlineconstexprexport

The empty reasoning channel that opens a fresh model turn.

Load-bearing rather than cosmetic: without it the 12B emits ghost thought channels and generation degenerates. It belongs at the START of a fresh turn and nowhere else – priming a second one mid-turn is off-distribution and the model parrots it back (measured: empty-channel echoes and one 682-character runaway).