Qwen.Protocol.ixx File Reference

Canonical Qwen 3.8 chat protocol: the ChatML turn structure, the reasoning gate and the tool-call grammar. More...

#include <format>
#include <optional>
#include <span>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
#include <memory>
import nlohmann.json;
import Dnn.Models.Conversation;

Namespaces
namespace	Mila
	Mila main API namespace.

Enumerations
enum class	Mila::Dnn::Qwen::ReasoningEffort { Low , Medium , Xhigh }
	How much reasoning to spend, as the checkpoint's template defines it. More...

Functions
std::string	Mila::Dnn::Qwen::formatPrompt (std::span< const Conversation::Turn > history, bool enable_thinking, ReasoningEffort effort=ReasoningEffort::Medium, const std::string &tool_signatures={})
	Render a history into Qwen's ChatML prompt, primed for generation.
std::string	Mila::Dnn::Qwen::formatToolCall (const std::string &name, const std::string &arguments)
	One tool call rendered in the grammar the model emits it in.
std::string	Mila::Dnn::Qwen::formatTurn (const Conversation::Turn &message)
	Render one completed turn, delimiters included.
std::optional< Conversation::ToolCall >	Mila::Dnn::Qwen::parseToolCall (const std::string &response)
	The first tool call in a response, or nothing when it holds none.
ReasoningEffort	Mila::Dnn::Qwen::reasoningEffortFromScale (int level)
	Map a 1..5 effort scale onto the three levels the model knows.
std::string_view	Mila::Dnn::Qwen::reasoningInstruction (ReasoningEffort effort)
	The instruction the template prepends to the system turn, verbatim. Empty for Medium.
std::string	Mila::Dnn::Qwen::serializeToolSignatures (const std::string &tools_json)
	The tool signatures as the template renders them: one JSON object per line.
std::string	Mila::Dnn::Qwen::toolsSection (const std::string &signatures)
	The tools section of the system turn, verbatim from the template.
nlohmann::json	Mila::Dnn::Qwen::typedParameterValue (const std::string &text)
	The value of one <parameter=...> block, typed the way the schema-free text allows.

Variables
constexpr std::string_view	Mila::Dnn::Qwen::kThinkClose = "</think>"
constexpr std::string_view	Mila::Dnn::Qwen::kThinkOpen = "<think>"
constexpr std::string_view	Mila::Dnn::Qwen::kToolCallClose = "</tool_call>"
constexpr std::string_view	Mila::Dnn::Qwen::kToolCallOpen = "<tool_call>"
constexpr std::string_view	Mila::Dnn::Qwen::kToolResponseClose = "</tool_response>"
constexpr std::string_view	Mila::Dnn::Qwen::kToolResponseOpen = "<tool_response>"
constexpr std::string_view	Mila::Dnn::Qwen::kTurnClose = "<|im_end|>"
constexpr std::string_view	Mila::Dnn::Qwen::kTurnOpen = "<|im_start|>"
	The turn delimiters.

Detailed Description

Canonical Qwen 3.8 chat protocol: the ChatML turn structure, the reasoning gate and the tool-call grammar.

The grammar is a property of the model, not of any single adaptor, so this runtime module is the one source of truth for it – the same rule that brought Gemma's down (see Gemma.Protocol.ixx). It was folded here from the Chat harness, where it was written first.

Rendered from the checkpoint's own chat template rather than from the Qwen 3 conventions it resembles, which differ in three places that all break silently: the call grammar is nested tags and not a JSON object, reasoning effort is a trained parameter with its own instruction text, and the generation primer leaves the reasoning span OPEN.

One file for the whole protocol because the pieces agree by construction – the markers here are the ones BpeVocabulary::loadQwen registers from the checkpoint vocabulary, so a template written apart from its parser could drift from it unnoticed.

String-level parse/format only, matching Gemma's module.

Enumeration Type Documentation

ReasoningEffort

enum class Mila::Dnn::Qwen::ReasoningEffort

exportstrong

How much reasoning to spend, as the checkpoint's template defines it.

Three levels rather than a continuum, and Medium is deliberately silent: the template emits no instruction for it, so the model runs at its trained default. Naming a fourth value, or inventing wording for Medium, would put text in the system turn that this checkpoint was not tuned against – which is what steering Gemma through prose has to do because Gemma's toggle carries no effort parameter at all.

Function Documentation

formatPrompt()

std::string Mila::Dnn::Qwen::formatPrompt ( std::span< const Conversation::Turn > history, bool enable_thinking, ReasoningEffort effort = ReasoningEffort::Medium, const std::string & tool_signatures = {} )

inlineexport

Render a history into Qwen's ChatML prompt, primed for generation.

There is no BOS: the checkpoint sets add_bos_token false, and a leading marker the model was not trained with costs a position and shifts every one after it.

The system turn is assembled here rather than taken from history, because the template orders it – reasoning instruction, then tools, then whatever the caller configured – and a caller that had already concatenated them could not produce that order.

Parameters

history	The conversation so far, oldest turn first. A system turn in it is ignored: this function assembles that turn itself, for the reason above.
enable_thinking	False makes the primer carry a CLOSED, empty reasoning span, which is the model's own suppression mechanism: a finished channel is one it continues past. True leaves the span OPEN, so the response begins inside the reasoning and carries a closing marker with no opening one.
effort	Selects the trained instruction text for the reasoning budget. Medium is the checkpoint's default and emits NOTHING – its instruction is the absence of one – so it is not a middle setting between the other two but the unmodified model. Ignored entirely when enable_thinking is false, since the instruction would then describe a channel that never opens.
tool_signatures	One JSON object per tool, newline separated – what serializeToolSignatures returns. Empty omits the whole tools section, which is what tells the model there are none.

Exceptions

std::invalid_argument

if history is empty or ends on an Assistant turn, since the primer would then open a second consecutive assistant turn.

formatToolCall()

std::string Mila::Dnn::Qwen::formatToolCall ( const std::string & name, const std::string & arguments )

inlineexport

One tool call rendered in the grammar the model emits it in.

Parameters

name	The function name, rendered into the <function=...> opener verbatim.
arguments	A JSON object as text. String values are spliced raw, everything else is re-encoded – which is the template's own rule, and it is what lets a parameter hold prose containing quotes or newlines without escaping.

formatTurn()

std::string Mila::Dnn::Qwen::formatTurn ( const Conversation::Turn & message )

inlineexport

Render one completed turn, delimiters included.

A tool result is a USER turn carrying a <tool_response> span, not a role of its own – rendering Conversation::Role::Tool by its own name would open a role Qwen was never trained to read. Consecutive results are not merged into one user turn the way the template does; no host dispatches parallel calls yet, so the case does not arise.

parseToolCall()

std::optional< Conversation::ToolCall > Mila::Dnn::Qwen::parseToolCall ( const std::string & response )

inlineexport

The first tool call in a response, or nothing when it holds none.

Anchored on the <tool_call> span, so ordinary prose is never routed here – and the template itself permits natural-language reasoning BEFORE a call, which a parser keyed on anything looser would swallow. An unterminated span means generation stopped inside the call: nothing is returned rather than half a call being dispatched.

Exceptions

std::runtime_error

if the span holds something that is not a call. The caller surfaces that – a malformed call inside correct delimiters is the model failing at the protocol, which is worth seeing rather than silently treating as prose.

reasoningEffortFromScale()

ReasoningEffort Mila::Dnn::Qwen::reasoningEffortFromScale ( int level )

inlineexport

Map a 1..5 effort scale onto the three levels the model knows.

The scale is a host's, shared across families, so the ends are pinned to the ends and the middle to the silent default. Here rather than in an adaptor so that two hosts asking for the same effort get the same prompt.

serializeToolSignatures()

std::string Mila::Dnn::Qwen::serializeToolSignatures ( const std::string & tools_json )

inlineexport

The tool signatures as the template renders them: one JSON object per line.

Not a JSON array – the template writes the objects newline separated inside <tools> with no array around them. Takes the array a host already holds, so a caller with tool schemas in any form converts once, to JSON, rather than to a private wire shape.

Parameters

tools_json

A JSON array of tool signature objects, as text.

Returns

Empty when the array is empty or does not parse, which omits the whole tools section – and that absence is what tells the model there are none.

toolsSection()

std::string Mila::Dnn::Qwen::toolsSection ( const std::string & signatures )

inlineexport

The tools section of the system turn, verbatim from the template.

The call format and the reminder block are protocol, not instructions we composed: the model is tuned to emit exactly this shape, and a paraphrase is a different prompt.

Parameters

signatures

One JSON object per tool, newline separated, as the template renders them inside <tools>.

typedParameterValue()

nlohmann::json Mila::Dnn::Qwen::typedParameterValue ( const std::string & text )

inlineexport

The value of one <parameter=...> block, typed the way the schema-free text allows.

Parsed as JSON when it parses, and kept as a string when it does not. The grammar carries no types – a parameter holding 4096 and one holding the word high look identical – so the choice is between guessing here and handing every handler a string it has to re-interpret. JSON-when-valid recovers numbers, booleans and nested objects, and prose is never valid JSON by accident.

Variable Documentation

kTurnOpen

std::string_view Mila::Dnn::Qwen::kTurnOpen = "<|im_start|>"

inlineconstexprexport

The turn delimiters.

Registered specials in the checkpoint vocabulary, so they are emitted as literal text here and encode atomically.