ModelManifest.ixx File Reference

The mila.json schema: what files compose a model, and for which Mila. More...

#include <charconv>
#include <cstdint>
#include <format>
#include <memory>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
import Mila.Version;
import nlohmann.json;

Classes
struct	Mila::Distribution::ModelFile
	One file composing a model, as the manifest declares it. More...
struct	Mila::Distribution::ModelManifest
	A model's published description. More...

Namespaces
namespace	Mila
	Mila main API namespace.

Functions
ModelManifest	Mila::Distribution::parseModelManifest (const std::string &text, std::string_view source)
	Parse a manifest, refusing anything a consumer would have to guess about.
bool	Mila::Distribution::parseVersion (std::string_view text, int &major, int &minor, int &patch)
	Parse "<major>.<minor>[.<patch>]", with an optional prerelease or build suffix.
void	Mila::Distribution::requireCompatibleMilaVersion (const ModelManifest &manifest, std::string_view source)
	Refuse an artifact that needs a newer Mila.
std::string	Mila::Distribution::toJsonText (const ModelManifest &manifest)
	Serialize a manifest.

Variables
constexpr std::string_view	Mila::Distribution::kLicenseRole = "license"
	The source model's license text, and any attribution file the license names.
constexpr std::string_view	Mila::Distribution::kNoticeRole = "notice"
constexpr std::string_view	Mila::Distribution::kTokenizerRole = "tokenizer"
	Optional, and separate from the weights so a reader can find it without knowing its name.
constexpr std::string_view	Mila::Distribution::kWeightsRole = "weights"
	The role every model must declare, because a model without weights is not one.

Detailed Description

The mila.json schema: what files compose a model, and for which Mila.

One repository is one model is one name. Precision is part of that name rather than a key inside this document, which is why there is no variant map here. See Specifications/ModelDistribution.md.

Function Documentation

parseModelManifest()

ModelManifest Mila::Distribution::parseModelManifest ( const std::string & text, std::string_view source )

export

Parse a manifest, refusing anything a consumer would have to guess about.

Parameters

text	The manifest's JSON text.
source	What is being parsed, for messages only – a name or a package path.

Exceptions

std::runtime_error

on malformed JSON, or a model that declares no usable files.

parseVersion()

bool Mila::Distribution::parseVersion	(	std::string_view	text,
		int &	major,
		int &	minor,
		int &	patch )

Parse "<major>.<minor>[.<patch>]", with an optional prerelease or build suffix.

std::from_chars rather than sscanf: no locale involvement, and trailing text is examined rather than silently discarded, so a manifest whose version field holds something that is not a version is refused instead of read as one this build happens to satisfy. A '-' or '+' suffix is accepted because Mila's own versions carry one.

requireCompatibleMilaVersion()

void Mila::Distribution::requireCompatibleMilaVersion ( const ModelManifest & manifest, std::string_view source )

export

Refuse an artifact that needs a newer Mila.

A version comparison here beats a parse error somewhere inside the tensor index, which is what a format change would otherwise look like.

toJsonText()

std::string Mila::Distribution::toJsonText ( const ModelManifest & manifest )

export

Serialize a manifest.

ordered_json rather than json: the emitted file is read by people, and a map that sorts its keys puts the tokenizer above the weights and the files above the architecture.

Variable Documentation

kLicenseRole

std::string_view Mila::Distribution::kLicenseRole = "license"

inlineconstexprexport

The source model's license text, and any attribution file the license names.

Declared rather than merely shipped in the package, because only a declared file is fetched: every license Mila redistributes under requires its copy travel with the work, and a pull that delivered weights alone would leave the recipient holding a copy that carries neither. Llama 3.x 1.b.iii is the strictest of them – the notice must sit in a file distributed as part of the copy – and Apache 2.0 4(a) and MIT ask the same of the license text.