Mila
Deep Neural Network Library
Loading...
Searching...
No Matches
ModelStore.ixx File Reference

The local store of installed models: content-addressed blobs plus the records that name them. More...

#include <algorithm>
#include <chrono>
#include <cstdint>
#include <cstdlib>
#include <filesystem>
#include <format>
#include <fstream>
#include <functional>
#include <ios>
#include <memory>
#include <optional>
#include <set>
#include <stdexcept>
#include <string>
#include <string_view>
#include <system_error>
#include <vector>
import Distribution.ModelPackage;
import Distribution.ModelManifest;
import Distribution.Environment;
import Distribution.Sha256;
import nlohmann.json;

Classes

struct  Mila::Distribution::FetchReport
struct  Mila::Distribution::InstallOptions
 How a package is to be installed. More...
struct  Mila::Distribution::ModelRecord
 An installed model: what the manifest published, plus how this copy came to be here. More...
class  Mila::Distribution::ModelStore
 The local store of installed models. More...
struct  Mila::Distribution::PruneOptions
struct  Mila::Distribution::RemovalReport
struct  Mila::Distribution::StoredModel
 A record together with where its bytes actually are. More...
struct  Mila::Distribution::StoreUsage

Namespaces

namespace  Mila
 Mila main API namespace.

Typedefs

using Mila::Distribution::BlobFetcher
 Fetches one blob from resume_from, handing each chunk to sink.

Enumerations

enum class  Mila::Distribution::FetchOutcome { Complete , RangeIgnored , RangeNotSatisfiable , Failed }
 What a fetch attempt reported back to the store. More...

Functions

std::filesystem::path Mila::Distribution::resolveStoreRoot ()
 Resolve the store root, first match wins.

Detailed Description

The local store of installed models: content-addressed blobs plus the records that name them.

A blob's path is its digest, so a file present in the store has been verified and an interrupted transfer cannot be mistaken for a complete one. Records are what make the store listable, and what removal is refcounted against. See Specifications/ModelDistribution.md.

Typedef Documentation

◆ BlobFetcher

Initial value:
std::function<FetchReport(
uint64_t resume_from,
const std::function<bool( const char*, size_t )>& sink )>
Definition ModelStore.ixx:62

Fetches one blob from resume_from, handing each chunk to sink.

The store supplies the sink so it can hash and write in one pass. What is being fetched is the fetcher's business: the store deals in digests, and a URL scheme belongs to whichever hub produced the fetcher.

Enumeration Type Documentation

◆ FetchOutcome

enum class Mila::Distribution::FetchOutcome
exportstrong

What a fetch attempt reported back to the store.

Deliberately narrower than HttpResult: the store cares about four outcomes, and keeping it decoupled from the HTTP client is what makes it testable offline.

Enumerator
Complete 

The requested byte range arrived in full.

RangeIgnored 

A resume was requested and the server sent the whole file.

RangeNotSatisfiable 

A resume was requested from at or past the end, so nothing was sent and the partial already holds every byte there is.

Whether they are the right bytes is the digest's question, not the transport's.

Failed 

Anything else; message carries the detail.

Function Documentation

◆ resolveStoreRoot()

std::filesystem::path Mila::Distribution::resolveStoreRoot ( )
inlineexport

Resolve the store root, first match wins.

MILA_CACHE_DIR, then the platform user cache. An explicit override exists because a multi-gigabyte store often does not belong on the same volume as the user profile.

The root holds models/, blobs/ and tmp/, so it does NOT end in "models" – appending it here as well as in recordPath() is what produced the Mila\models\models tree.