HuggingFaceHub.ixx File Reference

The HuggingFace implementation of IModelHub, and its token discovery. More...

#include <cstddef>
#include <cstdint>
#include <filesystem>
#include <format>
#include <fstream>
#include <functional>
#include <ios>
#include <memory>
#include <stdexcept>
#include <string>
#include <utility>
#include <vector>
import Distribution.HttpTransport;
import Distribution.HttpClient;
import Distribution.ModelStore;
import Distribution.HttpTransportBackend;
import Distribution.ModelHub;
import Distribution.Environment;
import Distribution.ModelCoordinate;
import nlohmann.json;

Classes
class	Mila::Distribution::HuggingFaceHub
	The HuggingFace hub. More...

Namespaces
namespace	Mila
	Mila main API namespace.

Functions
std::string	Mila::Distribution::discoverHuggingFaceToken ()
	HuggingFace token, or empty for anonymous access.
std::unique_ptr< IModelHub >	Mila::Distribution::makeDefaultModelHub (ProgressCallback progress={})
	The hub a consumer gets when it does not care how the bytes arrive.
std::vector< HubModel >	Mila::Distribution::parseHuggingFaceListing (const std::string &body)
	Parse a HuggingFace models listing.

Detailed Description

The HuggingFace implementation of IModelHub, and its token discovery.

The only place in the library that names a huggingface.co URL, and always compiled: URL shapes, the token file, the listing quirks and the meaning of a 403 are knowledge, not transport. It performs no I/O of its own – an IHttpTransport does that – so a build with no libcurl still knows everything about HuggingFace and needs only bytes from elsewhere. See Specifications/ModelDistribution.md.

Function Documentation

discoverHuggingFaceToken()

std::string Mila::Distribution::discoverHuggingFaceToken ( )

export

HuggingFace token, or empty for anonymous access.

First match wins: MILA_HF_TOKEN, HF_TOKEN, then the file huggingface-cli login writes. Gemma 4 is Apache 2.0 and needs none of this; Llama 3.1 and 3.2 are gated and do.

makeDefaultModelHub()

std::unique_ptr< IModelHub > Mila::Distribution::makeDefaultModelHub ( ProgressCallback progress = {} )

export

The hub a consumer gets when it does not care how the bytes arrive.

HuggingFace over whichever transport this build was compiled with. In a build without libcurl the hub still knows every URL and rule – what it gets back is a transport that refuses, so the failure names the missing dependency rather than a network error.

parseHuggingFaceListing()

std::vector< HubModel > Mila::Distribution::parseHuggingFaceListing ( const std::string & body )

export

Parse a HuggingFace models listing.

Exposed so the shape can be tested against a recorded response rather than a live one. Repositories that publish no mila.json are dropped: an owner may hold models this runtime cannot load, and reporting them as available would be a lie.