- Generated by
1.17.0
|
Mila
Deep Neural Network Library
|
The local store of installed models. More...
Public Member Functions | |
| ModelStore (std::filesystem::path root=resolveStoreRoot()) | |
| std::filesystem::path | adoptBlob (const std::string &description, const std::filesystem::path &source, const std::string &expected_sha256_hex, bool move_file=true) |
| Take a file that is already on disk into the store, verifying it on the way. | |
| std::filesystem::path | blobPath (const std::string &sha256_hex) const |
| bool | contains (const std::string &sha256_hex) const |
| StoredModel | describe (const ModelRecord &record) const |
| Resolve a record against the blob store without requiring it to be complete. | |
| std::filesystem::path | ensureBlob (const std::string &description, const std::string &expected_sha256_hex, const BlobFetcher &fetcher) |
| Ensure a blob is present, fetching it if it is not. | |
| StoredModel | install (const ModelPackage &package, const InstallOptions &options={}) |
| Install one variant of a package, and record it. | |
| std::vector< StoredModel > | list () const |
| Every installed model, in directory order. | |
| std::optional< StoredModel > | locate (const std::string &name) const |
| Where an installed model's files are, or nothing. | |
| RemovalReport | prune (const PruneOptions &options={}) |
| Reclaim blobs no record names, rejected transfers, and abandoned locks. | |
| std::optional< ModelRecord > | readRecord (const std::string &name) const |
| std::filesystem::path | recordPath (const std::string &name) const |
| RemovalReport | remove (const std::string &name) |
| Remove one installed model, then reclaim what nothing else references. | |
| bool | rename (const std::string &from, const std::string &to) |
| Rename an installed model. | |
| const std::filesystem::path & | root () const noexcept |
| StoreUsage | usage () const |
| What the store holds and what could be reclaimed. | |
| ModelRecord | writeRecord (ModelRecord record) |
| Write a record, stamping the install time. | |
Static Public Member Functions | |
| static std::string | foldName (std::string_view name) |
| The case-folded form of a model name, which is what keys the store. | |
The local store of installed models.
Layout:
* models/<owner>/<repository>/<variant>.json the records -- the index * blobs/sha256-<hex> the content * tmp/ in-flight transfers and their locks *
|
inline |
Take a file that is already on disk into the store, verifying it on the way.
The counterpart to ensureBlob for bytes that need no transfer. It hashes rather than trusting the caller, because the store's whole guarantee is that a path names its content – a blob adopted unverified would poison every later cache hit.
A move publishes by rename when the package and the store share a volume, which costs nothing whatever the file's size. Across volumes there is no atomic move, so the bytes go through tmp/ and are renamed from there: a partial copy must never occupy a path that implies verification.
| std::runtime_error | on a digest mismatch or a lock held by another process. |
|
inline |
Ensure a blob is present, fetching it if it is not.
Resumes from whatever a previous attempt left in tmp/, which is why the partial is named after the digest rather than randomly – a retry must be able to find it. The hash is recomputed over the resumed prefix before appending, because SHA-256 is sequential and cannot be restored from a byte offset alone.
A transfer lock arbitrates between processes. Chat and the inference server share one store, and the deterministic partial name that makes resume possible would otherwise let two of them append into a single file and interleave.
On a digest mismatch the partial is kept under a rejected name rather than destroyed: the bytes are known bad, but the byte count is the evidence that separates "altered in flight" from "a length bug", and destroying it destroys the diagnosis.
| description | What the blob is, for messages only – typically the file's path in the repository it came from. The store never interprets it. |
| expected_sha256_hex | The blob's digest, which is also its name under blobs/. |
| fetcher | Supplies the bytes when the blob is absent, resuming from any partial. |
| std::runtime_error | on fetch failure, digest mismatch, or a lock held elsewhere. |
|
inlinestatic |
The case-folded form of a model name, which is what keys the store.
A name is a filename, so without folding the store inherits the filesystem's opinion of case: /install Llama-3.1-8B resolves on Windows and fails on Linux, and the developer's platform is the forgiving one – so the failure only ever appears for someone else. Folding makes one name mean one model on both.
ASCII by hand rather than std::tolower, which is locale-dependent (a Turkish locale maps 'I' to a dotless form and would key the same model two ways). requireUsableName already restricts names to [A-Za-z0-9._-], so ASCII is the whole domain.
The record keeps the name as it was published – this folds the key, never the label.
|
inline |
Install one variant of a package, and record it.
Publishing to the local store and publishing to a hub take the same directory: what differs is only where the bytes go. The record is written last, after every file has verified, so a failed install leaves nothing that looks installed.
The package is not validated first on purpose. Adoption hashes each file as it takes it, so a separate validate() pass would read every byte a second time – at 6.8 GB that is not a cost worth paying for a check that already happened.
| std::runtime_error | if the variant does not exist, if it needs a newer Mila, if a file's digest disagrees with the manifest, or if the names do not form a coordinate. |
|
inline |
Every installed model, in directory order.
A record whose blobs have gone missing is reported with complete false rather than omitted: a store that silently hides a broken entry cannot be repaired by its owner.
|
inline |
Where an installed model's files are, or nothing.
Never consults a hub and never accepts a path: the store is the only thing a load reads from. A record whose blobs are incomplete resolves to nothing, because a caller that receives a path expects bytes behind it.
|
inline |
Reclaim blobs no record names, rejected transfers, and abandoned locks.
Mark-and-sweep over the record tree is exact and costs a directory walk: records are kilobytes, and the alternative – a reference count maintained by hand – is a number that can be wrong.
|
inline |
Remove one installed model, then reclaim what nothing else references.
The sweep is what makes this safe. Deduplication means a tokenizer blob may back several models, so removal cannot delete a model's files simply because that model is going.
|
inline |
Rename an installed model.
One record is rewritten and nothing else moves. The blobs are content-addressed, so what a model is called here has no bearing on where its bytes live; and origin is a field rather than a path segment, so a renamed model still says where it came from.
The install time is carried over: renaming is not reinstalling.
| std::runtime_error | if the new name is unusable or already taken. |
|
inline |
Write a record, stamping the install time.
Written to tmp/ and renamed, because a peer process may be listing the store while this one installs, and a half-written record must never be readable.
Persist a record, and hand back what was actually written.
Returns the record rather than void because the install time is stamped here: a caller that kept its own copy would hold one that disagrees with the store.
1.17.0