Core Directory Reference

Files
	FileHeader.ixx
	Common file header structure for Mila data files.
	TokenizerTrainer.ixx
	Abstract trainer interface for building tokenizers' vocabularies.
	TrainerFactory.ixx
	Factory helpers to construct tokenizer trainers and load vocabularies.