Improve docs/model_guide.md
with skeleton template code + description of utils like Collator
and Reorderer
#2358
Labels
documentation
Improvements or additions to documentation.
feature request
A feature that isn't implemented yet.
As per title, some of the machinery we use in HFLM around the ordering, batching, and caching of requests is still opaque to most users.
It'd be great if we could expand
model_guide.md
with:templates/new_template_model
, a filestub containing the rough outline of a fully-featured HF-like local model impl that uses reorderer, collator, partial caching, and so onThe text was updated successfully, but these errors were encountered: