Skip to content

New Optimized data pipeline (new tag, lower RAM, lighter image)

Compare
Choose a tag to compare
@MathiasExorde MathiasExorde released this 14 Oct 12:59
· 9 commits to main since this release
6dd1b2d

Optimized data pipeline (new ZS model, better RAM usage)

Update lab_initialization.py
Update pre_install.py
Update zero_shot.py
Update tag.py
Update Dockerfile
Update models.py
Removes a large zero shot model used until now, for a smaller, more efficient, with similar accuracy model.
We now are able to add it to the batch processing data pipeline and remove it from inter-item processing, accelerating the process & lot, reducing RAM usage over time.

The image will now be 1.2Gb smaller or so, and will have very fast item processing time, with all models loaded in RAM beforehand.