You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The fileformats are limited to huggingface generators, however there are cases where the generator can load a local checkpoint that would not be complatible with this probe. This is due the the expectation that the model under test is published for file enumeration via the huggingface_hub utility library.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/garak/cli.py", line 530, in main
command.probewise_run(
File "/usr/local/lib/python3.10/site-packages/garak/command.py", line 237, in probewise_run
probewise_h.run(generator, probe_names, evaluator, buffs)
File "/usr/local/lib/python3.10/site-packages/garak/harnesses/probewise.py", line 107, in run
h.run(model, [probe], detectors, evaluator, announce_probe=False)
File "/usr/local/lib/python3.10/site-packages/garak/harnesses/base.py", line 114, in run
attempt_results = probe.probe(model)
File "/usr/local/lib/python3.10/site-packages/garak/probes/fileformats.py", line 63, in probe
repo_filenames = huggingface_hub.list_repo_files(generator.name)
File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
validate_repo_id(arg_value)
File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/usr/local/checkpoints/Mistral-NeMo-Minitron-8B-Instruct/'. Use `repo_type` argument if needed.
This probe needs to be able to either skip for local model paths or follow an alternative file gathering approach.
The text was updated successfully, but these errors were encountered:
The
fileformats
are limited to huggingface generators, however there are cases where the generator can load a local checkpoint that would not be complatible with this probe. This is due the the expectation that the model under test is published for file enumeration via thehuggingface_hub
utility library.Contrived example:
Current trace reported in logs:
This probe needs to be able to either skip for
local
model paths or follow an alternative file gathering approach.The text was updated successfully, but these errors were encountered: