Fix/errors for some models including Mistral-7b and Flan-ul2 (#388)

* For some models like Mistral and Flan-UL2, when downloaded from the hub the binary files write to a snapshots/<uuid> subdirectory. Then when the files are committed, they cannot be found and the commit fails with an error. * Changed `local.split("/")[-1]` to os.path.basename(local) for compatibility with all OS including Windows. * Removed snapshots section * Removed additional line break. --------- Co-authored-by: Lachlan Cahill <[email protected]>
huggingface · Nov 27, 2023 · 9c81742 · 9c81742
1 parent 08dd5f2
commit 9c81742
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/bindings/python/convert.py b/bindings/python/convert.py
@@ -156,7 +156,7 @@ def convert_multi(
     local_filenames.append(index)
 
     operations = [
-        CommitOperationAdd(path_in_repo=local.split("/")[-1], path_or_fileobj=local) for local in local_filenames
+        CommitOperationAdd(path_in_repo=os.path.basename(local), path_or_fileobj=local) for local in local_filenames
     ]
     errors: List[Tuple[str, "Exception"]] = []