Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3: Kiko writes empty files #10

Open
ben-hawkyard-absolute opened this issue Apr 29, 2024 · 0 comments
Open

Python 3: Kiko writes empty files #10

ben-hawkyard-absolute opened this issue Apr 29, 2024 · 0 comments

Comments

@ben-hawkyard-absolute
Copy link

Hi there, I've been having trouble saving .kiko files correctly with python 3. As far as I can tell, when a kiko file is saved the data files that get added to the underlying archive are truncated at zero bytes. You can reproduce this with the following code snippet:

import os
import tempfile
from kiko.io.kikofile import KikoFile


handle, kiko_path = tempfile.mkstemp(suffix=".kiko")
os.close(handle)
kiko_file = KikoFile(kiko_path)
kiko_file.set_data({"test": "data"})
kiko_file.save()
kiko_file.parse()

When I do this I get an error that looks something like:

Traceback (most recent call last):
  File "/home/ben.hawkyard/Documents/scripts/testing/kiko_metadata_testing.py", line 11, in <module>
    kiko_file.parse()
  File "/opt/rez/rez_package_cache/kiko/1.0.0/a99e/a/python/kiko/io/kikofile.py", line 145, in parse
    self._metadata = json.load(v)
                     ^^^^^^^^^^^^
  File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/__init__.py", line 293, in load
    return loads(fp.read(),
           ^^^^^^^^^^^^^^^^
  File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I think this is caused by this bit of code in io/kikofile.py:

    @staticmethod
    def _add_to_tar(tar_file, name, f_obj):
        info = tarfile.TarInfo(name=name)
        if sys.version_info.major == 2:
            info.size = len(f_obj.buf)
            info.time = time.time()
        else:
            info.size = f_obj.tell()
            info.mtime = time.time()
        tar_file.addfile(tarinfo=info, fileobj=f_obj)

    @classmethod
    def _add_to_tar_from_dict(cls, tar_file, name, data):
        io = StringIO()
        json.dump(data, io)
        io.seek(0)
        cls._add_to_tar(tar_file, name, io)

calling io.seek(0) in _add_to_tar_from_dict means that f_obj.tell() in _add_to_tar returns zero and this causes us to write empty files to our kiko file when adding dicts to it.

I've done some testing and I think this could be fixed by changing the else branch in _add_to_tar:

        else:
            f_obj = BytesIO(f_obj.read().encode("utf-8"))  # TarFile expects file objects to be opened in binary mode.
            info.size = f_obj.seek(0, os.SEEK_END)
            f_obj.seek(0)
            info.mtime = time.time()

I'm happy to put in a pull request for this if there's interest. I can also see that mottosse fixes this in their pull request for kiko here: #6. Is this likely to be accepted, do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant