Nested JSON format serializing None as "None" instead of null

[vinceh121]

Describe the issue

In between the upgrade between 4.18.2 and 5.0.1, a bug (or an undocumented change?) was introduced that makes the nested JSON format serialize Python None as the string "None" instead of the JSON null as padding within arrays.

I'm unclear on whether this is a bug in Weblate or in the Translate library.

I already tried

• I've read and searched the documentation.
• I've searched for similar issues in this repository.

Steps to reproduce the behavior

1. Have a translation component using a nested JSON with arrays
2. Translate the first and last units of an array, leaving the ones in the middle untranslated.
3. Commit changes

Expected behavior

The padding within arrays should be done using nulls and not "None" strings.
Or this behaviour might be configurable, as I can see why some apps wouldn't play nice with nulls and may prefer a default string.

Screenshots

Exception traceback

No response

How do you run Weblate?

PyPI module

Weblate versions

• Weblate: 5.0.2
• Django: 4.2.3
• siphashc: 2.1
• translate-toolkit: 3.10.1
• lxml: 4.9.3
• Pillow: 9.5.0
• nh3: 0.2.14
• python-dateutil: 2.8.2
• social-auth-core: 4.4.2
• social-auth-app-django: 5.2.0
• django-crispy-forms: 2.0
• oauthlib: 3.2.2
• django-compressor: 4.4
• djangorestframework: 3.14.0
• django-filter: 23.2
• django-appconf: 1.0.5
• user-agents: 2.2.0
• filelock: 3.12.2
• rapidfuzz: 3.1.1
• openpyxl: 3.1.2
• celery: 5.3.1
• django-celery-beat: 2.5.0
• kombu: 5.3.1
• translation-finder: 2.15
• weblate-language-data: 2023.5
• html2text: 2020.1.16
• pycairo: 1.24.0
• PyGObject: 3.44.1
• diff-match-patch: 20230430
• requests: 2.31.0
• django-redis: 5.3.0
• hiredis: 2.2.3
• sentry-sdk: 1.28.1
• Cython: 0.29.36
• misaka: 2.1.1
• GitPython: 3.1.32
• borgbackup: 1.2.6
• pyparsing: 3.1.1
• ahocorasick_rs: 0.17.1
• python-redis-lock: 4.0.0
• charset-normalizer: 3.2.0
• Python: 3.11.2
• Git: 2.39.2
• phply: 1.2.6
• ruamel.yaml: 0.17.32
• tesserocr: 2.6.1
• boto3: 1.26.165
• zeep: 4.2.1
• aeidon: 1.12
• iniparse: 0.5
• Mercurial: 6.5
• git-review: 2.3.1
• PostgreSQL server: 13.11
• Database backends: django.db.backends.postgresql
• Cache backends: default:RedisCache, avatar:FileBasedCache
• Email setup: django.core.mail.backends.smtp.EmailBackend: localhost
• OS encoding: filesystem=utf-8, default=utf-8
• Celery: redis://localhost/2, redis://localhost/2, regular
• Platform: Linux 6.1.0-12-cloud-amd64 (x86_64)

Weblate deploy checks

No response

Additional context

No response

[nijel]

I think it always behaved this way. Anything in the json file is converted to a string, which is presented in weblate for translation. Until you touch these in weblate, they might be preserved, though.

[vinceh121]

The partially translated French file in the screenshot was fully created and managed by Weblate 4.18.2. This diff is from a commit right after we upgraded to 5.0.1 (also then 5.0.2)

I've confirmed with a quick local install that this is indeed how the format behaved in 4.18.2: https://gitlab.com/vinceh121/test9963/-/commit/75ea7019d80d8d170852031385beb50dcb1c56c6

Edit: this "None" behaviour in 5.0.2 seems to only apply when manually committing with the UI, and scheduled commits/push use null

[vinceh121]

After further research and debugging, I've traced this change down to this commit in translate, originally for handling JSON numbers: translate/translate@d8d5a3e

Perhaps it may make sense to add a special case for None?

[nijel]

Both translate-toolkit and Weblate expect strings as translatable content. Right now, it converts any value to a string. Before translate/translate@d8d5a3e it would fail to load such file with ValueError.

[vinceh121]

I understand that, but serializing a Python specific value into a generic JSON strikes me as odd.

But then again, I've again confirmed that this behaviour is inconsistent as it only happens with manual commits, and scheduled commits do the padding with null in 5.0.2.

Examples here, with scheduled commit: Nadekooo/YTTDFR@3ffa088
And here, with a manual commit: Nadekooo/YTTDFR@99f0336

[nijel]

The problem seems in preserving null values once they are present. The first commit adds them as placeholders for missing values. translate/translate#5034 should address this.

[github-actions]

The issue you've reported needs to be addressed in the translate-toolkit. Please file the issue there, and include links to any relevant specifications about the formats (if applicable).

[github-actions]

Thank you for your report; the issue you have reported has just been fixed.

• In case you see a problem with the fix, please comment on this issue.
• In case you see a similar problem, please open a separate issue.
• If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.