PICARD-2731: Update for managing translations on Weblate

.gitignore

@@ -42,3 +42,4 @@ win-version-info.txt
 *~
 appledev.p12
 build.cfg
+.weblate.ini

CONTRIBUTING.md

@@ -71,7 +71,7 @@
 
 ## Translations
 
-See po/README.md for information about translations.
+See [po/README.md](./po/README.md) for information about translations.
 
 
 ## Audio Metadata Specifications
@@ -122,4 +122,4 @@
 
 ### Other specs
 
-- [ReplayGain 2.0 specification](http://wiki.hydrogenaud.io/index.php?title=ReplayGain_2.0_specification)
+- [ReplayGain 2.0 specification](http://wiki.hydrogenaud.io/index.php?title=ReplayGain_2.0_specification)

RELEASING.md

@@ -49,12 +49,6 @@ This shouldn't be needed, but better to check before releasing
 git ls-tree --full-tree -r HEAD --name-only |while read f; do sed -i '1s/^\xEF\xBB\xBF//' "$f"; done && git diff --quiet || git commit -a -m 'Remove nasty BOM bytes'
 ```
 
-## Get latest translations from Transifex
-
-```bash
-python setup.py pull_translations && git diff --quiet || git commit -m 'Update .po files' -- po/
-```
-
 ## Synchronize generated consts
 
 ```bash

org.musicbrainz.Picard.appdata.xml.in

@@ -103,7 +103,7 @@
   <url type="faq">https://picard-docs.musicbrainz.org/en/faq/faq.html</url>
   <url type="help">https://picard-docs.musicbrainz.org/</url>
   <url type="donation">https://metabrainz.org/donate</url>
-  <url type="translate">https://www.transifex.com/musicbrainz/musicbrainz/picard/</url>
+  <url type="translate">https://translations.metabrainz.org/projects/picard/</url>
 
   <translation type="gettext">picard</translation>
   <developer_name>MetaBrainz Foundation</developer_name>

po/README.md

@@ -1,57 +1,77 @@
 Translations
 ============
 
-Picard translations are handled by [Transifex](https://www.transifex.com).
+Picard translations are handled by [Weblate](https://translations.metabrainz.org/projects/picard/). For translation instructions please see [Picard, Picard Website and Picard User Guide Internationalization](https://wiki.musicbrainz.org/MusicBrainz_Picard/Internationalization).
+
+The translation files are automatically synced between the Picard Github repository and Weblate. Translations can be done in Weblate or by updating the translation files directly.
+
+Below is a technical description for managing the translations as a Picard maintainer or developer.
 
-_Please do not manually edit the PO files._
 
 Required tools
 --------------
 
-* [Transifex client 1.x](https://developers.transifex.com/docs/cli)
+* [Weblate Client](https://docs.weblate.org/en/latest/wlc.html)
 * [Babel](https://babel.pocoo.org/)
 
 
 Picard source tree strings
 --------------------------
 
-Their translations are handled at <https://www.transifex.com/musicbrainz/musicbrainz/picard/>
+Their translations are handled at <https://translations.metabrainz.org/projects/picard/app/>
 
 One can update `picard.pot` using:
+
 ```bash
 $ python setup.py regen_pot_file
 ```
 
-Transifex will _automatically_ pick `picard.pot` from [Picard git repository master branch](https://github.com/metabrainz/picard/tree/master) once per day.
+Weblate will _automatically_ sync the changed `picard.pot` and update the translation files (`*.po`) with msgmerge.
 
 
-Attributes and countries strings
---------------------------------
+AppStream metadata translations
+-------------------------------
 
-Their translations are handled at <https://www.transifex.com/musicbrainz/musicbrainz/attributes/> and <https://www.transifex.com/musicbrainz/musicbrainz/countries/>
+Translations for the strings from `org.musicbrainz.Picard.appdata.xml.in` are handled at <https://translations.metabrainz.org/projects/picard/appstream/>.
 
-`attributes.pot` and `countries.pot` are updated by [musicbrainz-server project](https://github.com/metabrainz/musicbrainz-server), outside the Picard project.
+One can update `appstream/picard-appstream.pot` using:
 
-Picard maintainers can regenerate `picard/const/attributes.py` and `picard/const/countries.py`, which are using `attributes.pot` and `countries.pot` as base, using the command:
 ```bash
-$ python setup.py update_constants
+$ python setup.py regen_appdata_pot_file
 ```
-It will retrieve and parse latest `attributes.pot` and `countries.pot` to rebuild `picard/const/attributes.py` and `picard/const/countries.py`.
 
+Weblate will _automatically_ sync the changed `picard-appstream.pot` and update the translation files (`appstream/*.po`) with msgmerge.
+
+
+Windows installer translations
+------------------------------
 
-To fetch latest translations from Transifex
--------------------------------------------
+The translations for the Windows installer are inside the JSON files in `installer/i18n/sources`.
+Translation in Weblate is done at <https://translations.metabrainz.org/projects/picard/installer/>
+
+
+Attributes and countries strings
+--------------------------------
+
+Their translations are handled at <https://translations.metabrainz.org/projects/musicbrainz/attributes/> and <https://translations.metabrainz.org/projects/musicbrainz/countries/>
+
+`attributes.pot` and `countries.pot` are updated by [musicbrainz-server project](https://github.com/metabrainz/musicbrainz-server), outside the Picard project.
+
+Picard maintainers can regenerate `picard/const/attributes.py` and `picard/const/countries.py`, which are using `attributes.pot` and `countries.pot` as base. For this an Weblate API key is required, which can be found in your Weblate user settings under [API access](https://translations.metabrainz.org/accounts/profile/#api). The constants can then be updated with the following command:
 
-Use the following command:
 
 ```bash
-$ python setup.py pull_translations
+$ python setup.py update_constants --weblate-key={YOUR_WEBLATE_API_KEY}
 ```
 
-It will fetch all po files from Transifex, but the most incomplete ones.
+Instead of entering the Weblate API key each time you can also place a file `.weblate.ini` in the root of the repository with the following content:
 
-The minimum acceptable percentage of a translation in order to download it can be seen using:
-```bash
-$ python setup.py pull_translations --help
+```ini
+[weblate]
+url = https://translations.metabrainz.org/api/
+
+[keys]
+https://translations.metabrainz.org/api/ = YOUR_WEBLATE_API_KEY
 ```
-The percentage value is passed to the `tx pull` command.
+
+It will retrieve and parse latest `attributes.pot` and `countries.pot` to rebuild `picard/const/attributes.py` and `picard/const/countries.py`.

requirements-dev.txt

@@ -4,3 +4,4 @@ flake8
 isort>=5.0
 pycodestyle
 pylint>=2.6.0
+wlc

scripts/tools/pull-shared-translations.py

@@ -0,0 +1,125 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Picard, the next-generation MusicBrainz tagger
+#
+# Copyright (C) 2023 Philipp Wolfer
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+
+import argparse
+import logging
+import os
+import os.path
+import sys
+
+from wlc import (
+    Component,
+    Weblate,
+)
+from wlc.config import WeblateConfig
+
+
+WEBLATE_URL = 'https://translations.metabrainz.org/api/'
+PROJECT_NAME = 'musicbrainz'
+PROJECT_COMPONENTS = (
+    'attributes',
+    'countries',
+)
+MIN_TRANSLATED_PERCENT = 10
+
+
+logging.basicConfig(
+    force=True,
+    format="%(asctime)s:%(levelname)s: %(message)s",
+    level=logging.INFO,
+    stream=sys.stderr,
+)
+
+
+def fetch_translations(component_name: str, user_key: str = '', config: WeblateConfig = None):
+    weblate = Weblate(key=user_key, url=WEBLATE_URL, config=config)
+    component = Component(weblate, f'components/{PROJECT_NAME}/{component_name}/')
+    logging.info('Processing component %s...', component['name'])
+    translations = component.list()
+    source_language = component['source_language']['code']
+    output_dir = get_output_dir(component_name)
+    logging.info('Output dir: %s', output_dir)
+    for translation in translations:
+        # Skip source language and translation templates
+        language_name = translation['language']['name']
+        language_code = translation['language']['code']
+        if (language_code == source_language
+            or translation['translated_percent'] < MIN_TRANSLATED_PERCENT
+            or translation['is_template']):
+            logging.info('Skipping translation file for %s.', language_name)
+            continue
+
+        logging.info('Downloading translation file for %s...', language_name)
+        data = translation.download()
+        output_path = os.path.join(output_dir, f'{language_code}.po')
+        with open(output_path, 'bw') as output_file:
+            output_file.write(data)
+
+
+def get_output_dir(component_name: str) -> str:
+    path = os.path.join(os.path.dirname(__file__), '..', '..', 'po', component_name)
+    os.makedirs(path, exist_ok=True)
+    return path
+
+
+def load_config() -> WeblateConfig:
+    config_path = os.path.join(os.path.dirname(__file__), '..', '..', '.weblate.ini')
+    if os.path.exists:
+        config = WeblateConfig()
+        config.load(config_path)
+        return config
+    else:
+        return None
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        prog='pull-shared-translations',
+        description=(
+            'Fetches the translations for attributes and countries from '
+            'the MusicBrainz Server project on Weblate.'
+        ),
+        epilog=(
+            'Instead of passing the --key parameter the key can also be set in '
+            'a file .weblate.ini in the repositories root directory. See '
+            'po/README.md for details.'
+        ))
+    parser.add_argument('-k', '--key', help='Weblate user key')
+    args = parser.parse_args()
+
+    config = None
+    if not args.key:
+        config = load_config()
+        if not config:
+            parser.print_usage()
+            parser.error('No Weblate user key specified. See po/README.md for details.')
+        url, key = config.get_url_key()
+        if not key or url != WEBLATE_URL:
+            parser.print_usage()
+            parser.error('Invalid .weblate.ini. See po/README.md for details.')
+
+    for component_name in PROJECT_COMPONENTS:
+        fetch_translations(component_name, user_key=args.key, config=config)
+
+
+if __name__ == '__main__':
+    logging.debug("Starting...")
+    main()