Integrate Crowdin To RetroArch

RetroArch project uses *.h files to store translation strings. Here aren’t any tools to make the translation process easier. When source strings changed, you have to manually review the changes, locate and update translation strings. It is a hard work. As a big fan of RetroArch, I was thinking if it can be improved with modern i18n platforms.

Why crowdin?

1. Self-hosting solution is too expensive.
2. Crowdin has better editor than Weblate/Transifex.
3. Crowdin provides very good email support.

But *.h files are not supported…

Let’s convert *.h files to *.json files. (JSON is well supported by Python)

I have two Python scripts:

• h2json.py
• json2h.py

You can see they do two-way conversion.

Isn’t Bash script enough?

Bash scripts aren’t supported by Windows. Many Unix utilities are not existing in different Linux distros and macOS.

Python (3) is more platform independent. And it has built-in JSON support.

Crowdin project configuration

The only thing to notice is the language mapping.

First, I tried to use language_mapping in crowdin.yml file as below. But unluckily it doesn’t work.

"files":
  [
    {
      "source": "/msg_hash_us.json",
      "translation": "/msg_hash_%locale%.json",
      "translation_replace": { "_us": "" },
      "languages_mapping": { "zh-CN": "zhs", "zh-TW": "zht" },
    },
  ]

The only working solution is to go to Crowdin website, open project settings and add “Language Mapping” in first tab page. (The button is not very visible)

Glue up the whole workflow

1. git pull
2. Convert source (en) *.h to *.json
3. Upload source (en) *.json to Crowdin
4. Download translation *.json
5. Convert translation *.json to *.h
6. git commit and push

The only difficulty is that Crowdin CLI installation on Windows isn’t easy. So in the Python script, I download the crowdin-cli.jar into the project directory and execute it by Java.

Check the full workflow script.

Initial upload

1. Convert all *.h to *.json
2. Upload msg_hash_us.json as source
3. Upload other *.json as translations
4. Run the workflow script and compare the changes

Because the translation is managed as key-value pairs, some source string changes are not reflected in translation strings. They need to be reviewed in future.

In this process, I found some issues of my convert scripts:

• Strings shouldn’t be trimmed
• Replace should start from the right side
• It doesn’t escape " correctly
• It doesn’t support multi-line strings

Then I keep improving the scripts and repeat above process. The final result should be: you don’t lose existing translations.

README and PR

For developers and translators, Crowdin might be a new thing. I write a README to introduce how to translate on Crowdin and how to run the synchronization script on Linux/macOS/Windows.

Finally, make a PR and explain how it can improve the i18n workflow.