[UI Translation] Migrate to another tool, Pootle is unmaintained

[nielx]

Currently we are using Pootle (version 2.7). The project seems to be currently unmaintained, so at some point (post R1?) it may be a good idea to migrate to something else.

Wishlist:

• Online translation
• Offline translation
• Export of the result - perhaps even user friendly so that it can be tested on the fly

[nielx]

Configuration changes:

• In the DATABASES string the engine should be set to 'django.db.backends.postgresql'
• Add the POOTLE_FS_WORKING_PATH setting
• Add the POOTLE_CANONICAL_URL setting
• Change the POOTLE_MARKUP_FILTER setting

[nielx]

Error running migrations

Running migrations:
  Applying pootle_store.0027_unit_created_by_squashed_0055_fill_unit_source_data...Traceback (most recent call last):
  File "env/bin/pootle", line 11, in <module>
    sys.exit(main())
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/pootle/runner.py", line 357, in main
    run_app(project='pootle', django_settings_module='pootle.settings')
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/pootle/runner.py", line 338, in run_app
    management.execute_from_command_line(command)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 96, in database_forwards
    schema_editor.create_model(model)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 295, in create_model
    self.execute(sql, params or None)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 112, in execute
    cursor.execute(sql, params)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
django.db.utils.ProgrammingError: relation "pootle_store_suggestion_state" already exists

[nielx]

Next database issue

  Applying pootle_statistics.0011_cleanup_submissions_squashed_0023_remove_scorelog...Traceback (most recent call last):
  File "env/bin/pootle", line 11, in <module>
    sys.exit(main())
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/pootle/runner.py", line 357, in main
    run_app(project='pootle', django_settings_module='pootle.settings')
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/pootle/runner.py", line 338, in run_app
    management.execute_from_command_line(command)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/migrations/operations/fields.py", line 147, in database_forwards
    schema_editor.remove_field(from_model, from_model._meta.get_field(self.name))
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 445, in remove_field
    self.execute(self._delete_constraint_sql(self.sql_delete_fk, model, fk_name))
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 112, in execute
    cursor.execute(sql, params)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/srv/pootle-staging/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: cannot ALTER TABLE "pootle_app_submission" because it has pending trigger events

[nielx]

The haiku-sysadmin user no longer exists, changing to haiku-web.

[pulkomandy]

Pootle seems a bit unmaintained nowadays. Today at FOSDEM I was approached by someone from Weblate who could apparently help us migrate to their platform. Or we could consider KapiX's tool as well.

[nielx]

Indeed I have noticed. It seems dead.

I have modified the ticket.

[diver]

Somebody on Telegram suggested Transifex and Weblate.

[KapiX]

Replying to pulkomandy:

Or we could consider KapiX's tool as well.

It would be nice to have one system for everything, but I wouldn't say Polyglot currently solves "not maintained" problem we have with Pootle.

It also wasn't designed for relatively big projects like Haiku (UI would need to be redesigned) and wasn't tested with so many catkeys (I suppose Haiku is bigger in that regard than all projects currently there combined). A lot of features are missing too (although I want to tackle that soon).

In summary: yes, just not now. Go ahead with other system for now.

[nielx]

There are some good suggestions on here, but there is no need to actually start moving over right now. The current system is fragile but works well enough. I would say we reopen the conversation after a major release has been done.

[nielx]

Unassigning from me (back to the general todo list)

[jt15s]

Hmm, since Beta3 is out might be time to start seriously considering a migration? I think we should migrate to Weblate, since A) they offered to help us migrate and B) it's a solid and trusted platform.

[nielx]

Replying to jt15s:

Hmm, since Beta3 is out might be time to start seriously considering a migration? I think we should migrate to Weblate, since A) they offered to help us migrate and B) it's a solid and trusted platform.

In the description of this ticket I put what I think are the requirements (online translation, offline translation, downloads for on the fly). I wonder to what extend this tool will fulfill those requirements.

Also note that this is not a priority for me before R1, but feel free to start doing some of the ground work to engage the translator community to find out what their requirements are.

[jt15s]

Replying to nielx:

Replying to jt15s:

Hmm, since Beta3 is out might be time to start seriously considering a migration? I think we should migrate to Weblate, since A) they offered to help us migrate and B) it's a solid and trusted platform.

In the description of this ticket I put what I think are the requirements (online translation, offline translation, downloads for on the fly). I wonder to what extend this tool will fulfill those requirements.

Also note that this is not a priority for me before R1, but feel free to start doing some of the ground work to engage the translator community to find out what their requirements are.

I had a look at those requirements, and researched whether Weblate fulfills these or not:

• Online translation
• Offline translation
• Export of the result - perhaps even user friendly so that it can be tested on the fly

Weblate is an online translation tool and supports exporting and importing translations too. Translation files can be downloaded for offline editing: ​https://docs.weblate.org/en/latest/user/files.html

There's a full list of features provided by Weblate here: ​https://weblate.org/en/features/

[pulkomandy]

The main thing we need is export in our custom native format for catkeys files. Whichever tool we use, this will have to be added to it, since only Pootle (we added it there when we needed it) and KapiX's translation tool support this format.

As mentionned in my earlier post, I had briefly discussed this with someone from weblate at fosdem (apparently they had researched projects still using pootle before the event), and that person seemed ok with it. But that does not mean the project as a whole will agree, so we'll have to check that.

Additional requirement:

• ability to import/expomt haiku custom catkeys files
• ability to extract credits for each translation so we can put those in AboutSystem

[pulkomandy]

Adding this here, the Korean translation in Pootle seems to behave strangely:

​https://i18n.haiku-os.org/pootle/ko/haiku/translate/#filter=checks&category=critical&unit=124777

All \ characters are replaced with what looks like a strikeout W for me. I can't type a proper \ into the input field.

Not sure if that's just my web browser acting strange, are other people seeing that too? Is the exported file still correct?

[humdinger]

Works here under Haiku with hrev56563's Web+ and Falkon browser.

[jmairboeck]

Replying to pulkomandy:

All \ characters are replaced with what looks like a strikeout W for me. I can't type a proper \ into the input field.

This may be an encoding or a font problem (for historical reasons): ​http://archives.miloush.net/michkap/archive/2005/09/17/469941.html