Extension:Translate/Mass migration tools/Project updates

Project title: Tools for mass migration of legacy translated wiki content

Pre-Community Bonding Period

  • Read the existing documentation related to Translate extension
  • Prepared pages for translation and imported them manually
  • Went through the coding conventions for the language selected - JavaScript
  • Set up the coding environment - editor, gerrit ssh keys, etc. Ready to submit patches

Community Bonding Period Report


Goals for first half


As mentioned in the project schedule, the first half of the project aims to accomplish the following goals -

  • Coding of the confirmation and publishing screens related to Step 2 (importing translations), with basic initial import of old text is to be done by May 28
  • Once the initial import is done, enhancement bugs would be filed and fixed in the next few days until June 7
  • Desiging and coding of the algorithm for step 1 (preparing page for translation) would be done by June 23

Communication plan

  • IRC has been used so far for daily communication and mails when the mentors were not available
  • It has been decided to have daily meetings except on Sunday's at #mediawiki-i18n from 13.45 UTC to 14.00 UTC

Lessons learnt

  • Not to get stuck on a particular task and keep things simple for the first version
  • Strict use of Gerrit for code review
  • Second part of the project is more important and shall be done first

Work done


April 22 to May 5

  • Documented the requirements for the project
  • Designed an algorithm for aligning the translations
  • Wrote a script to fetch the source translations and align the section headers

May 6 to May 18

  • As per the meetings held, it has been decided that step 2 be done before step 1
  • The second step would be seen as a special page - Special:PageMigration. A skeleton of the same was set up on local wiki
  • Submitted patch sets for the first version of the special page. As of now, it -
  1. Fetches and displays the translation units created by Translate for the given page
  2. Imports the old translations for given language on paragraph level and displays them
  3. Feature can be now conditionally enabled by setting $wgTranslatePageMigration to true

Weekly Reports


Week 1: May 19 to May 25

  • Added features to edit, delete and swap the imported translations - Patch
  • i18n of messages used so far - Patch

Week 2: May 26 to June 2

  • Added feature for manual splitting of units by giving an option to add an empty unit after the current one - Patch
  • Created an instance on wmflabs and the tool can now be tested here.
  • Restricted access to the Special page to translation admins only on Wikimedia wikis - Patch
  • Fixed the IDs and class names adapting the BEM approach and revamped the interface using the MediaWiki grid layout - Patch

Week 3: June 3 to June 9

  • Wrote QUnit tests for the various API calls involved - Patch
  • Handled possible error cases like non-existence of a page, absence of old translations, etc - Patch
  • Added hover text for image icons to convey their meaning - Patch
  • Fixed the overall code architecture - Patch
  • Wrote a module for simplistic alignment based on h2 headers - Patch

Week 4: June 10 to June 16

  • Handled whitespace in input at Special:PageMigration by trimming them - Patch
  • Started with Step 1 of the project - preparing the page for translation
  • Wrote a script under Translate itself which does the basic work of preparing the page for translation. It includes adding ‎<translate> tags at the start and end of the page, adding newlines before and after section headers, handling of files, internal links as per the markup examples and saving of the page with the modified text - Patch
  • Handled internal links to sections on same page by linking them to language-independent anchor - Patch

Week 5: June 17 to June 23

  • Handled templates by keeping them as a part of the page template - Patch
  • Continued to work on the internal links to sections on same page patch

Week 6: June 24 to June 30

  • New icon images were introduced at Special:PageMigration - Patch
  • A new special page Special:PagePreparation was introduced which serves as the confirmation screen for step 1 - Patch

Week 7: July 1 to July 7


No work done due to relocation to Hyderabad

Week 8: July 8 to July 14

  • Continued to work on Special:PagePreparation. Added feature to show diff and save the page on user confirmation

Week 9: July 15 to July 21

  • Addressed code review for the page preparation patch
  • Addressed code review by Nikerabbit and Siebrand for Special:PagePreparation patch
  • Addressed code review for the categories related patch

Week 10: July 22 to July 28

  • Got the page preparation patch merged by removing the file handling part and addressing other issues.
  • Continue addressing CR for categories patch
  • Addressed code review for internal links to sections on same page patch
  • Added 'translation' as magic word under the Translate extension - Patch
  • Files (Images) were handled as a part of page preparation. The regex was built by first fetching the aliases of 'File' namespace. - Patch
  • i18n, L10n issues for Special:PagePreparation were addressed.
  • Punctuation added for error messages used at Special:PageMigration - Patch

Week 11: July 29 to August 4

  • Addressed code review for internal links to sections on same page patch
  • Got Special:PagePreparation patch merged by addressing the remaining issues
  • Addressed code review for translation as magic word patch
  • Changed 'translation' as magic word to '#translation' as parser function
  • Addressed code review for the new files with namespace aliases patch

Week 12: August 5 to August 11

  • Addressed code review for '#translation' as parser function patch
  • Addded the mw-ui-input class for input field at Special:PagePreparation - Patch
  • New icon images for Special:PageMigration patch was merged
  • Changed api call changed from .get to .post for the getDiff() function at Special:PagePreparation - Patch
  • Skipped source units on pressing tab key at Special:PageMigration - Patch
  • Addressed remaining issues for files patch and got it merged
  • Splitting of units was done after the h2 alignment - Patch
  • Amended the #translation as parser function patch to handle the case of source page or invalid language code
  • Submitted a patch to show messages after 'Import' and 'Save' at Special:PageMigration - Patch
  • Made input fields at both the special pages bigger and usable - Patch
  • Normalized page title used at Special:PageMigration - Patch
  • Submitted a fix to unwanted pixel for switch images - Patch

Week 13: August 12 to August 18

  • Wrote parser tests for the #translation as parser function patch
  • Addressed code review for page title normalized patch and got it merged
  • Addressed code review for input fields patch
  • Discussions and testing of split headers after alignment patch
  • Rebased and submitted to categories patch which uses #translation as parser function and namespace aliases from the files patch
  • A cancel button was added to Special:PagePreparation to abort the process - Patch
  • Relative links were ignored while adding the 'Special:MyLanguage/' prefix - Patch
  • Edit summary was changed to appear in content language in case of both the special pages - Patch
  • Added new feature to split headers from other wikitext on unit level - Patch
  • Did some testing of both the special pages on mediawiki.org
  • Wrote help pages for Special:PagePreparation and Special:PageMigration