Content translation/Exploratory Testing Tools

This page contains information and tools that can be used as a springboard for anyone engaged in exploratory testing of the MediaWiki Content Translation feature.

About Exploratory Testing

Link Descripton
Exploratory Testing This Wikipedia article gives you a good overview of Exploratory Testing, including a smattering of history and many useful links for investigating the subject further.
Four Schools of Software Testing Brett Pettichord compares Context-Driven Testing (which focuses on exploratory testing) to three other schools of testing: Analytical, Factory, and Quality Assurance.
You Are Not Done Yet Context-Driven or Exploratory testing starts with the acknowledgment that it is impossible to test everything about the software. If you don't believe that, read this article by Michael Hunter - it's an eye opener!
Context-Driven Software Testing Just go watch this. It's James Bach, one of the founders of Context-Driven Software Testing, describing what it's all about in under 5 minutes. Spoiler alert: He recommends three books, none of which have anything to do with testing -- or do they?
Why Exploratory Testing Matters In this short video, Ingo Philipp of Tricentis explains the importance of exploratory testing in ensuring complete test coverage. "We regard something as properly tested when it has been checked by efficient, formal test automation based on solid test case design AND when it is checked by the richness of the intellect of human beings."
Deployments The page tracks the planned deployments to different groups of Wikimedia projects (Wikipedia, Wiktionary, Wikiversity, Wikimedia Commons, etc.).
What is Exploratory Testing? Adding ET to your testing strategy for better coverage This resource explains provides an overview of Exploratory and Session-Based Testing, and discusses the importance of using Exploratory Testing in order to gain better QA coverage.
Please feel free to add more links!

About CX2

Link Descripton
Content Translation Main article on MediaWiki.
Talk: Content Translation This is where actual users are reporting their problems – with answers from staff! Very educational. (Make sure filter is set to Newest Topics!)
Help: Extension: Content Translation Help doc on MediaWiki. Feel free to add to it as you learn more about the product.
Content Translation / Machine Translation Describes the machine translation engines used in CX and provides links to information about the language pairs that each supports.
ISO 2-letter language codes You'll be speaking in 2-letter language codes before long! They're a great shortcut, whether you're using them as input in CX2 language search fields or in a Phabricator bug report to describe your testing.

CX2 Test Suites and Checklists


You can download these tools to help guide your initial testing. (They are particularly helpful when you are first learning Content Translation and want an overview of its capabilities.)

Test Suite file Notes
CX2 regression testing:

test cases

02-21-19 CX2 regression test

Content Translation test cases

Takes about 1.5 hours.
CX2 regression testing: checklist CX2-regression-testing-checklist v2.pdf A map of the different parts of CX2 to help guide your testing. More flexible than the regression testing spreadsheet; the sections are not interdependent, so you can pick and choose your area of focus.

Test Environments

Link Descripton A constantly-updated master. Bug fixes from developers’ local environments are merged here. A test copy of the most current PRODUCTION code. (The code is deployed from here to the wikis.) , Early deployment sites; the code is tested here before being deployed to the full set of wikis. (Be careful when testing! These are real wikis!)

Testing Tips


These are good to keep in mind whether you're testing a known bug, documenting a new one, or verifying a fix.

Tips... Details
Test against all machine translation engines CX2 supports five different machine translation engines: Apertium, Yandex, Google Translate, LingoCloud, and Youdao. The subset you see in the Translate page depends on the language pair you're using. Test against all engines that are available. If you discover an issue, test against "Copy Source Material" as well (as a control).
Test the full spectrum of languages For full coverage, test using the following language pairs:
  • en to Hebrew (or Farsi, Arabic - any of the right-to-left languages)
  • en to Chinese (or Korean, Japanese - the CJK language group)
  • en to Gujarati (or another scripted language)
Note the CX version you're using When you're posting test results, it's helpful to include the CX version. To find the version, navigate to page Special:Version in the wiki test environment.
Test in different browsers Remember to switch to different browsers from time to time. It's especially helpful to run through a complete regression suite or checklist using a browser that you and your coworkers tend to neglect! (IE, anybody? Firefox? Opera?)

Here is Wikipedia's page about browser compatibility.

* *

Example Pages - Wikitext Elements


The Content Translation test sites draw articles directly from Wikipedia and many other wikis, so there is always a wealth of source material for you to use in your testing. However, if you are focused on a particular scenario -- such as how Content Translation handles complex tables or particular templates -- it can be difficult to locate a page with the right elements. This list can help you find what you need. (And feel free to add any interesting examples you've come across yourself!)


Page section Notes
Singer 2018 All sections Many examples of simple and complex tables. Merged cells. Background colors. There is one example of embedded templates (see below).
Singer 2018 Results Table with embedded templates. (Template:Flagicon)
List of English football champions Football League First Division Error: Automatic Translation Failed. (Table too big to translate. See Phabricator T202283.)


Page section Notes
Demain nous appartient Top of page Skipped template. Generates the error message: Template unavailable in the destination wiki. Use French-to-English.


Page section Notes

Other templates

Page section Notes
Singer 2018 Results Template:Flagicon (embedded in table). Flagicon should display flag thumbnail image that links to article about associated country or organization. Template documentation is here.
Page section Notes
Singer 2018 Results Missing link. In English-to-Espanol translation, rows 2, 3, 10 & 11 (Chenyu, Feng, KZ & Chong) are "missing links" -- i.e., the target page is missing in the es wiki. Double click to open "Missing Link" dialog.

Math Formulas

Page section Notes
Free entropy Dependence of the potentials on the natural variables Lots of block-level <math> elements. There are also some formulas embedded in tables, as well as a few inline math elements (although these are mainly individual symbols rather than formulas).
Table of thermodynamic equations All sections Math formulas embedded in tables.

Musical Notation

Page section Notes
ABC Notation Example Single example of score element. (Note: though it looks like there are two scores titled "Legacy Jig," only the first is a <score> element. The second is just an image.
Jazz Many sections Many <score> elements scattered throughout the article.
Symphony No. 9 (Beethoven) Form Four examples of block-level <score> elements.


Page section Notes