VisualEditor/Planning and testing

This is a directory of pages related to the planning and testing process of the VisualEditor extension. Some pages document deprecated aspects of the software, and are kept for historical purposes.

Planning

"Forward-look" documents:

Agendas, meeting agendas for the development team
Roadmap, plan for VisualEditor as of August 21, 2012
Phase 2, description of the second phase of the VisualEditor development
Todo list
Todo later list, list of tasks de-scoped for the June 2013 release of the VisualEditor
FAQs from the June 2012 release
Undo and redo, planning for these functions (historical)

Testing

Test, a general test page
- Template test, a test page for templates
- References test, a test page for references
Typing, testing for languages requiring input methods
Indic, testing for Indic language characters
Right-to-left, testing for languages using right-to-left character placement
Testing Non-Latin Characters Input and Behavior, for non-Latin characters

Weekly Test Plans

The Visual Editor QA team uses the page VisualEditor/Weekly Test Plans to track features being added in each weekly product release cycle, and capture essential notes on what sort of testing will be needed to verify each feature for production release.

QA tests

JavaScript tests

The VisualEditor JavaScript QUnit tests are in test subdirectories of some modules. They need to run in a browser. You can visit Special:JavaScriptTest/qunit to run all tests (which will take a long time to complete in your browser), or choose an individual module from the dropdown. Append ?filter=ve to the URL to run all modules matching "ve" which will run all VisualEditor tests plus some others ( try it on test2wiki). You can also run VisualEditor's QUnit tests from the commandline using the Grunt JavaScript task runner, which will start up a PhantomJS headless browser to run them.

Grunt can also run the JSHint code checker. See Gruntfile.js.

Browser tests

modules/ve-mw/test/browser is a set of browser tests that perform some VE actions.

You can run these tests yourself by installing the necessary software, see Quality Assurance/Browser testing/Running tests; the easiest setup is to run a MediaWiki-Vagrant virtual machine and enable the roles browsertests and visualeditor.

For the tests to work

the test username (MEDIAWIKI_USER) needs an account on the test wiki (MEDIAWIKI_URL)
the test username's Preferences > Editing "Enable VisualEditor" must be checked
The test wiki must be configured so that VE is enabled for the User namespace (in $wgUseVisualEditorNamespaces).

Selenium browser tests

https://github.com/inez/VE-tests/ has additional browser tests that test basics such as typing, breaking a paragraph into two paragraphs, bold, list, etc.

These test are written in Java and depend on Maven and Selenium; they load the VE demo web page in VisualEditor/demos/ve . To run them you must install

Java JDK
Maven
chromedriver from http://chromedriver.storage.googleapis.com/index.html (and a compatible version of Google Chrome/chromium).

You can use FirefoxDriver instead, or load VE at a different URL, by editing test/java/BaseTest.java

Continuous integration tests

When developers submit changes to VisualEditor, and later when other developers approve ("+2") them in gerrit, Jenkins runs a subset of VE tests: jshint, PHPcs, qunit tests.

last failed lint check

WMF QA regularly runs browser tests in multiple browser at https://wmf.ci.cloudbees.com, visiting pages on wikis, including beta labs (which is running master) and test2wiki (which is running the most current wmf branch). Click the [r-ve] tab to see all VisualEditor runs.

Dirty diff checking

There is a bot that runs VisualEditor, picks a random article, saves it, and checks for diffs arising from round-trip editing and Parsoid parsing. Source here, output here. Any dirty diff is a bug in either VisualEditor or Parsoid, even normalizations with no visible change (whitespace, quoting arguments)

TODO should improve dirtydiffbot to indicated whether it triggered VE's dirtywarning message, issue filed on github

Other

Fail, (historical)
Quality Assurance/VisualEditor browser regression test backlog