Selenium/How-to/Run tests targeting Beta cluster using Sauce Labs

For this example, we will target English Wikipedia at the Beta Cluster using Sauce Labs.

Target beta cluster using Sauce Labs.png

AdvantagesEdit

  • You do not have to install or learn how to use MediaWiki-Vagrant.
  • While the tests are running, you can continue using your machine as usual, because everything is happening inside the virtual machine at Sauce Labs.
  • Debugging failed tests will be easier, since you will see the browser, logs, screenshots and video of the test run.
  • You do not have to have Chrome installed.
  • You can run tests using any operating system and browser combination that Sauce Labs supports.

DisadvantagesEdit

  • The tests will be slower since the target machine is not local. MediaWiki core Selenium test run takes about 12 minutes on my machine. A lot of tests fail. Subset of passing tests takes about 3 minutes. (As of September 2019.)
  • You will have to install Selenium and it's dependencies on your machine.
  • You will need internet connection while running the tests.

Clone MediaWiki coreEdit

There are several ways to clone the MediaWiki core repository (anonymous http, http, ssh). We will use anonymous http since it is the simplest.

$ git clone https://gerrit.wikimedia.org/r/mediawiki/core mediawiki

Go to mediawiki folderEdit

$ cd mediawiki

Install dependenciesEdit

Depending on your operating system, installing dependencies will be different.

Node.jsEdit

  • Install Node.js.
  • Install required npm packages.
$ npm ci

Sauce LabsEdit

Create Sauce Labs account if you do not have one.

Environment variablesEdit

Ensure the following environment variables are set. Either run this manually from the terminal or, to make it easier for next time, set them from shell configuration file (~/.bash_profile, ~/.zshrc...) which means the variables are always available from any terminal session:

export DISPLAY=1
export MEDIAWIKI_PASSWORD=mediawiki_password
export MEDIAWIKI_USER=mediawiki_user
export MW_SERVER=https://en.wikipedia.beta.wmflabs.org
export SAUCE_ACCESS_KEY=sauce_access_key
export SAUCE_USERNAME=sauce_username

Of course, replace mediawiki_user and mediawiki_password with your user and password. sauce_username is your Sauce Labs username. sauce_access_key is Access Key from Sauce Labs My Account page (https://app.saucelabs.com/users/sauce_username as of September 2019).

Run the Selenium testsEdit

$ npm run selenium-test

Please note that a lot of tests will fail. Tests are written for targeting MediaWiki on a Jenkins slave machine and MediaWiki-Vagrant virtual machine.

To run a subset of passing tests:

$ npm run selenium-daily