Reading/Browser tests/Setting up a browser test bot

This documents the steps required to get a bot running browser tests and posting test results to Gerrit. There is more information on wikitech-l.


  • Acquire a Gerrit account e.g. Barrythebrowsertestbot
  • Acquire a labs project
  • Create a labs instance with mediawiki
  • ssh into your instance e.g. ssh gather-browser-tests.eqiad.wmflabs
  • Install any extensions you want to test into the mediawiki install. It's likely you'll also need to install the vector skin.
  • Install phantomjs
  • Install Ruby (apt-get install ruby-dev && gem install bundler)
  • Install and configure arcanist for pasting failed test results to Phabricator (see below)
  • Install git review
  • Get from BarryTheBrowserTestBot
  • Register a public key with Gerrit and your labs instance.
  • Create a mediawiki account on your machine for your tests e.g. "User:Mr Selenium"
  • Update ~/.profile with information specific to your machine.
  • Navigate to the extension you want to run browser tests for and set up the browser test environment. Ensure the tests are passing green. Note that some extensions e.g. MobileFrontend require additional setup steps.
  • Run a trial run of the BarryTheBrowserTestBot without the --review flag
  • Set up a background task script and watch it review away.

Note: As of June 2015 this reviews any patches which do not have the browser bot attached to them. You can trigger a new review by removing the browser bot from the patchset.

Install and configure arcanist


For full instructions see

* cd /opt
* git clone
* cd /opt/arcanist/externals/includes/
* git clone
  • Edit /etc/profile and add the following line: PATH="$PATH:/opt/:/opt/arcanist/bin/" before export PATH
  • source /etc/profile


.profile sample code

BROWSER# Make sure you add phantomjs and Barry-the-Browser-Test-Bot to your path
# setup the variables needed by the browser tests
export MW_SERVER=
export MW_SCRIPT_PATH=/w
export MEDIAWIKI_USER=Mr_Selenium
export MEDIAWIKI_PASSWORD=password
# headless testing for the win
export BROWSER=phantomjs

Background task sample code

while :
  # Do Gather - trigger a review on the result. --project corresponds to the Gerrit project you want to test.
  # --core, --test --dependencies correspond to absolute directories on your machine. --core and --dependencies will be switched to master and updated before launching the browser tests.
  ./ --noupdates 1 --review 1 --paste 1 --project mediawiki/extensions/Gather --core /srv/mediawiki/ --test /srv/mediawiki/extensions/Gather/ --dependencies /srv/mediawiki/extensions/MobileFrontend
  # Do MobileFrontend but limit browser tests to those that are tagged @smoke
  ./ --review 1 --paste 1 --project mediawiki/extensions/MobileFrontend --core /srv/mediawiki/ --test /srv/mediawiki/extensions/MobileFrontend/ --tag smoke
  # sleep for 30 minutes
  sleep 1800

Run this in the background to start reviewing!:

./ &&

Current setup at gather-browser-tests.eqiad.wmflabs

  • The username used for pasting test results to is 'Barry-the-Browser-Test-Bot'. To get the password (which you probably won't need) run: cat /opt/barry_phab.txt
  • Arcanist and barry are cloned at /opt/arcanist and /opt/barry
  • The run script is located at /opt/ (which should probably be replaced by a cron job)