Selenium/Ruby/Target beta cluster using Sauce Labs

For this example, we will target English Wikipedia at the Beta Cluster using Sauce Labs. You will need internet access.

Advantages

You do not have to install or learn how to use MediaWiki-Vagrant.
While the tests are running, you can continue using your machine as usual, because everything is happening inside the virtual machine at Sauce Labs.
Debugging failed tests will be easier, since you will see the browser, logs, screenshots and video of the test run.
You do not have to have Chrome installed.
You can run tests using any operating system and browser combination that Sauce Labs supports.

Disadvantages

You will need internet connection while running the tests.
The tests will be slower since the target machine is not local.
You will have to install Selenium and it's dependencies on your machine.
It will also be slow.
- MediaWiki core Selenium test run takes about 3-4 minutes on my machine using local Firefox.
- The same tests take 12-13 minutes using Sauce Labs.
- It takes about 6-7 minutes for selenium-Core Jenkins job to run.

Clone MediaWiki core

There are several ways to clone the MediaWiki core repository (anonymous http, http, ssh). We will use anonymous http since it is the simplest.

$ git clone https://gerrit.wikimedia.org/r/mediawiki/core mediawiki
Cloning into 'core'...
...
Checking out files: 100% (6426/6426), done.

Go to `mediawiki` folder

$ cd mediawiki

Install dependencies

Depending on your operating system, installing dependencies will be different.

Ruby

Installing Ruby might be easy, or harder.

On Linux, please use appropriate package manager.
On Windows, please use RubyInstaller.
It should already installed on Mac machines.
Ruby 2.0 or newer is required.
Third-party tools like rbenv or RVM are not required.

After installation, check if everything is all right.

$ ruby -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin15]

Install required Ruby packages (gems).

$ gem install bundler --no-rdoc --no-ri
Fetching: bundler-1.13.6.gem (100%)
Successfully installed bundler-1.13.6
1 gem installed

$ bundle install
...
Use `bundle show [gemname]` to see where a bundled gem is installed.

Create an account at the beta cluster

You can skip this step if you already have an account there. If not, create an account at English Wikipedia at the Beta Cluster.

Run the Selenium tests

Please replace values of MEDIAWIKI_USER, MEDIAWIKI_PASSWORD, SAUCE_ONDEMAND_USERNAME and SAUCE_ONDEMAND_ACCESS_KEY.

$ MEDIAWIKI_ENVIRONMENT=beta MEDIAWIKI_USER=not-the-real-one MEDIAWIKI_PASSWORD=not-the-real-one SAUCE_ONDEMAND_USERNAME=not-the-real-one SAUCE_ONDEMAND_ACCESS_KEY=not-the-real-one bundle exec rake selenium
...
/usr/local/Cellar/ruby/2.3.1/bin/ruby -S bundle exec cucumber tests/browser --tags @chrome
...
20 scenarios (20 passed)
103 steps (103 passed)
12m27.113s