Selenium/Ruby/Setup instructions
This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date. See Selenium instead. |
Setup
editTo follow these instructions you should be comfortable at the command line; you should have a basic understanding of PHP, MediaWiki, and git; and you should be willing to do a little bit of hacking around to understand key Ruby things.
It is helpful to have a basic understanding of Cucumber, as well. Cucumber is the major ATDD technology in use for browser testing.
User
editCreate an account on beta labs if you don't already have one.
Git
editCreate a Gerrit account. Creation of a GitHub account is optional, but recommended.
On Mac install GitHub for Mac. Run the GitHub app and at the Configure Git step of the startup wizard click Install Command Line Tools button.
On Ubuntu install git with sudo apt-get install git
.
On Windows install GitHub for Windows. After the installation a Git Shell shortcut will appear on your desktop, and that will open a command line shell with Git configured.
Set up MediaWiki-Vagrant virtual machine
editTested on OS X 10.8.4 and Ubuntu 13.04 64-bit.
The easiest way to set up a MediaWiki server against which to run Cucumber tests is to use the MediaWiki-Vagrant virtual machine. Additionally, install the latest version of XQuartz on your Mac OS X host operating system. After setting it up, you'll be able to access a MediaWiki web server from localhost:8080 with a browser on your host operating system. Your host operating system is in effect connecting to a web server inside the virtual machine.
Set up your computer
editYou will need a few things set up in order to run the browser tests for MediaWiki using your host operating system.
Browser
editInstall Firefox (if you do not already have it installed). Firefox is the most convenient browser when using the browser automation tool Selenium/Webdriver.
Ruby
editTo run the tests you will need Ruby. The easiest way to install Ruby on Linux/Unix/Mac is RVM and on Windows RubyInstaller.
Bundler
Bundler is required for proper use of Ruby gems, including the mediawiki-selenium gem. After having installed RVM on your host operating system, at the command line in your shell, cd down into vagrant/mediawiki. You will be told at that point to use rvm to install a particular version of Ruby:
ruby-2.1.2 is not installed. To install do: 'rvm install ruby-2.1.2'
Run that command as told. This will download and set up Ruby at the appropriate version so that it can be used when in this directory. rvm nicely lets you run multiple versions of Ruby. rvm reports the required version of Ruby according to the setting in vagrant/mediawiki/Gemfile
. The Gemfile shows the required Ruby version as well as all of the shared libraries, called Ruby gems.
Now that you've obtained the version of Ruby needed for use while in this directory, install Bundler:
vagrant/tests/browser $ gem update --system; gem install bundler
Now that you've setup Bundler, while still in vagrant/mediawiki, run the following command. This command will get the other gems needed for executing browser tests, such as mediawiki_api, mediawiki_selenium, and rubocop.
vagrant/tests/browser $ bundle install
Code
editBrowser tests make use of the Ruby gem called page-object, which has become the standard implementation of the Page Object design pattern in Ruby.
The page-object Ruby gem makes use of the watir-webdriver API, which provides many sophisticated features not available in the raw selenium-webdriver API.
Now that you have things installed
editEnvironment variables
editThe browser tests make use several environment variables, which in a vagrant instance should appear as:
MEDIAWIKI_URL=http://127.0.0.1:8080/wiki/ MEDIAWIKI_API_URL=http://127.0.0.1:8080/w/api.php MEDIAWIKI_USER=Selenium_user MEDIAWIKI_PASSWORD=vagrant BROWSER=firefox
and if you are targeting beta labs should appear as
export MEDIAWIKI_URL=http://en.wikipedia.beta.wmflabs.org/wiki/ export MEDIAWIKI_API_URL=http://en.wikipedia.beta.wmflabs.org/w/api.php export MEDIAWIKI_USER=My_Beta_Labs_User export MEDIAWIKI_PASSWORD=My_Beta_Labs_Password export BROWSER=firefox
Running your tests
editA basic vagrant instance will have Mediawiki installed. Mediawiki follows the conventions of every mediawiki extension and keeps browser tests in the /tests/browser directory. In your vagrant instance you can cd to vagrant/mediawiki/tests/browser and run the following command
vagrant/mediawiki/tests/browser $ bundle exec cucumber features/main_page_links.feature
in order to run the main_page_links test.
When you run this command, the test framework will spawn a Firefox browser and examine the links on the Main Page of the target wiki.
If you wanted to run this test against a different domain name, as the mediawiki-selenium documentation recommends, you would need to change the MEDIAWIKI_URL
environment variable before running the cucumber test. How one changes environment variables varies from OS to OS and from shell to shell, but in bash, manipulating environment variables is done with export():
$ export MEDIAWIKI_URL=http://localhost:8080/wiki/
NOTE: we strongly recommend against running the browser test regression suite against any production Wikipedia targets. Many of the tests use the Mediawiki API to set up particular target pages with particular target data, and these can be construed as spam or garbage by wiki maintainers.
Setup for testing with other browsers
editIt is possible to run tests with other browsers (e.g., Chrome or the headless browser PhantomJS) by exporting the BROWSER environment variable.
- Chrome
- Install Chrome
- Install ChromeDriver
- PhantomJS
- Install PhantomJS
Further reading
editYou should have things working with the setup above. But if not, please visit the IRC channel #wikimedia-qa on freenode. Also, the QA mail list welcomes subscribers with questions.
Some of the best places to see up-to-date working examples for Cucumber are in a number of MediaWiki extensions. For example, the MobileFrontend and Flow extensions have significant tests in them. You will always find the browser tests in the tests/browser
directory of an extension. Each extension has its particular Gemfile, so be aware that you may need to bundle update
or bundle install
as you move from extension to extension running tests. RVM will keep track of all those changes automatically.
Take a look at Writing tests for a detailed discussion of the contents of the features
, features/step_definitions
and features/support
directories inside of the browser/tests
folder . Feel free to follow the examples from the extensions when creating your own tests.
More detailed documentation on various parts of the framework are available: