Selenium/How-to/Run tests using selenium-daily Jenkins job

For this example, we will target English Wikipedia at the Beta Cluster. The job runs once every day, but you can also start them manually.

Advantages

edit
  • You do not have to install anything on your machine.
  • You are probably already familiar with Jenkins.
  • While the tests are running, you can continue using your machine as usual, because everything is happening in Jenkins.
  • Debugging failed tests will be reasonably easy, since you will see logs and screenshots of the test run.
  • It will be reasonable fast. It takes 1-5 minutes for a Jenkins job to run. (As of October 2019.)

Disadvantages

edit
  • You will need internet connection while running the tests.
  • You might not be familiar with Jenkins, so you will have to learn how to use yet another tool.

Check if the Jenkins job is configured to run for a repository

edit
  • In mediawiki.yaml look for mediawiki-wdio-selenium project. (As of October 2019.)
- project:
    name: mediawiki-wdio-selenium
    project: MediaWiki
    recipients: betacluster-alerts@lists.wikimedia.org cparle@wikimedia.org etonkovidova@wikimedia.org qa-alerts@lists.wikimedia.org zfilipin@wikimedia.org
    repository: mediawiki/core
    jobs:
        - 'selenium-daily-{sitename}-{project}':
            sitename:
                - beta:
                    mw_server: 'https://en.wikipedia.beta.wmflabs.org'
                - betacommons:
                    mw_server: 'https://commons.wikimedia.beta.wmflabs.org'
- project:
    name: wdio-selenium-daily
    project:
        - CirrusSearch:
            recipients: discovery-alerts@lists.wikimedia.org qa-alerts@lists.wikimedia.org # @dcausse @EBernhardson
            repository: mediawiki/extensions/CirrusSearch
            site: en.wikipedia
...

Run the Selenium tests

edit
  • Log in to Jenkins.
  • Go to a job. (Example: selenium-daily-beta-MediaWiki.)
  • Click Build with Parameters.
  • Click Build.
  • In Build History a new job will be blinking at the top of the list, for example #106.
  • Click job number, in this case #106.
  • Click Console Output.
  • If the job has been running for a while, there will be Full Log link at the top of the page. You can click it to load the entire console output.
...
+ npm run-script selenium-daily
...
Spec Files:	 3 passed, 3 skipped, 6 total (100% completed) in 00:01:20
...