Continuous integration/Workflow 2014
This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date. Draft from 2013. |
The Workflow v1 (2012) is nearing completion. Time to take what we've learned, look at how the world has evolved and what new features are on the horizon. Here's what I hope we'll be doing before the end of 2014.
On submission of any patchset to any project on Gerrit by anyone:
- Run its tests in a virtual machine
- Gerrit notifies Zuul which distributes the job to a pool openstack of vagrant instances based on a pre-built image (these are teared down after a job and replaced with fresh ones automatically).
- Which tasks to execute is specified in a declarative file inside the repository (e.g. a bit like
.travis.yml, it would specify which of a handful of VM templates to use, and what commands to execute. such asnpm install package.json & grunt test)
- At the very least every repository runs lint checks (php lint, jshint, csslint, python pep8, puppet lint/validate)
- For those opting in, also coding style checks (phpcs, jscs, python pep8)
- MediaWiki core and extensions:
- Install MediaWiki on local webserver (+ any extensions installed if relevant)
- Run PHPUnit tests (optimised using phpunit-parallel)
- Run QUnit tests
- Run grunt/qunit (phantomjs)
- Run grunt/karma (cross-browser qunit testing in saucelabs or browserstack)
- Run selenium integration tests in saucelabs
- Run pdiff-based job to detect any visual differences on common urls (not an assertion, purely informative to detect unintended visual side effects and to easily review intended visual differences).
- Assert catch-all MediaWiki/php/apache error logs are empty
- For operations/apache-config
- Install the vhosts on the local instance and run integration tests to verify things work properly. Especially with regards to redirects which are easy to screw up (e.g. a rewrite case falling through and redirecting dozens of domains incorrectly, query string encoding etc.)
- For operations/puppet
- puppet validate
Tasks:
- Install csslint
- install jsc (bug 54218)
- Set up jobs for it and start using it in core and selected extensions
- Merge separate phpunit jobs back into one and use phpunit-parallel (bug 48217)
- Track and assert error logs for a high-level catching of issues (bug 48002)
- Implement virtual machine infrastructure
- Set up one or more templates
- Design and implement the travis-yml-like set up