Continuous integration/Workflow
This article describes how we trigger tests on changes being submitted to a project as well as the tools integrated to fulfill the workflow.
Tools
editWe are using third party open source software, the most important ones are:
- Gerrit
- Hosts the WMF git repository and provides an interface to review the code. Developers send a patchset which is then enqueued in the review queue and is not actually merged until it has been approved. Whenever a patchset is sent or is approved, Gerrit generates an event we react on.
- Jenkins
- A job control system which can be used to run unit tests, generate tarballs or whatever we can think about. Each job is a set of instructions (such as running a shell script or reporting test result) and can accept parameters to finely tune the job execution.
- Zuul
- listens for Gerrit events, according to a workflow specification file, it can trigger Jenkins jobs and pass them various parameters (ex: the Gerrit change number).
- Jenkins job builder
- craft jobs for Jenkins based on a flat file specification, save us from having to click checkboxes in Jenkins graphical interface.
The tools heavily interact with each other. The following sequence diagram illustrate what happens when a user propose a patch:
Roles
editWe identify two user roles:
- Contributor
- Willing to modify a git repository hosted by the WMF, be it code, translation, documentation update... Contributors send their changes to Gerrit using git. Any contributor is allowed to review any change, comment on it and gives its feedback by voting the code-review label (-1 to +1).
- Gate keeper
- They approve changes which will in turn trigger unit tests and merge the change in the repository. They can also prevent a change from being sent by voting code-review label -2.
From patch to merge
editWhenever an action is conducted in Gerrit, Zuul receives an event. It is passed through a set of rules that might end up triggering an event for Jenkins. Most projects have, or will have, a rule on patchset creation that will trigger a run of basic checks (linting, trivial errors). Whenever the checks fails, Zuul will comment on Gerrit change and list the URL pointing to Jenkins jobs consoles. Moreover, Zuul is able to vote on Verified / Code-Review labels, when basic checks are successful, Zuul can flag the change as Verified which also mean the change is ready to be reviewed by a human being. Developers, testers or even a non coders will have a look at the code sent, test it and comment about possible issues.
Each person can vote under the Code-Review with a score varying from -2 (do not submit/merge) to +2 (change approved). Only projects owner and trusted people are able to actually approve a change. They have the power to decide how the software is evolving and guarantee the nothing bad will happen in production. Whenever a +2 vote is cast, unit tests are run. For MediaWiki core this includes attempting to install MediaWiki and running the full PHPUnit test suite. If the test suite fails, the change is verified and prevented from being submitted (Code-Review -2). If all tests are successful, change is ready to be merged.
Basically:
- Static analysis (linting and coding style) is done on patch submission.
- Execution of unit tests is run once changes are reviewed (Code-Review +2).
Overview:
