Continuous integration/Codehealth Pipeline

The codehealth pipeline is an initiative of Code Health Group/projects/Code Health Metrics. Currently it is enabled for a handful of MediaWiki extensions, while we collect feedback from participating projects.

tl;dr

Patches (pre-merge and post-merge) are sent to a service which analyzes the code for various health checks, then that service gives a score, and that score is reported to Gerrit. A list of repositories that have been integrated into the codehealth pipeline can be found in this Phabricator Ticket

How it works

The code health reports are generated by checking out the code for a patch (or if it's postmerge, for the master branch), then running the sonar-scanner application.

That application reads from a .sonar-project.properties file (more on that later) which tells Sonar which directories have source code, which directories have test files, which directories/files should be excluded, what type of project (PHP, Java, and so on) it is, etc.

The sonar-scanner application sends all of those files to a remote server.^[1] It will also send over any code coverage reports that have been generated by other scripts – sonar-scanner doesn't do that on its own. For PHP, the code coverage is generated by PHPUnit unit tests only (not integration tests), and for JavaScript it is generated with node-qunit tests which currently only exist in a handful of repositories (GrowthExperiments, Popups, MobileFrontend).

The destination for sending the analysis results could be the self-hosted version of SonarQube. But in our case we currently use the hosted version of SonarQube which is at https://sonarcloud.io.

When the analysis reaches the SonarQube application, it is stored in its database and SonarQube provides a grade for the patch. Then it issues a POST request to SonarQube Bot.

SonarQube Bot

SonarQube Bot is a Symfony application that listens for incoming POST requests from SonarQube. It looks to see if the quality gate in SonarQube for the patch was pass or fail. Then it marks a patchset as Verified +1 if the quality gate passed. It also leaves a comment with a summary of the quality gates, for both success and failure cases.

Currently the bot leaves inline comments with issues found, using Gerrit's robot comment feature. Robot comments are not well supported in Gerrit 2.15, so it is an experimental feature. If you want to have your extension in the whitelist of projects that has inline comments please let us know via the talk page.

The configuration for the bot is managed via .env.local.php in Toolforge, after executing become sonarqubebot. Its logs are visible in var/log/prod.log. Currently the maintainers with SSH access are User:KHarlan (WMF) and User:GLederrey (WMF).

Feedback

Feedback via the talk page here is very welcome.

References

↑ From the announcement: "While SonarQube is an open source project, we currently use the hosted version at sonarcloud.io. We plan to eventually migrate to our own self-hosted SonarQube instance, so we have full ownership of tools and data." https://sonarcloud.io/ is described as "the public instance of SonarQube" at https://docs.sonarqube.org/latest/requirements/requirements/ and might be running the code published at https://github.com/SonarSource/sonarqube under LGPL, although that's not stated on the website.

[1] From the announcement: "While SonarQube is an open source project, we currently use the hosted version at sonarcloud.io. We plan to eventually migrate to our own self-hosted SonarQube instance, so we have full ownership of tools and data." https://sonarcloud.io/ is described as "the public instance of SonarQube" at https://docs.sonarqube.org/latest/requirements/requirements/ and might be running the code published at https://github.com/SonarSource/sonarqube under LGPL, although that's not stated on the website.

[1]