Continuous integration/Entry points
We have standardized on the following tools for testing our code:
Language | Command runner | Linting | Code style | Static analysis | Unit tests | Documentation |
---|---|---|---|---|---|---|
PHP | composer | php-parallel-lint
|
PHP CodeSniffer with mediawiki-tools-codesniffer | Phan with mediawiki-phan-config | PHPUnit | Doxygen |
JavaScript | npm & grunt | grunt-eslint with eslint-config-wikimedia | grunt-eslint with eslint-config-wikimedia | QUnit | JSDoc | |
JSON | N/A | json-schema | N/A | N/A | ||
i18n | grunt-banana-checker | N/A | N/A | N/A | Localisation Message documentation | |
CSS/LESS | grunt-stylelint with stylelint-config-wikimedia | N/A | N/A | N/A | ||
Java | maven wrapper (maven) | ? | ? | ? | ? | ? |
Python | tox | flake8 | unittest
pytest[1] |
sphinx? | ||
Ruby | bundler | rubocop | ? | rake test | ? |
Documentation on how to configure and set up these tools can be found below.
JavaScript
editTesting JavaScript
editWe are using npm test
as an entry point.
If your project has any JavaScript files, it should at least have a package.json
file which defines a test
script, and the related package-lock.json
file to ensure consistency in CI runs and security upgrades.
For MediaWiki extensions and skins, the npm test
script should not run project tests, only linters.
Anything other than linters (e.g. unit/integration tests) are run through the normal MediaWiki channels, tested on Special:JavaScriptTest; if you want a stand-alone runner, you should put that into another script.
You will need the .eslintrc.json
configuration file in your project (see Manual:Coding conventions/JavaScript#Linting).
Look at one of the projects listed in the example section below for an example of these files.
npm
commands fails with "node: No such file or directory", you may need to install the "nodejs-legacy
" package.Grunt task runner
editIf your project has complex build processes, or is an extension or skin that will benefit from i18n checking and JSON file linting, the convention is to use Grunt as a task runner.
Your project still has a package.json
file, which has a dependency on grunt
and sets "test": "grunt test"
.
In turn, a Gruntfile.js
file implements grunt test
, and this can run a wide variety of tools and tests:
eslint
, which checks both JS and JSON files.stylelint
, which checks both CSS and LESS files.banana-checker
, which checks messages in MediaWiki i18n files.
You can specify configuration settings for these tools in Gruntfile.js
.
However, it should contain little to no configuration for tools that can run outside grunt
so that they operate the same when run standalone or from a text editor plugin.
Always use native configuration files where possible, including .eslintrc.json
mentioned above.
JavaScript documentation
editUse npm run doc
as the entry point.
The convention is to use JSDoc .
The predoc
and postdoc
script hooks in package.json
can be used to run any additional scripts (e.g. build files for inclusion beforehand, or copy additional files for publication afterwards).
Examples
editAdvanced setup using Grunt
editpackage.json
edit{
"private": true,
"scripts": {
"test": "grunt test"
},
"devDependencies": {
"eslint-config-wikimedia": "0.15.0",
"grunt": "1.0.4",
"grunt-banana-checker": "0.8.1",
"grunt-eslint": "22.0.0",
"grunt-stylelint": "0.12.0",
"stylelint-config-wikimedia": "0.7.0"
}
}
Gruntfile.js
edit/* eslint-env node, es6 */
module.exports = function ( grunt ) {
grunt.loadNpmTasks( 'grunt-banana-checker' );
grunt.loadNpmTasks( 'grunt-eslint' );
grunt.loadNpmTasks( 'grunt-stylelint' );
grunt.initConfig( {
eslint: {
options: {
extensions: [ '.js', '.json' ],
cache: true
},
all: [
'**/*.{js,json}',
'!{vendor,node_modules}/**'
]
},
stylelint: {
all: [
'**/*.{css,less}',
'!{vendor,node_modules}/**'
]
},
banana: {
all: 'i18n/'
}
} );
grunt.registerTask( 'test', [ 'eslint', 'stylelint', 'banana' ] );
grunt.registerTask( 'default', 'test' );
};
Example projects
edit- Extension:BoilerPlate has a
Gruntfile.js
that runs jshint, jscs, and banana-checker (for MediaWiki'si18n
JSON files). - jquery-client: package.json (jshint, jscs, karma; no Grunt needed)
- CSSJanus: package.json / Gruntfile.js (jshint, jscs, custom test)
- TemplateData: package.json / Gruntfile.js (jshint, jscs, banana-checker)
Further reading
edit- package.json format on docs.npmjs.org
- package "scripts" lifecycle on docs.npmjs.org
PHP
edit- Use the BoilerPlate extension as starting point for a new MediaWiki extension.
- Use the mediawiki-tools-cookiecutter-library for creating a new PHP library.
Testing PHP
editWe are using composer test
as an entry point.
If your project has PHP files it should list the test framework packages it needs in composer.json
under require-dev
and list the commands to be run in the scripts.test
property:
{
"require-dev": {
"mediawiki/mediawiki-codesniffer": "31.0.0",
"mediawiki/minus-x": "1.1.0",
"ockcyp/covers-validator": "1.2.0",
"php-parallel-lint/php-console-highlighter": "0.5.0",
"php-parallel-lint/php-parallel-lint": "1.2.0",
"phpunit/phpunit": "^8.5"
},
"scripts": {
"test": [
"parallel-lint . --exclude vendor --exclude node_modules",
"minus-x check .",
"phpunit",
"covers-validator",
"phpcs -p -s"
],
"cover": "phpunit --coverage-html coverage",
"fix": [
"minus-x fix .",
"phpcbf"
]
}
}
See composer.json of the cdb project for a good example.
Note that MediaWiki extensions are not standalone projects and cannot run their own PHPUnit test suite from composer.
Those repositories have a separate mediawiki-extensions
job.
PHPCS and PHP lint are still run via composer.json
and composer test
:
{
"require-dev": {
"mediawiki/mediawiki-codesniffer": "31.0.0",
"mediawiki/minus-x": "1.1.0",
"php-parallel-lint/php-console-highlighter": "0.5.0",
"php-parallel-lint/php-parallel-lint": "1.2.0"
},
"scripts": {
"test": [
"parallel-lint . --exclude vendor --exclude node_modules",
"minus-x check .",
"phpcs -p -s"
],
"fix": [
"minus-x fix .",
"phpcbf"
]
}
}
See composer.json of the AbuseFilter MediaWiki extension for a good example.
PHP Documentation
editSee: Doxygen .
Use the doxygen program to generate a Doxyfile
file in the project root.
Testing Python
editRuby
editRake
editUse Rake to define your commands, they will be executed via Bundler.
Example Rakefile
:
require 'bundler/setup'
require 'rubocop/rake_task'
RuboCop::RakeTask.new(:rubocop) do |task|
# if you use mediawiki-vagrant, rubocop will by default use it's .rubocop.yml
# the next line makes it explicit that you want .rubocop.yml from the directory
# where `bundle exec rake` is executed
task.options = ['-c', '.rubocop.yml']
end
require 'mediawiki_selenium/rake_task'
MediawikiSelenium::RakeTask.new
task default: [:test]
desc 'Run all build/tests commands (CI entry point)'
task test: [:rubocop]
The above code will create following Rake targets.
$ bundle exec rake -T rake rubocop # Run RuboCop rake rubocop:auto_correct # Auto-correct RuboCop offenses rake selenium # Run Cucumber features rake test # Run all build/tests commands (CI entry point)
The Jenkins job rake-jessie
invokes test
target by running bundle exec rake test
.
Reference: phab:T104024
ruby debug tip
editYou can use the gem pry
to break on error and get shown a console in the context of the failure.
To your Gemfile add gem 'pry'
then to break:
require 'pry'
binding.pry
your call that fail
You will then be in a console before the breakage that let you inspect the environment (ls
).
See https://github.com/pry/pry for details.
ci.yml
editWe have a set of Jenkins jobs that run daily and execute Ruby + Selenium tests.
The jobs are named selenium*
.
Each repositories has only a single job defined in Jenkins.
It is a multi configuration job that spawns one or more child job based on a configuration in each repositories: tests/browser/ci.yml
.
The main job will then spawn child jobs based on its content.
Example of a simple ci.yml
is in mediawiki/core
.
BROWSER:
- firefox
MEDIAWIKI_ENVIRONMENT:
- beta
PLATFORM:
- Linux
As you can see, there are three variables, BROWSER
, MEDIAWIKI_ENVIRONMENT
and PLATFORM
.
BROWSER
and PLATFORM
can be any valid Sauce Labs browser/OS/version combination.
MEDIAWIKI_ENVIRONMENT
can have values beta
, mediawiki
and test
, or any other environment configured in environments.yml
.
For example:
BROWSER:
- chrome
- firefox
- internet_explorer 9.0
- safari
MEDIAWIKI_ENVIRONMENT:
- beta
- mediawiki
- test
PLATFORM:
- Linux
- OS X 10.9
- Windows 8.1
Example of a complicated ci.yml
is in mediawiki/extensions/MultimediaViewer
.
For more information see Jenkins Yaml Axis Plugin.
Reference: phab:T128190