MediaWiki API integration tests

The API testing tool is a library for end-to-end integration tests for the MediaWiki Action API and REST API. You can run tests locally by installing the NPM package and configuring it to access a test wiki or a service. The library is implemented in JavaScript for node.js, using the SuperTest HTTP testing library, the Chai assertion library, and the Mocha testing framework.

Setting up a test environmentEdit

InstallationEdit

To run the API test suite, you need to have node.js and npm installed.

Installation of the testing environment can then be done with npm:

$ npm install --save-dev api-testing mocha

Test wiki setupEdit

To run the tests, you need a MediaWiki installation to run them against. Ideally, tests would run against an empty, freshly initialized wiki each time, but tests should be written to function when run against a wiki that already has content from previous test runs and manual testing.

The recommended way to set up a wiki instance to test against is to spin up a docker container that provides a wiki. MediaWiki-Docker-Dev uses docker-compose to provide all necessary services for MediaWiki, such as a database and caching.

ConfigurationEdit

The testing tool requires a .api-testing.config.json configuration file that tells it how to access the target wiki. An example configuration is contained in the configs/example.json file:

{
  "base_uri": "http://default.web.mw.localhost:8080/mediawiki/",
  "main_page": "Main_Page",
  "root_user": {
    "name": "Admin",
    "password": "dockerpass"
  },
  "secret_key": "",
  "extra_parameters": {
    "xdebug_session": "PHPSTORM"
  }
}

The above values were chosen to work with a default setup of MediaWiki-Docker-Dev. Copy the configuration file into the root folder of your application, and supply the target wiki's $wgSecretKey in the secret_key configuration option. $wgSecretKey can be found in the wiki's LocalSettings.php - if you are using MediaWiki-Docker-Dev, this file can be found under config/mediawiki/.

To use a custom configuration file, create a .api-testing.config.json file in the root folder of your application. For a MediaWiki extension, include the config file in the root of MediaWiki Core. For automated testing, set the API_TESTING_CONFIG_FILE environment variable to point to the correct configuration file.

The configuration file is evaluated in the following order:

  1. API_TESTING_CONFIG_FILE if set
  2. .api-testing.config.json if it exists

For a custom setup, provide the following configuration settings:

  • base_uri: Full base URI of the MediaWiki installation to target. Must end with a slash (/).
  • main_page: The name of the wiki's main page.
  • root_user: Login credentials for a user that has bureaucrat privileges (most importantly, the right to add users to groups to grant them privileged access).
  • secret_key: $wgSecretKey can be found in the wiki's LocalSettings.php.
  Caution: The content of the wiki you are running the tests against will be polluted with test content! Do not run tests against a wiki with valuable content.
  Caution: Be careful about running tests against a wiki that is publicly accessible.

Tests should be written to use randomized passwords for all accounts they create, but there is no guarantee that no tests creates a privileged user with a known or easy to guess password. Also, if your test wiki is publicly accessible, be careful not to publish the root_user credentials, and to not use the default credentials. Even if the wiki itself doesn't have valuable content, having your test wiki

compromised may open you up to attacks if it shares a domain or host with a real wiki.

Service setupEdit

For non-MediaWiki applications, set the environment variable REST_BASE_URL to point to your service. For example:

$ REST_BASE_URL=http://localhost:3000/

Running testsEdit

Resetting the target wikiEdit

Before running tests, it's advisable to ensure a known state of the wiki the tests run against. While tests should be written to be robust against pre-existing content, e.g. by randomizing all resource names, a known base state is useful. Also, test runs tend to pollute the wiki a lot, so a reset is bound to save space, even if not done for every test run.

The easiest way to achieve a known state of the wiki is to take a snapshot of a known state, preferably right after installation when the wiki contains just one page and one user, and then load that dump into the database before running tests. For convenience, two pairs of scripts are supplied to achieve this: one pair for use with a local MediaWiki installation and another pair for a MediaWiki-Docker-Dev environment.

Local snapshotsEdit

If you have MediaWiki installed locally, you can use:

$ node_modules/api-testing/bin/take-snapshot <name.tar> [db] [host]

This saves a snapshot of a wiki in the given tar file. The [db] parameter is the database name. If not given, "wiki" is used, which is the default name proposed by the MediaWiki installer. The [host] parameter allows the database host to be specified, in case it's not localhost.

$ node_modules/api-testing/bin/medd-load-snapshot <name.tar> [db] [host]

This restores the snapshot in the given tar file. The tar file contains the name of the wiki database the snapshot was taken from. If the [db] parameter is not given, the dump will be loaded into that same database. The name of the database is also shown in the confirmation prompt.

Before you can use these scripts, you need to configure the location of your MediaWiki installation in bin/local.env:

MW_DIR="../../mediawiki"

Set this to something like /var/www/html/mediawiki/ or wherever you have installed MediaWiki.

MediaWiki-Docker-Dev snapshotsEdit

If you have your wiki instances managed by MediaWiki-Docker-Dev, you can use:

$ node_modules/api-testing/bin/mwdd-take-snapshot <name.tar> [db]

This saves a snapshot of a wiki in the given tar file. The [db] parameter is the database name, which is the name you gave your wiki when running the addsite script. If not given, "default" is used, which is the name of the wiki pre-installed by MediaWiki-Docker-Dev.

$ node_modules/api-testing/bin/mwdd-load-snapshot <name.tar> [db]

This restores the snapshot in the given tar file. The tar file contains the name of the wiki database the snapshot was taken from. If the [db] parameter is not given, the dump will be loaded into that same database. The name of the database is also shown in the confirmation prompt.

Before you can use these scripts, you need to configure the location of your MediaWiki-Docker-Dev installation in bin/local.env:

MWDD_DIR="../../mediawiki-docker-dev"

Set this to something like $HOME/opt/mediawiki-docker-dev/ or wherever you have installed MediaWiki-Docker-Dev.

Running specific testsEdit

You can run individual test files or directories containing test files by invoking Mocha directly and pointing it to the desired path:

$ ./node_modules/.bin/mocha <test-file-or-dir> --timeout 0

For more information on running Mocha tests and controlling the output, see the Mocha docs.

Writing testsEdit

Action API tests are stored in the /tests/api-testing/action directory; REST API tests are stored in the /tests/api-testing/REST directory. Each file corresponds to a test suite covering an area of MediaWiki functionality. A test suite is defined by a top-level describe() function that takes two parameters:

  • a string describing the feature being tested
  • a function containing the test code

Within the describe() function, you can use the before() hook to set up preconditions and the it() function to create individual test cases. You can break up a long, complex test suite by nesting additional describe() functions under the top-level function.

Within a test suite, each test case is executed as an async function. These asynchronous functions contain await expressions to execute test steps in sequence and assert expressions to evaluate responses. Assertions can use any methods supported by the Chai assert interface, such as assert.equal, assert.match (using a regular expression), and assert.include.

An await expression pauses the execution of an async function until the expression can return a resolved Promise. Because of this, await expressions are only valid inside async functions. For more information about Promises, see MDN's guide to using Promises.

Here's a template that shows the main sections of a test suite:

// Load required modules
const { assert, action, utils } = require('api-testing');

// Define a test suite
describe('The feature being tested', function () {
    // Define global variables
    ...

    // Set up preconditions
    before(async () => {
        await ...
    });

    // Define a test case
    it('should perform the action being tested', async () => {
        // Execute test steps
        await ...
        // Validate output using assertions
        assert ...
    });
});

Generating random stringsEdit

When writing tests, use random values whenever possible. This allows tests to function when run against a wiki that already contains content from previous test runs. To generate a random string for use in a test, use the uniq() function.

// Generates a unique, 20-character string of random alphanumeric characters
// Defaults to 10 characters
utils.uniq(20)

Handling page titlesEdit

The API testing tool includes functions to help you manage wiki page titles.

// Returns a random, 10-character, alphanumeric page title
utils.title()

// Returns a random page title with the prefix 'Test:'
utils.title('Test:')

// Returns the provided title with spaces replaced with underscores
// This example returns 'My_Wiki_Page'
utils.dbkey('My Wiki Page')

// Returns true if the provided titles are equal
assert.sameTitle('Test Page 1', 'Test Page 1')

Creating accounts and logging inEdit

FixturesEdit

To manage accounts and sessions, the API testing tool provides convenient fixtures that you can reuse across tests.

fixture name account type
alice user
bob user
robby bot
mindy admin

To open a wiki session and log in using a fixture, instantiate the account asynchronously using a fixtures function.

// Defines a global variable to use in the test suite
let alice;

// Opens a session and logs in as alice
before(async () => {
    alice = await action.alice();
});

Custom accountsEdit

If you're planning to make a permanent change to an account (for example: blocking an account) or if you need an account with a custom prefix, you can create a custom account instead of using a fixture. To open a wiki session and log in using a custom account, define a session using the getAnon() function, and log in using the account method. Applying a prefix to the username is optional.

// Defines global variables for two custom account sessions
const fiona = action.getAnon();
const franky = action.getAnon();

// Logs in and applies the Fiona_ and Franky_ prefixes to the usernames
before(async () => {
    await Promise.all([
        fiona.account('Fiona_'),
        franky.account('Franky_')
    ]);
})

Account propertiesEdit

Once logged in, you can access information about the account using the username, userid, and password properties.

parameter name example description
username alice.username
myUser.username
Randomly generated username tied to the account. Usernames for accounts created using a fixture are prefixed with the name of the fixture (for example: alice_dRUET7xhKQ). Usernames for custom accounts can have an optional prefix; for example, myUser.account('User1_') results in a username in the format User1_nD9EUfYXgR.
userid alice.userid
myUser.userid
User ID tied to the account
password alice.password
myUser.password
Randomly generated password tied to the account

Anonymous usersEdit

To create an anonymous user, define the account using the getAnon() function, but omit the account function. This opens a new session without logging in, resulting in an anonymous user.

// Creates a session for an anonymous user
const anonymousUser = action.getAnon();

Working with wiki pagesEdit

The API testing tool provides helpful methods for interacting with wiki pages, including editing a page, exploring page history, and retrieving page HTML. To create and edit a page, use the title() function to generate a random title and the edit() method to edit the page. To validate the edit, you can use the getHtml() method to get the HTML of the page and the assert.include() method to check for the edited text.

describe('Page editing', function () {
    let alice;
    // Generates a random page title
    const title = utils.title();

    before(async () => {
        alice = await action.alice();
    });

    it('should edit a page', async () => {
        // Has alice edit the page with "Hello, world!"
        const editPage = await alice.edit(title, { text: 'Hello, world!' });
        
        // Returns the HTML of the page
        const pageHtml = await alice.getHtml(title);
       
        // Validates whether the text is present in the HTML
        assert.include(pageHtml, 'Hello, world!');
    });
});

Edit a pageEdit

edit()
arguments
  • Page title (string)
  • API:Edit parameters (object)
response API:Edit#Response response
You can also access edit properties using the param_user, param_text, and param_summary parameters.
example
edit(title, { text: 'Hello, world!' })

Return a revision record for a pageEdit

getRevision()
arguments
  • Page title (string)
  • Revision ID (optional) - If revision ID is 0 or not provided, returns the latest revision.
  • API:Revisions parameters (object)
response API:Revisions#Response response
example
getRevision(title)

Return HTML for a page with comments strippedEdit

getHtml()
arguments Page title (string)
response API:Parsing_wikitext#Response response
example
getHtml(title)

Return the most recent changes entry matching the given parametersEdit

getChangeEntry()
arguments API:RecentChanges parameters (object)
response API:RecentChanges#Response response
example
getChangeEntry({ rctitle: page })

Return the newest log entry matching the given parametersEdit

getLogEntry()
arguments API:Logevents parameters (object)
response API:Logevents#Response response
example
getLogEntry({ letype: 'delete', letitle: title })

Calling the Action APIEdit

The Action API list, meta, and prop modules provide access to information about wiki pages and users. The testing tool provides methods that let you make GET requests to these modules within tests.

The List APIEdit

list()
description GET request to list items that match select criteria. See the Lists API docs for available submodules.
arguments
  • API submodule (string)
  • Submodule-specific parameters
response See the Lists API docs for submodules responses.
example
list('usercontribs', {
    ucuser: `${fiona.username}|${franky.username}`,
    ucprop: 'ids|user|comment|timestamp'
})

The Meta APIEdit

meta()
description GET request to fetch information which is not associated with pages(metadata). See the Meta API docs for available submodules.
arguments
  • API submodule (string)
  • Submodule-specific parameters
response See the Meta API docs for submodules responses.
example
meta('userinfo', { uiprop: 'options' })

The Properties APIEdit

prop()
description GET request to list properties of selected pages. See the Properties API docs for available submodules.
arguments
  • API submodule (string)
  • Page title (string)
  • Submodule-specific parameters
response See the Properties API docs for submodules responses.
example
prop('links', pageX, { plnamespace: 0 })

The Upload APIEdit

upload()
description POST request to upload a file. See the API:Upload API docs for available submodules.
arguments
  • Submodule-specific parameters (object)
  • file (string)
response See the Upload API docs for submodules responses.
example
upload({"filename": "file_1.jpg", "token": await mindy.token()}, '~/file.jpg')

Other Action API callsEdit

To call any module in the Action API, use the action() method.

action()
description Executes an HTTP request to the Action API and returns the parsed response body. This method fails if the response contains an error code. See the API docs for available actions.
arguments
  • Action name (string)
  • Submodule-specific parameters
  • POST (for POST requests)
response See the API docs for action responses.
example
action('parse', {page: pageTitle })

Some Action API calls require a token. See individual API action docs for token requirements. For example, to patrol a page, make a POST request to the patrol action. In the API:Patrol docs, we can see that this call requires a token, which we can get using the token() method.

const result = await mindy.action(
    'patrol',
    {
        title: pageTitle,
        revid: edit.newrevid,
        token: await mindy.token('patrol')
    },
    'POST',
)

The action() method fails if the response contains an error code. To test for expected errors, you can use the actionError() method.

actionError()
description Executes an HTTP request to the Action API and returns the error stanza of the response body. This method fails if there is no error stanza.
arguments
  • Action name (string)
  • Submodule-specific parameters
  • POST (for POST requests)
response See the API docs for error responses.
example
actionError('query', { list: 'recentchanges', rctitle: pageTitle, rcprop: 'ids|flags|patrolled' }, )

Action API example testEdit

The Recent Changes test suite contains two tests cases that validate a user's ability to access recent changes for a page.

// Loads required modules
const { action, assert, utils } = require('api-testing');

// Defines a test suite called 'Recent Changes'
describe('Recent Changes', function () {
    // Defines a random page title prefixed with 'Recent_Changes_'
    const title = utils.title('Recent_Changes_');
    // Defines the account we'll use in this test suite
    let alice;

    // Logs in to a wiki session using the alice fixture
    before(async () => {
        alice = await action.alice();
    });

    // Defines the first test case
    it('should create page and get new page recent changes', async () => {
        // Has alice add the text 'Recent changes testing' to the randomly
        // defined page title, only if it does not already exist
        const edit = await alice.edit(title, { text: 'Recent changes testing', createonly: true });
       
        // Has alice request the most recent changes for the same page
        const results = await alice.list('recentchanges', { rctype: 'new', rctitle: title });

        // Validates that the most recent change creates a new page
        assert.equal(results[0].type, 'new');
       
        // Validates that the recent change has the same titled as the edited page
        assert.sameTitle(results[0].title, title);
        
        // Validates that the page ID in the recent change is the same as the page ID that was edited
        assert.equal(results[0].pageid, edit.pageid);
        
        // Validates that the revision ID in the recent change is the same as the revision ID that was edited
        assert.equal(results[0].revid, edit.newrevid);
    });

    // Defines a second test case
    it('should edit page and get most recent edit changes', async () => {
        // Has alice make a second edit to the page with the text 'Recent changes testing..R1'
        const rev1 = await alice.edit(title, { text: 'Recent changes testing..R1' });
        // Has alice request the most recent changes for that page
        const results = await alice.list('recentchanges', { rctype: 'edit', rctitle: title });

        // Validates the same data points as the previous test case
        assert.equal(results[0].type, 'edit');
        assert.sameTitle(results[0].title, title);
        assert.equal(results[0].pageid, rev1.pageid);
        assert.equal(results[0].revid, rev1.newrevid);
    });
});

Calling the REST APIEdit

The testing tool provides methods that let you make requests to the MediaWiki REST API. To open a REST API session, use the REST() function within the top level describe() function. REST API sessions are anonymous.

To make a request, use the corresponding method for the HTTP request method:

  • get(): Make an HTTP GET request to the REST API
  • post(): Make an HTTP POST request to the REST API
  • put(): Make an HTTP PUT request to the REST API
  • del(): Make an HTTP DELETE request to the REST API

These methods can take up to three arguments:

  • The endpoint path as a string, starting after the version number (Example: /revision/${revId1}/compare/${revId2} for the compare revisions endpoint)
  • If supported by the endpoint, a request body as an object
  • If supported by the endpoint, the content-type as a string. Defaulting to application/json

See the REST API docs for endpoint paths and request body requirements.

REST API example testEdit

Here's an example test suite with two tests for the get revision endpoint.

// Loads required modules
const { action, assert, REST, utils } = require('../../index');


// Defines a test suite called 'Get revision'
describe('Get revision', () => {
    // Creates a REST API session
    const client = new REST();
    // Defines the account we'll use in this test suite
    let mindy;

    // Logs in to a wiki session using the mindy fixture
    before(async () => {
        mindy = await action.mindy();
    });

    // Defines a successful test case
    it('should successfully get information about revision', async () => {
        // Defines a random page title prefixed with 'Revision'
        const page = utils.title('Revision');
        // Has mindy add the text 'Hello World' to the randomly defined page title
        const { newrevid, pageid, param_summary } = await mindy.edit(page, {text: 'Hello World'});
        // Makes a request to the REST API to get information about mindy's edit
        const { status, body } = await client.get(`/revision/${newrevid}/bare`);

        // Validates the API response properties
        assert.strictEqual(status, 200);
        assert.strictEqual(body.id, newrevid);
        assert.deepEqual(body.page, { id: pageid, title: page });
        assert.nestedPropertyVal(body, 'user.name', mindy.username);
    });

    // Defines a failed test case
    it('should return 404 for revision that does not exist', async () => {
        // Makes a request to the REST API with a known invalid parameter
        const { status } = await client.get('/revision/99999999/bare');

        // Validates that the API returns a 404
        assert.strictEqual(status, 404);
    });
});

ContributingEdit

The API tests are hosted on Gerrit and mirrored on GitHub. To open a patch request, see the guide to using Gerrit for Wikimedia projects.

To review open tasks or file a bug report, visit Phabricator.

Sharing feedbackEdit

To share your feedback about this page or to ask a question about API tests, leave a comment on the talk page.