Design System Team/Test environments

The Design System Team uses a range of test environments for its work with Codex, both outside and inside of MediaWiki. This page documents these environments.

Codex repository

Test suite

The Codex monorepo contains various tests for the different packages. You can run npm run test locally to run all tests. Jest is used for unit testing and will output a coverage report in the console after running the tests. Read more about unit testing in Codex.

Netlify previews

When a patch is pushed to the Codex repository in Gerrit, an instance of the documentation site will be generated and launched via Netlify. Access the site by visiting the URL with the pattern https://PATCH_ID_NUMBER--wikimedia-codex.netlify.app/, swapping out PATCH_ID_NUMBER for the Gerrit patch ID (the number at the end of the URL, not the Change-Id). For example, the Netlify preview for this patch is available at https://953740--wikimedia-codex.netlify.app/.

You can also view the Sandbox page by adding /sandbox to the URL, e.g. https://953740--wikimedia-codex.netlify.app/sandbox.

Continuous integration (CI)

The Codex repository in Gerrit relies on Jenkins to run a set of tasks for patches. Currently there are 2 jobs:

Tests: Builds the library, runs all tests, and diffs the compiled CSS
Netlify preview: Deploys the current branch to Netlify

You can click on the console output of these jobs for an individual patch to view things like:

Test build:
- Test failures
- The Jest coverage report
- The CSS diff, which is an un-minified, prettified diff of the compiled CSS. This is useful to ensure that style or token changes have no unintended effects.
Netlify preview:
- Build failures
- The link to the Netlify preview (at the bottom of the output)

You can always view the status of in-progress jobs on the Zuul status page.

MediaWiki integrations

VueTest

VueTest is a MediaWiki extension meant to be a testing ground for Vue and Codex work, internal to the Design System Team. VueTest uses an internal version of Codex, allowing us to update the version past the version that's included in MediaWiki. It contains a special page of component demos that are pulled from the codex-docs package. This page can be used to view Codex components in a MediaWiki context, and it is used for visual regression testing with Pixel (see below). See the VueTest README for more information.

CodexExample

CodexExample is a MediaWiki extension meant to be an example of how to use Codex within MediaWiki. It contains uses of Vue components, CSS-only components, icons, and design tokens, along with comments explaining exactly how to use them. The README also identifies where these things are used in the code.

Pixel

Pixel is a visual regression testing tool developed by WMF engineers for testing with BackstopJS in MediaWiki. Pixel contains a dockerized MediaWiki environment, including a set of skins and extensions. Config can be added to run snapshot testing of a reference versus a test case. This can be done locally by pulling down the repository, installing it, taking reference snapshots, and running tests. There is also a live instance with a web UI that runs tests of the latest release branch of core and all extensions and skins versus the master branch, once per hour.

For Codex, we test a SpecialPage generated by the VueTest extension. To easily run the tests, you can open and merge a patch to update the version of Codex in VueTest (see VueTest README). The next time the live Pixel tests run, you will see results based on the newly updated master branch of VueTest.

Patchdemo

Patchdemo is an invaluable tool for generating a sharable MediaWiki instance based on a patch or set of patches. One use case is testing the core patch to update Codex. Some tips for using patchdemo:

You can add more extensions by expanding the "Choose included repos" toggle. For example, you might include VueTest if you want to see the component demos.
If you want to test TypeaheadSearch or any project that relies on content, check the "Proxy articles from wikipedia.org" box to import content into your test instance
You can log in to several existing accounts in your test instance with password "patchdemo":
- Patch Demo (admin)
- Alice
- Bob
- Mallory (blocked user)

Test and beta wikis

There are various test wikis available to test features in a safe environment that is at least somewhat similar to production. The test wikis run the latest production code, while the beta versions run the master branch of core and other repositories. Beta wikis are useful for testing features after they have been merged but before they have ridden the deployment train. Some useful beta sites include: