MediaWiki

Core Platform Team/Initiative/Add API integration tests/Initiative Description

< Core Platform Team‎ | Initiative

< Add API integration tests

Summary

Create a suite of end-to-end tests for all modules of MediaWiki’s action API.

Significance and Motivation

We need to ensure we don’t break anything while refactoring core code for the Decoupling and HTTP API projects. Unit test coverage of MediaWIki core is insufficient for this purpose. Instead of writing unit tests for code what we plan to dismantle, we should instead ensure that the behavior of the outward facing API remains stable. Since all key functionality of MediaWiki can be accessed via that API, testing that API could cover all critical code paths, assuming tests are written for all relevant actions and combinations of parameters.

Outcomes

Ultimately: Increase predictability and safety of deploying code changes

Immediately: Provide a standard framework for testing HTTP services and APIs.

Enables/unblocks: Allow confident refactoring by improving test coverage.

Baseline Metrics
  • No end-to-end API tests exist.
Target Metrics
  • Measure: Percentage of modules covered, percentage of parameters covered for each module, number of parameter permutations covered, number of cross-module use cases tested.
  • API actions that cause database updates have tests covering all relevant modes of operation (combinations of parameters).
  • End-to-end API tests can easily be run in a local development environment
  • Stretch/later: All API actions have tests covering all relevant modes of operation.
  • Stretch/later: End-to-end API tests are run automatically as part of CI
  • Stretch/later: End-to-end API tests are run periodically against deployment-prep ("beta") projects.
Stakeholders
  • API users
  • Core code developers
  • Release Engineering
Known Dependencies/Blockers

None

Retrieved from "https://www.mediawiki.org/w/index.php?title=Core_Platform_Team/Initiative/Add_API_integration_tests/Initiative_Description&oldid=4846415"