Requests for comment/Standards for external services
This document is an attempt to formalize the output of the "Architecting Core: Standalone Services" session from the 2018 Wikimedia Technical Conference.
| Standards for external services | |
|---|---|
| Component | General |
| Creation date | |
| Author(s) | Ian Marlier, Giuseppe Lavagetto |
| Document status | accepted See Phabricator. |
Proposal
editThe following proposals aim to set architectural principles and standards for when, how and why to to extend MediaWiki via an external service for use in the WMF production environment.
Definition
editStandalone services are applications that extended MediaWiki's functionality, but that are operationally distinct in some meaningful way. The mechanism of extension is not specified, beyond saying that it's necessarily interprocess as opposed to intraprocess; queues, API calls and other types of RPC mechanisms, XHR from a client, are all valid examples, while shelling out from MediaWiki itself is not. It is important that the core business logic is implemented in the service, rather than within MediaWiki itself.
Selection Criteria: Deciding whether an external service is appropriate
editThe properties listed here are intended to be a guide as to whether a given feature can be provided externally to MediaWiki or not. They are intended to be necessary, but not sufficient. If a proposed feature has one or more properties that appear in the left column, but no properties that appear in the right column, then that feature could be implemented as a standalone service. If the proposed feature has one or more properties that appear in the right column, then a wide consensus must be reached before implementing it as a standalone service.
| Properties that make a feature suitable for implementation as a standalone service | Properties that make a feature unsuitable for implementation as a standalone service |
|---|---|
| State is independent - the functionality provided does not require view of MediaWiki state that is guaranteed to be current. | Feature only works correctly with a consistent and current view of MediaWiki state. |
| 3rd party library or service exists that can provide the needed functionality with minimal integration | Feature requires direct access to the MediaWiki database, and cannot use an API to retrieve or update data. |
| A non-PHP language or framework exists that significantly simplifies implementation | Requires features/functionality provided by other MediaWiki extensions that are implemented internally |
| Functionality gracefully degradates if the external service is unavailable | Unavailability of the external service compromises the general availability of the site to the user (e.g. results in a MediaWiki fatal error) |
| The feature is independently useful, and is likely to have non-MediaWiki use cases | The Feature involves directly parsing wikitext, or accessing MediaWiki i18n messages (also wikitext) |
If any of the following are true, then the feature absolutely should be implemented as an external service, with appropriate architectural changes made elsewhere to eliminate disqualifying properties.
| Properties that require that a service is provided externally to MediaWiki |
|---|
| Elevated security need: Due to data isolation or other operational requirements, a given feature cannot be provided in the same operational environment as MediaWiki itself. |
| Excessive or potentially unbounded resource needs: Image thumbnailing, video transcoding, and machine translation are all examples of features where unpredictable properties such as request rates and input size have a significant impact on the resources required, and based on factors that the operator can't control may result in resource contention and denial of service. |
| Long-running processes involved. |
| The feature in question is used to triage or fix MediaWiki in the case of failures |
| The application is going to be run in a separate environment from MediaWiki itself |
Given MediaWiki is not just part of the Wikimedia production infrastructure but also a software used my many third-parties, we can't just delegate some of its fundamental functions to an external service completely. Whenever an external service provides a functionality, we also need to ask ourselves if said functionality is fundamental or optional to MediaWiki.If the functionality replaced by the external service is fundamental, a fallback solution must be present within MediaWiki that substitutes what is being implemented in the service. As an example: for Wikimedia purposes, async processing for MediaWiki is implemented via a series of external services; for simple installations, a MySql-backed version of the same mecahnism exists and cannot be dropped from MediaWiki. What constitutes a core functionality of MediaWiki and what doesn't will need to be further defined elsewhere, and is beyond the scope of this document.
Architectural Guidelines for external services
edit- This section was moved to Wikimedia services guideline