RESTBase/deprecation

This document summarises the rationale and plan for deprecating RESTbase.

Decision

The RESTbase service should be deprecated and removed from WMF sites. Services that use RESTbase should be migrated to a different platform.

Rationale

This was already decided in 2019, see Core Platform Team/Initiatives/Sunsetting RESTBase. So quote: "Remove the need to support a complex piece of service architecture replacing it with a well supported third party application reducing cost of ownership".

See also the notes from the Technical Conference 2019.

In summary:

• The rationales for the design of RESTbase are obsolete. Needs fulfilled by RESTbase are better met by other means. Examples:
  • Load balancing and life cycle management should by done with Kubernetes.
  • API routing and instrumentation should be done with Envoy. See also the design document for rate limiting in the API gateway.
  • Response caching should be done with Varnish.
  • Parsing wikitext should be done by Parsoid in MediaWiki core.
  • Persistent storage should be done by each service individually, supported by a generalized data storage platform.
  • Event handling should be done by individual services or using a stream processing framework.
• RESTbase has major architectural problems. Most importantly, it serves as a central hub that may be hit multiple times recursively while handling a single request. This creates a cyclic dynamic dependency which incurs latency, makes errors hard to analyze, and causes issues with one service to impact other services.
• Several of the ideas that drove and informed the development of RESTbase, never materialized. Examples:
  • include storing HTML for all versions of a page
  • ...
• The RESTbase code base is unmaintained. All engineers with a deep knowledge of the code base have left WMF. This makes it a critical risk factor to operational reliability.
• RESTbase is in need of an update. It depends on old versions of npm packages. Updating it would be a significant effort, especially since all engineers with a deep knowledge of the code base have left WMF.
• The RESTbase code is complex. The effort level that would be required for a team of engineers to develop a level of understanding sufficient to update and maintain RESTbase is at least comparable to the effort level required to retire it in favor of alternatives that are technically preferable anyway.
  • The storage abstraction layer seems to add more complexity rather than removing it.

Migration Plan

A checklist for individual services is available under RESTBase/service migration.

A migration plan for each service has been layed out in the Services Architecture Recommendations (2019).

An overview can be found in the Services Re-architecture document.