RESTBase

RESTBase is a caching / storing API proxy backing the Wikimedia REST API . Its configuration is based on Swagger specs, and its primary storage backend is using Cassandra. It powers "rest_v1", the Wikimedia REST content API used by VisualEditor to retrieve page HTML for editing. For performance (task T95229) service endpoints are also available on each wiki, e.g. on this wiki.

As a proxy, RESTBase does not perform any significant content processing itself. Instead, it requests content transformations from backend services when needed, and typically (depending on configuration) stores it back for later retrieval. For high-volume static endpoints, most requests will be satisfied directly from storage.

Its storage backends expose a RESTful storage API similar to Amazon DynamoDB and Google DataStore. The primary implementation uses Apache Cassandra. Notable features include automatically maintained secondary indexes and some lightweight transaction support. A SQLite backend has been developed and is the default backend in the package.

RESTBase automatically emits statsd metrics about all storage and backend requests. This provides a good baseline level of performance and error instrumentation in a micro-service architecture.

Use cases

Our first use case is speeding up VisualEditor by reducing HTML size, and eliminating Varnish cache misses. RESTBase stores Parsoid metadata separately from the HTML of the page, which reduces the size of the latter by about 40%. RESTBase provides only this HTML to VE, which reduces network transfer and processing latency significantly. In the longer term, we are aiming to bring down the size of the HTML to that of current PHP parser output to make it suitable for regular page views. This has the potential to make switching to visual editing instantaneous and free of any scrolling.

If parse time is not a pressing concern for your wiki (for example it does not have complex templates or large transclusion counts), then accessing Parsoid directly may make more sense than introducing a dependency on RESTBase.

Another use case we are strongly interested in is providing a section-level editing API for micro-contributions and extremely fast VisualEditor saves, even faster than wikitext.

Documentation

• Browse the API endpoints
• Overview
• Architecture
• Browse the documentation
• Deployment process
• List of current users

Installation

• RESTBase/Installation

See also

HyperSwitch Group: Wikimedia Services Start: 2014-04

• Browse the source code
• Follow architectural discussions about RESTBase by joining & watching the RESTBase-architecture project
• Report issues on Phabricator, project 'RESTBase'
• RESTBase/Alternative architectural options considered
• RESTBase/Table storage backend options
• Original requests for comments: Storage service (talk) and Content API (talk)