Wikimedia Services/status

Last update on: 2014-09-monthly

2014-07-monthly

The brand new Services group (currently Matt Walker and Gabriel Wicke) started July with two main projects:

PDF render service deployment
Design and prototyping work on the storage service and REST API

The PDF render service is now deployed in production, and can be selected as a render backend in Special:Book. The renderer does not work perfectly on all pages yet, but the hope is that this will soon be fixed in collaboration with the other primary author of this service, C. Scott Ananian.

Prototyping work on the storage service and REST API is progressing well. The storage service now has early support for bucket creation and multiple bucket types. We decided to configure the storage service as a backend for the REST API server. This means that all requests will be sent to the REST API, which will then route them to the appropriate storage service without network overhead. This design lets us keep the storage service buckets very general by adding entry point specific logic in front-end handlers. The interface is still well-defined in terms of HTTP requests, so it remains straightforward to run the storage service as a separate process. We refined the bucket design to allow us to add features very similar to Amazon DynamoDB in a future iteration. There is also an early design for light-weight HTTP transaction support.

Matt Walker is sadly leaving the Foundation by the end of this month to follow his passion of building flying cars. This means that we currently have three positions open in the service group, which we hope to start filling soon.

2014-08-monthly

August was mostly a month of travel and vacation for the service team. We deployed a first prototype of the RESTBase storage and API service in Labs. We also presented on both Parsoid and RESTBase at Wikimania, which was well received.

Later in August, computer science student Hardik Juneja joined the team as a part-time contractor. Working from Delhi, he dived straight into complex secondary index update algorithms in the Cassandra back-end. At the end of the month, design work resumed, with the goal of making RESTBase easier to extend with additional entry points and bucket types.

2014-09-monthly

September saw a lot of activity on the RESTBase storage and API service. A new 'pagecontent' composite bucket type using revisioned blob buckets was introduced. This uses the by-now fairly rich table storage backend to provide functionality similar to MediaWiki's revision table, and supports any number of revisioned types of content (like HTML, wikitext, JSON metadata) associated with each revision.

Work on secondary index updates continued at full steam, and is now close to being merged.