MediaWiki

Core Platform Team/Initiative/API Gateway/Documentation Plan

< Core Platform Team | Initiative


View the prototype Track this project on Phabricator Share feedback Subscribe to project updates

Overview

edit

The API Gateway is a CPT initiative to create a rate-limited API proxy at api.wikimedia.org as part of the Platform Evolution program. The MVP scope includes the MediaWiki REST API (including page, media, history, and search endpoints) and the Feeds API, with the intention of adding APIs over time. See the initiative page and Phabricator tag.

Approximate list of endpoints for MVP
  • GET /feed/v1/{project}/{language}/announcements
  • GET /feed/v1/{project}/{language}/featured/{yyyy}/{mm}/{dd}
  • GET /feed/v1/{project}/{language}/onthisday/{type}/{mm}/{dd}
  • GET /core/v1/{project}/{language}/file/{title}
  • GET /core/v1/{project}/{language}/search/page
  • GET /core/v1/{project}/{language}/search/title
  • POST /core/v1/{project}/{language}/page
  • GET /core/v1/{project}/{language}/page/{title}
  • PUT /core/v1/{project}/{language}/page/{title}
  • GET /core/v1/{project}/{language}/page/{title}/bare
  • GET /core/v1/{project}/{language}/page/{title}/html
  • GET /core/v1/{project}/{language}/page/{title}/with_html
  • GET /core/v1/{project}/{language}/page/{title}/links/language
  • GET /core/v1/{project}/{language}/page/{title}/links/media
  • GET /core/v1/{project}/{language}/page/{title}/history
  • GET /core/v1/{project}/{language}/page/{title}/history/counts/{type}
  • GET /core/v1/{project}/{language}/revision/{id}/bare
  • GET /core/v1/{project}/{language}/revision/{from}/compare/{to}

The purpose of this document is to manage the planning and feedback process for the documentation for this project.

Components

edit
 
Wikimedia API Gateway architecture diagram
MediaWiki
The API Portal will be a MediaWiki instance at api.wikimedia.org (beta: api.wikimedia.beta.wmflabs.org)
WikimediaApiPortal skin (Phabricator project)
A MediaWiki skin providing the look-and-feel of the API Portal
WikimediaApiPortalOAuth extension (Phabricator project)
A MediaWiki extension running on the API Portal that interfaces with the OAuth extension on Meta-Wiki
OAuth extension on Meta-Wiki
Responsible for storing and managing OAuth clients

Implementation plan

edit
ID Description Owner Status

Phase 1: Planning and development

1 Finalize development contract Cindy   Done
2 Security approval for the Chameleon skin (phab:T246949) Cindy Cancelled (See phab:T252462.)
3 Collect feedback from Technical Engagement Alex   Done (Notes)
4 Create a wiki instance Cindy   Done
5 Submit request to create a beta wiki instance (phab:T254185) Cindy   Done
6 Create a MediaWiki skin ("WikimediaApiPortal") that matches the provided design, and create documentation for the extension on mediawiki.org Cindy   Done
7 Define technical specification for interactions between the API Portal and the OAuth extension Bill   Done
8 Update OAuth extension to support configuration via web API, including documentation updates
  • phab:T257982 (Update the OAuth extension to support the API Portal)
 Bill   Done
9 Create a MediaWiki extension ("WikimediaApiPortalOAuth") that provides a simplified user interface for managing OAuth 2.0 consumers through the API Portal, matching the provided design, and update documentation Cindy: UI

Bill: API

  Done
10 Draft documentation for the API portal Alex   Done

Phase 2: Beta testing

Prerequisites:
  1. WikimediaApiPortal skin  Done, WikimediaApiPortalOAuth extension  Done, and updates to OAuth extension  Done are code complete
11 Deploy WikimediaApiPortal (phab:T257959) and WikimediaApiPortalOAuth to api.wikimedia.beta.wmflabs.org Cindy   Done
12 Enable feature flag for new endpoints in the OAuth extension on beta Cindy   Done
13 Configure and test permissions on beta site (phab:T249834) Alex   Done
14 Test WikimediaApiPortal skin on beta site Alex   Done
15 Test WikimediaApiPortalOAuth extension and interactions with OAuth extension on beta site with beta-Meta Technical: Bill
Design: Alex		   Done

Phase 3: Private launch

Prerequisites:
  • Completion of Phase 2
  • Viable Envoy configuration in Kubernetes (See phab:T246945#6274493)
  • Security reviews for WikimediaApiPortal skin (phab:T246949)  Done, WikimediaApiPortalOAuth extension (phab:T254947)  Done, and updates to OAuth extension (phab:T254948)  Done are complete
16 Deploy WikimediaApiPortal and WikimediaApiPortalOAuth to api.wikimedia.org Cindy   Done
17 Configure permissions on api.wikimedia.org to enable private launch (Homepage and login page are publicly visible; other content is private) (phab:T249834) Cindy   Done
18 Add content to api.wikimedia.org, review, and iterate Alex   Done

Phase 4: Content testing

Prerequisites:
  • Completion of Phase 3
  • Completion of phab:T235276 (Client Developer uses MediaWiki REST API) and phab:T246265 (Client Developer uses Wikifeeds API)
19 Enable feature flag in the OAuth extension to enable new functionality Cindy   Done
20 Test tutorials and code samples Alex   Done

Phase 5: Public launch

Prerequisites:
  • Completion of Phase 4
  • Public launch of API Gateway (phab:T255032)
  • Performance reviews for WikimediaApiPortal skin (phab:T252462)  Done, WikimediaApiPortalOAuth extension (phab:T254950)  Done, and updates to OAuth extension (phab:T254951)  Done are complete
21 Update permissions to allow public read access to all content on api.wikimedia.org Cindy   Done
22 Work with engineers to plan, draft, and review documentation for implementers of the API Gateway (phab:T251440) Hugh   Done wikitech:API Gateway
23 Pass end-to-end testing for the Gateway, Portal, and OAuth extension in production (phab:T268257) Alex   Done
24 Announce launch to the wider movement (phab:T268257) Alex Postponed following product management handoff

Documentation for end-users of the API (API Portal)

edit

Requirements

edit

Documentation requirements

edit
  • How to use the api.wikimedia.org API
  • How to use the OAuth 2.0 flow with the api.wikimedia.org API
  • Rate limiting policies
  • Links to other Wikimedia APIs and resources
  • Visual design similar to the prototype
  • Shared Wikimedia account login

OAuth 2.0 client management requirements

edit
  • Developers can create, list, and disable OAuth 2.0 clients using a simplified interface
  • Visual and interaction design similar to the prototype

Scope

edit

MVP scope

edit
  • Manually created API reference docs and guides in English
  • Editing restricted to a docs-editors group while we stabilize the content and set up contribution flows (See phab:T249834)
  • User feedback via talk pages and “Was this page helpful?” gadget
  • OAuth 2.0 client management interface
  • Strategy for reducing duplication between the the API portal and the existing docs for the MediaWiki REST API and Feeds API.
  • Pageview data for the API portal

Future scope

edit
  • API sandbox
  • Translate extension
  • Contribution flow for volunteers
  • Automated reference docs integrated with translatewiki
  • Asynchronous review process for contributors

Out of scope

edit
  • Updates to the UI of the OAuth special pages on Meta

Implementation strategy

edit
  • Use MediaWiki as the content platform
  • Implement the visual design using a new MediaWiki skin, configured to look similar to the prototype. Certain items such as the behaviors/flyouts may differ since they would use styles from Bootstrap. In the future, the visual style can be implemented using the improved Vector skin.
  • Create an extension that provides a simple UI to create simple OAuth 2.0 clients as shown in the prototype.
    • The new extensions will only be installed on the API Portal.
    • The new extension will interact with the OAuth extension installed on Meta through an API.
    • The new extension will be named WikimediaApiPortal and could also contain other UI elements needed for the API Portal.
  • Attempt to limit any additional libraries necessary for the UI.

Naming and location

edit

Proposed location: MediaWiki instance at:

  • api.wikimedia.org (preferred)
  • Alternatives:
    • developer.wikimedia.org (could be confusing due to the fact that these docs won’t have the comprehensive scope of a Wikimedia Developer Portal)
    • restapi.wikimedia.org
    • apiportal.wikimedia.org
    • webapi.wikimedia.org
    • apidev.wikimedia.org
    • apidoc.wikimedia.org
    • dev.wikimedia.org (exists as a redirect to a page on mediawiki.org)

Considerations

edit

Due to the maintenance effort required to sustain a new public wiki, we should seek to balance the design requirements of this project with the principles and processes of the Wikimedia movement.

Open questions

edit

Prototype

edit

View the API Portal prototype here, including:

Documentation for implementers of the API Gateway and rate limiting system

edit

Documentation requirements

edit
  • Overview of the implementation and configuration of the API Gateway and rate limiting system
  • How to manage rate limits
  • How to add and remove APIs from the Gateway

Implementation strategy

edit

Proposed location: Wikitech TechOps docs

Objectives and metrics

edit
Create a great developer experience for the Wikimedia API
Metric: Qualitative feedback from developer interviews
Make it easy for developers to get started
Metric: Percentage of visitors to the site who register an app
Metric: Percentage of registered apps with at least one API call
Make it easy for developers to build apps
Metric: Percentage of users with registered apps that have an app with more than five API calls
Retrieved from "https://www.mediawiki.org/w/index.php?title=Core_Platform_Team/Initiative/API_Gateway/Documentation_Plan&oldid=5895528"