Core Platform Team/Initiatives/API Gateway/Documentation Plan

View the prototype Track this project on Phabricator Share feedback Subscribe to project updates

OverviewEdit

The API Gateway is a CPT initiative to create a rate-limited API proxy at api.wikimedia.org as part of the Platform Evolution program. The MVP scope includes the MediaWiki REST API (including page, media, history, and search endpoints) and the Feeds API, with the intention of adding APIs over time. As of July 24, 2020, the initiative is in the implementation phase. See the initiative page and Phabricator tag.

The purpose of this document is to manage the planning and feedback process for the documentation for this project.

ComponentsEdit

Architecture diagram working draft

MediaWiki
The API Portal will be a MediaWiki instance at api.wikimedia.org (beta: api.wikimedia.beta.wmflabs.org)
WikimediaApiPortal skin (Phabricator project)
A MediaWiki skin providing the look-and-feel of the API Portal
WikimediaApiPortalOAuth extension (Phabricator project)
A MediaWiki extension running on the API Portal that interfaces with the OAuth extension on Meta-Wiki
OAuth extension on Meta-Wiki
Responsible for storing and managing OAuth clients

Implementation planEdit

ID Description Owner Status

Phase 1: Planning and development

1 Finalize development contract Cindy   Done
2 Security approval for the Chameleon skin (phab:T246949) Cindy Cancelled (See phab:T252462.)
3 Collect feedback from Technical Engagement Alex   Done (Notes)
4 Create a wiki instance Cindy   Done
5 Submit request to create a beta wiki instance (phab:T254185) Cindy   Done
6 Create a MediaWiki skin ("WikimediaApiPortal") that matches the provided design, and create documentation for the extension on mediawiki.org Cindy   Done
7 Define technical specification for interactions between the API Portal and the OAuth extension Bill   Done
8 Update OAuth extension to support configuration via web API, including documentation updates
  • phab:T257982 (Update the OAuth extension to support the API Portal)
Bill   Done
9 Create a MediaWiki extension ("WikimediaApiPortalOAuth") that provides a simplified user interface for managing OAuth 2.0 consumers through the API Portal, matching the provided design, and update documentation Cindy: UI

Bill: API

  Done
10 Draft documentation for the API portal Alex In progress

Phase 2: Beta testing

Prerequisites:
  1. WikimediaApiPortal skin  Done, WikimediaApiPortalOAuth extension, and updates to OAuth extension  Done are code complete
11 Deploy WikimediaApiPortal (phab:T257959) and WikimediaApiPortalOAuth to api.wikimedia.beta.wmflabs.org Cindy   Done
12 Enable feature flag for new endpoints in the OAuth extension on beta Cindy   Done
13 Configure and test permissions on beta site (phab:T249834) Alex   Done
14 Test WikimediaApiPortal skin on beta site Alex   Done
15 Test WikimediaApiPortalOAuth extension and interactions with OAuth extension on beta site with beta-Meta Technical: Bill
Design: Alex
In progress

Phase 3: Private launch

Prerequisites:
16 Deploy WikimediaApiPortal and WikimediaApiPortalOAuth to api.wikimedia.org Cindy WikimediaApiPortal  Done
17 Configure permissions on api.wikimedia.org to enable private launch (Homepage and login page are publicly visible; other content is private) (phab:T249834) Cindy   Done
18 Add content to api.wikimedia.org, review, and iterate Alex In progress

Phase 4: Content testing

Prerequisites:
  • Completion of Phase 3
  • Completion of phab:T235276 (Client Developer uses MediaWiki REST API) and phab:T246265 (Client Developer uses Wikifeeds API)
19 Enable feature flag in the OAuth extension to enable new functionality Cindy
20 Test tutorials and code samples Alex

Phase 5: Public launch

Prerequisites:
  • Completion of Phase 4
  • Public launch of API Gateway (phab:T255032)
  • Performance reviews for WikimediaApiPortal skin (phab:T252462)  Done, WikimediaApiPortalOAuth extension (phab:T254950), and updates to OAuth extension (phab:T254951)  Done are complete
21 Update permissions to allow public read access to all content on api.wikimedia.org Cindy
22 Work with engineers to plan, draft, and review documentation for implementers of the API Gateway (phab:T251440) Alex

Documentation for end-users of the API (API Portal)Edit

RequirementsEdit

Documentation requirementsEdit

  • How to use the api.wikimedia.org API
  • How to use the OAuth 2.0 flow with the api.wikimedia.org API
  • Rate limiting policies
  • Links to other Wikimedia APIs and resources
  • Visual design similar to the prototype
  • Shared Wikimedia account login

OAuth 2.0 client management requirementsEdit

  • Developers can create, list, and disable OAuth 2.0 clients using a simplified interface
  • Visual and interaction design similar to the prototype

ScopeEdit

MVP scopeEdit

  • Manually created API reference docs and guides in English
  • Editing restricted to a docs-editors group while we stabilize the content and set up contribution flows (See phab:T249834)
  • User feedback via talk pages and “Was this page helpful?” gadget
  • OAuth 2.0 client management interface
  • Strategy for reducing duplication between the the API portal and the existing docs for the MediaWiki REST API and Feeds API.
  • Pageview data for the API portal

Future scopeEdit

  • API sandbox
  • Translate extension
  • Contribution flow for volunteers
  • Automated reference docs integrated with translatewiki
  • Asynchronous review process for contributors

Out of scopeEdit

  • Updates to the UI of the OAuth special pages on Meta

Implementation strategyEdit

  • Use MediaWiki as the content platform
  • Implement the visual design using a new MediaWiki skin, configured to look similar to the prototype. Certain items such as the behaviors/flyouts may differ since they would use styles from Bootstrap. In the future, the visual style can be implemented using the improved Vector skin.
  • Create an extension that provides a simple UI to create simple OAuth 2.0 clients as shown in the prototype.
    • The new extensions will only be installed on the API Portal.
    • The new extension will interact with the OAuth extension installed on Meta through an API.
    • The new extension will be named WikimediaApiPortal and could also contain other UI elements needed for the API Portal.
  • Attempt to limit any additional libraries necessary for the UI.

Naming and locationEdit

Proposed location: MediaWiki instance at:

  • api.wikimedia.org (preferred)
  • Alternatives:
    • developer.wikimedia.org (could be confusing due to the fact that these docs won’t have the comprehensive scope of a Wikimedia Developer Portal)
    • restapi.wikimedia.org
    • apiportal.wikimedia.org
    • webapi.wikimedia.org
    • apidev.wikimedia.org
    • apidoc.wikimedia.org
    • dev.wikimedia.org (exists as a redirect to a page on mediawiki.org)

ConsiderationsEdit

Due to the maintenance effort required to sustain a new public wiki, we should seek to balance the design requirements of this project with the principles and processes of the Wikimedia movement.

Open questionsEdit

PrototypeEdit

View the API Portal prototype here, including:

Documentation for implementers of the API Gateway and rate limiting systemEdit

Documentation requirementsEdit

  • Overview of the implementation and configuration of the API Gateway and rate limiting system
  • How to manage rate limits
  • How to add and remove APIs from the Gateway

Implementation strategyEdit

Proposed location: Wikitech TechOps docs

Objectives and metricsEdit

Create a great developer experience for the Wikimedia API
Metric: Qualitative feedback from developer interviews
Make it easy for developers to get started
Metric: Percentage of visitors to the site who register an app
Metric: Percentage of registered apps with at least one API call
Make it easy for developers to build apps
Metric: Percentage of users with registered apps that have an app with more than five API calls