Wikimedia Engineering/2016-17 Goals/New MediaWiki-focused team/Platform proposal

MediaWiki Platform team

This is a draft proposal for a new team consisting of seasoned MediaWiki developers, focusing on making MediaWiki more useful as a platform for developing new functionality desired by the Wikimedia movement. The proposal builds on (but does not necessarily reflect any consensus of) an officewiki discussion and a WikiDev2016 unconference session about the problems created by the disbanding of the MediaWiki Core team, without anyone picking up its key responsibilities. At that session, held with the participation of many tenured WMF developers, there was wide agreement that the reorganization of WMF Engineering in 2016 spring created systemic problems with the maintenance and evolution of MediaWiki, and those cannot be effectively addressed without better institutional representation for MediaWiki. This proposal is a follow-up.

Problem statement

MediaWiki is the centerpiece of the Foundation's software stack, and serves as a platform or an API provider for most new features the Foundation is working on; for such a fundamental piece of software, the amount of resources dedicated to its improvement are disproportionately low. There is a widespread feeling amongst people relatively new to MediaWiki development that MediaWiki is outdated or hard to develop for; at the same time, experienced MediaWiki developers feel that the software is deteriorating due to insufficient maintenance efforts and product teams adding technical debt and cutting corners. Those two problems reinforce each other and form a vicious loop.

If developers find the main tool in the WMF's software toolset difficult to use, that's a fundamental strategic problem that needs to be addressed. If the people who are the most knowledgeable about MediaWiki feel that it's not on a sustainable path, that should also be cause for alarm. To fix those issues, a dedicated team is needed.

Team scope

The team would focus on MediaWiki as a collaborative document editing platform that enables development of user-facing features which serve the Wikimedia mission, and on improving other developers' ability to work with it. More specifically, that would include three focus areas:

• Refactoring MediaWiki into loosely coupled, well-documented components with clean interfaces and easy-to understand interactions, so that developers working on some aspect of MediaWiki only have to deal with the amount of complexity that's necessary for their task. This could involve things like the librarization project, evolving MediaWiki towards a cleaner separation of UI and backend logic, turning web APIs (often called, for lack of a better option, from internal code) into internal APIs with a thin web frontend, or rearchitecting content handling to work with structured data instead of text blobs.
• Directly support other developers working with core. This could involve code review for key changes, consulting or even more intensive training (such as temporarily embedding a team member in an audience team, or vice versa).
• Improving developer experience by making MediaWiki's developer-facing features more intuitive and convenient to use (e.g. better logging and debugging features, self-contained documentation sandbox for web APIs, IDE support).

Due to the nature of the above areas, this would typically mean focusing on MediaWiki core and on the backend, but neither would be a hard rule. Conversely, a task would not necessarily be within the scope of the team just because it is implemented in the MediaWiki core backend. (Some examples: OAuth is not part of core but is very much about allowing developers to build on MediaWiki as a platform. ResourceLoader is frontend, but is similarly part of the "platform offering". Special pages are part of core, and mostly backend logic, but most of them have nothing to do with providing a platform.)

Q&A

Where would the team belong on the org chart?

Like other teams which are one step removed from the end users and serve them by enhancing the productivity of other developers, it would probably be a part of the Technology department.

How is this related to the old Platform Engineering?

By similarity of name only. Platform Engineering was the predecessor of the current Technology department; this team would only be a small part of it.

How is this related to the old MediaWiki core team?

This proposal was initiated in a discussion about the old core team, and problems caused by disbanding that team. The proposal attempts to continue that team's successes while learning from its faults. It posits that the main problem was lack of focus due to an ill-defined and overly wide scope, and tries to address this by having a smaller and tighter scope and by defining an audience (Wikimedia developers, especially the ones in the product verticals) whose needs would orient the team.

How is this related to support of third parties using MediaWiki?

The team's mandate would be to serve the Wikimedia mission by evolving MediaWiki as a platform. That means that defining and prioritizing goals would be based on the needs of Wikimedia developers, not third-party developers. (In practice that would mostly mean WMF teams, but include other developers focusing on the Wikimedia sites, such as WMDE's Wikidata team, and volunteer developers.)

The team would factor third-party usage into its decision making in two ways: by trying to preserve backwards compatibility, where economical (as a relatively cheap way of preserving the health of the open-source ecosystem around MediaWiki, and thus retaining the free support we receive from it), and by surveying third-party use cases before deciding on interfaces (good interfaces consider the widest range of use cases because which ones will be important for the WMF is hard to predict years in advance, and redesigning interfaces is much more costly than designing them inclusively in the first place).

An argument can be made that MediaWiki itself should be seen as a product that directly serves the Wikimedia mission, as the most advanced free software for open, collaborative knowledge curation, that third parties can use to collect and share free knowledge; and thus addressing third-party MediaWiki needs should be part of the WMF's activities. While that might be true, including it in the Platform team's mandate would threaten the clarity of its scope and the simplicity of its prioritization process, so a different arrangement should be sought for third-party support. (That said, third parties are still expected to benefit a lot from the existence of the team because the architecture-level needs of developers inside and outside the Wikimedia movement are not that different.)

Would the team "own" MediaWiki?

Being perceived as a "gatekeeper" of MediaWiki was one of the problems with the old core team that this proposal tries to learn from. While helping other developers work with MediaWiki would be the main goal of the Platform team, MediaWiki core would not be the team's "product", and the team would have no special say in core changes. As before, those decisions would be made through ArchCom and the code review consensus process. The team would rely on those processes to get authorization for its own plans.

Shouldn't this be done by the product verticals?

As the products needed by the Wikimedia movement grow in number and complexity, MediaWiki developers need new utilities and abstractions to remain efficient. The vertical structure has been set up to minimize cross-team collaboration and allow a strong focus on key user needs and metrics; that's great for feature development but not so much for common utilities and interfaces, which tend to end up siloed to a particular product or department, and designed with a narrow use case in mind. At best that results in duplication of effort and wasted resources; at worst in a fractured, inconsistent and unintuitive codebase.

Wouldn't such a team "lock in" the Foundation's strategy to MediaWiki even if in the future some other approach might make more sense?

Different people might have different long-term visions about the Wikimedia movement's software stack, but in the mid-term, MediaWiki will remain the centerpiece of it. Even if we decide to eventually replace MediaWiki (and no one with any level of familiarity with our software stack seems to advocate for that), that would require a huge shared effort over several years, and would involve a step-by-step process of replacing components - exactly the kind of refactoring the organization struggles with due to the lack of attention on MediaWiki as a platform. So, whatever the long-term strategy, the need for such a team right now is clear.