Technical debt/Map
This page is currently a draft.
|
Here's a quick review of the major pain points and areas of on-going migration and change in the MediaWiki technical ecosystem
Theme | Action needed | Task | Criticality | Challenge | Status |
---|---|---|---|---|---|
- Key
- 🟢 – Actively being worked on by staff as part of the Annual Plan
- 🟡 – Actively being worked on by staff, but not as part of the Annual Plan
Work that is planned/hoped-for but currently on hiatus waiting for other things is not listed as "being worked on".
Technical debt by conceptual theme
editFix things that are significant risks to production
edit- As well as the well-understood short-term reactive work that occurs, there are areas where our systems are currently unsustainable, and which need to be addressed or at least ameliorated to de-risk. The risk horizon of these areas varies from months to years, depending.
- Reduce load and growth on the production systems
Reduction in complexity and tightly-coupled code
edit- MediaWiki is a Labyrinthine monolith that has developed over more than two decades, which makes it hard to learn, hard to reason about, and hard to alter – changes in one area can often break or have other unexpected results elsewhere. This is an area where a lot amount of work has been done, especially by a handful of volunteers and staff, and yet there remains lots more to do.
- (T376615) Migrate away from heavy classes to lighter-weight, smaller ones
- Language
- (T376565) Split out retrieving language code from Language object
- Title
- Switch uses of Title to PageIdentity etc., e.g (T278459) Switch uses of Title to PageIdentity where simple
- User — (T231930) Replacement of User with Authority
- (T271463) Migrate uses of PermissionManager to Authority
- WikiPage — Switch uses of WikiPage to PageStore
- Language
- Make code more isolated and easy to test
- Drop global configuration reading/writing
- (T159283) Drop global non-configuration objects
- (#dependency_injection/) Migrate all code where possible to use dependency Injection for better isolation and testability
- (T20654) Refactor EditPage.php to split logic from UX concerns
Address developer productivity issues
edit- Connected to but distinct from the complexity/connectedness issues from tightly-coupled code, it's vital that we have high-quality tools and engineering processes that make it easy to do the right thing, to increase system stability, ecosystem depth, and developer productivity in general.
- Test our code better and more consistently to give greater confidence
- Upgrade our CI tools to the current versions, and consolidate on only having one at once
- …
Remove use of two technical systems where one will suffice
edit- There are many technical systems that have been over-taken by other systems but the replacement has stalled (or in some cases, never been started). Maintaining both is an unnecessary burden and on-boarding/complexity cost.
- Roll-out wider use of newer, better systems (and ultimately remove the old ones)
- 🟢 Replace the old parser with Parsoid
- 🟢 Replace RESTBase with the REST API
- (T341775) Discourage, deprecate and stop using Xml methods for building HTML markup
- Switch all Action API users we control to requesting JSON in formatversion=2
- (T338439) etc.
- Generally: come up with a deprecation policy for the Action API
- to allow us to turn off
formatversion=1
eventually - to allow "safe" changes to API results, notification of clients, etc.
- to allow us to turn off
- (T167246) Complete the migration from "user" to "actor"
- 🟡 (T28741) Migrate the file tables to a modern, scalable database structure
- Drop the old dynamic extension registration system, rely on extension.json only
- (T98668) Migrate all known extensions and skins to static extension registration
- …
- (T293710) Switch from having two JSON Schema libraries in PHP to only one
- (T297498, T346829) Use a standard JSON codec
- (T278278) Namespace all our PHP classes so we can use PSR-4 to load them more easily
- 🟡 (T166010) Namespace all of MediaWiki core's PHP classes
- (T198901) Migrate all production services from bare metal to Kubernetes
- 🟢 (T290536) Migrate all production MediaWiki traffic to Kubernetes (aka "mw-on-k8s")
- Switch front-end code to all use Codex, not various systems
- (T281930) Switch mobile from bespoke code to Codex
- Switch Visual Editor from OOUI
- 🟢 Gerrit -> GitLab migration
- 🟢 (T343098) Migrate direct SQL queries to proper API
- (T237773) Make wikitech.wikimedia.org a regular production wiki
- …
Remove / reduce use of old user-facing features from production, ideally migrating their use to newer ones
edit- There are many features that have been developed over the past quarter-century and which are no longer things we wish to support, either because they never achieved their full vision, their work was over-taken by other work, or they proved to be less popular than appropriate for the level of usage they attracted. Maintaining them is generally a minor burden individually, though collectively larger, but their true cost occurs in emergencies or when they finally become so obsolete that they block wider work, and is borne mostly by ad hoc teams.
- (T158181) Consolidate our mobile and desktop Web experiences
- (T65504) Consolidate our mobile and desktop media overlay
- …
- (T49145) Drop jQuery UI
- (T100270) Migrate code using jQuery UI to Codex
- (T332022) Undeploy StructuredDiscussions (Flow)
- 🟡 Migrate uses of StructuredDiscussions to regular discussion pages
- (T350164) Undeploy LiquidThreads
- 🟡 Migrate uses of LiquidThreads to regular discussion pages
- Undeploy EasyTimeline
- (T137291) Migrate uses of EasyTimeline to Graph extension or similar
- 🟡 Fix the Graph extension
- (T137291) Migrate uses of EasyTimeline to Graph extension or similar
- (T347972) 🟡 Undeploy MachineVision
- (T161553) Undeploy OpenStackManager
- (T290759) Undeploy VipsScaler
- (T344534) Undeploy wikihiero
- (T318522) Reduce complexity and brittleness of PageTriage
- (T340117) Rewrite PageTriage toolbar in Vue
- (T277883) Reduce complexity and brittleness of FlaggedRevisions
- (T37704) Move use of inline style attributes to TemplateStyles
- (T306043) Replace api.php as a user-facing front-end with the API Gateway
- Migrate use of SpamBlacklist into AbuseFilter and undeploy
- Migrate use of TitleBlacklist into AbuseFilter and undeploy
- (T224922) Undeploy Collection extension and PDF render service
- Remove the use of
$wgRawHtml
on donatewiki/internalwiki/thankyouwiki/collabwiki - [Controversial] Switch from having four-ish template systems (simple transclusion, ParserFunctions, Wikifunctions, Scribunto, plus a number of ad hoc partial re-implementations used for UX messages and in client-side JS) to fewer-than-that
- (T125073) Be able to rename wikis where they use old or inaccurate language terms.
- Migrate use of core parser functions and the ParserFunctions extension to Scribunto or Wikifunctions and undeploy
- …
Keep our technology stack current to keep up deployability, avoid security issues, and retain pace
edit- XYZ
- Migrate production from Debian bullseye to bookworm
- (T291916) Migrate from Debian buster to bullseye
- Migrate production to PHP 8.2 or newer
- (T189767) Migrate Scribunto to a more recent/supported Lua (5.4?)
- (T364779) Migrate production Node services to Node 20
- 🟡 Replace the Graph extension
- …
Complete major feature development and land in a "good" state that is used and supported
edit- There are several major, partially-completed technical changes which should be disposed of in some form. This might be to complete the original vision, park them in a more complete form, or decide they will never happen.
- (T174022) Multi-Content Revisions
- (Mostly UX work, not well-tracked)
- (T96384) Use MCR for image pages
- Use MCR for TemplateData
- Use MCR for language converter glossaries
- (T215306) Structured Data on Commons
- (T31272) Gadgets 2.0
- (T43716) Have Parsoid do language script conversions, rather than re-using the old Parser
- Real time collaboration within Visual Editor
- Extend/re-write Content Translation to provide a parallel translation tool that can make updates after the initial page creation
- TemplateData
- …
Work on new technical improvements for planned / desired future features / capabilities
edit- There are a number of proposed technological changes that would enable significant user-facing features and/or capabilities, but are not currently resourced.
- Provide a Server-Side Rendered form of Codex for server-built pages and non-JS clients
- (T282585) Asynchronous Content Fragments
- Switch wikitext parsing into "incremental" mode, significantly reducing load and increasing speed
- Export MediaWiki's language system as a library in multiple stacks, to power the Web's language tools wherever
- Replace MediaWiki's language converter with industry-standard tools (reuse of tools known by academic linguists and/or libicu/OpenCC and/or a NLP/machine translation based approach, glossaries in MCR, etc)
- Progressively alter wikitext for consistency
- Provide a new wikitext syntax more suitable for discussions
- [Controversial] (T90914) Provide a new wikitext syntax for semantically including illustrations, to replace the current directive-based one.
- [Controversial] Global Templates
- (T149667) Provide readers and/or editors the ability to author/use annotations on articles, enabling inline comments, pronunciation markup for text-to-speech (Wikispeech), translation correspondences, citation regions, etc
- Mark type of MediaWiki messages (wikitext, plain text, raw html, …) to ensure they are authored and used appropriately.
- Security-relevant w/r/t stronger checks for double-escaping, use of raw html, etc.
- [Controversial] Unify databases between projects to make cross-wiki initiatives easier, for example dependency tracking between updates in multiple language versions of the same article (more).
- [Controversial] (T113004) More flexible revision mechanism in core to integrate flagged revisions, unsaved stashed edits, proposed edits in the Draft namespace, storing unresolved edit conflicts, session checkpointing, etc.
- …