MediaWiki Product Insights/Reports/May 2024

Hi All -

Welcome to the monthly MediaWiki Insights email!

We’re beginning this edition with a big thanks to Sportzpikachu, Bill (Wbm1058), xtex, Suffusion of Yellow, TuukkHa, theknightwho, Akiel, Maunikashekar, DreamRimmer, Seawolf35, Likibp, NicoleLBee, SIri and Weebney who got their first patch in MW core or Wikimedia deployed extensions and services merged during the month of May! Congratulations :-)

Goal accomplished: 20% increase in numbers of authors who have submitted more than 5 patches to MediaWiki core

A key goal in this years’ annual plan of the Wikimedia Foundation is to increase the number of authors who have committed more than 5 patches to MediaWiki core. In May 2024 we crossed the magic bar of a 20% increase (baseline = 70, the number of authors who have submitted more than 5 patches to MediaWiki core between July 2022 and June 2023).

At the end of May, we were at 86 people with more than 5 patches - which is a 22.85% growth!

The push to the finish line came at the MediaWiki track at the Wikimedia Hackathon in Tallinn. Some people contributed to MediaWiki core for the first time, other people came back to MediaWiki after a pause. The Wikimedia train stats show an impressive increase of patches for the week after the hackathon (481 patches in 70 repos by 106 authors vs 318 patches in 76 repos by 75 authors in the week before the Hackathon), and about 200 out of the 481 patches were for MediaWiki core alone. Deep appreciation for Jeena Huneidi, who was the train conductor that week! You can learn more about the MediaWiki work done during the Hackathon under the corresponding Phabricator project.

We also continued to observe an impressive decrease in median (>30%) and average time to first review data (>50%), and an increase in the number of patches to MediaWiki core compared to last fiscal year. All of these data points are the best that we have had in years and are strong indicators that our development process around MediaWiki is improving.

The health of our platform relies on our ability to maintain the platform, and a key part of that is to share knowledge and enable people in making improvements to the core software.

A big thanks to all the people who contribute to MediaWiki core and invest time in code review and consultancy! Turning the curve around from a decrease to an increase for the first time in years is such a great success, which was only possible thanks to all of you. <3

MediaWiki as a product: Reflections and questions

As shared in the previous MediaWiki Insights email, we wanted to come back with some reflections and insights on the direction of MediaWiki after the presentations given at the Wikimedia Hackathon in May and the MediaWiki users and developers conference in April. To get more information, you can go over the slides as well as the notes in this ticket.

The MediaWiki product strategy is anchored in the larger strategy of the Wikimedia Foundation, which is all about the question of how we can sustain the success of the Wikimedia projects into the future – while the internet around us is changing.

MediaWiki, the software platform and interfaces that allow Wikipedia and other projects to function, is at the heart of that question. What decisions and platform improvements can we make to ensure that the MediaWiki platform is sustainable and can support the creation, moderation, storage, discovery, and consumption of open, multilingual content at scale for the next decade?

This means, for example: the platform needs to scale for very high global read traffic (billions of page views/month) and high global edit traffic (millions of edits/month), which may coincide and spike when there is a current event. It means the platform needs to support a large contributor base and the creation, moderation and discovery of open content in more than 300 languages – content is not only text, but also data, images, or compositions of different types of content. Last, the platform needs to enable teams and technical contributors in a way that we can balance platform and feature needs, and in a way that the customization offering is curated and thus manageable and useful to its users. The use cases and scale of Wikipedia guide the design goal and platform mission.

Manual:What is MediaWiki? describes some of that well: MediaWiki has been built to support and scale for Wikipedia’s needs and use cases (and then all the Wikimedia projects) and decisions are made with that in mind. The flip-side of this is that there are things MediaWiki does not do so well - because it’s not designed for those. One thing we need to get better at is providing clarity about that to users – so that people can make informed decisions whether the software is right for their use case. The release cycles and related communication present an opportunity to do so. And one thing we need to ask ourselves is what our core software architecture should look like and what core concepts the platform needs to support inherently to effectively serve as the knowledge production platform for Wikipedia’s scale and use cases. What can we learn from the use cases served through the extension ecosystem? Are there patterns in what people have tried to enable? Which of those patterns may be something we should enable differently, and what may not be in scope?

The magnitude of the challenges we’re tackling is huge. We started this journey by learning about people's experiences, challenges and hopes, and continue to talk and learn. Explorations around questions such as how Wikipedia’s essential workflows are currently supported through the MediaWiki software ecosystem, or what is our understanding of the purpose of each of the mechanisms available to customize MediaWiki have likewise helped us gain a better sense for challenges and opportunities of the platform. Last, it has been insightful to go back in time and reflect on how it all started (and whether and how we lost focus and track).

Turning lessons learned into actions

A key early decision has been to set a focus on platform sustainability: We invested in contributor retention and growth, allocated a product manager focused on sustainability (Mateus Santos), prioritized work on several multi-year initiatives that are critical for platform sustainability and evolution, designed sustainability goals for the Foundation’s next annual plan (Wiki Experiences key results 5.1 and 5.4), and are currently preparing for the handover of the release responsibility to MediaWiki Engineering. There is a deep correlation between platform sustainability and release cycles, and an opportunity to align better with other infrastructure management tasks (e.g. PHP upgrades) and product strategy.

As we’re making this transition, it is also time to “release” James Forrester and Sam Reed from the release responsibility: A big shoutout to James and Sam for all the work they have done around the releases over the past years and many thanks for all their support, kindness and knowledge transfer as they are handing things over! <3 The security releases will continue to be handled by the Security team.

Building on the research from this year, we also plan to start exploring how we can clarify the relationship of core and extensions better, what should exist (or not) within core, extensions, or the interfaces between them, and how that may translate into architectural needs and decisions. This in itself is a big question and scope, so we’re trying to approach it with small steps – and conduct more research and experiments first (KR 5.2).

As part of KR 5.3, we want to leverage Parsoid's capabilities which implicitly models a page as a composition of well-structured fragments. We aim to use this capability to derive speedy HTML updates on edits to templates and pages, as well as unblock WikiFunctions' integration on wikis whose page renderings are served by Parsoid.

Getting ready for next year’s work: A database to query dependencies in MediaWiki and an overview on MediaWiki’s customisation interfaces

In an earlier MW Insights email, we talked about our work to explore how the MediaWiki software ecosystem currently supports essential workflows on Wikipedia and Wikidata. While we’re still working on another report about this exploration, we want to share two other pieces of work that we believe to be very helpful for the MediaWiki goals under the Foundation’s annual plan 2024/2025 (WE5.1-5.4):

Amir Sarabadani surprised us with a wonderful project at the Wikimedia Hackathon: Huma – a bird’s eye view of MediaWiki. Huma is a postgreSQL database that allows querying dependencies and usage of certain methods in MediaWiki. Want to know which are the most implemented hooks in MediaWiki? - ask Huma. Interested in cycle dependencies? - ask Huma. You can find more information on the project on this page.

MediaWiki’s customisation interfaces: Earlier this year, we started talking about all the different ways that users can build software on top of MediaWiki, and which technique is best suited for what purpose - from templates to Toolforge and from extensions to gadgets. We tried to capture our thoughts with sticky notes and arrows on the wall, to create a “feature development map”. Daniel Kinzler then picked up the ball and started a spreadsheet with all the interfaces we have for customization and automation, and the capabilities and properties each of them offers. After many improvements by engineers and product managers from MediaWiki Engineering, we now have a concise chart that can serve as an overview of the different customization mechanisms, and provide guidance on the advantages and disadvantages of each approach. The chart is available as a PDF on Commons, and as a spreadsheet on Google Drive.

My apologies for the length of this email – there was a lot to share this time! And please join me in thanking all the wonderful contributors and code reviewers in MediaWiki core! :-)

Thanks all for reading! --BMueller (WMF) (talk) 21:40, 6 June 2024 (UTC)