User:TBurmeister (WMF)/Sandbox/Ecosystem intro

This page provides an overview of Wikimedia software and infrastructure for new technical contributors. Its goal is to help developers understand the major areas where you can apply your technical skills to help support and grow the Movement.

Core principles and technology

Open source

The Wikimedia technical ecosystem is guided by free and open source software (FLOSS) principles. Contributors around the world volunteer their time to maintain and extend the technology behind Wikimedia projects. This means that the Wikimedia technical ecosystem consists of many unique projects with varying contributing guidelines, design patterns, hosting locations, and code review processes.

Open source values like shared ownership and strong privacy commitments are central to Wikimedia's technical infrastructure, software, and services. For example:

For version control and code review, the Wikimedia Foundation (WMF) uses Git and Gerrit. Git is a free and open source distributed version control system, and Gerrit is a free, web-based collaborative code review tool that integrates with Git.
Wikimedia Cloud Services (WMCS) provides cloud computing infrastructure, shared hosting (platform as a service), and data services for contributors to Wikimedia software projects. You can use these services to host your software tools for the Wikimedia movement, for free, instead of using proprietary, paid, or non-open-source services.

In addition to providing tools and services for technical contributors, WMF maintains core infrastructure like data centers, search, networking, and global traffic routing. You don't need to learn about those in order to contribute to wiki projects, but you should know about the core software that powers Wikipedia and other wikis: MediaWiki.

MediaWiki

Wikimedia projects, including Wikipedia, run on the MediaWiki software platform. MediaWiki is free and open source software, built specifically for collecting and organizing free and open knowledge. It's multilingual, extensible, customizable, and reliable. MediaWiki enables users around the world to edit, organize, maintain, and collaborate on wikis. It manages the display of content like text, photos, formatting, and revisions. MediaWiki is more than just a content management system: it also provides tools to handle spam and vandalism, and other functionality that supports community curation of freely-reusable knowledge on an open platform.

	Cool fact: NASA uses MediaWiki as its wiki software.

Key areas where developers can contribute

Develop MediaWiki, apps, and extensions

MediaWiki core and extensions

The MediaWiki software is developed collaboratively by people from all around the world. The Wikimedia technical community has added many key features to MediaWiki and built a large ecosystem of extensions to the core software. Due to its complexity, contributing to MediaWiki may be more challenging than other types of technical contributions. As a new contributor, focus on features and bugs that only touch a small region of code. Avoid building new MediaWiki skins and extensions: every new extension adds to the software's overall maintenance burden, and can block the launch of new, crucial features. Instead, look into working on one of the many existing extensions.

Extension example: translate text and manage multilingual wikis
The Translate extension enables localization of MediaWiki and translation of every kind of text. It supports web-based translation, proofreading, and collaboration between translators all over the world, while also making back-end integration with source code as easy as possible. Screenshot of a translation page generated by the Translate extension.

Things to know before you start:

Writing code for MediaWiki generally requires knowledge of PHP, MySQL/MariaDB, JavaScript, and CSS.
The MediaWiki codebase is large – don't be overwhelmed by it! When you're first starting, aim to write features or fix bugs that only touch a small region of code.
Follow the MediaWiki development guidelines and best practices for extensions.

To learn more, visit How to become a MediaWiki hacker.

Wikimedia mobile apps

Mobile apps expand the reach of Wikimedia content by allowing users to interact with Wikimedia without using a browser. There are apps for the Android, iOS, and KaiOS platforms, apps for different Wikimedia projects like Commons, Wikipedia, and Wiktionary, and apps for activities like reading, contributing, and more. Many apps have vibrant developer communities and welcome new contributors.

Mobile app example: read Wikipedia without internet access
Kiwix is an offline reader – it makes Wikipedia accessible to people without internet access. It provides content in more than 100 languages from Wikipedia, Wikiquote, Wiktionary, and other online resources. Kiwix development uses Java/Kotlin (Android), Swift (iOS), Qt (desktop), NodeJS (mediawiki offliner) and C++.
Kiwix flyer	Screenshot of Kiwix on iOS

To learn more, visit Wikimedia Apps.

Customize MediaWiki through on-wiki development

Templates and modules

Templates enable one piece of content to appear on multiple wiki pages ("transclusion"). Many wikis rely heavily on templates: there are thousands of them, they're used on almost every page, and they're integral to wiki functionality. You can pass parameters to templates to customize how content is handled in different contexts.

Template example: format blockquotes
The Blockquote template provides formatting for block quotations, and includes preformatted attribution parameters for a quote's author and source. This enables editors to use wikitext markup to easily and consistently style how block quotes display on wiki pages. Screenshot of the Template:Blockquote documentation showing how it works.

You don't need a developer account to edit or create a template, and they're written in wikitext. That makes templates an easy way to get started with customizing wiki behavior.

Modules are similar to templates, but instead of being written in wikitext, they use the Lua programming language. While templates are part of the core MediaWiki application, support for modules is implemented by an extension called Scribunto.

Module example: format complex route maps
The Routemap module supports a special, complex, custom language for displaying rail maps. It is used in thousands of articles about railways in multiple languages. Using Lua enables this module to replace legacy route map templates with a faster and more mobile-friendly implementation.
Screenshot of an article using the Routemap module on Korean Wikipedia.	Example code and rendering from Routemap documentation.

Things to know before you start:

Some template functionality depends on other MediaWiki extensions, CSS, and/or Javascript that must be deployed on a given wiki in order for functions to work.
Templates are a very powerful feature of the MediaWiki software, so you should make edits to complex templates in a sandbox and include test cases to verify that they work as intended.
To lower the risk of mistakes breaking hundreds or thousands of pages, wiki projects may have policies that restrict or "protect" edits to widely-used templates or modules.
The MediaWiki software has parameters that limit the complexity of a page, and the amount of data that can be included. This means there are some limits to what you can do with templates and modules.
Modules should follow the Lua coding conventions.

To learn more about templates, visit Help:Templates. To learn more about modules and how Lua is used on MediaWiki, visit the Scribunto Lua manual.

User scripts and gadgets

MediaWiki supports front-end customization, or "on-wiki" development, to enhance the software's core functionality. User scripts are Javascript and/or CSS that provide additional features on top of MediaWiki. Code lives on the wiki, and is injected from the client side on MediaWiki via user preferences. You can modify your local Javascript and CSS to change the appearance and behavior of your own interface. Individual users can add features for themselves as long as they follow expectations and guidelines around doing so. If you build something that others can use, you can share your scripts informally, or work with a wiki community to investigate making your customizations available to other users and projects as a "gadget".

User script example: display calculated content

This user script calculates the reading time of an article and displays it on the page.

var numWords = $("#mw-content-text > div").text().split(" ").length;
var headerWords = $("h1").text().split(" ").length;
var totalWords = numWords + headerWords;
var timeInMinutes = totalWords / 200;
var header = $("h1").text();
$("h1").text(header + " (it will take you " + timeInMinutes + " mins to read this article)");

Screenshot of English Wikipedia article displaying reading time.

Gadgets are user-generated customizations that are available for installation on a wiki, via the Gadgets MediaWiki extension. Many gadgets originated as user scripts that became widely-used on their home wiki, and admins wanted to make them easier for everyone to find and use. Admins can configure which gadgets are available and enabled by default for an entire MediaWiki site or skin.

Gadget example: help editors categorize content
HotCat helps users easily remove, change, and add categories to Wikipedia pages. It has a suggestions list that proposes existing categories for auto-completion.
Screenshot of HotCat functionality on French Wikipedia.

Gadget example: help moderators fix content
This gadget deployed on Persian Wikipedia helps content moderators fix Arabic letters, grammar, numbers, URLs, article structure, and more.
Screenshot of gadget JavaScript from Persian Wikipedia.

Things to know before you start:

For user scripts to work, $wgAllowUserJs variable in the site's configuration file needs to be set as true.
For gadgets to work, the Gadgets and CodeEditor extensions must be installed.
Gadget code should follow the JavaScript coding conventions.
Each wiki community determines how and whether to review gadget code. The process for making a gadget available to all users of a wiki is not streamlined. It requires a series of steps, including recruiting an interface administrator and providing deployment instructions to them.
On Wikimedia wikis, you must be an interface admin to edit gadgets.

To learn more about user scripts and gadgets, visit "Gadget kitchen".

Build tools and bots to help Wikimedia projects

MediaWiki APIs provide direct access to the data contained in the MediaWiki database. This includes wiki content, metadata, analytics data, and more. You can use these APIs to build apps that contribute content or new functionality to the Wikimedia ecosystem. The API modules provide programmatic access to much of the same functionality provided by the MediaWiki web interface.

Bots

Bots are automated programs that help wiki projects by doing crucial but repetitive tasks. Bots save human effort by correcting missing or extra spaces, suggesting articles to newcomers, fixing broken links, and more. Bot automation supports Wikimedia volunteers by allowing them to spend more of their time on tasks that require human intelligence, like writing articles, finding sources, fact checking, and synthesizing multiple points of view.


Bot example: fight link rot
InternetArchiveBot (IABot) is an advanced bot that identifies and replaces broken external links. It also completes reference templates and adds links to resources, where appropriate. This bot is operated as a service of the Internet Archive in partnership with the Wikimedia community. As of October 2022, IABot is running on 150+ Wikimedia wikis. IABot also has a browser-based interface (web app) that provides easy access to some of the bot's services and configuration.

Bots can request pages and submit edits to wikis just like how human users do the same tasks in a web browser. Because bots access pages the same way people do, they can get caught in edit conflicts, have page timeouts, or run across other unexpected complications while requesting pages or making edits. Since bots generally complete high-volume work, you should consider these situations when writing a bot.

Things to know before you start:

Each Wikimedia project has a policy that governs the creation and implementation of bots. For example, see English Wikipedia's bot policy. You must understand and follow the bot guidelines established by the wiki project you're developing for.
Before creating a new bot, explore the bots that are currently active on the wiki project, and how you can contribute to them.
Multiple programming languages, like Haskell, Java, Perl, PHP, Python, and Ruby have libraries available for creating bots. Pywikibot is a commonly used Python package developed specifically for creating MediaWiki bots.

To learn more about bots, visit Manual:Creating a bot.

Tools

Tools are software applications that help people working on Wikimedia projects. Tools can do all kinds of tasks, like help editors discover content to improve, simplify edit-a-thon planning, visualize wiki content, help fight vandalism, and much more.

Tool example: improve images
CropTool is a MediaWiki tool for cropping images at Wikimedia Commons and other Wikimedia sites. It supports JPEG, PNG, TIFF and (animated) GIF files, and can also extract single pages from DJVU and PDF files as JPEG for cropping.
Screenshot of the CropTool menu item on a wiki.

The MediaWiki Action API gives you programmatic access to content on any Wikimedia wiki. You can use the Action API to search and display wiki pages, get media files, edit pages, and manage wikis. Client libraries implementing the MediaWiki API are available in many languages, including Python and JavaScript.

Tool example: connect Wikipedia articles and Wikidata items
The Duplicity tool picks a random wiki article that doesn't have an associated Wikidata item, and offers some possible matches on Wikidata. It makes it easy to add structured data to capture the topics, concepts, or entities discussed in an article.
Screenshot of the Duplicity tool interface.

To explore more Wikimedia tools, visit Toolhub.

Things to know before you start:

You need a Wikimedia Developer Account to host tools on Toolforge, and commit code to Wikimedia's Git repositories.
Toolforge tools must be open source software licensed under an OSI approved license, and all projects must follow the Wikimedia Cloud Services Terms of Use.
Take advantage of client libraries for the MediaWiki API in many programming languages.

Join the Wikimedia tech community

A great way to find ways to contribute is to get connected to the technical community. You can meet people and discuss development work on various communication channels and at events like Hackathons. The Wikimedia Foundation also offers outreach programs to mentor new developers.

Attendees at the Wikimedia Hackathon in Prague, 2019.

Get started as a new contributor

Visit the New Developers page to learn more about getting started with development and contributing to Wikimedia open source.
Use the Developer Portal to find curated resources and tutorials for your area of interest.