Open main menu

Upstream projects

MediaWiki developers and users[1] rely on several upstream projects for their architecture or processes. At the same time, they develop software projects used by others, which are downstream to them. These links define our location in the free software map, our neighbor communities, and the flows of feedback, patches, and contributors.

Let's identify our upstream projects, our neighbor communities, and the contributors that connect us with them.

MotivationEdit

This is a first step to improve our relations with other communities, to increase the contributions received, and our influence in the projects that matter to us.

Our mid-term goals include:

  • identify the projects where we want to see significant development, to the point of sending patches as well
  • identify the communities where Wikimedia should be regularly active and heard
  • identify the people in the Wikimedia and upstream communities that know each other and act as bridge
  • identify organizations and events we should get in touch and be part of
  • get involved in bigger development efforts regularly, become a regular FOSS player

Invented hereEdit

Projects we maintain that we want others to use and contribute to.

Own libraries used by MediaWiki core
Name Description Language Repository Issues Main contacts
MediaWiki The collaborative editing engine that runs Wikipedia. PHP Diffusion Phabricator MediaWiki Platform team, Tim Starling, Brion Vibber
OOjs Library for working with objects in JavaScript. JavaScript Gerrit Phabricator Timo Tijhof, Roan Kattouw
OOUI OOjs-based UX toolkit. JavaScript, PHP Gerrit Phabricator James Forrester, Bartosz Dziewoński
CSSJanus Convert stylesheets from left-to-right to right-to-left. PHP GitHub GitHub Timo Tijhof, Roan Kattouw
CLDRPluralRuleParser Parse plural rules specified in the CLDR project. PHP Gerrit Phabricator
RemexHtml HTML5 parsing library to replace Tidy. PHP Gerrit Phabricator Tim Starling
wikimedia/cdb Pure PHP replacement for dba functions. PHP Gerrit Phabricator Tim Starling, Chad Horohoe
wikimedia/at-ease Safe replacement to PHP's @ for suppressing warnings. PHP Gerrit Phabricator Tim Starling, Legoktm
wikimedia/utfnormal Unicode normalisation routines. PHP Gerrit Phabricator Brion VIBBER
wikimedia/ip-set Match IPs against CIDR sets. PHP Gerrit Phabricator Brandon Black
wikimedia/running-stat Efficient way to compute statistics over many observations. PHP Gerrit Phabricator Performance Team
wikimedia/base-convert Convert an arbitrarily-long string from one numeric base to another PHP Gerrit Phabricator
ScopedCallback Automatically run code once a reference is out of scope. PHP Gerrit Phabricator
WaitConditionLoop Poll state changes while executing other code concurrently. PHP Gerrit Phabricator Performance Team, Aaron Schulz
WrappedString Compact strings with common start/end pairs. PHP Gerrit Phrabricator Performance Team, Timo Tijhof
composer-merge-plugin Merge multiple composer.json files at runtime. PHP GitHub GitHub Bryan Davis
testing-access-wrapper Helper library for unit tests. PHP Gerrit Phabricator
php-session-serializer Abstract session_ functions and related INI settings. PHP Gerrit Phabricator Brad Jorsch
purtle Fast streaming RDF serializer PHP Gerrit WMDE, Stas Malyshev
Own libraries used by Wikimedia-deployed extensions and services
Name & URL Description Language(s) Repository Bug tracker Main contacts
Parsoid Two-way MediaWiki wikitext↔HTML parser. Node.js Diffusion Phabricator (under "Parsoid") Subbu Sastry,

C. Scott Ananian

VisualEditor Rich HTML editor. JavaScript Gerrit Phabricator James Forrester, Ed Sanders
UnicodeJS String manipulation based on Unicode standards (word breaks, graphemes, character classes). JavaScript Gerrit Phabricator David Chan, James Forrester, Ed Sanders
wikimedia/css-sanitizer A CSS tokenizer, parser and grammar matcher. PHP Gerrit Phabricator
wikimedia/equivset Detect visually equivalent UTF-8 characters. PHP Gerrit Phabricator Brion VIBBER
Milkshake jQuery.uls, Language selection JavaScript GitHub GitHub Santhosh Thottingal
jQuery.IME, input method editor library JavaScript GitHub GitHub Santhosh Thottingal
jQuery.i18n, Internationalization library JavaScript GitHub GitHub Santhosh Thottingal
jQuery.webfonts, Font embedding capabilities JavaScript GitHub GitHub Santhosh Thottingal
Offline content tools mw-ocg-bundler, article/resource spider tool Node.js GitHub GitHub C. Scott Ananian
mw-ocg-latexer, article PDF conversion tool Node.js GitHub GitHub C. Scott Ananian
textcat Text categorization library in PHP PHP Gerrit Phabricator Trey Jones
Other projects and libraries maintained by the MediaWiki engineering community
Name & URL Description Language(s) Repository Bug tracker Main contacts
translatewiki.net Where we organise localisation of MediaWiki. PHP phab:diffusion/GTWN/ #translatewiki.net in Phabricator, translatewiki:Support Nikerabbit (founder), Siebrand (co-maintainer), Raymond (MediaWiki updates responsible)
Pywikibot Python Wiki Robot Framework to automatize tasks Python Diffusion Phabricator (under "Pywikibot") xqt
Orgchart tool Generates organisational charts. JavaScript, Node.js Diffusion Phabricator (under "Utilities") MarkTraceur
Bingle Pipes bugs form Bugzilla into tools like Mingle and Trello; obsolete after migration to Phabricator Python Github Github User:Awjrichards

More stuff (hundreds of projects) by fellow MediaWiki developers and Wikimedians is also listed at:

ComponentsEdit

Projects that contribute to the architecture of MediaWiki, key extensions, and applications. They are included in the downloadable packages of key Wikimedia projects, or they are identified as required dependencies.

Upstream components
Name & URL Description Language(s) Repository Bug tracker Main contacts
GNU diff and diff3 Powers recent changes, history and undo button (i.e. most editing features).
Elasticsearch Distributed real-time search and analytics engine. Java GitHub GitHub
jQuery Lots of DOM traversing, and other DOM-related helpers, for JavaScript. JavaScript GitHub GitHub (jQuery Foundation), Timo Tijhof
OpenStack Nova OpenStack compute service. Python OpenStack Gerrit Launchpad Ryan Lane
OpenStack Glance OpenStack image service. Python OpenStack Gerrit Launchpad Ryan Lane
OpenStack Keystone OpenStack identity service. Python OpenStack Gerrit Launchpad Ryan Lane
OpenStack Swift OpenStack object storage service. Python OpenStack Gerrit Launchpad Wikimedia SRE
Puppet Configuration management system. Ruby GitHub Jira Wikimedia SRE
SaltStack Orchestration and remote execution system. Python GitHub GitHub Ryan Lane
Unihan Database For Chinese language conversion Used as database
Smart Common Input Method For Chinese language conversion Used as database
libtabe For Chinese language conversion Used as database
ImageMagick general purpose image scaling C No repo ImageMagick.tar.gz Forum
Lua Scripting language C
ACE Web based code editor JavaScript, HTML GitHub GitHub User:TheDJ (has signed contributor license)
VIPS Image scaling of large images (VipsScaler) GitHub GitHub User:Bryan
libRsvg PNG renderer for SVG images C GNOME git GNOME Gitlab
MathJax Math rendering in HTML/SVG/MathML; used by Math extension JavaScript GitHub GitHub
FFmpeg / libav Video conversion and scaling C ffmpeg / libav ffmpeg / libav
MWLib mediawiki parser and converter Python GitHub GitHub
Pygments.org Syntax highlighting Python Bitbucket Bitbucket
Unicode CLDR (Common Locale Data Repository) All we manage to take: language names, plural rules, language-territory information, ... [1] Nemo (CLDR Survey Tool manager)
CLDRPluralRuleParser Plural support based on CLDR JavaScript GitHub GitHub Santhosh Thottingal
Apertium Free software, rule-based machine translation; used by Translate, ContentTranslation and others. Soureforge [2] Francis Tyers, Firespeaker, Unhammer
Ceph Distributed storage system C++ GitHub Ceph
Cortado Media streaming Java applet Java xiph [3]
GlusterFS Distributed file system [4]
HHVM Alternate PHP engine. C++ GitHub GitHub (Facebook)
Kaltura / mwEmbed used by TimedMediaHandler [5]
Leaflet Maps on Mobile JavaScript [6]
Lilypond used by the Score extension [7]
MariaDB Relational database. [8]
memcached GitHub [9]
multilightbox used by Extension:MultimediaViewer [10]
MySQL Relational database. [11]
PHP Programming language that MediaWiki is written in [12]
Rangy JavaScript range and selection library, used by VisualEditor [13]
Swift Distributed storage system [14]
lightncandy HTML templating (mustache and handlebars) in PHP PHP GitHub GitHub EBernhardson, Kaldari
Composer Dependency manager for PHP libraries - used for managing MediaWiki libraries and CI PHP getcomposer.org GitHub Legoktm, Bryan Davis, Reedy
Blazegraph RDF-graph database supporting RDF and SPARQL - used by WDQS Java Blazegraph.com Jira / GitHub
(what else?)

MediaWiki development processEdit

Projects that contribute to our development process. They are installed in Wikimedia servers and they are used in the release process of key Wikimedia projects.

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
Jenkins Continuous integration Java GitHub Jira Hashar, MarkTraceur
Jenkins Job Builder Job creation system Python Git Launchpad Hashar, MarkTraceur
Zuul Gerrit/Jenkins bridge trigger Python Git Launchpad Hashar, MarkTraceur
Gerrit Code review tool Java Gerrit Monorail QChris
Etherpad Lite Real-time collaborative document editing JavaScript (Node.js) GitHub GitHub MarkTraceur, Mutante
PHPUnit Unit testing framework for PHP PHP GitHub GitHub
Node.js Programming language, e.g. used by ESLint, and Karma. C++, JavaScript GitHub GitHub (Node.js Foundation), Timo Tijhof
QUnit Unit testing framework for JavaScript. JavaScript GitHub GitHub (jQuery Foundation), Timo Tijhof
Karma Automation for running unit tests in web browsers and capturing their results. JavaScript GitHub GitHub (Google), Timo Tijhof
Grunt Task runner for build and test scripts. JavaScript GitHub GitHub (jQuery Foundation)
Vagrant Github GitHub
Logstash A tool that can be used to collect, process and forward events and log messages JRuby GitHub Jira bd808
Kibana A browser based analytics and search interface for Elasticsearch that was developed primarily to view Logstash event data JavaScript GitHub GitHub bd808
Graphite/Carbon Real-time graphing system and time series Python GitHub GitHub
Grafana Metrics visualisation Go, JavaScript GitHub GitHub
JSDuck Static site generator for documentation. Ruby GitHub GitHub
Mailman Mailing list server / management (see also Mailing lists administration) Python Gitlab Gitlab
git-review Command line tool for using Gerrit GitHub [15]
Phabricator Task tracker and software development tool used by Wikimedia, see https://phabricator.wikimedia.org PHP GitHub #phabricator-upstream and phabricator.com Andre Klapper
PHP CodeSniffer Continuous integration PHP GitHub GitHub Legoktm (?)
(what else?)

Non-MediaWiki but still WMF-used stuffEdit

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
Varnish Reverse proxy server [16]
nginx Reverse proxy / web server [17]
RT (Request Tracker) Ticket system used by Wikimedia Operations team [18]
Selenium Automated Browser Testing [19]
Icinga Network system monitoring (icinga.wikimedia.org) [20]
OTRS Ticket system used for answering queries, see https://ticket.wikimedia.org/otrs [21]
CiviCRM Used for WMF Fundraising (now upstreaming patches) [22]
WordPress https://blog.wikimedia.org PHP Trac (mailing lists)
Rawdog https://en.planet.wikimedia.org/ etc Python http://offog.org/git/rawdog/
(what else?)

Client-side interactionEdit

Software heavily interacting or interconnected with MediaWiki functionality and work, but user side.

Integrated on Wikimedia projectsEdit

Projects that are used in some way (embedding, linking) on Wikimedia projects, often installed in Wikimedia servers like ToolLabs.

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
CodeMirror a browser based text editor (Used by the CodeMirror extension to provide syntax highlighting in the 2010 wikitext editor.) Javascript GitHub [https://github.com/codemirror/codemirror/issues Github
IIPImage Viewing and zooming of ultra high-resolution images (linked by default on Wikimedia Commons) C++ SourceForge SourceForge
Hundreds of other packages on Tools
(what else?)

Editing toolsEdit

For a full listing, see the lists m:Open Source Toolset, m:FLOSS-Exchange, d:Q6584911.

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
Multimedia software and SVG tools Software used to contribute/work on multimedia in MediaWiki wikis, like Inkscape and GIMP. May play well with MediaWiki or not.
LibreOffice (formerly OpenOffice) Productivity suite, can e.g. produce charts and export/publish documents to wikitext (via libreoffice-wiki-publisher). [23]
Browser spellcheckers Aiding editors to prevent errors before they're saved. Over 120 in Firefox, see respective descriptions.
KIPI uploader and other third party upload tools Multimedia contribution software by third parties, usually made for integration in their software (e.g. digiKam, Gwenview).
(what else?)

Web browsersEdit

Name & URL Description Language(s) Repository Bug tracker Main contacts
Chrome Browser by Google chromium.org (Wikimedia issues are specifically tracked) Performance Team
Firefox Browser by Mozilla mozilla.org Performance Team
Opera Browser by Opera Software opera.com
Internet Explorer Browser by Microsoft microsoft.com
Safari Browser by Apple webkit.org
(what else?)

NotesEdit

  1. For instance, whether volunteers or for work: maintainers and contributors of MediaWiki core and extensions and other developers; Wikimedia projects editors and sysadmins; sysadmins of MediaWiki wikis.

See alsoEdit