Upstream projects

MediaWiki developers and users[1] rely on several upstream projects for their architecture or processes. At the same time, they develop software projects used by others, which are downstream to them. These links define our location in the free software map, our neighbor communities, and the flows of feedback, patches, and contributors.

Let's identify our upstream projects, our neighbor communities, and the contributors that connect us with them.

MotivationEdit

This is a first step to improve our relations with other communities, to increase the contributions received, and our influence in the projects that matter to us.

Our mid-term goals include:

  • identify the projects where we want to see significant development, to the point of sending patches as well
  • identify the communities where Wikimedia should be regularly active and heard
  • identify the people in the Wikimedia and upstream communities that know each other and act as bridge
  • identify organizations and events we should get in touch and be part of
  • get involved in bigger development efforts regularly, become a regular FOSS player

Invented hereEdit

Projects we maintain that we want others to use and contribute to.

Wikimedia-deployed services, extensions and libraries
Name Repository Issue tracker Main contacts
MediaWiki Diffusion Phabricator MediaWiki Platform team, Tim Starling, Brion Vibber
OOjs Gerrit Phabricator Timo Tijhof, Roan Kattouw
OOUI Gerrit Phabricator James Forrester, Bartosz Dziewoński
CSSJanus GitHub GitHub Timo Tijhof, Roan Kattouw
CLDRPluralRuleParser Gerrit Phabricator Niklas Laxström
RemexHtml Gerrit Phabricator Tim Starling
wikimedia/cdb Gerrit Phabricator Performance Team, Tim Starling, Timo Tijhof
wikimedia/at-ease Gerrit Phabricator Tim Starling, Legoktm
wikimedia/utfnormal Gerrit Phabricator Brion VIBBER
wikimedia/ip-set Gerrit Phabricator Brandon Black
wikimedia/running-stat Gerrit Phabricator Performance Team
wikimedia/base-convert Gerrit Phabricator
ScopedCallback Gerrit Phabricator
WaitConditionLoop Gerrit Phabricator Performance Team, Aaron Schulz
WrappedString Gerrit Phrabricator Performance Team, Timo Tijhof
composer-merge-plugin GitHub GitHub Bryan Davis
testing-access-wrapper Gerrit Phabricator
php-session-serializer Gerrit Phabricator Brad Jorsch
purtle Gerrit WMDE, Stas Malyshev
Name Repository Issue tracker Main contacts
Parsoid Gerrit Phabricator (under "Parsoid") Subbu Sastry,

C. Scott Ananian

VisualEditor Gerrit Phabricator James Forrester, Ed Sanders
UnicodeJS Gerrit Phabricator David Chan, James Forrester, Ed Sanders
wikimedia/css-sanitizer Gerrit Phabricator
wikimedia/equivset Gerrit Phabricator Brion VIBBER
Milkshake GitHub GitHub Santhosh Thottingal
GitHub GitHub Santhosh Thottingal
GitHub GitHub Santhosh Thottingal
GitHub GitHub Santhosh Thottingal
Offline content tools GitHub GitHub C. Scott Ananian
GitHub GitHub C. Scott Ananian
textcat Gerrit Phabricator Trey Jones
Other projects from the MediaWiki engineering community
Name & URL Description Language(s) Repository Bug tracker Main contacts
translatewiki.net Where we organise localisation of MediaWiki. PHP, Puppet phab:diffusion/GTWN/ #translatewiki.net in Phabricator, translatewiki:Support Nikerabbit (founder), Abijeet Patro (co-maintainer), Siebrand (co-maintainer), Raymond (MediaWiki updates responsible)
Pywikibot Python Wiki Robot Framework to automatize tasks Python Diffusion Phabricator (under "Pywikibot") xqt
Orgchart tool Generates organisational charts. JavaScript, Node.js Diffusion Phabricator (under "Utilities") MarkTraceur
Bingle Pipes bugs form Bugzilla into tools like Mingle and Trello; obsolete after migration to Phabricator Python Github Github User:Awjrichards

More stuff (hundreds of projects) by fellow MediaWiki developers and Wikimedians is also listed at:

ComponentsEdit

Projects that contribute to the architecture of MediaWiki, key extensions, and applications. They are included in the downloadable packages of key Wikimedia projects, or they are identified as required dependencies.

Upstream components
Name & URL Description Language(s) Repository Bug tracker Main contacts
GNU diff and diff3 Powers recent changes, history and undo button (i.e. most editing features).
Elasticsearch Distributed real-time search and analytics engine. Java GitHub GitHub
jQuery Lots of DOM traversing, and other DOM-related helpers, for JavaScript. JavaScript GitHub GitHub (jQuery Foundation), Timo Tijhof
Vue.js Vue is a progressive framework for building user interfaces. JavaScript GitHub GitHub
Puppet Configuration management system. Ruby GitHub Jira Wikimedia SRE
Unihan Database For Chinese language conversion Used as database
Smart Common Input Method For Chinese language conversion Used as database
libtabe For Chinese language conversion Used as database
ImageMagick general purpose image scaling C No repo ImageMagick.tar.gz Forum
Lua Scripting language C
ACE Web based code editor JavaScript, HTML GitHub GitHub User:TheDJ (has signed contributor license)
VIPS Image scaling of large images (VipsScaler) GitHub GitHub User:Bryan
libRsvg PNG renderer for SVG images C GNOME git GNOME Gitlab (Wikimedia tagged issues)
MathJax Math rendering in HTML/SVG/MathML; used by Math extension JavaScript GitHub GitHub
FFmpeg / libav Video conversion and scaling C ffmpeg / libav ffmpeg / libav
MWLib mediawiki parser and converter Python GitHub GitHub
Pygments.org Syntax highlighting Python Bitbucket Bitbucket
Unicode CLDR (Common Locale Data Repository) All we manage to take: language names, plural rules, language-territory information, ... [1] Nemo (CLDR Survey Tool manager)
CLDRPluralRuleParser Plural support based on CLDR JavaScript GitHub GitHub Santhosh Thottingal
Apertium Free software, rule-based machine translation; used by Translate, ContentTranslation and others. Soureforge [2] Francis Tyers, Firespeaker, Unhammer
Ceph Distributed storage system C++ GitHub Ceph
Cortado Media streaming Java applet Java xiph [3]
GlusterFS Distributed file system [4]
Kaltura / mwEmbed used by TimedMediaHandler [5]
Leaflet Maps on Mobile JavaScript [6]
Lilypond used by the Score extension Gitlab.com
MariaDB Relational database. [7]
memcached GitHub [8]
multilightbox used by Extension:MultimediaViewer [9]
MySQL Relational database. [10]
PHP Programming language that MediaWiki is written in [11]
Rangy JavaScript range and selection library, used by VisualEditor [12]
Swift Distributed storage system [13]
lightncandy HTML templating (mustache and handlebars) in PHP PHP GitHub GitHub EBernhardson, Kaldari
Composer Dependency manager for PHP libraries - used for managing MediaWiki libraries and CI PHP getcomposer.org GitHub Legoktm, Bryan Davis, Reedy
Blazegraph RDF-graph database supporting RDF and SPARQL - used by WDQS Java Blazegraph.com Jira / GitHub
(what else?)

MediaWiki development processEdit

Projects that contribute to our development process. They are installed in Wikimedia servers and they are used in the release process of key Wikimedia projects.

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
Jenkins Continuous integration Java GitHub Jira Hashar, MarkTraceur
Jenkins Job Builder Job creation system Python Git Openstack Storyboard Hashar, MarkTraceur
Zuul Gerrit/Jenkins bridge trigger Python Git Openstack Storyboard Hashar, MarkTraceur
Gerrit Code review tool Java Gerrit Monorail QChris
Etherpad Lite Real-time collaborative document editing JavaScript (Node.js) GitHub GitHub MarkTraceur, Mutante
PHPUnit Unit testing framework for PHP PHP GitHub GitHub
Node.js Programming language, e.g. used by ESLint, and Karma. C++, JavaScript GitHub GitHub (Node.js Foundation), Timo Tijhof
QUnit Unit testing framework for JavaScript. JavaScript GitHub GitHub (jQuery Foundation), Timo Tijhof
Karma Automation for running unit tests in web browsers and capturing their results. JavaScript GitHub GitHub (Google), Timo Tijhof
Grunt Task runner for build and test scripts. JavaScript GitHub GitHub (jQuery Foundation)
Vagrant Github GitHub
Logstash A tool that can be used to collect, process and forward events and log messages JRuby GitHub Jira bd808
Kibana A browser based analytics and search interface for Elasticsearch that was developed primarily to view Logstash event data JavaScript GitHub GitHub bd808
Graphite/Carbon Real-time graphing system and time series Python GitHub GitHub
Grafana Metrics visualisation Go, JavaScript GitHub GitHub
JSDuck Static site generator for documentation. Ruby GitHub GitHub
Mailman Mailing list server / management (see also Mailing lists administration) Python Gitlab Gitlab
git-review Command line tool for using Gerrit GitHub [14]
Phabricator Task tracker and software development tool used by Wikimedia, see https://phabricator.wikimedia.org PHP GitHub #phabricator-upstream and phabricator.com Andre Klapper
PHP CodeSniffer Continuous integration PHP GitHub GitHub Legoktm (?)
Phan Continuous integration PHP GitHub GitHub
(what else?)

Non-MediaWiki but still WMF-used stuffEdit

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
Debian Free Operating System used on all production servers and cloud VPS instances. various [15] [16] multiple Debian devs in SRE
Let's Encrypt non-profit certificate authority - used for TLS (SSL) certificates for many services github/letsencrypt SRE Traffic team
Varnish Reverse proxy server [17]
nginx Reverse proxy / web server [18]
RT (Request Tracker) Ticket system used by Wikimedia Operations team [19]
Selenium Automated Browser Testing [20]
Icinga Network system monitoring (icinga.wikimedia.org) [21]
OTRS Ticket system used for answering queries, see https://ticket.wikimedia.org/otrs [22] for bug reports; [23] for feature requests
CiviCRM Used for WMF Fundraising (now upstreaming patches) [24]
WordPress https://blog.wikimedia.org PHP Trac (mailing lists)
Rawdog https://en.planet.wikimedia.org/ etc Python http://offog.org/git/rawdog/
Django Python web framework Python https://github.com/django/django
OpenStack Nova OpenStack compute service. Python OpenStack Github Launchpad Wikimedia Cloud Services team
OpenStack Neutron OpenStack software-defined-networking service. Python OpenStack Github Launchpad Wikimedia Cloud Services team
OpenStack Glance OpenStack image service. Python OpenStack Github Launchpad Wikimedia Cloud Services team
OpenStack Keystone OpenStack identity service. Python OpenStack Github Launchpad Wikimedia Cloud Services team
OpenStack Swift OpenStack object storage service. Python OpenStack Github Launchpad Wikimedia SRE
Kubernetes Cloud platform for running containers Go Kubernetes Github Kubernetes issues on Github Wikimedia SRE and Wikimedia Cloud Services team
Linux Kernel The core that runs all of our servers C Git source tree TBD, probably better to use the Debian BTS. Wikimedia SRE and Wikimedia Cloud Services team and others.
Netfilter Linux packet filtering framework C Netfilter source repositories Netfilter bugzilla Wikimedia SRE and Wikimedia Cloud Services team and others.
Son of Grid Engine Distributed and scheduled computing platform Several SGE source repositories SGE mailing list Wikimedia Cloud Services team
AJV (Another JSON Schema Validator) JSONSchema Validator that powers WMF's streaming Event Platform components like EventGate and schema repositories. NodeJS AJV Github AJV issues on Github Wikimedia Analytics Engineering and Core Platform teams
Apache Superset Dashboarding and data exploration tool Python Superset Github Wikimedia Analytics Engineering
Turnilo Data exploration tool Javascript Turnilo Github Wikimedia Analytics Engineering
(what else?)

Client-side interactionEdit

Software heavily interacting or interconnected with MediaWiki functionality and work, but user side.

Integrated on Wikimedia projectsEdit

Projects that are used in some way (embedding, linking) on Wikimedia projects, often installed in Wikimedia servers like ToolLabs.

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
CodeMirror a browser based text editor (Used by the CodeMirror extension to provide syntax highlighting in the 2010 wikitext editor.) Javascript GitHub Github
IIPImage Viewing and zooming of ultra high-resolution images (linked by default on Wikimedia Commons) C++ SourceForge SourceForge
Hundreds of other packages on Tools
(what else?)

Editing toolsEdit

For a full listing, see the lists m:Open Source Toolset, m:FLOSS-Exchange, d:Q6584911.

Upstream tools
Name & URL Description Language(s) Repository Bug tracker Main contacts
Multimedia software and SVG tools Software used to contribute/work on multimedia in MediaWiki wikis, like Inkscape and GIMP. May play well with MediaWiki or not.
LibreOffice (formerly OpenOffice) Productivity suite, can e.g. produce charts and export/publish documents to wikitext (via libreoffice-wiki-publisher). [25]
Browser spellcheckers Aiding editors to prevent errors before they're saved. Over 120 in Firefox, see respective descriptions.
KIPI uploader and other third party upload tools Multimedia contribution software by third parties, usually made for integration in their software (e.g. digiKam, Gwenview).
(what else?)

Web browsersEdit

Name & URL Description Language(s) Repository Bug tracker Main contacts
Chrome Browser by Google chromium.org (Wikimedia issues are specifically tracked) Performance Team
Firefox Browser by Mozilla mozilla.org Performance Team
Opera Browser by Opera Software opera.com
Internet Explorer Browser by Microsoft microsoft.com
Safari Browser by Apple webkit.org
(what else?)

NotesEdit

  1. For instance, whether volunteers or for work: maintainers and contributors of MediaWiki core and extensions and other developers; Wikimedia projects editors and sysadmins; sysadmins of MediaWiki wikis.

See alsoEdit