Requests for comment/Replacing extension distributor

ExtensionDistributor was in need of some major rework. We completed this in early January 2013.

Request for comment (RFC)

Replacing extension distributor
Component	General
Creation date	10 December 2012
Author(s)	^demon, Krinkle
Document status	implemented

Problem

1. The code is overly complex (shelling out to git-archive & gzip is error-prone^{[citation needed]})
2. The current system requires NFS on the apaches for clones of git repositories and storing/caching the tar packages.
3. The current system was built for our SVN-based branching model, and doesn't suit users well
4. Ownership and permissions mismatches have meant that the extension is often broken in practice for our users.

Proposal #1: Gitweb with core branch timestamps for guesstimates of extension branches

• Rewrite the code to fetch tars/zips from a url pattern (configurable: Gitweb, Gitblit, Github ..). This already handles the "package and tar" logic for us.
  • Put something in front of Gitweb/Gitblit to lessen the load since (at least Gitweb) does not cache (e.g Varnish). We don't actually need to store the archives, long-term caching will suffice.
  • For third party users (if requested): Add on-disk cache as a feature.
• Versions: master and MediaWiki core branch names (extension snapshots of their master branch at the time of the MediaWiki core branch point timestamp). This was more or less what we accomplished in SVN.

Proposal #2: Gitweb with git branch points

• Rewrite the code to fetch tars/zips from a url pattern.
  • Put something in front of Gitweb/Gitblit to lessen the load
• Versions: the git branches that exist for the selected extension.
  • Periodically we can run a script (opt-out base) that automatically creates REL* branches in extensions from the master history at the MediaWiki core branch point timestamps. This way extensions can have their own branch names if they want to (such as SemanticMediaWiki), they can work on the REL branches if they backported fixes (which they can't if the branches don't really exist), and it removes the need to do the timestamp calculation at runtime.

Solution: Github with git branch points

• Rewrite the code to fetch tars from a url pattern.
  • Using github's API to fetch the tars.
• Versions: the git branches that exist for the selected extension.