Wikimedia Release Engineering Team/Deployment pipeline/2020-09-02

2020-09-02 edit

Always edit

TODOs from last time edit

  • Meeting with Fandom happened yesterday...
    • TODO Tyler to ask Fandom if they have a recording of the meeting
    •   Done they do :)

General edit

RelEng edit

  • Dan has been thinking about container composition and workflows.
    • See https://phabricator.wikimedia.org/T259817#6395133 for current thinking.
    • Creating from scratch images and https://phabricator.wikimedia.org/T255595
      • Start integrating for MediaWiki for production early
      • Satisfies the need for a build step
      • Current status: l10n cache rebuilding updates. Currently depends on a full MediaWiki installation. Did a deep dive on rebuildLocalisationCache.php with xdebug.
        • Clears resourceloader cache as part of rebuild, but is likely incorrect since we update Wikiversions *after* l10n update
        • Scap purges cache for resourceloader as well post update -- that seems correct
        • removing resourceloader purge from rebuildLocalisationCache removes the need for a database connection
        • the next step is localisation hooks, but no production extensions use those hooks
        • deprecating those hooks, removing the resourceloader clearing, makes the l10nupdate much faster (hopefully), but it's a deep rabbithole
  • Feedback for image building
    • Scratch images in registries to integrate images later on
    • Makes use of local caches and blubber
    • Would love additional review
    • Alex will send to team
    • Blubber patch to support scratch images
    • Investigating MW LocalisationCache things. So far:
      • Process is database independent if we remove the ResourceLoader cache clear (doesn't seem like a good thing to do in that process anyhow)
      • No production-deployed extensions actually implement the two hooks defined by LC rebuild
      • These things mean it may be more possible to factor out LC rebuild from MW into a separate/faseter (maybe non-MW-PHP specific) process and compile LC files earlier in an MW image building pipeline
      • Need to dig deeper.

Serviceops edit

  • SRE will be reinitializing the eqiad kubernetes cluster next week since all of eqiad is depooled
  • Working on a policy document for k8s and calico version upgrades
    • focused on k8s upgrades, Docker upgrades later
  • Work on push-notifications resumed.
  • Resuming work on apertium moving to the pipeline
    • this and graphoid the final scb services

As Always edit