Wikimedia Release Engineering Team/Checkin archive/20150414

2015-04-14 edit

Team Business edit

Team Quarterly Goals edit

https://phabricator.wikimedia.org/maniphest/query/O9isnUt5IGLP/#R

Scrum of Scrums edit

https://phabricator.wikimedia.org/project/board/64/
Blocked on us: https://phabricator.wikimedia.org/maniphest/?statuses=open%2Cstalled&allProjects=PHID-PROJ-arpazvuktn2l647rb6us#R

Beta Cluster stability / Staging edit

https://phabricator.wikimedia.org/project/board/497/?order=priority
Quarterly Priority: Green nightly builds on staging: https://phabricator.wikimedia.org/T88701
Quarterly Priority: Stable uptime metrics of the Staging cluster: https://phabricator.wikimedia.org/T88705
  • rin tin tin tin
  • Yuvi has stuff for oids, in discussion


Deployment Cabal edit

First meeting with Marco. Been very productive, Marco highlighted issues Service team had/have with Trebuchet
Trebuchet as building block. Seems accurate for the organization needs

Information leaked from the project super secret etherpad (NOFORN):

  • Services need better visibility and hookability:
    • Better feedback and interactivity from the git deploy cli
      • one aspect of this is non-blocking interactivity of git deploy, it should report thorough, real-time feedback and allow deployer to take actions during the deployment process, not just after
    • Needs to have:
      • rolling restarts
      • 'canary' testing
      • easy semi-automatic rollback when things go bad
      • emergency abort if deployer notices something went wrong
      • ability to isolate and investigate failed nodes when the majority of nodes succeed but a few failed.

Homework:

  • Mukunda and Chad will work further on the mediawiki/core and extensions release branching strategy
    • Mukunda is experimenting with git-subtree to create a composite release repo with all deployed extensions merged into one place
  • Tyler will attempt to document the hooks in trebuchet which we can use to provide custom behavior for individual services or projects
    • This is already in the trebuchet core and it's configurable, just not very discoverable right now
  • Dan to sit in on deployments to get a better sense for the current practices

Test history edit

Quarterly Priority: By team test history: https://phabricator.wikimedia.org/T88706
  • prototyped with Riojs + Elasticsearch
    • need feedback from potential consumers (James F, Jon Robson, Elena, ... ?)

Isolated CI instances edit

https://phabricator.wikimedia.org/tag/continuous-integration/board/?order=priority
Quarterly Priority: Disposable VMs - https://phabricator.wikimedia.org/T47499
  • antoine whistes*
  • nodepool packaging
  • moving jenkins & zuul to new servers - will cause delay but worth it
  • puppet work to be paired by Antoine/Zeljko with reviews by Andrew/Chase

MediaWiki Releases edit

Quarterly Priority: Release MediaWiki 1.25: https://phabricator.wikimedia.org/T88709
List of open 1.25 bugs: https://phabricator.wikimedia.org/maniphest/query/BPlQRzYIEE31
Ones with patch-for-review https://phabricator.wikimedia.org/maniphest/query/tfwMAVRECGaZ/#R

Should the mediawiki release be an actual projet with a workboard? ie https://phabricator.wikimedia.org/tag/MW-1.25-release/

Other Work edit

Mukunda had a Phabricator meeting with Chase, Andre and Quim.

  • Chase will be less available soonish
  • Mukunda & Chad are getting root on Phabricator server, Andre getting admin privs to handle user/repo maintenance
  • We need to take over responsibility for Phabricator upgrades (monthly) after this month
  • There is a lot of ongoing Phabricator process work, we might take an interest in some of it (looking for links)
  • Phacility Paid prio: https://secure.phabricator.com/T7711

Hiring edit

  • delayed...

Vacations/Confs/etc edit

  • Antoine: observing french holidays: Fri May 1, Fri May 7-8, Thurs May 14
  • Dan in France the week before offsite (May 11-15)
  • Week before hackathon: Team offsite in France - https://phabricator.wikimedia.org/T89036
  • May: Hackathon in Lyon, France
  • Chad vacation after offsite/hackathon (through 5/31)
  • Elena on vacation May26-June07