Wikimedia Release Engineering Team/Checkin archive/20151123





How to do it:

  • Nov 24: Mukunda - day off (requested in ADP)
  • Nov 25: Greg - Half day (working that morning)
  • Nov 25: Tyler - Half day (need to update ADP still)
  • Nov 25: Mukunda - half day
  • Nov 26: US HOLIDAY - Thanksgiving
  • Nov 27: US HOLIDAY - Day after Thanksgiving
  • Dec 2?-Jan 0x - Antoine, no clue yet
  • Dec 4th: Greg - disconnected, leaving Thursday evening, returning Sunday :)
  • Dec 2x--XX: Antoine, unscheduled yet
  • Dec 21-Jan 1: Greg - vacation, at least (maybe part of the week before)
  • Dec 22-29: Chad - Christmas
  • Dec 23–25: Tyler — Hopeful, probable, Christmas in Kansas!
  • Dec 24-Jan 3: Dan - Holidays
  • Dec 25: US HOLIDAY - Christmas Day - December 25
  • Jan 1: US HOLIDAY - New Year's Day
  • Jan 4 - 8: WikiDev16 + All Hands
  • Jan 16-18: Chad - another music festival
  • Jan 18: US HOLIDAY - Martin Luther King Day
  • Feb 15: US HOLIDAY - President's Day
  • May 17-(?): Dan - paternity leave :D
    • PO Box for pastries? - Antoine
  • May 30: US HOLIDAY - Memorial Day
  • June-ish: Chad - EDC
  • August: France holiday - because french. :)

Team Business


Actions from last meeting

  • DONE - Greg: Reword the differential dogfooding goal
  • DONE - Greg: Get perferred physcian form from HR for US staff
  • TODO - Antoine + Mukunda should sit down and talk CI/Harbormaster/Nodepool
    • Mukunda just needs to find time to finally test out harbormaster triggering jenkins jobs.
  • IN PROGRESS - Chad: re 1.26 release: "Going to push on all the assigned ones and find assignees for the remainder this week."
    • Dan to continue to bring up in SoS
    • Release is next week, the 25th of Nov. before thanksgiving (US Nov 26th).
  • TODO - Antoine: re KPIs, do mw core gate-processing time for this quarter, reassess for next quarter if needed
  • TODO - No One Yet: investigate carbon aggregation of stats >1 month old behavior
    • ACTION: Antoine to create a task





Q3 Goals


Goals timeline:

  • December 3: Group goal scoped and drafted on for Technology team.
  • December 10: Group goal + all individual team *drafts completed* on; discuss at Infra+Tech group and identify dependencies.
  • December 17: individual team goals + group goal *finalized* on; discuss at Monthly Eng Staff.
  • Some discussion in scap meeting Friday
  • scap3 for mediawiki?

  • Beta Cluster Cleanup (questionable)
    • Incorporate the work from the previous Staging effort into the current Beta Cluster
      • specific outputs?

  • Differential
    • Garner more early adopters (people and projects)
      • how many?
      • one proj per team (at WMF) in differential
        • reading
        • editing
        • discovery
        • "technology"?
          • operations repos that don't have much CI (eg: deb packages)
          • research? performance?


  • Log spam:
    • Can we explain the two bumps (up and down) of the INFO lines?
      • RunJobs logging changes? See engineering-l/ops-l? (greg was lazy this morning, will look later)
    • Looking good, actually. The log(n) hides it a bit
      • on 9/12 we were at 1k warning, 2k errors
      • last week we've been between 100-800ish for both
      • we should make some graph that shows that more clearly
      • ACTION: Chad take a look at making the >=Warning graph (mostly done, trying to figure out right log)

New vs Maint time spent




Scrum of Scrums

Blocked on us:

Project Updates


CI Scaling

Quarterly Goal: "CI cluster responds to spike in queued builds by starting and registering additional jenkins slaves" -
  • zuul-cloner being added
  • instances now have git mirror of mediawiki/core and operations/puppet. Will add more depending on size / jobs frequency.
  • Have jobs to clone source code and work from /srv/ .
  • Then get a convention of capturing artifacts in /log/** /build/** .

Beta Cluster

  • Antoine / Jaime chatted about databases performances. Random items:

=> Antoine to summarize discussion with Jaime and fill tasks accordingly

  • Team: might have to watch slow query logs / strict errors etc

Deployment Cabal

Quarterly Goal: "Migrate all Service team owned services and MW deploys to scap3" -
  • scap 3.0?
    • no idea / pending production use

Diff[usion|erential] migration

Quarterly Goal:
  • Find some Differential Ambassadors for each vertical
    • bd808 for Reading
  • Antoine has arcanist installed!  ;)
  • gitblit redirect stuffs
    • chad wants more out of mukunda :)
    • merge the first part (today), still work on the extra bits

Release MediaWiki 1.26

Release date 11/25
  • Do we have all bundled extensions/skins migrated to extension.json/skin.json?

Developer Tooling (MW-Vagrant, MW-Selenium, etc.)


Other Work

  • Mukunda is working on subtree merging mediawiki extensions for long-lived release branch.