Wikimedia Release Engineering Team/Checkin archive/20180611
2018-06-11
editVacations/Important dates
edit- June 19: WMF Monthly Holiday - https://en.wikipedia.org/wiki/Juneteenth
- June 18–22: Dan – Vacation
- July 2: Greg's BDay!!!
- July 4: US Holiday
- July 16: Mukunda's bday.....funtimes
- August 15: WMF Monthly Holiday
- Middle of August...a few days somewhere - thcipriani
Rotating positions
editTrain
edit- Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R
- May 21 - wmf.5 - Mukunda
- May 28 - wmf.6 - Tyler
- June 4 - wmf.7 - Tyler
- June 11 - wmf.8 - Dan (with Tyler doing Thursday)
- June 18 - wmf.9 - Dan (no train, SRE summit)
- June 25 - wmf.10 - Dan
- July 2 - wmf.11 - Zeljko - no train, Fourth of July
- July 9 - wmf.12 - Zeljko
- July 16 - wmf.13 - Antoine
- July 23 - wmf.14 - Antoine
SoS
edit- May 21 - Chad
- May 28 - Mukunda
- June 4 - Mukunda
- June 11 - Tyler <-----
- June 18 - Tyler
- June 25 - Tyler
- July 2 - Dan
- July 9 - Dan
- July 16 - Zeljko
- July 23 - Zeljko
- July 30 - Antoine
- August 6 - Antoine
Team Business
editUpdates
edit- Train: Decided: Dan this week
- Train EU time:
- EU SWAT 1300-1400 CEST (1100-1200 UTC), 2 hours earlier than now
- break 1400-1500 CEST (1200-1300 UTC), my usual lunch time
- train 1500-1700 CEST (1300-1500 UTC)
- break 1700-1800 CEST (1500-1600 UTC)
- Puppet SWAT 1800-1900 CEST (1600-1800 UTC)
- SWAT window on Wednesday
- bad timing (with SoS)
- was moved back for the pre-Train sanity hour
- move it back to 11 Pacific?
- DECIDED: move back to 11 Pacific time
- Backfill needs/desires
- Developer Productivity JD: https://docs.google.com/document/d/1Jnt9gCZ0zk0lC_aYYzzE9x3JxnLsxhm6QKAjXCaGrho/edit
- proposal: add a list of keywords of things (software/languages) we use
- Greg will make a Sr Engineer focused version
- https://phabricator.wikimedia.org/T196840 - fyi mmodell
Offsite notes and action items
editData Data Data
edit- Talk with Analytics - JR
- Talk with CE/Bitergia - JR
- Explore Bitergia - JR
- Identify data sources we want to collect - RelEng (who know what systems)
- https://etherpad.wikimedia.org/p/releng-data-sources
- TODO: Greg: email the etherpad out to the team list
- Erik Bernhardson / Guillaume Lederrey
- After we get our list of data sources talk to them
SWATs/Trains
edit- TODO: Tyler reasses scap swat in mw-config from Mukunda
- Look into parsing scap messages for known patterns and pulling out the data
- Look into enabling scap start/done
- This should be trivial
- Look into recording if mwdebug was used during the deploy (eg: 'scap stage')
- H/Now will we get time for this?
- Have Mukunda do a couple weeks of SWATs
- Mukunda has a lot to say about this subject.... writeup incoming
TODO tl;dr: Tyler taskify the above and talk to Mukunda
Staging
edit- Greg to talk with Deb about what to do next with talking to Victoria
- Done
- Greg to figure out how we can better market what we are accomplishing (eg "monthly showcase")
- TODO
- Get a k8s cluster from SRE for CI to deploy to.
- Dan/Tyler discuss later today
- potentially poke/get a commitment on a timeline soon
- Dan/Tyler discuss later today
Scrum of Scrums
edit- Greg to copy to etherpad after meeting: https://etherpad.wikimedia.org/p/Scrum-of-Scrums
This week
editRelease Engineering
edit- Blocking
- Blocked
- Updates
- Quarterly cross-dependencies
Last week
editRelease Engineering
edit- Blocking
- Blocked
- Updates
- Please take the Beta Cluster survey:
- Heads up: There will be more people in the normal MW Train deployment rotation (namely: Antoine, Zeljko, and Dan to start) and we'll be doing some Train deployments during EU hours some weeks. Exact timing TBA (soon).
- Heads up: There is a new primary deployment host, deploy1001.eqiad.wmnet (tin is gone)
- Reminder: just use deployment.eqiad.wmnet for ssh
- Quarterly cross-dependencies
Train status and happenings
edit
Past week status updates
edit- All of it in table form: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q4
Quaterly Goals for Q4
editProgram 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure
edit- Migrate away from Nodepool - task T190097
- Migrate MediaWiki PHPUnit tests to Shipyard (docker-based CI) (~40% of Nodepool usage) - task T183512
- Add Composer support to Blubber - task T186547
- Add Python/Tox support to Blubber - task T186545
- Add Ruby/Gem/Bundler/Rake support to Blubber - task T188950
Program 3: Outcome 1: Objective 2: Identify and find stewards for high-priority/high use code segment orphans
edit- Broad role out of Code Stewardship model.
- Update Maintainers/Developers page with currently known Code Stewards.
- Use Code Stewardship review process to address gaps.
- Deploy dashboard of Code Stewardship Coverage
- Reviewed feedback and met with Toby and Victoria
- Zotero issue seems to be addressed
Program 3: Outcome2: Objective 2: Define and implement a process to regularly address technical debt across the Foundation
edit- Roll out of technical debt reduction approaches.
- Identify early adopter engineering teams to test approaches.
- Reviewed Platform technical debt tasks in phab
- Reviewed general technical debt tasks in phab
Program 6: Outcome 2: Objective 2: Prove viability of testing staged service containers alongside MediaWiki extension containers
edit- Add Composer support to Blubber - task T186547
- Small, standalone, MediaWiki containers built using Blubber
- Limited scope with: Debian Stretch, php7, composer dependencies, Mariadb, Apache (or last two with standalone containers)
- Build MW base containers upon branch cut at master branch point from core
- Build ext specific containers using MW image as base
Few discussions last week:
- Math extension post-merge job
- Need to break-up further (see bullet points)
- CI namespace for staging k8s
- SRE asked for a task last Thursday
- Hopefully Soon™
Quaterly non-goal "Work"
editProgram 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)
edit
Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure
editProgram 1: Outcome 6: Milestone 1: Maintain Gerrit
editProgram 1: Outcome 6: Milestone 2: Maintain Phabricator
edit- Streamline logspam workflows by adding some integration with phabricator
- Store git-lfs (and other phab uploads) in swift: task T182085
- Making some progress on a disaster recovery plan for phabricator:
** https://phabricator.wikimedia.org/T190572
Other work
editStandup!
editAntoine
edit- What I plan to do this week
- Migrate mediawiki-extensions jobs
- What I'm blocked on
- Whatever tests is going to end up failling :/ Eg Selenium triggering a bug in mediawiki https://phabricator.wikimedia.org/T191537
- Other?
Dan
edit- What I plan to do this week
- Train
- Docker image build for Math extension
- What I'm blocked on
- Other?
Greg
edit- What I plan to do this week
- A couple high prio HR/Fiscal year things
- Announce EU Train window and deploy schedule changes
- Make a Senior Eng focused version of the DevProd JD
- email the "what data sources do you know of" etherpad out to the team list
- What I'm blocked on
- nothing yet
- Other?
Jean-Rene
edit- What I plan to do this week
- More on Search platform and platfom technical tech
- Q4 stewardship reviews
- Q3 stewardship sync-ups
- offsite tasks
- What I'm blocked on
- Other?
Mukunda
edit- What I plan to do this week
- Fix phabricator importing new gerrit meta commits
- Work on scap swat stuff (documentation, clean up cli api)
- support git-lfs stuff as needed
- Try to get some movement on phabricator+swift puppetization
- What I'm blocked on
- Waiting on filippo to review puppet patches for swift
- Should be nearly ready to merge and the patches got some review early this morning.
- https://gerrit.wikimedia.org/r/c/operations/puppet/+/432528
- Waiting on filippo to review puppet patches for swift
- Other?
Tyler
edit- What I plan to do this week
- "Train"ing
- Math extension pipeline sense-making (see bullet points on task)
- Scap 3.8.2-1 help/fallout/whatever's needed
- What I'm blocked on
- None?
- Other?
- Filed Automate the Train
Zeljko
edit- What I plan to do this week
- T190710 Minerva Ruby and Node.js browser tests running side by side
- T194252 Configure the CI job that runs WikibaseLexeme's browser tests against beta wikidata
- What I'm blocked on
- Other?
- T195496 mediawiki-core-qunit-selenium-jessie sometimes fails for extensions - Antoine and Anomie made great progress https://gerrit.wikimedia.org/r/c/mediawiki/core/+/439289
Grooming
editTeam Kanban Board Review and Triage
edit- closed and touched in the 7 days
- No update for 4 weeks
- No update for 3 weeks
- No update for 2 weeks
- No update for 1 week
- All Open
- Review To Triage column of #releng
Once / month-ish review of backlog(s)
edit- releng Review To Triage column of #releng
- releng-kanban Review unassigned in kanban
- releng-kanban Review 'backlog' colum of -kanban
- releng-next - Review for things we need to put on our kanban backlog
- releng-backlog - oh my, the huge backlog of things...