Wikimedia Release Engineering Team/Checkin archive/20181119
2018-11-19
editVacations/Important dates
edit- November 19 - Jeena's start date
- November 21 - Dan vacation (road trip!)
- November 22+23 - Holidays (Thanksgiving)
- November 25-december 2nd: Mukunda vacation (in California ahead of the offsite)
- Week of December 3rd - Team offsite
- December 24-28 - Holidays (Christmas)
Rotating positions
editTrain
edit- Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R
- Oct 08 - wmf.25 - Dan (No train due to DC switchover)
- Oct 15 - wmf.26 - Mukunda (last 1.32 wmf.XX release, 1.33 starts the next week)
- Oct 22 - wmf.1 - Mukunda (warning, TechConf happening, ping Greg if you need responses from anyone there...)
- Oct 29 - wmf.2 - Tyler
- Nov 05 - wmf.3 - Tyler
- Nov 12 - wmf.4 - Antoine
- Nov 19 - wmf.5 - No Train (Thanksgiving) <----
- Nov 26 - wmf.6 - Antoine
- Dec 03 - wmf.7 - No Train (Offsite)
- Dec 10 - wmf.8 - Zeljko
- Dec 17 - wmf.9 - Zeljko
- Dec 24 - wmf.10 - No Train (Holiday break)
- Dec 31 - wmf.11 - No Train (Holiday break)
- Jan 07 - wmf.12 - Dan
- Jan 14 - wmf.13 - Dan
- Jan 21 - wmf.14 - Mukunda
- Jan 28 - wmf.15 - No Train (All Hands)
- Feb 04 - wmf.16 - Mukunda
- Feb 11 - wmf.17 - Tyler
- Feb 18 - wmf.18 - Tyler
- Feb 25 - wmf.19 - Antoine
SoS
edit- Zeljko for ever :)
Team Business
editWelcome Jeena!
edit- Hi Jeena :)
- Welcome!
December Offsite
editDetails:
- Week of December 3rd
- At the Queen Mary hotel in Long Beach
- Deb T will be facilitating
- REMINDER: Deadline to book travel is Nov 8th!
- Topics!
Activity decision: Both the Aquarium of the Pacific and the USS Iowa on Friday.
- https://en.wikipedia.org/wiki/USS_Iowa_(BB-61)
- https://en.wikipedia.org/wiki/Aquarium_of_the_Pacific
All Hands
edit- Registration: https://office.wikimedia.org/wiki/All_hands/2019/Registration
- Needed for everyone
- NOTE: There's a way to request a hotel room for semi-local people (commutes longer than 1.5 hours)
OMG Surveys
edit- Engagement survey
- Please take it! :) You should have an email from "Culture Amp".
- Internal Support for Open Source Software
Incoming Triage/Needs attention
edit- deploy1001:/srv/mediawiki out of date?
- https://phabricator.wikimedia.org/T207602
- Found because the Security team noticed that a previously deployed security patch was no longer deployed, should sync up with them this week about that (Reedy or Brian)
- See: https://phabricator.wikimedia.org/T207600
- 2018-10-22: no idea, thcipriani will look, I guess
- 2018-10-29: scap updated, needs release this week
- 2018-11-05:
- Need to poke Reedy re:T207600
- scap still needs release - mukunda will take care of it
- 2018-11-19:
- Poked Reedy re:T207600
- Scap 3.8.9-1 running in beta
- deployment-prep region migration
- See email with same subject on releng@lists
- Question: incrementally or not?
- looks like "however Andrew wants to do it"
- REMINDER: send an email update to wikitech-l@/qa@ with the planned timeline/outage
- 2018-10-29: ACTION: Tyler to reply saying "take it away, andrew, and when are you going to do it?"
- 2018-11-05: Email response Done -- blocking task from Krenair https://phabricator.wikimedia.org/T208101 -- Dan and Mukunda graciously volunteered ;)
- 2018-11-19: squeeze in the migration this week? Yes. Mukunda to work on that with Andrew and use paring with Tyler to discuss any breakages.
- Migrate the Integration cloud project to eqiad1-r
- https://phabricator.wikimedia.org/T208803
- 2018-11-12: Need a point person to work with Andrew on this
- 2018-11-19: Tyler and Andrew migrated a few, no issues so far. integration-publishing migrated. castor02 can be migrated the same ways as other slaves, culprit might happen see task for details.
- Migrate the contintcloud cloud project to eqiad1-r
- Nodepool stopped on Thursday. Antoine asked for project to be deleted.
Scrum of Scrums
edit- Greg to copy to etherpad after meeting: https://etherpad.wikimedia.org/p/Scrum-of-Scrums
Incoming from last week
edit- Blocking: nothing...
Outgoing this week (wrong section heading is on purpose for copy/pasting into Scrum of Scrums etherpad
editRelease Engineering
edit- Blocked by:
- Blocking:
- Updates:
- Train Health:
- Last week: 1.33.0-wmf.4 deployment blockers https://phabricator.wikimedia.org/T206658
- This week: No train due to the Thanksgiving Holiday
- Next week:
- Log Health:
- Code Health:
- Bring in 'da noise, bring in defunct. It's a zombie party! Peering into the CI cauldron – October 2018 https://phabricator.wikimedia.org/phame/post/view/127
- Train Health:
Callouts
edit- Release Engineering
- Blubber build with Blubber
- Tyler: Gerrit is alright. Got JavaMelody for monitoring now. InitialiseSettings.php is too big.
- Zeljko/Gehel: https://sonarcloud.io/organizations/wmftest/projects
- Antoine: Nodepool stopped yeah! #technical-debt
Train status and happenings
edit
Quarterly Goals for Q2
editTEC1 (Maint): Outcome 1 / Output 1.1
edit- GOAL: Release MediaWiki 1.32
- WHO: Mukunda, (Tyler on backup)
- 2018-11-05: Mukunda to release MediaWiki 1.32 RC.0
- rc.0 Done
TEC1 (Maint): Outcome 1 / Output 1.1
edit- GOAL: Determine the procedure and requirements for an automated MediaWiki branch cut.
- WHO: Mukunda, Tyler, Antoine
- mukunda + tyler to pair on deploy notes this week
TEC3 (Pipeline): Outcome 1 / Output 1.2
edit- GOAL: Formalize the collection of CI infrastructure and tooling metrics
- WHO: Dan, Antoine
TEC3 (Pipeline): Outcome 2 / Output 2.3
edit- GOAL: Develop set of metrics to assess incident reports/post mortems - task T206622
- WHO: Greg, Zeljko
TEC3 (Pipeline): Outcome 3 / Output 3.1
edit- GOALS:
- Adopt more services into Deployment pipeline - task T205919
- Migrate graphoid to the Deployment pipeline
- Deploy zotero v2 to the Deployment pipeline
- Deploy blubberoid
- Adopt more services into Deployment pipeline - task T205919
- WHO: Dan, Tyler, Lars
- Lars created blubber production variant blubber file!
- Saw movement on zotero this weekend, to prod this week
- graphoid -- thcipriani to poke this week
TEC12 (DevProd): Outcome 2 / Output 2.1
edit- GOAL: The Annual Developer Productivity Survey results are synthesized and shared, creating a first year baseline.
- WHO: Mukunda, Greg
- 2018-11-13:
- 45 responses so far
- 35 from staff
- 9 volunteer
- 1 other / 3rd party
- Overall Satisfaction:
- Local Development: 3.37 / 5 (67%)
- Beta Cluster: 3.51 / 5 (70%)
- Testing: 3.37 / 5 (67%)
- Code Review: 3.37 / 5 (67%)
- Deployments 4.0 / 5 (80%)
- Production Systems: 4.0 / 5 (80%)
- Tools: 4.0 / 5 (80%)
- Documentation: 3.1 / 5 (62%)
- Lots of good feedback given
- I'm very happy with the results of this survey.
- 45 responses so far
TEC13 (Code Health): Outcome 1 / Output 1.1
edit- GOAL: Update/refresh review queue (review process for initial code deployment)
- WHO: JR
No activitiy.
TEC13 (Code Health): Outcome 2 / Output 2.2
edit- GOAL: 5 of the 15 prioritized repositories have at least 1 end-to-end test - task T206621
- WHO: Zeljko
Top 15 defined. Deeper dive into data to see if it supports other course of action.
TEC13 (Code Health): Outcome 2 / Output 2.3
edit- GOAL: Assess Platform unit test practices and define improvement plan
- WHO: JR, Core Platform Team
Planned working session with Corey/Cindy Review Unit testing activities of Core
TEC13 (Code Health): Outcome 3 / Output 3.2
edit- GOAL: Core Platform and Search Platform teams are using TDM PoC
- WHO: JR, Core Platform Team
Planned working session with Corey/Cindy
TEC13 (Code Health): Outcome 3 / Output 3.4
edit- GOALs:
- Identify key Tech Debt areas
- Put in place Tech Debt management process for PEP
- WHO: JR, Core Platform Team
Planned working session with Corey/Cindy
TEC13 (Code Health): Outcome 4 / Output 4.1
edit- GOAL: Metrics defined and deployed for all 4 Code Health areas.
- WHO: JR, Code Health Metrics Working Group
Post-merge SonarQube scanner setup and running.
Other non-goal work
editTEC3 (Pipeline): Outcume 2 / Output 2.3
edit- GOAL: Outline options for managing incident reports creation, follow-ups, and analysis
- WHO: Greg, Mukunda, Zeljko
- Task: https://phabricator.wikimedia.org/T208632
- 2018-11-05: mukunda to make some comments
Selenium
editGerrit
edit- Problem shortly after 2.15.6 upgrade
- Antoine noticed a bunch of bots in logs
- Mukunda + tyler did troubleshooting
- Installed javamelody monitoring: https://gerrit.wikimedia.org/r/monitoring
- Added a robots.txt that blocks indexing gitiles
- seems stable
- Followup action adding prometheus exporter to gerrit https://phabricator.wikimedia.org/T184086
- 2.15.7 out
- let's do the prep work at the offsite!
Phabricator
editJenkins
editQA
editSCAP
editStandup!
editAntoine
edit- What I plan to do this week
- Maintenance
- Doc about MediaWiki extensions testing. Hoping to publish draft by end of week
- (stretch) fill tasks to migrate jobs from the old legacy Jenkins slaves
- (stretch) push to get Nodepool decomissioned entirely
- What I'm blocked on
- Magic?!
- Other?
Dan
edit- What I plan to do this week
- Prometheus! (for realz)
- Be Jeena's onboarding buddy
- What I'm blocked on
- Knowing stuff
- Being gone on Wednesday
- Other?
Greg
edit- What I plan to do this week
- Onboarding :)
- some workboard grooming (is a hope/desire)
- Staging discussion tomorrow morning at 7am
- warning on future meeting with Victoria
- Review Zeljko's blog post
- What I'm blocked on
- determining when I should head back south/home :)
- Other?
Jean-Rene
edit- What I plan to do this week
- Code Health Newsletter
- RoO/Review queue
- What I'm blocked on
- Not blocked, but needed to postpone work with Core Platform Team to next week.
- Other?
Jeena
edit- What I plan to do this week
- What I'm blocked on
- Everything :)
- Other?
- Computer arrived!
Lars
edit- What I plan to do this week
- Blubberfile for Blubber
- Try to understand how Blubberoid could be deployed via delivery pipeline
- What I'm blocked on
- nada
- Other?
- nada
Mukunda
edit- What I plan to do this week
- Help andrewbogot with moving deployment-prep to the new cloud region
- Assuming the above goes smoothly, release mediawiki 1.32.0-rc.1
- What I'm blocked on
- Other?
Tyler
edit- What I plan to do this week
- pairing with Jeena if she has time this week
- GPG signing and pwstore setup
- Pipeline image versioning update
- Graphoid poking
- Move to no-helm-test pipeline to create base image post merge
- Pairing on deploy notes
- Gerrit 2.16 stable branch
- Patches for docker on releases-jenkins
- pairing with Jeena if she has time this week
- What I'm blocked on
- Other?
Zeljko
edit- What I plan to do this week
- T199133 Find top 15 target projects that could use Selenium tests to prevent incidents
- You'll know more than you want to know by the end of the meeting
- T207046 Code health metrics spike
- I might need somebody's help (Antoine?) on Gerrit bots
- T199133 Find top 15 target projects that could use Selenium tests to prevent incidents
- What I'm blocked on
- Other?
Grooming
editTeam Kanban Board Review and Triage
edit- closed and touched in the 7 days
- No update for 4 weeks
- No update for 3 weeks
- No update for 2 weeks
- No update for 1 week
- All Open
- Review To Triage column of #releng
Once / month-ish review of backlog(s)
edit- releng Review To Triage column of #releng
- releng-kanban Review unassigned in kanban
- releng-kanban Review 'backlog' colum of -kanban
- releng-next - Review for things we need to put on our kanban backlog
- releng-backlog - oh my, the huge backlog of things...