Wikimedia Release Engineering Team/Checkin archive/20190123
2019-01-23
editVacations/Important dates
edit- January 28 - February 1 - All Hands
- February 2 - February 9 - Lars on vacation
- February Fri. 8th - Mon. Feb. 17th - Antoine, school vacations
- February 18 (President's Day) - US Staff
- February 19 - March 1 - Dan, vacation
- March 11 (WMF Holiday) - US Staff
- April 22 (WMF Holiday) - US Staff
- April 22nd - Antoine, Easter
- May 1st - Antoine, labor day
- May 8th - Antoine, 1945 victory
- May 30th-31th - Antoine, Feast of the Ascension
- June 10th - Antoine, Pentecost -- see https://en.wikipedia.org/wiki/Eastertide for Antoine/France Easter holidays
- May 27 (Memorial Day) - US Staff
- June 19 (Juneteenth) - US Staff
Rotating positions
editTrain
edit- Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/query/s3KW8bpsXhYF/#R
- Jan 07 - wmf.12 - Dan
- Jan 14 - wmf.13 - Dan
- Jan 21 - wmf.14 - Mukunda
- Jan 28 - wmf.15 - No Train (All Hands)
- Feb 04 - wmf.16 - Mukunda
- Feb 11 - wmf.17 - Tyler
- Feb 18 - wmf.18 - Tyler
- Feb 25 - wmf.19 - Antoine
SoS
edit- Zeljko 4eva! :)
Team Business
editAll Hands
editTuesday (1/29): Team day (mini-offsite)
edit- Location: 251 Post St, Room 610
- Put any must-talk-abouts at: https://etherpad.wikimedia.org/p/releng-allhands-teamday-topics
- Tyler and JR: You were selected (by Deb?) to pick up our materials (pens, paper, snacks) from the WMF Office that morning
- I have an invite to our "breather space" on 2019-01-29
- TODO: Greg will forward email after meeting.
- Looks like Deb already took care of that.
- thcipriani: is anything happening re:pipeline cabal? Should I follow up on that email?
- Yes.
Photos!
edit- https://drive.google.com/drive/folders/1XsUXBlLbs5sofosxg9nsurLSd7kWStKW
- delete any you don't want shared by EOD today.
Lightning talks
edit- Greg: Internal Support for Free/Open Source Tools Working Group
- JR/Greg: Code Health
- Željko: Code Health Metrics and SonarQube https://phabricator.wikimedia.org/phame/post/view/133
Book club
edit- It's now a thing: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Book_club
- Buy your copy and expense it.
- SUMMARY: Read Part I
- TODO: Greg to find a magical time to discuss
- During team day on Tuesday of All Hands? Otherwise it'd be the week of March 4th
- There's almost no chance I'll read it by then, I think some people didn't buy yet (Antoine?)
- I just ordered one for Antoine, will give it to him at All Hands
- Thcipriani: I have read the foreword/ToC so far
- Jeena: I read some :)
- Lars: I seem to be ahead with my reading (finished ch2 today)
- There's almost no chance I'll read it by then, I think some people didn't buy yet (Antoine?)
- During team day on Tuesday of All Hands? Otherwise it'd be the week of March 4th
- PLAN
- Discuss what people have read thus far on Tuesday
- March 4th (TODO: Greg put in calendar): discuss Part I plus Chapter 5.
Spring Offsite
edit- Location: Chicago, IL (Central timezone, UTC-5 while we're there)
- Dates: Arriving on Sunday (4/21), Flying out on Friday (4/26)
- That's a really bad time for me :( Easter (4/21) is a big family event in Croatia.
- TODO: Greg to ask Travel to move it to Mon-Sat.
- Activity day: Send your suggestions to me if you have them :) I'll make the voting spreadsheet later.
- Chicago Bulls!!!11!oneone
- I've heard there's good pizza :P
- Greenfield park conservatory?
- Program: Haven't started yet :)
Technical Advice IRC meeting
edit- https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
- "Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project."
- They're looking for hosts on February 6th (and this morning, 1 hour ago....)
Draft Gerrit policy from Tim
edit- TechCom discussion today at 2pm Pacific in #wikimedia-office
- https://www.mediawiki.org/wiki/User:Tim_Starling_(WMF)/Draft_Gerrit_privilege_policy
Incoming/Needs attention
editQuibble should run `npm install` and `npm run selenium-test` for each extension/skin that has Selenium tests
edit- https://phabricator.wikimedia.org/T199116
- This is blocking both Wikidata and Readers Web teams from running their ruby and JS tests side-by-side
LFS objects are not mirroring from Github through Phab to Gerrit consistently
edit- https://phabricator.wikimedia.org/T212818 -> now: https://phabricator.wikimedia.org/T212962
- We need to figure out the correct long term solution here.
- Contraints:
- They want git-lfs support
- They also want to have development happening on github ("external contributors")
- :(((((((
- question: can we just use git-fat?
- Antoine: probably because ORES models stored outside of git predate our adoption of git-fat and at the time we went with lfs
- Possible outcome: migrate to git-fat and archiva?
- references/see also:
- https://phabricator.wikimedia.org/T181678 - Plan migration of ORES repos to git-lfs - Nov 2017 - June 2018
- https://phabricator.wikimedia.org/T180627 - Support git-lfs in scap
- 2019-01-14:
- LFS doesn't support mirroring automatically at all
- Mirroring ***from*** github may be broken in other ways, not sure
Mukunda to follow up on: https://phabricator.wikimedia.org/T212962 - articlequality repo mirroring is broken
- this is more complex than it seemed initially, I'm working on sorting this out
- TODO: Greg, make time on Tuesday for this to be a discussion topic, especially wrt to Continuous Deployment and mid-term planning
- TODO: Let's verify how services are handled.
Scrum of Scrums
edit- Greg to copy to etherpad after meeting: https://etherpad.wikimedia.org/p/Scrum-of-Scrums
Incoming from last week
edit- Blocking: nothing...
Outgoing this week (wrong section heading is on purpose for copy/pasting into Scrum of Scrums etherpad
editRelease Engineering
edit- Blocked by:
- Blocking:
- Updates:
- Train Health:
- Last week: 1.33.0-wmf.13 - https://phabricator.wikimedia.org/T206667
- This week: 1.33.0-wmf.14 - https://phabricator.wikimedia.org/T206668
- Next week: No train, WMF ALL HANDS!
- Log Health:
- Code Health:
Callouts
edit- Release Engineering
Train status and happenings
edit- Completely uneventful so far this week
- Change logs generated by jenkins: FTW 🍻
Quarterly Goals for Q3
edithttps://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2018-19_Q3
TEC1 (Maint): Outcome 1 / Output 1.1
edit- GOAL: Automate the generation of change log notes
- WHO: Mukunda, (Tyler on backup)
- Changelog generated from Jenkins job
- Mukunda and I created a plan for updating automagically
- The main challenge is automatically determining the correct version numbers for the job parameters.
TEC1 (Maint): Outcome 1 / Output 1.1
edit- GOAL: Investigate notification methods for developers with changes that are riding any given train
- WHO: Mukunda, Tyler
- maybe rethink this plan given curren situation....
- Let's comment on the patch in gerrit!...or something
- best to send out an email so that folks can be involved in the planning phase
TEC3 (Pipeline): Outcome 1 / Output 1.2
edit- GOAL: Instrument Quibble for data collection
- WHO: Mukunda, Antoine
TEC3 (Pipeline): Outcome 1 / Output 1.2
edit- GOAL: Create a graph where time is spent and make a prioritized list for improvements.
- WHO: Mukunda, Antoine
TEC3 (Pipeline): Outcome 2 / Output 2.1
edit- GOAL: Select and integrate a code health metric solution into our tooling.
- WHO: JR, ...
No activity
TEC3 (Pipeline): Outcome 3 / Output 3.1
edit- GOALS:
- Adopt more services into Deployment pipeline - task T212801
- cxserver, ORES (partially), citoid, changeprop, cpjobqueue (stretch)
- Deploy eventgate
- Adopt more services into Deployment pipeline - task T212801
- WHO: Dan, Tyler, Lars
- blubberoid.wikimedia.org is a thing https://gerrit.wikimedia.org/r/#/c/operations/dns/+/485814/
- FYI, we talked about this a bit yesterday (package-lock.json): https://phabricator.wikimedia.org/T179229
- Lars: we do need to be able to rebuild a Docker image without it having substantial changes, or else we need to archive all image versions we use in production
- Lars: being able to rebuild is better as it allows, say, Debian package security fixes to be changed
- Lars: this is going to be thorny problem to solve
- Some talk on task https://phabricator.wikimedia.org/T214431
TEC12 (DevProd): Outcome 1 / Output 1.1
edit- GOAL: Conduct interviews with development stakeholders and compile a report that informs future work creation of a rubric.
- WHO: Jeena, Mukunda
- Pretty much finished up interviews with everyone who has contacted me
- Working on the analysis step
TEC13 (Code Health): Outcome 1 / Output 1.1
edit- GOALs:
- Develop and communicate guidelines and best practices for successful Code Stewardship.
- (Continued from Q2) Update/refresh review queue (review process for initial code deployment)
- WHO: JR
Started developing guidelines/best practices this week. Plan to continue working on that the remainder of the week.
TEC13 (Code Health): Outcome 2 / Output 2.2
edit- GOAL: 5 of the 15 prioritized repositories have at least 1 end-to-end test - task T206621
- WHO: Zeljko
- contacted all teams, twice in couple of weeks
- no reply from most teams
- one team said "no thanks" for one repo
- one team said "let's do it" for one repo
- one team already has a patch in gerrit for one repo
- out team discussing phab/phab
TEC13 (Code Health): Outcome 2 / Output 2.3
edit- GOALs:
- Evolve/develop tools and processes to support the PE refactoring effort to improve code health.
- Develop common test strategy that enable teams to engage in more effective and efficient testing practices. (maybe should be output 2.4?)
- WHO: JR, Core Platform Team
Will be meeting with Kunal and Corey re Code Coverage tooling.
TEC13 (Code Health): Outcome 3 / Output 3.2
edit- GOALs:
- Speak at All Hands on the status of Technical Debt
- Engage and coach development teams on their approach to managing technical debt.
- WHO: JR, Core Platform Team
No activity
TEC13 (Code Health): Outcome 4 / Output 4.1
edit- GOALs: Code Health Dashboard with 50% of repositories covered.
- WHO: JR, Core Platform Team
No activity
Other non-goal work
editSelenium
edit- T206621 5 of the 15 prioritized repositories have at least 1 end-to-end test - see TEC13 (Code Health): Outcome 2 / Output 2.2
Gerrit
edit- 2.15.9 announced
- https://groups.google.com/d/msg/repo-discuss/FFdu8h4twuM/RZfXps1ZFQAJ
- a fix for a memory leak
- ETA of after all hands
- Good presentation on gerrit concepts: https://docs.google.com/presentation/d/1C73UgQdzZDw0gzpaEqIC6SPujZJhqamyqO1XOHjH-uk/edit#slide=id.g4d6c16487b_1_808
Phabricator
edit- Mukunda is meeting with Daniel today to discuss server migration and disaster-recovery planning
Jenkins
editQA/Code Health
editEdward Tadros started in Audiences
SCAP
editStandup!
editAntoine
edit- What I plan to do this week
- docker-pkg improvement and tracking dependencies/outdated containers
- Upgrade npm containers to s/python-minimal/python2.7/ required by node-gyp which dies with compiler.ast: no such module
- if time allow get Quibble to run 'npm run selenium-test' / https://phabricator.wikimedia.org/T199116
- What I'm blocked on
- CD book not available here. But will get it next week from Greg
- Other?
- Landing on Sunday noon or so.
Dan
editexcused
- What I plan to do this week
- What I'm blocked on
- Other?
Greg
edit- What I plan to do this week
- All Hands lightning talk prep
- Finalize hiring
- Get a rough agenda for us on Tuesday
- Get the ISSOSTWG recommendation done
- Respond to email from JR to Corey
- What I'm blocked on
- Other?
Jean-Rene
edit- What I plan to do this week
- Focus on Stewardship activities
- Code Review workgroup prework
- What I'm blocked on
- Other?
Jeena
edit- What I plan to do this week
- Analyzing the interview data
- Reading the book
- What I'm blocked on
- Other?
Lars
edit- What I plan to do this week
- continue reading CD book
- continue pondering on Blubberoid CD pipeline, and discuss it with Tyler, Dan, others
- try to find San Fransisco on a map (good read on Oakland and Yerba Buena island https://tinyletter.com/vruba/letters/6-92-small-cat )
- What I'm blocked on
- n/a
- Other?
- gave up trying to deploy locally modified Blubberoid with helm to local minikube: this doesn't seem like a reasonable toolset without a lot of step-by-pasteable-step instructions
Mukunda
edit- What I plan to do this week
- Plan phabricator server migration and disaster recovery with Daniel
- Conduct the train
- Buy a house (Closing this evening!) \o/ time to fix all the things!
- A bit of automating the train
- What I'm blocked on
- Other?
Tyler
edit- What I plan to do this week
- Continue CI image rebuild -- only npm left -- maybe antoine is doing this? cf: line 308 -- tl;dr: I'll work on this today
- Code review:
- Blubber
- Keyholder
- Scap
- FInish moving scap -> Gerrit (wrap up existing patches on phab)
- What I'm blocked on
- If I get 1/2 of what I plan to do this week accomplished time doesn't work like I think it does.
- Other?
- Have only read Foreword of CD book
Zeljko
edit- What I plan to do this week
- T206621 5 of the 15 prioritized repositories have at least 1 end-to-end test
- T214478 The first Selenium test for AbuseFilter
- T214480 The first Selenium test for TimedMediaHandler
- T207044 Give a code health metrics talk at All Hands
- T206621 5 of the 15 prioritized repositories have at least 1 end-to-end test
- What I'm blocked on
- People not replying to my mails :( re T206621
- Other?
- new toy arrived today, mini guitar amplifier, mouse in photo for scale :D https://usercontent.irccloud-cdn.com/file/eesHK5yV/amp.jpeg oh boy :)
- landing in SF on Sunday at 1pm
Grooming
editTeam Kanban Board Review and Triage
edit- closed and touched in the 7 days
- No update for 4 weeks
- No update for 3 weeks
- No update for 2 weeks
- No update for 1 week
- All Open
- Review To Triage column of #releng
Once / month-ish review of backlog(s)
edit- releng Review To Triage column of #releng
- releng-kanban Review unassigned in kanban
- releng-kanban Review 'backlog' colum of -kanban
- releng-next - Review for things we need to put on our kanban backlog
- releng-backlog - oh my, the huge backlog of things...