Wikimedia Release Engineering Team/Checkin archive/2024-07-24
2024-07-24
edit🏆 Wins/winterrogation
edit- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
- July 2024
- "[WE6.2.1] Publish pre-train single version containers" is now on Phab <https://phabricator.wikimedia.org/T369115>
- Submitted a WMCS themed talk proposal for WikiCon North America (<https://wikiconference.org/wiki/Submissions:2024/What%27s_new_with_Wikimedia_Cloud_Services>)
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/367 kubernetes: Make k8s deployment failures fatal
- https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/101 Allow empty requirements list to enable Python builder
- Cleaned up scap, kokkuri and mediawiki/services/machinetranslation
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/368 Prompt for log message if not supplied on command line
- Deployed gitlab-runner v17.0.0. on gitlab-cloud-runners.
- https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1047158 mw-web: Add traindev environment — testing canary deployments should work!
- https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/76 enable mw-web canary / main k8s deployment
- Fixed train-dev for 1.43.0-wmf.11
- Blubber v1.0.0 has been published
- Native BuildKit LLB (Low-Level Build) instructions (no more reliance on Dockerfile)
- Refactored to support all `docker build` and `docker buildx build` options
- Supports attestations stored alongside images in the registry, provenance and SBOM
- Looks like phorge 2024.19 stable release merges cleanly, as does phorge/master
- Settled on a squash commit template: https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/64
- Enabled image diff in Gerrit: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/902211/1/static/images/project-logos/dkwikimedia.png
- Further refinement in upstream code would allow some more tuning, screenshots at https://phabricator.wikimedia.org/T341291#9939660
- Diff is based on Resemble.js library, demo at http://rsmbl.github.io/Resemble.js/
- MediaWiki train!!!! Win win!
- Started building out an on-wiki space to keep track of group-1 things: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Group_-1
- https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/100: Change kokkuri to use v2.0.3 (newest buildctl)
- https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/107 Fix duplicate pipelines when pushing merge request
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/377 sync: Refactor deployment stages
- thx to swfrench for reviews (nice)
- Dropped logstash_checker.py from puppet (nice :))
- Filed https://phabricator.wikimedia.org/T369532: Update ldap-sync-bot token
- We have a director
- Met with Guillaume re: archiva migration, publishing doesn't seem to be working but we'll debug that this afternoon.
- Discussed gitlab-behind-loadbalancing ideas with Collab Services, they may be dissuaded from changing SSH remotes
- Blubber can now build directly from Git URLs. This makes the provenance more "complete"
- Adding support for remote build contexts to Kokkuri. In the GitLab CI context this means passing the MR Git URL directly to BuildKit
- Rephrased the GitLab account approval banner because confusion: https://phabricator.wikimedia.org/T369698
- Updated the GitLab approval form to remove with the "specific reason" for GitLab
- Continued with writing random Phab downstream and Phorge upstream tech debt patches and bug fixes
- Talked with folks who like quarterly Phab account metrics about lies, damned lies, and statistics (thanks Tyler)
- https://phabricator.wikimedia.org/T369862 Upgrade to buildkit v0.15.0
- ldap-sync-bot token has been renewed.
- Braggedlogged about latest Phabricator improvements in https://phabricator.wikimedia.org/J316
- Phab/Phorge up to date with upstream stable (next step: rolling release) oooh
- invented https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Group_-1/Progress_reports/2024-07-11 to put weekly reports on-wiki
- dzahn got Gerrit on nftables, no obvious breakage
- Sorta got Maven publishing working at https://gitlab.wikimedia.org/repos/maven/maven-test-project
- Open question: Any reason to care about just using a single project for all kinds of packages?
- https://phabricator.wikimedia.org/T367322#9987048
- Swapping out VMs for CI/Integration-project --- cumin done! deb building in progress.
- Explored running old-timey patchdemo in k8s alongside catalyst
- Another routine Phab/Phorge deploy
- Some less dependencies upon EOL Debian Buster
- Merged modifications for repos/releng/release
- (make-container-image), operations/mediawiki-config
- (MWMultiversion.php), and scap to support FORCE_MW_VERSION, to support
single image container images.
- Andre's "scap train" mods deployed.
- scap changes to better support alternate stage_dir:
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/390 cli.py: Change default path of mediawiki sync lockfile
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/389 Make default history_log follow stage_dir
- Southparkfan hammering away at beta mess
- streaming logs from Catalyst environments
Stuff from last time
edit📅 Vacations/Important dates
edit- https://office.wikimedia.org/wiki/HR_Corner/Holiday_List#2024
- https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off
- July 1-9: Jaime
- July 4-5: US staff holiday
- July 4-5: Andre (festival; CZ Holiday)
- July 12: Bryan
- July 12–15: Ahmon
- July 12: Brennen WF Las Vegas
- July 12-22: Dan
- July 19: Bryan
- July 25-31: Ahmon
- July 26: Bryan
- Aug 02: Bryan
- Aug 05-08: Dan
- Fri 09 Aug – Global holiday: International Day of the World’s Indigenous Peoples
- Aug 12: Dan
- Mon 12 Aug-16: Ahmon out
- Mon 12 Aug - Fri 23 Aug: Antoine
- Aug 16: Bryan
- Aug 23: Bryan
- Sun 25 Aug - 03 Sep: Brennen 🔥
- Aug 30: Bryan
- Sept 02: US Labor day (WMF US holiday)
- Sept 06: Bryan
- Sept 13: Bryan
- Sept 18-19: Brennen Winfield
- Sept 19-20, 23: Bryan Riot Fest in Chicago!
- Sept 27: Bryan
- Sept??? Jeena
- Oct 03-06: WikiCon North America (Indianapolis)
Future
edit🔥🚂 Train
edit- https://versions.toolforge.org/
- https://train-blockers.toolforge.org/
- https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
Rotation
edit- 13 May (13-17) - 1.43.0-wmf.5 – Antoine + Andre (Jeena as backup)
- 20 May (20-24) - 1.43.0-wmf.6 – Andre + Antoine (you can do it Andre!) (Antoine out Mon, Ahmon out Fri)
- 27 May (27-31) - 1.43.0-wmf.7 – Ahmon + Andre (Memorial day Monday)
- 03 Jun (03-07) - 1.43.0-wmf.8 – Dduvall + Ahmon (Brennen out)
- 10 Jun (10-14) - 1.43.0-wmf.9 – Brennen + Ahmon
- 17 Jun (17-21) – 1.43.0-wmf.10 – Jaime + Brennen
- 24 Jun (24-28) – 1.43.0-wmf.11 – Jeena + Jaime
- 01 Jul (02–04) – 1.43.0-wmf.12 – Antoine + Jeena
- 08 Jul (08-12) – 1.43.0-wmf.13 – Andre + Antoine (Ahmon + Dan out Friday)
- 15 Jul (15-19) – 1.43.0-wmf.14 – Ahmon + Andre (Dan Out)
- 22 Jul (22-26) – 1.43.0-wmf.15 – Dan + Ahmon (Ahmon out Thurs)
- 29 Jul (19-Aug 02) – 1.43.0-wmf.16 – Brennen + Dan (Ahmon out)
- 05 Aug (05-09) – 1.43.0-wmf.17 – Jaime + Brennen (Dan out, Global holiday Friday)
- 12 Aug (12–16) – 1.43.0-wmf.18 – Jeena + Jaime (Ahmon out, Antoine out)
- 19 Aug (19–23) – 1.43.0-wmf.19 – Andre + Jeena (Antoine out)
- 26 Aug (26–30) – 1.43.0-wmf.20 – Antoine + Andre (Brennen out)
- 02 Sep (02–06) – 1.43.0-wmf.21 – Ahmon + Antoine (US holiday Monday, Brennen out Tues)
- 09 Sep (09–13) – 1.43.0-wmf.22 – Dan + Ahmon
- 16 Sep (16–20) – 1.43.0-wmf.23 – Jaime + Dan (Brennen out)
- 23 Sep (23–27) – 1.43.0-wmf.24 – Brennne + Jaime
Team Discussions
editFill your day with webforms
edit- Jul 24 (today): All-Hands 2025 head-count due: https://docs.google.com/forms/d/1Pdq9KKWBHdCja7hQ97aCqgtEO2b_GhI_y-3mijt_qTI/viewform?edit_requested=true
- Jul 30 (2 weeks-ish): Culture Amp "Culture of Review" 10 questions: https://wikimedia.cultureamp.com/adhoc/wikimedia?code=66844329fe50970033be6876
- Offsite dates: https://docs.google.com/forms/d/e/1FAIpQLSdFM8C7RaRSXYgxefMN3ak03A8Vj1v8N-1wtOjOfCMlAtap5Q/viewform
K8s Deployment Prep
edit- Seems like a thing we're going to have to do as MW ceases to bare metal in prod
- Talked about a k8s in deployment-prep for misc services/sca/scb
- Magnum
- Using opentofu to do things
- we should get into the task tree re:removing all the bare metal things from puppet
Let's do some inbox triage: https://phabricator.wikimedia.org/maniphest/query/7vRDrcVnt8OI/#R
🌻 Open source/Upstream contributions
edit