Wikimedia Release Engineering Team/Checkin archive/2023-01-25

2023-01-25

📅 Vacations/Important dates

https://office.wikimedia.org/wiki/HR_Corner/Holiday_List#2023

https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off

January 2023

• 2 Mon Jan: New Year's day observed
• 16 Mon Jan: Martin Luther King Jr Day
• 1-15 Jan: Jaime
• 20 Fri Jan: Dan 🎂
• 20 Fri Jan: Brennen ½ day

February 2023

• 20 Mon Feb: U.S. Presidents' Day

March 2023

• 17 Fri: Tyler (guessing, running https://www.madmooseevents.com/canyonlands-half-marathon on the 18th 😬)

🔥🚂 Train

https://tools.wmflabs.org/versions/

https://train-blockers.toolforge.org/

https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar

• 2 Jan - wmf.17 - Dan + Antoine (Jaime out)
• 9 Jan - wmf.18 - Jeena + Dan (Jaime out)
• 16 Jan - wmf.19 - Jaime + Jeena
• 23 Jan - wmf.20 - Brennen + Jaime
• 30 Jan - wmf.21 - Ahmon + Brennen
• 6 Feb - wmf.22 - Chad + Ahmon
• 13 Feb - wmf.23 - Antoine + Chad (todo move antoine to week after)

Team discussions

Quickies

• Chad demo of harbor
  • Harbor evaluated 5 years ago
  • GC with younggen
  • Helm chart repo
  • OIDC
  • Preseeding with dragonfly
  • Stable + mature
  • Terraform provider as well
  • Webhooks for events
  • Prometheus endpoint
  • Running on test k8s in DO
  • Questions:
    • Configurable outside GUI? Yep, full API, programatic managment

• Quarter goals: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/202223Q3
• Mise-en-place goals: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Sprint/GitLab_mise_en_place
• Still waiting on this? give releng access to logs to debug buildkit-to-wmf-registry publishing?

Backport forbidden patch types

• "Forbidden patch types": https://wikitech.wikimedia.org/wiki/Backport_windows
  • "Single patches that require more than one sync - in other words, changes to multiple files which depend on each other."
  • Do we care with k8s + scap backport
  • New problems: extension.json parsed outside of php restart :D https://wikitech.wikimedia.org/wiki/Incidents/2023-01-17_MediaWiki
  • Ignored in practice for years
  • TODO: tweak language, still somewhat needed. Goes away on Full K8s -- thcipriani

GitLab Migrations

• https://phabricator.wikimedia.org/T273525
• Still needed: branch renaming for master→main for tools-release
• How/when do we want to handle that in the process of migration?

Discussion

• We have to do it :)
• In between time: useful to have master branch in gerrit, have two branches in gitlab, then change default upstream
• Moving over to main a pre-req for gitlab?
• Zuul will only work with master in gerrit
• Leave master branch for a while? Create new refs/heads/main?
• TODO: create a "main" on repos/releng/release, leave master for now, deal with fallout later -- thcipriani

Shared runners

• One of our goals: Launch instance-wide runners
• Same sprint as scaling buildkitd
• Interest in IRC/Slack in using Kokkuri, blocked on runner availability
• Talked to a few of you in 1-on-1s about ideas for this, better to talk together

Discussion

• FInish out the sprint, it's the last thing, then we'll be done
• Value in getting kokkuri usage
• Let's let people use WMCS with the knowlege that they'll run into space issue?
• Better to let people have early access knowing their jobs will be slower?
• We can continue to promise things will get better'
  • Plus things get better magically
• Approach: have instance-wide runners pick them up (which would be WMCS), then after sprint have scalable buildkitd, then what do with WMCS runners?
  • Leave around for special cases or retire?
  • Alternative: up replicaset and make DO available now and continue working on autoscaling seperately?
• Proposal: Enable WMCS runners site-wide, continue hacking on buildkitd hacking, reassess
• TODO: dancy to make wmcs instance-wide/work with brennen
  • May be a matter of changing the registration token for the runners