MediaWiki

Wikimedia Release Engineering Team/Monthly notable accomplishments

< Wikimedia Release Engineering Team

This page lists notable accomplishments for the month as we come up with them during our weekly team meetings.

23/24 Q3 edit

Apr edit

  • new puppetserver in devtools!
  • phorge inbound mail patches upstream
  • Phorge PoC patch to thwart webcrawlers—every line number is a separate page
  • Phabricator 6 mo stale ticket pings
  • Hiding non-canonical diffusion links WIP
  • Phab deploy!
  • Gerrit 3.7 upgrade had a slow down ( https://phabricator.wikimedia.org/T355529 ) Upstream fixed it. I have upgraded on Monday morning and... it works!
  • Scap user interactions consolidated into two functions
  • /srv/mediawiki/php symlink uses is dead \o/
  • Train-dev gerrit auto-upgrade
  • Scap handles pending rollback state from helm
  • SpiderPig demo
  • Patchdemo is running in k8s
  • Phab/Phorge translations are going again, ish
  • scap release process: Added checks to avoid surprises
  • scap backport: Improved behavior on bad change number input.
  • train-dev: Up-to-date docker compose plugin is required now
  • scap backport: support deployable non-production branch backport \o/
  • phabricator upstream patches for error log explosions
  • phabricator -- patch for restricting visibility of priority field
  • catalyst deploying extensions with a MediaWiki environment

Mar edit

  • Train: Nightly security patch failures updating phabricator tasks merged, ready to release
  • GitLab CI: Merged deploys-in-progress reset script
  • Scap3: Two repos have patches for git-fat → git-lfs
  • scap: replaced canary swagger checks with test server httpbb checks
  • Phorge integration with GitLab in its third round of review
  • GitLab webhooks also still going, looks like it'll go through
  • People like scap backport - more patches, fewer things typed into terminals.
  • Train: Security patch notification now working!
  • GitLab webhooks have a more accurate regex for "Bug: TXX"
  • Working towards getting rid of the /srv/mediawiki/php symlink
  • Upgraded GitLab k8s/cloud cluster to new k8s version and documented the process
  • Phab deploy is out (but stuff is broken (not terribly (probably)))
  • scap backport now works for non-extension submodules
  • gitlab cloud runner dependencies
  • scap backport -2 fix merged, need to release
  • scap web demo
  • catalyst project has a patchdemo vm
  • Jenkins is upgraded to the latest version
  • Logstash dashboard for Phab errors is nice and clean

Feb edit

  • Phabricator dump script works again (but also probably isn't necessary?)
  • extended docpub to run multiple doc build jobs
  • Sandeep able to run train-dev environment
  • Gerrit 3.8 upgrade prepped
  • buildkitd security upgrade on gitlab-runners.
  • logstash_checker.py updated to check mw-on-k8s canaries. Merged this morning. Working on scap part today.
  • python2 wheels for zuul2 for bullseye - unlock ability to upgrade Debian on contint boxes
  • Patch for scap backport branch about to be live! (paired w/Jeena)
  • Prepped new Phorge release: https://phabricator.wikimedia.org/T358610
  • Deployed new scap with support for canary checks and git-lfs fixes
  • In progress: updating security issues with patch issues

Jan edit

  • Gerrit train-dev fixed!
  • bd808's account approver thing seems working, maybe?
  • Requested extra Phab/Phorge hardware as backup
  • Deployed persistent volume cleaner
  • Docker images with pyenv with multiple versions of python
  • Worked with upstream phorge to hide the audit application (the post-merge review application for source code)
  • Webhook payloads for GitLab merge request changes—tracked down a bug in upstream
  • phab1004 distro upgrade is done
    • We know we can run in codfw if we have to (it sucks though, let's not)
  • scap clean now works better
  • PVC story nearing completion
  • Helm pending release story, in progress, testing needed
  • Sandeep a very happy linux user
  • We're on Gerrit 3.7
  • Added GitLab support to Git/Reviewers mediawiki.org page.
  • scap stage-train working! We're deleting old versions.

23/24 Q2 edit

Dec edit

  • https://extloc.toolforge.org/ is live and works
  • Fixed cas3 → openid providers in gitlab
  • Differential is dead!
  • Andre is now a "blessed committer" in upstream Phorge and can +2 other folks' patches
  • Not my win, but bd808 has a gitlab account approver bot just about working
  • https://phabricator.wikimedia.org/T351478 in progress!
  • Jenkins job builder jobs into gitlab-ci.yml jobs
  • Systemd-managed dockerfiles for zuul

Nov edit

Oct edit

  • Bitergia user data is now a webapp and we can use it
  • Project Catalyst is underway -- https://wikitech.wikimedia.org/wiki/Catalyst
  • RelEngers can downtime things for phabricator deploys without SRE (although cookbooks are less fiddly)
  • Ran train with Andre!
  • Regular Phab deploy!
  • libs/metrics-platform moved over
  • Clarified language on backports: https://phabricator.wikimedia.org/T344409
  • Investigated hidden repositories in GitLab
  • Prettified puppet compiler output
  • fresnel updates for dependencies
  • Jaime working on project catalyst
  • Gerritlab revised branch naming landed

23/24–Q1 edit

Sep edit

  • Image published for Blubber that is native LLB, no dockerfile anymore
    • implications
      • dockerfile is unnecessary since no one sees the dockerfile—we can customize each llb instruction and what it displays to the users: a name that corresponds to the blubber.yaml config
      • now we have the ability to create our own instructions
      • dockerfile2llb gone! No more external helper images that haven't been maintained just to copy files around—no more cross-platform compatibility/emulation issues
      • llb gets new stuff first—ex: diffop/mergeop https://www.docker.com/blog/mergediff-building-dags-more-efficiently-and-elegantly/
  • Phorge working on the scap3∞ deployment environment
  • Landed 3 upstream phorge patches, 1 is one we've had for years the blocks some tasks rendering (T284397)
  • Patch for T&S could outputs the MediaWiki SUL account along with the phab username (T344303)
  • Wrote a plugin for tox to keep supporting [tox:jenkins] CI config with tox v4 https://gerrit.wikimedia.org/g/integration/tox-jenkins-override unblocking part of the migration from tox v3.
  • Moved Civi CRM CI from Stretch to Bullseye and to php 7.4 (aligning with prod). Paired with Ejegg from FR-tech.
  • Ahmon refactoring GitLab nodepools, buildkitd persistent volume, and containerd debugging
  • Dan deep debugging of source code for containerd
  • GerritLab uses the git credential helper
  • Trusted runners now not running untrusted jobs
  • Authentication working in phorge dev environment
  • Phabricator housekeeping for open tickets assigned for more than two years
  • Phabricator logstash dashboard with filters
  • We've got a weekly Phorge deploy window, of sorts (and can ask for other things)
  • Workaround for security patches touching l10n—fixes bug!
  • Tox v4 migration in progress
  • Added phorge to the scap3 development environment!
  • Fixed a logspam-watch bug (SIGPIPE)

Aug edit

  • Developer Satisfaction Survey got presented
  • Gerrit repo archiving script for GitLab migrations \o/
  • Gerritlab adoption
  • JWT auth changes
  • T272693 - reviewed non-standard phabricator policies
  • Downstream phabricator patches for php8 + logspam
  • Upstream phorge patches for logspam
  • Overwrote feed transaction default query in conduit (T344232#9092848)
  • Scap3 can now be configured to disable the service on secondary hosts: https://phabricator.wikimedia.org/T343447
  • Kokkuri is now using the new gitlab id tokens: https://phabricator.wikimedia.org/T337474
  • We're on Phorge!
  • Gitlab CI-built kask container image deployed today. (https://phabricator.wikimedia.org/T335691)
  • Gitlab local hacks in progress
  • Ahmon passed his CKA! Read Kubernetes in action
  • Merged 3 fixes to Phorge upstream for phab logspam
  • 🎉 Delayed announcement: Jeena's back, and she's a senior software engineer
  • Blubber refactor ripping out dockerfile passing acceptance tests—straight to llb
  • Added another pool to our DO cloud runner pull—memory-optimized
  • Refactored the patch to tune-down staging substatually(sp?)
    • Now there are 4 runner-controller runners running + 4 nodes ready to go
  • GerritLab commits merged to speed up sending patches and does the right things given GitLab's weirdness
  • scap backport bugfix

Jul edit

  • git::clone puppet resource updated
  • LDAP group sync to GitLab
  • Git blame on stack traces within Phatality
  • Buildkitd allowed image list deployed
  • Onboarding Andre Klapper, all sorts of new permissions: phab-root, contint-root, gerrit-root, gerritadmin
  • Bunch of GitLab accounts created ~200 accounts for the mediawiki/* namespace
  • Assisted in GitLab migrations, notably, [[ https://gitlab.wikimedia.org/repos/data-engineering/datahub | datahub ]]

22/23–Q4 edit

April edit

  • Mr. Widget doesn't seem to have broken again.
  • Job to test train branch cut on a daily basis
  • Successfully debugged an obscure buildkitd -> registry interaction
  • A plan exists for Phorge migration
  • Abstract Wikipedia showed up asking for help with a GitLab migration
  • Jelto deployed the privileged buildkitd commit
  • Moving scap backport tests, win in progress
  • Aphlict on a new box---nothing exploded, nobody yelling
  • Jenkins releases configuration fully automated
  • scap train

May edit

Jun edit

  • Blubber acceptance tests
  • docpub in Jenkins
  • Antivandalism patch deployed! (one down; one to go)
  • Learned that we needed to restart php
  • git::clone changes in puppet for specifying a tag
  • git::clone upstream changes now changes the origin
  • WMCS instance caches for NPM via "npm cache verify" to GC the cache
  • buildctl --wait
  • dev-images image rebuilding
  • train backport on Saturday

22/23-Q3 edit

Jan edit

Feb edit

  • Buildkit upstream patch to make client connections more robust in case of loss
  • Smooth wmf.22 train
  • docker-pkg build --list \o/
  • Isolating build containers in buildkit in privilege mode
  • Mr. Widget should be deployable, I think (as soon as secrets are actually stashed)
  • Pushed up patches for JWT (hopefully the final ones :D)
  • Deployed production releases-jenkins using scap3!
  • Patched scap to be smarter about interrupted helm deploys!
  • See Slack://WikiLove thread re: scap backport
    • "90% reduction in time spent in existential dread" ← going on a slide deck somewhere!
  • fixed scap backport for dependencies
  • Buildkit upstream patch to make client connections more robust in case of loss
  • Smooth wmf.22 train
  • docker-pkg build --list \o/

Mar edit

  • New scap self-install in production
  • Learned too much about iptables
  • Moved gitlab-cloud-runner Helm stuff to Terraform :)
  • Thundering herd testing passed—k8s can handle 100 concurrent job
  • Phab release prepped
  • docker-gc is blubberized and kokkorized
  • found and deleted docker image based on obsolete debian version (stretch)
  • made terraform plan run before merge
  • Tentative optimism for CKA
  • Monte's having success using the phab api to build different views of tasks
  • Mr Widget deployed
  • gitlab-cloud-runner stress tests successful!
  • Dockerhub mirror admission controller
  • Reggie JWT auth enabled in gitlab cloud runners
  • Gitlab cloud runners ready to be made available instancewide
  • We made a staging cluster
  • Docker-gc repo using kokkori
  • Gerrit progress bars

22/23-Q2 edit

Oct edit

  • Blubber's gitlab ci file is enough to move the project over
  • Setup in gitlab ci: jwt auth and buildkit
  • New features in blubber: builders improvement from Jaime + feature for running variant from Jaime
  • Jaime + Antoine upgraded scap in the dev-tools project
  • Team likes each other
  • Train feels less stressful lately
  • GitLab runners in K8s now running buildkit with caching—nothing to a k8s cluster
  • Scap builds docs on GitLab

Nov edit

  • Scap repo is fully moved over to GitLab
  • Internship opportunities!
  • Dan's daughter can now pedal a bike
  • Critical systems list
  • Antoine fixed mixed-case usernames in Gerrit
  • Gerrit upgrade
  • Scap3 dev env: https://gitlab.wikimedia.org/repos/releng/scap3-dev
  • Phabricator is now hosted on a new box at phab1004 and deployed with scap
  • registry-based caching
  • reggie is in use and working
  • Autoscaling
  • Kokkuri

Dec edit

22/23-Q1 edit

July edit

  • We don't specifically have any reason to think our GitLab instance has been owned, necessarily.
  • Small merges for mwpresync
  • GitLab runner config management changes merged!
  • Increased the team knowledge of scap3
  • Pending major update for GitLab
  • Scap-o-scap installed in beta! \o/

August edit

  • Train-blockers toolforge scrapes from phab \o/
  • Nagged GitLab into updating their FAQ: https://gitlab.com/gitlab-org/gitlab/-/issues/363212#note_1066797431
  • Clare used scap backport for real
  • Phabricator (probably) deploys from scap 3
  • Beta exists still
  • Chad re-earning t-shirt
  • Upgraded Gerrit from 3.4.4 to 3.4.5
  • Scap-backport improvements, seeing increased use
  • Renewed GitLab relationship!
  • Moved Gerrit replica server!
  • Yet another successful train, automatic edition this time!
  • Team reviews are fast!
  • Gitlab JWT STUFF MERGEDDDDDD \o/

Sept edit

21/22-Q4 edit

April edit

  • Jamie deployed the train
  • Jaime rolled back the train
  • GPG keysigning
  • Fixed bug in proxy balancing in scap
  • Scap deploy-promote
  • Scap 4.7.0-fully out; 4.7.1 going out this week!
  • Scap 4.7.1 fixes cross-datacenter pulls!
  • New Phatality deploy
  • Scap backport

May edit

  • Our long, grinding efforts at deployment training are finally starting to result in more people doing deploys (well, ok, they've resulted in Clare doing deploys, anyway) \o/
  • Rolled back train five times
  • Deployment tooling just kind of sucks less than it used to
  • Merged scap backport
  • scap stage-train \o/
  • Finally got rid of the generic service-pipeline-* jobs and migrated remaining 23 projects to use bespoke `.pipeline/config.yaml` based jobs
  • gitlab-a-thon
    • We found a a whooooole lot of blockers
    • Dan being a good open-source citizen: https://github.com/moby/buildkit/pull/2868
      • JWT implements oauth2
      • Could be used to authorize push access to namespace based on project path
  • Root access on phabricator
  • Updated Jenkins for Security—which broke Jenkins for a while
  • I think I finally remember how to use a standalone puppetmaster
  • ERC going well and DEI moving ahead
  • Dan got changes to buildkit merged upstream
  • Seems like we're pretty close to how auth will work for publishing images from GL
  • serviceops are plodding ahead on GitLab physical machines
  • CI for blubber in gitlab
  • update scap backport to work with new zuul plugin
  • new tests for scap backport
  • scap tests run without deprecation warnings (for stretch, buster, and bullseye)
  • Giuseppe plans to enable always-restart-php-fpm on Thursday.
  • Docs for GitLab are somewhat less crappy than they were a week ago
  • Upgraded Gerrit in train-dev to match production
  • Hired Backfill
  • Phabricator deployment runbook https://wikitech.wikimedia.org/wiki/Phabricator/Deployment

June edit

  • GitLab Sprint summary by Brennen https://phabricator.wikimedia.org/phame/post/view/288/gitlab-a-thon/
  • We have GitLab on new metal, and can probably enable GL Container Registry \o/
  • We know more about git than we did in May
  • Functional scap already self-installed in prod
  • JWT presentation!
  • Phab deployment has a runbook https://wikitech.wikimedia.org/wiki/Phabricator/Deployment
  • scap scap
    • successfully used it
  • ITCs getting done
  • somebody noticed phab
  • scap backport revert
  • scap rollback
  • scap pushing commits with a shared ssh key
  • Chad has some equipment: monitor and docking station required
  • Dan's rested
  • uneventful train
  • Antoine got rest api in gerrit working for jsonschema

21/22-Q3 edit

January edit

  • New release of Java Gearman plugin
  • Gerrit upgrade! (3.3.6 → 3.3.9)
  • Ahmon debugging of contint1001
  • mwcli coolest tool honorable mention
  • Production benchmarks of yaml parsing for MW
  • New test environment progress for GitLab
  • Plan for trusted runners coming together
  • Scap prep auto working in beta (on the way to prod)
  • Gerrit 3.3→3.4 prework to solve javascript incompat
  • CI-agents upgrade prep for stretch -> bullseye
  • MWCLI progress and GoLand for IntelliJ

February edit

  • logspam-watch is fast now because Ahmon is good at computers and can tolerate modifying Perl
  • Wrapping up MediaWiki settings loader expedition caching—abstractions for MediaWiki config settings—they can declare how they're cached and how long (with probablistic early expiry to help with lock contention)
  • Mukunda has freed himself from train :D
  • Jaime joined, already has a scap patchset
  • Things we didn't do but benefit from: New GitLab test instance in devtools project: https://gitlab.devtools.wmcloud.org/ --> same puppet as prod!
  • Tyler's DigitalOcean exploration spike
  • Scap prep auto! is neat <3
  • Antoine's qemu learnings!
  • moving train-dev to helm3

March edit

  • Scap 4.4.1 released (includes container image building stuff) \o/
  • Brennen talked publicly and was not shamed by it
  • Onboarding Jaime!
  • Dan's blubber demo!
    • docker build can use a blubber file directly now
    • Supports bd808 and developer tooling
    • Opens up for a more flexible release model
  • Job post posted
  • scap backport exits if change is not mergeable
  • Trainsperiment was instructive AND SUCCESSFUL!
  • Got a working mw container image deployed from deploy1002 woohoo 🐳🎉
  • Fixed check-new-errors script (extremely tiny win) \o/
  • Jaime's first scap release!
  • I don't have to press enter any more! (added -n)
  • Train schedule worked!
  • deploy-promote upstreamed to scap!

21/22-Q2 edit

December edit

November edit

  • Deploy MediaWiki manually in Train-Dev to k8s!
  • Pipeline supports copying files out of containers as published artifacts in Jenkins!
  • Deployed Tuesday with a script thing!
  • Scap 4.0.3 release!
    • record time for scap release
  • Worked?!l out a path forward for GitLab runner architecture with SRE; moved projects to top-level group with runners: https://gitlab.wikimedia.org/repos \o/

October edit

  • Scap 4 release!! Now with more Python
  • GitLab upgrades
  • Gerrit added to train-dev unblocks scap backport dev
  • Data^3d is functional and ready for demos
  • More people using train dev
  • Antoine uses chrome^Hium |not sure that is a win|
  • Gerrit upgrade to 3.3.6 (fixes some minor ui glitches)
  • Client side errors are blocking the train
  • GitLab is open to all
  • dashboards getting close to demo-worthy?  http://173.17.185.55:8001/-/dashboards/project-metrics?project=PHID-PROJ-uier7rukzszoewbhj7ja

21/22-Q1 edit

September edit

August edit

  • Started dev-images to buster
  • Gerrit 3.3
  • Successful php_fpm_always_restart: true test (https://phabricator.wikimedia.org/T266055)
  • GitLab soft launch 
  • migrated mw-cli to gitlab, got docker-in-docker integration tests working (thanks addshore)
  • Finished dev-images to buster
  • Merged workboard metrics code!
    • Reviewed on GitLab
    • GitLab code review experience ftw
  • A successful (?) interaction with GitLab upstream
    • GitLab upstream merge request in progress
  • Node 14 patch updated
  • Emacs installed on releasesXXXX servers
  • Mukunda learned how to extend datasette with ddd/phab functionality

July edit

  • Projects exist on GitLab
  • Gerrit upgrade pairing
  • Published local dev cli

20/21-Q4 edit

April edit

  • Scap 3.17.1 tagged
  • GitLab Ansible code review
  • Deployment trainings

May edit

  • Quibble 0.0.47
  • Jenkins upgrade to latest LTS
  • Released new upstream Jenkins Gearman plugin
  • Wikitech Gerrit docs updated
  • data³ used successfully to extract train blocker stats from Phabricator
    • Added transaction metadata to Phabricator task transactions api so that tools can get more detailed transaction details required for the train blockers analysis.
  • Quibble weekly meetings
  • gitlab.wikimedia.org is running (still needs cas registration)
  • Documented the process for adding languages to phabricator, as well as maintaining the translation strings from translatewiki. All of this is now documented in the README for the phabricator translation repo. That change can be seen here: https://phabricator.wikimedia.org/rPHTR0de9c13ef996326a99d6320f4c26669901f3aff4

June edit

  • Knowledge transfer on Gerrit deployments
  • Running gitlab.wikimedia.org, real use now
  • Guiseppe reports: curl -H 'Host: en.wikipedia.org' https://staging.svc.eqiad.wmnet:4444/wiki/Main_Page works
  • Automatic notification of security patch application failures.  One real use so far.

20/21-Q3 edit

January edit

  • Update dev images to split apache and php containers for local dev
  • Gerrit security bug discovery and deployed fix by Antoine
  • In sync with Gerrit upstream war (Java compiled code)
  • Target releases for apt packages in blubber deployed so wuvi can use npm

February edit

  • PipelineLib fully working on releases-jenkins.wikimedia.org
  • Rust introduction talk (not strictly RelEng business)
  • logspam-watch
    • Minimum hits consolidation feature
    • Error histograms, at-a-glance status indicators (emoji, it's emoji), improved UTF-8 handling and terminal resizing
  • Gearman plugin deployed. Merged bunch of pending changes + a fork from GoodData company which adds support for Pipeline jobs

March edit

  • PipelineLib fully working on releases-jenkins.wikimedia.org
  • Credentials added to pipelinelib
  • S&F contractors underway with production GitLab configuration
  • Terrible script for finding status of production errors on logstash dashboard
  • Ability to deploy phatality updates
  • scap apply-patches much improved

20/21-Q2 edit

October edit

  • GitLab consultation

November edit

  • Gerrit security upgrade
  • Gerrit grafana dashboard
  • Created pipelinelib-experimental cloud project for working on pipelinelib
  • Scap 3.16.0 release (tagged, waiting on SRE now)
  • logspam-watch improvements
  • apparently scap apply-patches may possibly work in some circumstances
  • Upstream fix for shallow cloning in git: https://github.com/git/git/commit/fb3d1a083f776f02caa514cad8b232d8b974641f

December edit

  • Scap 3.16.0 released and deployed
  • Dropped scap plugins from mw-config
  • unconditional restart on deploy for opcache corruption deployed
  • https://doc-stage.wmcloud.org/ , staging area for doc.wikimedia.org. Next prod then update related docs.
  • Scap source formatted with Black now
  • Runnable runbook blog

20/21-Q1 edit

July edit

  • CI now supports REL1_35 branches (and ignores REL1_33).
  • Eliminate elasticsearch dependency from Phabricator search engine
  • Cassandra Docker image
  • Jenkins node Docker image cleanup & re-onlining after disk space recovers
  • Collection of disk space stats on Jenkins workers
  • Credentials and environment variables in PipelineLib
  • Blubber now correctly supports multi-stage artifact copies

August edit

  • Reduced the number of non-failure FAILURE messages in CI
  • After 9 months, Aphlict is finally back.
  • Scap version 3.15.0 released (in git, if not as .deb yet)

September edit

19/20-Q4 edit

April edit

  • Docker images published on buster-based contint2001 (as part of general temporary switch-over from contint1001 to 2001 for buster migration)
  • Composer is now authenticated with github
  • Dropped basic PHP 7.1 testing from CI
  • Published Kubernetes migration tutorial
  • Phabricator milestone columns can now be moved on workboards
  • Phabricator workboards can be sorted by most recent activity.
  • Tech talk on PGP basics
  • "Cache of wmf-config/InitialiseSettings often 1 step behind" fixed! - task T236104

May edit

  • The release train branch cut is now an automatic job
  • Wikimedia Portals build and WDQS data release jobs moved to docker
  • The Continuous Integration instances on WMCS have been fully migrated off Jessie! T236576
  • Scap 1.14.0 released (by releng) and deployed (by serviceops)
  • Documentation for setting up a local dev environment for Phabricator: https://www.mediawiki.org/wiki/Phabricator/Local_Dev_Environment
  • CI server (contint) migrated to buster!

June edit

  • Scap plugins will move from mediawiki-config to scap git repository with the next release.
  • Deployment script added to deployment-charts for deploying to k8s
  • MediaWiki branch cuts are fully automated, at last!!!!
  • TMH job runner works in MediaWiki-Docker
  • Interactive logspam-watch
  • Gerrit 3.2.2

19/20-Q3 edit

January edit

February edit

March edit

  • scap has its first integration test
  • MediaWiki tarball / Wikimedia production are now PHP 7.4-compatible.
  • All extension and skin repos are now being tested against PHP 7.4.
  • Analytics Refinery release job isolated into a Docker container.

19/20–Q2 edit

December edit

November edit

  • branch.py for cutting the branch for train
  • logspam-watch for tailing logfiles

October edit

19/20-Q1 edit

September edit

  • Scap 3.12.1-1 released/deployed
  • Refactored Zuul layout to use per-branch pipelines
  • quibble -c Lets you run arbitrary code against a working MediaWiki install
  • The phabricator "Report Error Code" form (https://phabricator.wikimedia.org/maniphest/task/edit/form/46/ ) has been updated with separate fields for the stack trace and error code/request id.
  • T232608 Delete selenium-daily-beta-EXTENSION Jenkins jobs that are broken more than 30 days
  • Write cached config to JSON as well as serialised PHP https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/533592/ (first step towards a saner config)
  • MediaWiki PHP support target modernised from 7.0+ to 7.2+ for 1.34 onwards. https://phabricator.wikimedia.org/T228342
  • Quibble 0.0.35 release
  • 1.34.0-wmf.24 branch cut was done /mostly/ with branch.py instead of make-wmf-branch.php (some small bugs remain to work out but it's very close)
  • Creating accounts was broken on beta cluster since 2019-09-08. It was fixed today (2019-09-25). https://phabricator.wikimedia.org/T232796
  • Phatality extension for Kibana deployed to production and used for reporting production errors into Phabricator.
  • Train blocker tasks created for 1.35.0-wmf.1—1.35.0-wmf.25
  • Dev images are now automatically created as part of postmerge via the pipeline for MediaWiki

August edit

  • Read only "gerrit-replica" active, handling 10% of all traffic (read from phab)
  • https://time.releng.team ¯\_(ツ)_/¯
  • Scap 3.12.0-1 in production

July edit

18/19-Q4 edit

June edit

May edit

April edit

  • Phabricator vandalism rollback tool completed 🎉 (blog post? 😉)
  • Upgrade Zuul to 2.5.1-wmf6 (which unblocks the Gerrit upgrade to 2.16) - https://phabricator.wikimedia.org/T208426
  • Team offsite in Chicago

18/19-Q3 edit

March edit

Feb edit

Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments&oldid=6498510"