MediaWiki

SQL/XML Dumps/Becoming a dumps co-maintainer/Deploying MW changes

< SQL/XML Dumps

Sooner or later you will want to backport and deploy a patch to MediaWiki, or to deploy configuration changes. This is my "short" version of how to get that done.

Bare bones of backporting

edit

Before you start

edit

You can go to trainings on how to do this! See https://wikitech.wikimedia.org/wiki/Deployments/Training

Official docs on deployment:

NOTE: at this writing mwdebug1003 is on stretch and mwdebug1001,2 are on buster; use the buster ones for testing unless there is a stretch-specific thing you are checking.

Creating the backport patch from a change merged in core

edit

Backporting in gerrit is a matter of figuring out which versions are deployed where that need the backport; you can look at https://versions.toolforge.org/ and then if there are entries that list multiple versions for a group of wikis, you can go to any current copy of the mediawiki-config repo, grep for the version(s) that interest you in wikiversions.json, and determine which specific wikis run those versions. once you've determined which versions should get the backport, you can do the backport directly in gerrit, by going to the merged changeset in master, in the upper right corner clicking the three dots for the extended menu, selecting "cherry pick", typing "wmf" to get the suggestion list, and as much needed of the branch to see the desired branch in the suggestion list, and choose that. The rest should be self-explanatory.

If you are the +2-er on master, it is fie for you to also do the backport in gerrit (wait until master +2 post-merge jenkins completes, then you can do it) and +2 it there

Anyone in the Platform Engineering team can grab a ubn ticket w/ a patch and herd it along, there is no set procedure, but it is preferred that someone on the clinic duty team grab and do it, including the deploy.

Using backport windows or not

edit

You don't have to wait for a window if it's really OMG UBN (as opposed to "we'd like it done soon but things are not on fire"), but then you do need to make sure someone from sre is around in case there are issues with the deploy, and you should announce to releng too. channels for these: #wikimedia-sre (to ask for someone to pay attention to the deploy), #wikimedia-releng (to notifiy releng), #wikimedia-operations (to discuss each step of the deploy as it happens).

If you are using a backport/deploy window, you don't need to ask for an sre since these windows are scheduled on the deployment calendar and known in advance, but you do need to add your info to the calendar: https://wikitech.wikimedia.org/wiki/Deployments

you can +2 and merge the backport ahead of time but you should time it so that the merge finishes up around the start of the window or later, instead of completing well beforehand. If there is more than one patch in the window, and you're touching the same file, don't do this.

Make sure you know what to test

edit

Verify that you can reproduce the bug that's being fixed, before you start, so that you know the procedure to check if the error is gone on the appropriate mwdebug host.

If it's not an error but a feature, verify that you know how to use the feature and test for its presence on the right mwdebug host.

Working with the backport window managers, if any

edit

If you are deploying in a window, there is a list of deployers who might manage the window on the Deployments calendar. In practice one of those people will announce that they are managing the window for that time. They will call out each person in the slot on the calendar and verify that the person is present for the deployment. Then they will go through the deployments, typically in the order listed on the calendar. This may be shuffled around if folks are waiting for patches to merge. Although the person managing the window can deploy your patch, they are usually quite happy for you to do it yourself.

Actually doing the deploy: steps on deploy host

edit

On deploy1002 in /srv/mediawiki-staging/php-your-version, git status and see what is outstanding. If there are security patches committed locally but not merged, this is ok. Such patches will be listed first before the top commit of the branch, and marked as SECURITY in the commit message. If some extensions have unmerged commits, you can ignore them. IF THERE ARE OTHER CHANGES NOT MERGED OR NOT COMMITTED, STOP and get advice.

git log to see what is going on, same as above.

If all is good, git fetch to pull in your backported changes

git log to be sure only your change got pulled in; you can specifically compare your local HEAD to upstream by "git log -p HEAD..@{u}"

git merge

If there were unmerged security changes on top, git rebase (no args needed)

Testing on mwdebug host

edit

ssh to mwdebugX (decide if you want buster or stretch), and, as your regular user, scap pull from any directory; NOTE THAT this does an rsync so it picks up everything in the directory.

Verify that your test of the bug or feature does what is expected.

Finishing up

edit

Back on deploy1002, from /srv/mediawiki-staging you can now do scap sync-file php-version/.../path-to-change.php "message here"

A nice format for the message with the linked gerrit commit and the linked phab task is available at https://wikitech.wikimedia.org/wiki/Backport_windows/Deployers and you should use it! Basically it is this:

'Backport: [[gerrit:GERRIT-NUMBER|COMMIT-MESSAGE (PHABRICATOR-TASK)]]'

Wait a bit for things not to explode

Profit!

Config changes

edit

These are trickier because you may be deploying multiple files in one patch. In general, this is discouraged because it is too easy to get the order of the sync-file commands wrong and to push the wrong file out first, breaking the wikis.

However there are no branches to worry about and so no version checking that needs to be done, so in that sense config changes are simpler. Additionally, merges of config changes are relatively fast, so they should not be done ahead of time.

Config changes that impact user-facing behavior should have been approved by community concensus; as dumps maintainers, that's not usually an issue for our work.

Retrieved from "https://www.mediawiki.org/w/index.php?title=SQL/XML_Dumps/Becoming_a_dumps_co-maintainer/Deploying_MW_changes&oldid=4496666"