SQL/XML Dumps/Useful skills checklist for dumps co-maintainers
Some skills you'll want to have to become a dumps co-maintainer, not in any particular order
DRAFT and just the start of a list etc. Don't take it too seriously.
Part one: Keeping things working while the primary maintainer is away
edit- Be able to run page content generation dumps manually for wikidata at the right time of the full run
- Be able to determine which snapshot host runs which dumps and shoot them or restart them manually in case of problems
- Be able to run 7z recompression manually for page content dumps for a given wiki, if needed
- Be able to run md5 hash generation for bz2 or 7z files for a given wiki manually, if needed
- Be able to determine whether a report on Phabricator is our issue or someone else's
- Example: a report that new analytics files are not available for public download)
- ...
Part two: Well-defined tasks
edit- Be able to handle mirror requests with the aid of someone with puppet merge rights:
- Be able to guide other teams through the process of setting up dumps for a new service or extension (with the aid of someone with puppet merge rights)
- Be able to bring up a new snapshot host with the testbed role (with the aid of someone with puppet merge rights)
- Be able to bring up a new dumpsdata host in a secondary role (with the aid of someone with puppet merge rights)
- Be able to swap roles of snapshot hosts
- Be able to swap roles of dumpsdata hosts
- Be able to decommission snapshot and dumpsdata hosts
- Be able to schedule maintenance on a given host or hosts according to when dumps of particular types are not running
- Be able to work on capacity planning, understanding current and past baselines and being able to project growth
- ...
Part three: General troubleshooting
edit- Be able to examine db queries in dumps code and verify that they are optimized well
- Be able to do performance tests of db queries in production on servers in the inactive dc (in the dumps group!!)
- Be able to track down the source of a dumps slowdown
- Example: a change in MW code that causes page content to be loaded upon retrieval of the revision metadata
- Be able to watch incoming gerrit changes that could impact the dumps
- ...
Part four: Testing
edit- Be able to run dumps tests against a local install
- Be able to test batches (etc) in docker dumps testbed when that code is ready
- Be able to fix broken mw code sync or puppet sync on beta, to facilitate dumps testing
- Be able to test both XML/SQL and "misc" dumps on beta
- Be able to test a single dump step at scale on the production cluster (small wikis only, writing to a temp area)
- Be able to test a single "misc" (not XML/SQL dump) in production (small jobs only, writing to a temp area)
- Be able to do code review of gerrit changes that add to dumps or impact dumps code
- ...
Part five: Deploying
edit- Be able to rebuild the mwbzutils package after making updates to it, and deploy it to the snapshot hosts
- Be able to deploy a change to the XML/SQL dumps repo to the snapshot hosts
- ...
Part six: Enhancements
edit- Be able to write a new maintenance script for the "misc dumps"
- Be able to write a new dumps step for the XML/SQL dumps
- Be able to restructure the design of rsyncs of dumps data out to the secondary and labstore hosts
- Be able to add new dumps appropriately to the infrastructure
- Example: OKAPI HTML dump service from the labstore boxes
- Be able to redesign part or all of the XML/SQL dumps process, understanding the rationale (lol) for the current architecture and being able to adhere to the important parts of that in the redesign
- ...