Wikimedia Release Engineering Team/SSD Sync Up/2019-07-16
2019-07-16
editLast Time: 2019-07-02 Goals: https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2019-20_Q1#Release_Engineering
Deployment Pipeline
edit- Last week's Tech Talk by Alexandros: https://www.youtube.com/watch?v=i0FTcG7PxzI
GOALS
edit- Migrate restrouter
- https://phabricator.wikimedia.org/T223953
- (Stretch): MobileContentService
- (Stretch): Preparatory MediaWiki config clean-up & static loading work
- Moving configuration to static configuration (e.g., json files rather than a bunch of PHP)
TODOs from Last Time
edit- Done thcipriani -- Pipeline image build cleanup for contint1001 (not that it's important anymore :))
- Done contint1001 store docker images on separate partition or disk
- In progress thcipriani -- Base Blubber policy file for CI
- blocked on deployment
- TODO thcipriani to ping alexandros
- In progress brennen -- pipeline config validation
- Not done TODO Pipeline docs
Other Work
editNew CI
editGOALS
edit- POCs of GitLab and Zuul3 systems; evaluate options
- Document an implementable architecture for what we want in new CI
TODOs from Last Time
edit- TODO Talk to SRE about Zuul v3 needs
- Needed: conversation about phasing out Zuulv2
- Meeting for zuulv2 thursday, TODO ping greg about antoine going
- Lots of ansible to setup Zuulv3
- 'TODO Discuss with SRE Thursday
- Needed: conversation about phasing out Zuulv2
Other Work
edit- v2 of CI arch doc: https://docs.google.com/document/d/1EQuInEV-eY_5kxOZ8E1qEdLr8fb6ihwOD9V_tpVFWuU/edit
- Only a few comments received, no significant changes. v3 someday, but not immediately.
- No news, but s'ok
- Lars is hacking up new components around GitLab (mostly independent of what CI engine we choose, GitLab is just the first one to be tried)
- going slowly this week, due to train
- All but one component done (the controller)
- other components tested manually
- prototype: http://git.liw.fi/wmf-ci-arch/tree/
- gitlab-ci uploads build artifacts to the artifact store
- Testing with http://git.liw.fi/heippa/tree/.gitlab-ci.yml -- simulates the commit stage
- uses the ick artifact store
- VCS Worker and Deployment Worker in api.py
- Deployment via ansible currently (git clone)
- Aim: get controller working, then get a volunteer -- i.e., build blubber for a prototype
Local Development
editGOALS
edit- Migrate local-charts to deployment-charts
- https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/522584
- Did not handle any concurrency with 30 requests/sec
- Didn't use too many resources
- Blocked on: Brennen working on moving mw-base image to apache
- https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/522584
- Instantiate testing and linting of helm charts
- Preliminary work on a CLI for setup/management
TODOs from Last Time
edit- mediawiki/core blubber: Jam a shell script into the builder?
- TODO: the actual jamming of shell script
- Haven't been able to make that work
- Job to run docker-pkg nightly to make up-to-date dev image
- thcipriani/brennen pair Friday on working on this
- TODO: the actual jamming of shell script
Other Work
edit- In progress Porting from local-charts to deployment-charts ( https://phabricator.wikimedia.org/T224935 ):
- Add restbase chart (port from local-charts) TODO Thursday meeting
- In progress Move dev-images PHP image from php -S to Apache + php-fpm