Wikimedia Release Engineering Team/Deployment pipeline/2018-06-07
Last Time edit
General edit
- First meeting back from hackathon
- Alexandros: Scaffold demo + K8s session https://asciinema.org/a/cG2EGGbS3G9LT86xseDqrmucD
- Dan + Tyler: pipeline session
- Rough docs: https://wikitech.wikimedia.org/wiki/Blubber
RelEng edit
Question for Alexandros: Is graphoid blubberized? https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/services/graphoid/+/refs/heads/master/
- Yes and no. Changes are up and I think are working fine, need to be merged though and tested https://gerrit.wikimedia.org/r/#/q/status:open+project:mediawiki/services/graphoid+branch:master+owner:self
- Update MediaWiki in docker to exercise services via MediaWiki
- Currently looking at Math v. Mathoid
- We get to shell out to npm, but deps for MediaWiki extensions are different
- Using composer is not, evidently, the right path (RfC in the pipes)
- Current plan is just to use integration/config CI's hash of extension dependencies: other thoughts? https://github.com/wikimedia/mediawiki-containers
- dist/pipeline/blubber.yaml
- needs to move for python projects
- top-level seems standard, but ugly (seealso scap)
- Suggestion from giuseppe: .blubber.yaml .helm.yaml
- Pipeline modifications
- pushing before verify stage ( https://upload.wikimedia.org/wikipedia/mediawiki/9/9b/Containerized_continuous_delivery_2017_concept.png )
- Currently: build test (CI), run test (CI), build prod (CI), verify prod (CI), build test (Prod), run test (Prod), build prod (Prod), push prod (Prod)
- Ideally: build test (CI), run test (CI), build prod (Prod), verify prod (CI), push prod (Prod)
- This is more sane with a k8s cluster v minikube
- sound sane? no? ok.
- TODO: Need to be able to clean up failing images from the registry
- Need to test for resource contention/traffice (bottleneck is swift)
- TODO: create task add #ops + @Joe
