Wikimedia Release Engineering Team/Deployment pipeline/2018-06-07

Last Time

• 2018-04-26

General

• First meeting back from hackathon
  • Alexandros: Scaffold demo + K8s session https://asciinema.org/a/cG2EGGbS3G9LT86xseDqrmucD
  • Dan + Tyler: pipeline session
  • Rough docs: https://wikitech.wikimedia.org/wiki/Blubber

RelEng

Question for Alexandros: Is graphoid blubberized? https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/services/graphoid/+/refs/heads/master/

• • Yes and no. Changes are up and I think are working fine, need to be merged though and tested https://gerrit.wikimedia.org/r/#/q/status:open+project:mediawiki/services/graphoid+branch:master+owner:self

• Update MediaWiki in docker to exercise services via MediaWiki
  • Currently looking at Math v. Mathoid
  • We get to shell out to npm, but deps for MediaWiki extensions are different
  • Using composer is not, evidently, the right path (RfC in the pipes)
  • Current plan is just to use integration/config CI's hash of extension dependencies: other thoughts? https://github.com/wikimedia/mediawiki-containers

• dist/pipeline/blubber.yaml
  • needs to move for python projects
  • top-level seems standard, but ugly (seealso scap)
  • Suggestion from giuseppe: .blubber.yaml .helm.yaml

• Pipeline modifications
  • pushing before verify stage ( https://upload.wikimedia.org/wikipedia/mediawiki/9/9b/Containerized_continuous_delivery_2017_concept.png )
  • Currently: build test (CI), run test (CI), build prod (CI), verify prod (CI), build test (Prod), run test (Prod), build prod (Prod), push prod (Prod)
  • Ideally: build test (CI), run test (CI), build prod (Prod), verify prod (CI), push prod (Prod)
  • This is more sane with a k8s cluster v minikube
  • sound sane? no? ok.
  • TODO: Need to be able to clean up failing images from the registry
  • Need to test for resource contention/traffice (bottleneck is swift)
  • TODO: create task add #ops + @Joe

As Always

• Release Pipeline Workboard
• Meeting notes