Wikimedia Release Engineering Team/Deployment pipeline/2019-05-23

Last Time

• 2019-05-09
• Archive

General

• Deploy the RESTBase front-end service (RESTRouter) to Kubernetes
  • Splitting restbase, the api router will go into k8s

• • CI pipeline for it
    • Access to cassandra to perform tests can be simulated via sqlite since front-end will not have direct access to storage

• • actually deploying to k8s -- may not be possible due to hardware restrictions
    • Alex has benchmarking instructions and we can learn more after running those benchmarks
    • We may not have hardware to support
    • If we don't have hardware to support, it'd be next quarter
    • this front-end service should (hopefully) be pretty lightweight
    • TODO Marko to do benchmark + helm chart

• • Changeprop vs RESTBase front-end
    • Marko would like to focus on RESTBase front-end as opposed to changeprop
    • Agreed: we can't rush changeprop just to meet goals

Q1 things

• serviceops
  • Calico this quarter, next quarter etcd v3
  • Joe more available, potentially
  • upgrading k8s itself
  • remaining q4 things
  • docker image upgrade pipeline
  • authorization model upgrade -- kube.config files vs cluster creds
  • moving scb services -- will need machines before end of q1 next year

• services
  • changeprop
  • mcs -- maybe
  • live debugging work for services -- documentation/generalization needed (aotto has a nice wikipage)

• Releng
  • .pipeline/config.yaml expansion
  • self-service stuff next quarter
  • lars cooking up future CI document

Questions

• Secure publishing Jenkins
  • jobs on current Jenkins cluster trigger jobs on secure cluster
  • Limit access to Jenkins (including Read Only)
  • Minimum, secure Jenkins
• New CI system Coming Soon™, but not soon enough, probably
  • Probably need to spec this out -- what do we need?
    • Last step of the pipeline on secure jenkins
    • docker-pkg, blubber, debs

TODOs from last time

•  Done TODO what are our annual plans WRT to this project
  • Outcome: A secure and sustainable platform that empowers a thriving developer community with the ease of software-as-a-service tooling.
  • Key Deliverable: "Strengthen next generation testing and deployment pipeline to support more services, code health indicators, and local development
  • Projects
    • All applicable new and existing services (and partially MediaWiki) exist in the Deployment Pipeline
    • Actionable code health metrics are provided for code stewards
    • Provide a standardized local MediaWiki development environment

• Template:Stalled TODO various attack vectors document to start

• In progress TODO: support documention like the one tyler did for the portal and pipeline/helmfile and deployment
  • Martyav reached out on wiki https://wikitech.wikimedia.org/wiki/Talk:Deployment_pipeline

• TODO docs for service docker container in beta

RelEng

• Pipeline .pipeline/config.yaml working
  • https://integration.wikimedia.org/ci/blue/organizations/jenkins/blubber-pipeline-test/detail/blubber-pipeline-test/5/pipeline/42/
  • https://gerrit.wikimedia.org/r/#/c/integration/config/+/510602/6/jjb/project-pipelines.yaml
  • There are bugs and other glaring issues at the moment. :) Fixing up this week and next.
    • For example: https://gerrit.wikimedia.org/r/c/blubber/+/511784/4/.pipeline/config.yaml
    • Lots of repetition. Maybe pipelines need an `includes` field? (include stage definitions from one pipeline section into another?)
    • Default blubberfile should probably be `blubber.yaml`, not `[pipelinename]/blubber.yaml`

• Kask integration testing with Cassandra via the Deployment Pipeline
  • Sounds like we don't want to use the cassandra instance from the chart
  • Will need to provide ability to override values during helm install via pipeline

Services

—

As Always

• Release Pipeline Workboard
• Meeting notes