Wikimedia Release Engineering Team/Deployment pipeline/2019-04-11

Last Time

• 2019-03-28
• Archive

General

• Q4 Tracking tasks
  • Thank you Alexandros for creating

TODOs from last time

•  Done TODO merge/deploy changeprop patch for zuul https://gerrit.wikimedia.org/r/#/c/integration/config/+/496387/

• N Not done TODO various attack vectors document to start

• In progress TODO: support documention like the one tyler did for the portal and pipeline/helmfile and deployment
  • https://wikitech.wikimedia.org/wiki/Deployment_pipeline now exists, https://wikitech.wikimedia.org/wiki/Continuous_Delivery has been deleted.
  • reached out to technical writer, in very preliminary stages for next quarter
  • think about other documentation needs beyone Deployment_pipeline main page

• TODO: Joe & James_F to work on eventual 2019-04-01 email
  • Beware: announces on 04/01 can be considered an April's fool [and James is out on 1 April itself. <-- this is an april fool's joke for sure]
  • postponed until there are docs ~2019-05-xx
  • Draft: https://etherpad.wikimedia.org/p/LZKuNSFpM4iKbuHkdPRf
  • Shall we send one soon, more a "hey, what do you think?" e-mail?
  • a preliminary email for people to be able to comment on work so far and how that jives with their needs
  • wikitech-l, engineering-l, services@lists.wm.org
  • what is the call to action?
    • Is there documentation you expect to see?
    • Poke around this project?
  • Overview
  • Who cares about this email?

To do send email about email

RelEng

• Additional NPM packages during container image build
  • Looks like optional dependencies is where this landed?
  • packages included as optionalDependencies
  • once services updated to newest servicerunner it should be automatic
  • first deployment of this happened today
• Latencies and duration of GCs are wrong until https://phabricator.wikimedia.org/T220709 is fixed

• Implementing `.pipeline/config.yaml`
  • https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/502917
  • https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/502918
  • a single pipeline can operate on two seperate paths working on two seperate repos
  • you can publish different docker images for different services
  • stages have a flow:

 build  run  publish  deploy

• • each stage step exports binding with the results of a step, subsequent steps can access ancestor bindings
  • Mulitple images with multiple charts with a single repo
  • config.yaml: orchestrating steps for CI
    • can specify different blubber configs to different services
    • refers to blubber variants internally
  • blubber.yaml: creates build manifests
  • Timelines: start experimenting possibly next week
  • experiment with ORES possibly(?) since they need this functionality
  • Might use for restbase possibly
  • MCS split into MCS / PCS (?)

Serviceops

•  Done cxserver training. Deployment done today
• ongoing Upgrades to infrastructure ongoing, kubernetes 1.12.7 next week
• started Started work on kask+wikidata termbox SSR
• ongoing Need to resume work on changeprop
• started A tech talk proposal (late june per organizers' request)

Services

• ChangeProp is ready to go.
• RB split is coming
• MCS/PCS are eager to move to node10. Maybe for Q1?

As Always

• Release Pipeline Workboard
• Meeting notes