Wikimedia Release Engineering Team/SSD Sync Up/2017-09-06

2017-09-06

• Define feedback for deployers for the test and production build stage in CI
• Complete test container build with developer feedback
• Complete container build cycle within CI
• Complete build phase of release pipeline ( https://www.mediawiki.org/wiki/File:Containerized_continuous_delivery_2017_concept.png )
  • Build test variant
  • Run test entrypoint w/developer feedback - services dependency
  • Build production variant w/developer feedback - services dependency
  • Tag production container
  • Push to production docker registry - ops dependency - staging namespace

2017-09-05

• helm chart to push

• task to put 17.05 on CI image
  • apt repo for that? Maybe?
  • may have to aptly for  updated repo
  • to target stretch
• Jenkins pipeline job

Blubber MVP

• Basic config specification
  • example yaml in blubber repo
• Support for multi-stage Dockerfile output
  • Should only require a small amount of config (current state of blubber requires manually defined artifacts)
  • The benefits of multi-stage should be asessed (i.e. is the resulting image sufficiently smaller in size than a development image)
• Optimized node dependency installation
  • Does not require any finagling of config from user
  • Image layers are small
  • Image layers are invalidated only when package.json changes
• Configurable runtime info
  • Working directory, unprivileged user, uid, etc.
• Support for "global" defaults - dependency on ops
  • Configuration that can be defined by ops and applied to any injested blubber config (aka policies)
  • From example, the `runs` config from mathoid looks like but doesn't seem mathoid specific:

      runs:         in: /srv/service         as: runuser         uid: 666         gid: 666         environment:           APP_BASE_PATH: /srv/service

Deploying images

• helm config
  • feedback needed from ops/services
  • there is PoC config dan has
  • Using helm to manage staging k8s applications
  • redefine as PoC use of helm

Blubber in CI

• https://www.mediawiki.org/wiki/File:Containerized_continuous_delivery_2017_concept.png
• puppetization/CI Infra
  • docker on contint1001 
  • new packaging for docker version 17.05
  • secret storage on contint1001 for the registry password (later)
• CI workflow definition/CI job definition/Jenkins pipeline
  • blubber build test image
  • docker run test image
  • decision/feedback fork:
    • test entrypoint passes
    • test entrypoint fails
  • passing case
    • another CI job
      • blubber generates production Dockerfile
      • docker production build
      • docker container push to repo
  • failing case
    • raise this ...somehow... to developer

quarterly goals

• Define method for monitoring and reacting to the mathoid functional tests
  • provide developer with useful feedback from results of test entrypoint failure/success and production image build
    • re-use jenkins pipeline dashboard/shit
• Define functional tests for Mathoid running  on the staging Kubernetes cluster for use in future gating decisions
  • way too far out
  • next quarter probably
  • small bit

• Have a build phase working with CI

Q2

• deploying to staging cluster with helm