Wikimedia Release Engineering Team/Deployment pipeline/2017-02-21
2017-02-21
editAntoine, Jean-René, Giuseppe, Marko, Tyler
Background Notes
edit- Problems
- Releng October 2016 offsite
- CI pipeline != production != development
- CI/Beta situation -- we want to make them closer to production, want developers to be able to run something close locally
- Magic containers fix everything magically — also, lots of work: https://www.mediawiki.org/wiki/File:Containerized_continuous_delivery_2017_concept.png
Notes from Giuseppe:
- Need a single cli tool that generates a standard, WMF-specialized dockerfile for at least: php/hhvm, python, node, java
- Existing debian packages of libraries we already use should be preferred by the tool whenever possible
- Only use images coming from https://registry.wikimedia.org/ (official docker registry, public already)
- Images build on and published from copper (machine ops use to build packages/images etc)
- NO private data in container images, ever. They are all going to be public by default
- Integration layers need a *lot* of thought
- Dockerfile generation can be tricky
- Images will not be uploaded by users, they just upload the repo, the official artifact will be built by CI or "releases"
- How to mark releases?
- gerrit-specific?
- CI pipeline from pearson https://github.com/pearsontechnology/deployment-pipeline-jenkins-plugin
Base images to be maintained by ops
- includes limited APT sources, etc.
- what happens when a new base image is built/registered?
- rebuild all dependent images and upload to registry (triggered/automated in CI)
- do we need a private registry? seems there's some facility for this already
Some kind of unified/standardized "build manifest" that abstracts away the problematic Dockerfile
- Example given https://github.com/pearsontechnology/deployment-pipeline-jenkins-plugin#buildbitesize
- Exposes limited instructions for installing dependencies
- k8s does rolling deploy
- Tests with private data (eg MediaWiki security patches)? Part of Docker registry can be made private.
Last Time
edit- Questions for operations
- What do the base images look like? How are they created? Are they signed?
- Re: Some Build Manifest Abstraction, do we limit package installation from certain apt sources?
- What do they need from this pipeline
- Base image updates trigger new builds/tests/everything
- Example: firejail updates
- Currently manual testing in beta and rollout