Wikimedia Release Engineering Team/SSD Sync Up/2019-02-19

Last Time: 2019-02-12

TODOs

•  Done TODO liw to create wikipage with questions cf: https://phabricator.wikimedia.org/T213090
  • https://wikitech.wikimedia.org/wiki/Deployment_pipeline has an initial list of questions
•  Done TODO thcipriani to review https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/480689
  • In production now... I think

Discussion

• Lars: how closely should we be working with service developers when developing the pipeline?
  • We should aim at developing something the service developers will use and like
  • A good way to achieve that is to apply continuous deployment and tight iterations with developer feedback on the pipeline itself
  • Who are / should be the people who give us feedback?
    • Marko, Andrew Otto, Pchelolo, Dan
    • Andrew Otto feedback: https://phabricator.wikimedia.org/T211247#4860256
  • Lars: whatever is going into the pipeline next expands the list of users from whom we solicit feedback to eventually expand to encompass all service developers
  • Lars: do this iteratively via adding new service to the pipeline

---

• Lars: What is the current status of the pipeline? Can we code review +2 a change to Blubberoid and have it end up in production?
  • curl -T blubberoid.json -H "Content-Type: application/json" https://blubberoid.wikimedia.org/v1/test # Blubberoid runs in some form of production, is that K8s?
  • Tyler: deployment is missing
    • At this point deployment is running scap helm upgrade
  • Lars: Should services, once hitting production, be deployed automatically?
  • Tyler: we need a K8s token on a CI machine to do the deployment automatically
  • TODO thcipriani to followup to make task for adding token for auto-deployment
  • Tyler: we should have automated deployment of at least Blubberoid by the end of this quarter

---

• thcipriani: more .pipeline/config.yaml talk
  • https://phabricator.wikimedia.org/T209106
  • https://phabricator.wikimedia.org/T209106#4958830
    • Antoine: for CI we have standardized to a directory named "log". Exposed in jobs/containers via the LOG_DIR environment variable. It used to be set via "$WORKSPACE/log".  With the CI Docker containers it is usually "/log" (and hence we do mkdir -p log docker run --env LOG_DIR=/log" -v "$(pwd)/log":/log).
  • https://phabricator.wikimedia.org/T210267#4958266

---

• Jeena: made some changes to local-charts for customizations, started work on RESTBase
• Lars: where are we with local dev, can I setup something to develop Wikimedia?
• Jeena: there are helm-charts in at https://gerrit.wikimedia.org/g/releng/local-charts this allows you to setup MediaWiki, Parsoid, and MariaDB
• Brennen: I have scripts for pre-requisites and I'll try to get a changeset for that today
• Lars: I will be happy to alpha/beta test instructions
• Brennen: wait until I submit instructions and then work from there
• Jeena: will let you know when ready

---

• James: does a config change mean that there is a new image? (Q. from Eric E. in IRC)
• thcipriani: currently that would mean a new image, new pipeline build
• Jeena: isn't that part of a deployment? Would that require a new image?
• James: there are pros and cons to both, for instance mw-config could make everything fall over
• Lars: I think we need the same images for local/prod/staging and the way for that to work is to keep all configuration somewhere else
• thcipriani: there is a .pipeline/config.yaml that would affect what the pipeline produces that creates a new docker image, there is other configuration in helm-charts that doesn't.
• Antoine: need to run and e2e test when helmcharts change?