Wikimedia Release Engineering Team/Checkin archive/20150317



Team Business

  • split the team/meetings up? (moved to end of meeting)
    • The team overall likes being informed of all others activities.
    • Peoplel can always set their own meetings to go deeper in details.
    • Our team is slightly different than features, so one per week meeting is just fine.
  • RelEng/QA things at the Hackathon



Team Quarterly Goals


Scrum of Scrums

Blocked on us:

Beta Cluster stability

Quarterly Priority: Green nightly builds on staging:
Quarterly Priority: Stable uptime metrics of the Staging cluster:
  • groundwork done mostly: ENC, Autosigner, autopuppetmaster, tin and palladium are functional
  • Chad has ES patch (I think) Tyler has MX patch for operations puppet
  • Next steps--think it's time to move on to apps

Test history

Quarterly Priority: By team test history:

Slamed by MW-V things last week.

Isolated CI instances

Quarterly Priority: Disposable VMs -
Ops responds with a question: How many VMs, and what other hardware will this require?
Roughly 10 at start, then ~ 50 which is the number of executors we have on current Jenkins. The pool size will depends on the speed to spawn a new instance and replenish the pool being consumed.
The idea is to start with a few projects (ex: releng repos), then extend to the rest progressively starting end of June.
10 is not a problem, but for 50 we'll need to allocate new virt hardware.
Some ops (_joe_, specifically) want to discuss using chroots instead. Andrew is struggling to get someone to respond to Antoine directly.
We had exchanges last summer about isolation. chroots can be escaped, lxc/docker needs some additional isolation work. kvm is isolated out of the box and using openstack as a provider let us reuse the NodePool software already written. Though ideally we would use OpenStack + LXC to speed up the vm spawning.
  • Zuul package ready for precise and trusty
    • need a review by an opsen (fillipo/godog)
    • Antoine to contact fillipo
  • Antoine to create procurment ticket
    • This is the next big timesink/blocker
    • Might need servers to be physically moved between racks.
  • Antoine to loopback with Andrew B / Yuvi to present the arch and more or less validate it.

MediaWiki Releases

Quarterly Priority: Release MediaWiki 1.25:

Other Work





  • Dan in France the week before offsite (May 11-15)
  • Week before hackathon: Team offsite in France -
  • May: Hackathon in Lyon, France
  • Chad vacation after offsite (maybe, dubious now)
  • Elena on vacation May26-June07
  • Antoine: observing french holidays: Mon April 6th, Fri May 1, Fri May 8, Thurs May 14