Is it a hard requirement that we use something off the shelf or are we open to forking or maintaining our own thing?
For the latter I'm specifically wondering about where this puts us with Zuul v2, which we're already effectively maintaining.
Topic on Talk:Wikimedia Release Engineering Team/CI Futures WG/Candidates
@Hashar is probably the best person to answer this.
Our own system would make sense for an unsolved problem. We did that with OpenStackManager to offer end user managements of their cloud projects, since then upstream created Horizon which solved the same use case and we are dismissing our home grow solution.
CI is a solved problem. There are plenty of solutions available, we do not need to roll out and maintain our own. Well if the WMF had the meant to create and advocate softwares reusable, surely we could set up a product to address the CI requirements, but I don't think it is anywhere near the WMF core duties nor are we any good at producing software reusable by others (beside MediaWiki).
The Zuul version we use is a fork of upstream v2.5.1 which got released in September 2016. It has 19 patches and a few bugs and required features that have been addressed upstream. The reason we haven't moved forward is the next version is a major overhaul that requires a non trivial migration (change of the layout config, new daemons to be added, job definitions entirely overhauled, no more support for Jenkins). Release engineering has been busy with other duties since then (deployment tooling, maintenance, docker, quibble etc).
Anyway, the requirement for off the shelf could be seen as lets not reinvent the wheel. Instead adopt some upstream system that mostly match our needs, work closely with upstream to push our own agenda (features, bugs etc) and benefit from others maintaining and enhancing the system. We are very good at that.
There are very good chance we will fork the selected upstream system, at least to be able to quickly apply custom patches that are pending upstream. We are already doing that with Gerrit and Phabricator ;)
In short, this 'CI Futures Working Group is about phasing out the legacy / monkey patched Zuul v2 we are running in favor of a new system (which might be Zuul v3 or not) :)