Wikimedia Release Engineering Team/Checkin archive/20170315

2017-03-15

Vacations/Important dates

https://office.wikimedia.org/wiki/HR_Corner/Holiday_List

How to do it

• March 23-28: Chad in Miami
• March 30, 31: Tyler in Wichita
• March 30-Apr 3: Greg in Berlin (Wikimedia Conference)
• April 10-17 (ish): Dan out
• April 14: Monthly day off
• May 15-18: Team Offsite in Vienna
• May 19-21: EU Hackathon

Team Business

Time spent spreadsheet

• week 10: https://docs.google.com/spreadsheets/d/1OZzvtlc116UIsFFbic1zX34LXgAJddPTXMGVE6uMsJw/edit#gid=938789717

Rotating positions and absences

Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/u/blockers

Mar 6 and Mar 13

• Train: Mukunda
  • wmf.15
  • wmf.16
• SoS: Tyler
• Out:
  • Mon Mar 13th: Monthly day off

Mar 20 and Mar 27

• Train: Tyler (plus Chad for goup2 on March 30th)
  • wmf.17
  • wmf.18
• SoS: Chad
• Out:
  • March 23-28: Chad in Miami
  • March 30, 31: Tyler in Wichita
  • Thu March 30 - Mon Apr 3: Greg in Berlin (Wikimedia Conference)

Apr 3 and Apr 10

• Train: Chad
  • wmf.19
  • wmf.20
• SoS: JR
• Out:
  • April 10-17: Dan
  • April 14 (Friday): US Staff holiday

Actions from last meeting

• TODO: Sign up for the Product and Technology consultation if you have any slight amount of interest: https://office.wikimedia.org/wiki/Product_and_Tech_consultation
  • Mukunda and Antoine sync up

Scrum of Scrums

https://phabricator.wikimedia.org/project/board/64/

Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

This week

• Blocking
• Blocked
• Updates

This week

• Blocking
• Blocked
  • T139740 Port Selenium tests from Ruby to Node.js, the patch is almost ready, please review https://gerrit.wikimedia.org/r/#/c/328191/
• Updates

Other Team Business

Keyholder blogpost

• https://etherpad.wikimedia.org/p/keyholder-blog
• https://docs.google.com/document/d/1kNjUbdI92J1SwENnvBcDdvE4IbqsIb8HzIAj1lwQW4s/edit
  • Feedback from Faidon, Ori, bd808: all is well
  • Sent ^ google doc to blogteam

Logspam

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

Annual Planning

Technology-wide planning

E_TOOMANYGDOCS

Q3 goal/project check-in

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201617Q3

Continuous Integration Infrastructure modernization

Gather requirements for new CI system - task T153270

• Need to dust off specifications that we worked on early, use to evaluate deployment pipeline
• Evaluation of Pearson Deployment Pipeline continues
  • k8s infra being build on ci-staging
    • kubectl cluster-info => "Kubernetes master is running at http://localhost:8080" !!!
  • plugin (god help us all) is installed and ready

• Random aside: can MediaWiki Config live in a containerized world?
  • No. Maybe.

• Update from Antoine on yesterday's (Tuesday's) CI backlog
  • tl;dr: tons of jobs flooding the system, not directly due to the migration of jobs that happened on Monday (which was a 10% increase)

Migrate MediaWiki deploys to scap3 (part 2 — This time, it's personal; A.K.A Scappy Strikes Back)

Build fanout support in scap3

https://phabricator.wikimedia.org/T121276

Have a clear course of action to get actionables done by end of quarter (sync git data to proxies post-canary check and just prior to using proxies; also need some minor puppet tweaks for some symlinks and a new vhost)

Lots of stuff getting done in scap3-MW-MVP (https://phabricator.wikimedia.org/project/view/1823/ )

Gotta fix up the mergeMessageFileList crap (extension-list)

rewriting updateWikiversions as scap plugin

Improved lock handling

Non-Quarterly goal work

Browser tests

• T139740 Port Selenium tests from Ruby to Node.js
  • Almost there. No major changes to the commit in the last few weeks. I really think somebody should just push the big red +2 button. We can fix the minor things later.
• T159945 Selenium/WebdriverIO tests in JavaScript/Node.js
  • Submitted skill share session for Vienna hackathon.
• T158074 Update Ruby tests to Selenium 3
  • Done as far as we are concerned. 3 repositories left to teams to update.
• T152963 Increase in failures caused by Saucelabs
  • Looks like T160086 (Upgrade Cucumber from version 1 to version 3) is the simplest way to rerun tests
• T1331 Remove lines from Gemfile that are used by RVM
  • Done. An intern did all the work, Antoine and me were reviewing.

People status updates

Antoine

Last week

• Qunit thing from SoS
  •  Done Chromium upgrade solved it. https://phabricator.wikimedia.org/T153038
• Reviews of Javascript/Selenium for MediaWiki
  •  Done
• Prepare to upgrade Jenkins on contint2001 (copy data, upgrade, see what happens)

• As time allow (since 02/27)
  • migrate PHP 5.3 jobs to PHP 5.5 (due end of march)
  • Think about getting more Jenkins instances (beta, one for browser tests, maybe tools-labs)
• Investigated puppetization of Jenkins .xml file. The encryption of secret with a randomly generated salt blocks me. Not sure how to provision that in puppet.

This week

• {done} read doc about bootstrapvz
• PHP5.3 jobs migrated. Precise instance will be shutdown/deleted on Monday March 20th
• migration to Nodepool mostly complete. Leftover jobs have to be either deleted, move to other jenkins or integrated in existing jobs
• Upgrade Jenkins on contint2001

Chad

Last week

• Scap 3
• Quick gerrit downtime (LDAP change)

This week

• scap scap scap (more scap)
  • Did I mention scap?

Dan

Last week

• Help break out CI/CD build PoC tasks (set up registry, image config/builder, etc.)
• Play with Pearson Jenkins plugin
• Grab a task and start hacking

This week

• Continuing to set up and test k8s

JR

• quality interviews
• CI knowledge building
• bug/defect data wrangling

Last week

• quality interviews
  • met with Andre, Bernd, Giles
• test reporting dashboard
  • requirements/highlevel design
• CI knowledge building

Mukunda

Last week

• Deployment Train
• leftover from last week
  • dust off the phab milestone scap plugin
    • look at making it update the deployment schedule (in addition to the deployment blocker tasks)
    • this probably won't happen this week
• Search engine overhaul
  • Another week, another round of code review

Tyler

Last week

• Polish up blog post
• scap3 fanout stuff w/chad
• CI digging deep

This week

• Scap3 fanout stuff
• Hooking together pieces of Pearson Pipeline w/Dan
• Dusting off critera doc for Pearson Pipeline eval

Željko

Last week

• T158074 Update Ruby tests to Selenium 3
• T152963 Increase in failures caused by Saucelabs
• T139740 Port Selenium tests from Ruby to Node.js
• T159931 Provisioning visualeditor role fails with `Unable to locate package zookeeper-server`
• T159945 Selenium/WebdriverIO tests in JavaScript/Node.js
• T160086 Upgrade Cucumber from version 1 to version 3
• T1331 Remove lines from Gemfile that are used by RVM

This week

• T139740 Port Selenium tests from Ruby to Node.js
• T150435 Selenium tests for VisualEditor in Node.js
• T152963 Increase in failures caused by Saucelabs => * T160086 Upgrade Cucumber from version 1 to version 3