Wikimedia Release Engineering Team/Checkin archive/20170405

2017-04-05

Vacations/Important dates

https://office.wikimedia.org/wiki/HR_Corner/Holiday_List

How to do it

• April 10-13: Dan out
• April 14: Monthly day off
• April 13 and April 16th: Antoine travel
• April 17: Željko local holiday (Easter Monday)
• April 27-29th: Antoine in Amsterdam (unreacheable)
• May 15-18: Team Offsite in Vienna
• May 19-21: EU Hackathon
• July 10th: Tyler doing things and stuff.

Team Business

Time spent spreadsheet

• week 13: https://docs.google.com/spreadsheets/d/1OZzvtlc116UIsFFbic1zX34LXgAJddPTXMGVE6uMsJw/edit#gid=1831444635

Rotating positions and absences

Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/u/blockers

Mar 20 and Mar 27

• Train: Tyler (plus Chad for goup2 on March 30th)
  • wmf.17
  • wmf.18
• SoS: Chad
• Out:
  • March 23-28: Chad in Miami
  • March 30, 31: Tyler in Wichita
  • Thu March 30 - Mon Apr 3: Greg in Berlin (Wikimedia Conference)

Apr 3 and Apr 10

• Train: Chad
  • wmf.19
  • wmf.20
• SoS: JR
• Out:
  • April 10-17: Dan
  • April 14 (Friday): US Staff holiday

April 17 and Apr 24

• Train: Mukunda
  • [ wmf.21]
  • [ wmf.22]
• SoS: Tyler
• Out:

Scrum of Scrums

https://phabricator.wikimedia.org/project/board/64/

Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

This week

• Blocking
• Blocked
  • May need Ops help deploying 3d2png via scap3 (worked on Beta Cluster), need some +2 in puppet
• Updates

Last week

• Blocking
• Blocked
• Updates

Other Team Business

Annual Planning

• https://office.wikimedia.org/wiki/Product_and_Tech_consultation

Logspam

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

Q3 goal/project FINAL check-in

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201617Q3

Continuous Integration Infrastructure modernization

Gather requirements for new CI system - task T153270

• Meeting notes from yesterday
  • JR did an awesome

Migrate MediaWiki deploys to scap3 (part 2 — This time, it's personal; A.K.A Scappy Strikes Back)

Build fanout support in scap3

task T121276

Phabricator

==== New Features

• Show snippets with highlighted matching terms in highlighting in phabricator global search results (show document body snippets, not just titles)
  • Screenshot: https://phabricator.wikimedia.org/F7268129
  • Task: https://phabricator.wikimedia.org/T162284
  • Try it on labs: https://phab-01.wmflabs.org/search/query/QO5UI2mJp8M7/#R
  • This is a fairly involved change which adds a new "Engine Extension" infrastructure to phabricator.
    • This foundation can be used to add various enhancements to the search results views (e.g. customized views for each object type)
    • Also lays the foundation for resolving https://secure.phabricator.com/T8646

Bug Fixes

• Make paging work for results > 100
  • WMF Task: https://phabricator.wikimedia.org/T92960
  • Upstream Task: https://secure.phabricator.com/T8285
  • Upstream Diff: https://secure.phabricator.com/D17615

These should both get deployed tonight.

Q4 goal/project check-in

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201617Q4

Streamlined Services Delivery

Build a runnable image from tested and confirmed code that is suitable for promotion.

define 'tested and confirmed' / determine the entry-point for building a container

generate or consume a docker file for building

build image, upload and tag it in docker registry

• POC with JenkinsFile stuff
• POC docker registery, working on building and pushing to it

Train 2.0

Move MW+Extension deploys to scap3 - task T114313 (part 3)

Integrate scap with etcd/pybal to automatically depool and repool servers - task T104352

Release MediaWiki 1.29

A quality MediaWiki 1.29 is successfully released - task T153271

We're redoing the build queue

Non-Quarterly goal work

Browser tests

• T161103 Write documentation on Selenium tests in Node.js
  • Done https://www.mediawiki.org/wiki/Selenium/Node.js
• T150435 Selenium tests for VisualEditor in Node.js
  • Announcement sent
• T161697 Run WebdriverIO tests using Firefox
  • Stalled. Upstream working on it.
• T150435 Selenium tests for VisualEditor in Node.js
  • https://gerrit.wikimedia.org/r/#/c/345146/
  • Setup done. To make it useful, VE people need to get involved.

People status updates

Antoine

Last week

• Little Steps Sprint https://phabricator.wikimedia.org/project/view/2676/
  • merge some jobs
    • low hanging fruits done.
• CPU usage on beta cluster Swift backend servers
• Basic support for Moritz upgrade of HHVM from 3.12 to 3.18
• gdnsd config files from puppet - https://gerrit.wikimedia.org/r/#/c/343747/

This week

• Schedule Jenkins upgrade early next week during European morning
• CI instances out of RAM when running Wikibase
  • TLDR: https://phabricator.wikimedia.org/T125050#3153282 PHPUnit uses ~2GB ram and fork() fails due to not enough memory (instance only has 1.5G free and linux refuses to fork)
• merge some jobs (eg for extensions: composer test + phpunit -- qunit + npm)

Chad

Last week

• vacation
• scap

This week

• scap
• train
• security release

Dan

Last week

• Brain dump the k8s cluster stuff and start working on Build PoC
  • Assess existing docker-based builds as a jumping off point

This week

• Take a closer look at build config specification
  • Possibly start building PoC command-line tool that converts to Dockerfile

JR

Last week

• quality interview
  • polish up report out and schedule some time to discuss
• bug/defect data wrangling
• test results data wrangling
• CI Pipeline requirements

This week

• quality interview
  • Talking to a couple for folks this week
  • writing up report out.
• bug/defect data wrangling
• test results data wrangling
• CI Pipeline requirements
  • Consolidating Ops/RelEng/Services requirements

Mukunda

Last week

• Work on further issues with phab search, hopefully making a few more improvements
  • tracked upstream https://secure.phabricator.com/T12450
• Built search result snippets / highlighting

This week

• Deploy snippets / highlighting
• Further work upstream
• Polish `scap tag` for creating template-based phabricator milestones from the CLI
• Evaluate the contributed "Release" app for phabricator (this is a thing that might replace Releeph, for cutting release branches in phabricator...)
  • This is a series of patches from https://secure.phabricator.com/p/avivey/ which is mostly finished but not really well integrated yet
  • I got some tips from avivey about how to try it out locally
  • The patches:
    • https://secure.phabricator.com/D16981
    • https://secure.phabricator.com/D17020
    • https://secure.phabricator.com/D17004

Tyler

Last week

• Train
• CI goal peacemaking
• Wichita

This week

• 3d2png helping
• RESTBase helping
• Push built images to ci-staging docker registry
• wrap service-runner docker generation in groovy? Maybe...will look

Željko

Last week

• T158074 Update Ruby tests to Selenium 3
• T137112 migrate mwext-mw-selenium to Nodepool instances
• T161603 Provisioning MediaWiki Vagrant VisualEditor role does not result in working VisualEditor
• T161103 Write documentation on Selenium tests in Node.js
• T150435 Selenium tests for VisualEditor in Node.js
• T139740 Port Selenium tests from Ruby to Node.js
• T161697 Run WebdriverIO tests using Firefox

This week

• T139740 Port Selenium tests from Ruby to Node.js
  • Run WebdriverIO jobs in test and gate-and-submit pipelines for mediawiki/core and extensions, voting.
• T160086 Upgrade Cucumber from version 1 to version 3
• T152963 Increase in failures caused by Saucelabs