Scrum of scrums/2017-06-07

2017-06-07

contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering

callouts

• RelEng: MW 1.29 Release blocked on the tasks in: https://phabricator.wikimedia.org/project/view/2400/
• TechOps: codfw Row D switch upgrade on Tuesday 20th 1500UTC: https://phabricator.wikimedia.org/T167274
• RelEng: WARNING: Ops will be removing Salt near the end of next quarter, that means no more Trebuchet as well. See https://phabricator.wikimedia.org/T129290#3245438 for a list of things still needing migration to scap3. (may see some strange updates from us :))

Technology

Analytics

Productionizing code to count project-wide unique devices (unique devices on *.wikipedia.org) Still working on eventlogging purging so data is in compliance with 90 day retention, should be done by end of quarter. Still working on prep work to replace kafka cluster and add TLS support, some kafka clients (like mediawiki) do not support this natively (this work will expand into next quarter) Added data for ops to druid: requests sampled 1/128, can be used for troubleshooting Pivot no longer open source: Confirmed that last open source clone of pivot doesn’t have any of the bugfixes we need (due to litigation pivot is closed source now), we will try to migrate users to Superset next quarter (similar but less optimal tool open sourced by Airbnb), that will eat time from our next quarter plans.

Research

• Working on deploying Recommendation API based on ServiceTemplateNode
  • https://phabricator.wikimedia.org/T165760
• Building Spark job with MLlib for finding translation recommendations
  • https://github.com/schana/recommendation-translation
  • https://phabricator.wikimedia.org/T162912

Services

• Blockers: none
• Updates:
  • Summary endpoint now contains HTML extract along with plain text
  • Working on an experiment with services kubernetes development setup

Discovery/Search

• Tuna reorg is in effect
• No blockers
• Chinese/Hebrew analyzers ready, deploying. Next is Japanese.
• Upgraded kibana to v 5.3.3
• Continuing work on ML-assisted ranking
• Special:Undelete search deployed for admin testing (https://phabricator.wikimedia.org/T163235)
• Sister wiki search is being deployed (https://phabricator.wikimedia.org/T162626)

RelEng

• Blocking: none?
• Blockers: none
• Updates:
  • MW 1.29 Release blocked on the tasks in: https://phabricator.wikimedia.org/project/view/2400/
  • WARNING: Ops will be removing Salt near the end of next quarter, that means no more Trebuchet as well. See https://phabricator.wikimedia.org/T129290#3245438 for a list of things still needing migration to scap3. (may see some strange updates from us :))

Security

• Reviews:
  • TemplateStyles is almost complete
  • psy/psysh use on WMF servers
  • Verification of whitelisted.yaml / graylisted.yaml
  • Auto-approval of low-risk OAuth applications
  • Ex:JsonConfig/Ex:Kartographer

Tech Ops

• Blocking: none?
• Blockers: none
• Updates:
  • codfw Row D switch upgrade on Tuesday 20th 1500UTC: https://phabricator.wikimedia.org/T167274
  • enwiki API overload https://wikitech.wikimedia.org/wiki/Incident_documentation/20170607-WikiScrape
  • New ops person joining the team: Keith Herron

Reading

web

• Warnings in place for pdf generation. Talks continue around backend.
• Updating page previews to consume and render HTML previews
• Page previews on Wikidata

iOS

• Finishing up 5.5 (Places, Explore feed updates) - https://phabricator.wikimedia.org/project/view/2602/
  • Regression testing & fixing remaining issues
  • Public beta this week

Android

• Current release work complete, beta release soon, perhaps today/Wednesday, assuming no trouble in QA: https://phabricator.wikimedia.org/project/view/2352/
• On deck: https://phabricator.wikimedia.org/project/view/2763/
• New engineer candidate interviews underway.

Reading Infrastructure

• EL problem https://phabricator.wikimedia.org/T67508
• OCG Vagrant role
• TemplateStyles test server
• MCS: Fixed featured article titles for French WP in aggregated feed endpoint.

Multimedia

• No blockers, not blocking
• Work on 3D progressing, though still waiting on proper reviews on design and usability

Community Tech

• Deployed LoginNotify to Test Wikipedia (https://www.mediawiki.org/wiki/Extension:LoginNotify)
• Polishing up CodeMirror extrension for deployment as a Beta Feature
• Still working on XTools rewrite
• No blockers

Editing

UI Standardization

• This week:
  • Clean-up/patches of style guide workboard https://phabricator.wikimedia.org/tag/wikimediaui_style_guide/
• Updates:
  • WikimediaUI Style Guide https://wikimedia.github.io/WikimediaUI-Style-Guide/
    • Semi-automated SVG export of widgets overview and widgets/components to provide open format for designers' use
  • OOjs UI https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md
    • v0.22.1 released (James D. Forrester)
      • Continued work on icons: Drop the core icon pack – please check your extensions for needed icon packs

Language

• No blockers/blocking
• New feature in ContentTranslation: CX will allow publishing to User (or Draft if available) namespace easily.

Collaboration

• Enabling saved filters in production. This has been gated off for a while, it is now moving to enabled along with the train (and some related bug fixes)
• RC Filters fixes
  • Variety of UI fixes
  • A couple backend-of-the-frontend fixes
• Echo
  • Fixed an exception in the Echo blacklist functionality (still dark-launched and only available on test wikis)
  • Another Echo bug fix
• A few other small or not user-visible fixes

Parsing

• Linter is being re-enabled on large wikis next week - the core patch that blocked this has now been merged and will be deployed this week
• Now that wmf2 is deployed on the cluster, we are ready to do final reviews of red link support in Parsoid, merge and test it.
• Parsoid side patch to parse language variants is now going through final reviews.

Fundraising Tech

• No blockers/blocking AFIK
• Offsite this week
• Consolidation and improvements in config for SmashPig and DonationInterface
• Usual onslaught of minor fixes for payment processor integrations
• Deploy of CentralNotice feature coming up