Scrum of scrums/2016-03-09

2016-03-09

Product

Community Tech

• Blocking:
  • None
• Blocked:
  • Waiting for Niklas to get back from vacation to merge patch to add pageviews tool into translatewiki.net
• Updates:
  • Deadlink fixing bot is going well. Just built a logging interface for Internet Archives to monitor progress: http://tools.wmflabs.org/deadlinks/
  • Continuing to collaborate with MusikAnimal on Pageviews tool: http://tools.wmflabs.org/pageviews/
    • New topviews interface: http://tools.wmflabs.org/pageviews/topviews/

Discovery

Discovery Engineering

• Blocking:
  • ???
• Blocked:
  • Security: SVG sanitizer
  • Ops: Maps Hardware https://phabricator.wikimedia.org/T125126
• Updates:
  • New portal version deployed on beta, up for general deployment soon
  • Completion suggester to be deployed on lower-volume wikis on 10th
  • Working on Relevance Forge (nee Relevance Labs)
  • WDQS GUI now has maps for queries returning coordinates
  • Deployed Maps to Wikivoyage - listening to community feedback
  • WDQS query caching problem root cause found, https://phabricator.wikimedia.org/T128813
  • Discovery data access guidelines published: https://meta.wikimedia.org/wiki/Discovery/Data_access_guidelines

Editing

Collaboration

• Blocking:
  • External Store - Continuing work
• Blocked:
  • No new blockers. Matthias put up an XSD for the Flow dumps.
• Updates:
  • Rolling out cross-wiki notifications beta feature to all wikis tomorrow
  • There have been some issues with duplicate thank-you notifications (for edit milestones, e.g. 100th edit), which are being worked on.
  • Also working on some Echo bundling i18n issues and enhancements to the Notifications special page

Fundraising Tech

• No blockers
• preparing real donor test of backup worldwide credit card processor
• Working out final details of Latin American payment processing expansion
• Steady stream of CiviCRM enhancements for us and upstream
• More experiments in fraud monitoring and blocking
• Investigating firewall rules for PCI scan
• Investigating possibility of using current PHP/MW
• Replaced dying drive in ActiveMQ box, cursed SPOF

Reading

Android

• Updates:
  • 2.1.142 beta publishing very soon.
  • Reading list endpoint needs - depending on timing/approach may need security review

iOS

• Blocked:
  • Need Piwik access for Josh Minor - done. https://phabricator.wikimedia.org/T124218
• Updates:
  • iOS v5.0 is schedule to release this Thursday March 10th
  • We need to do additional updates to the Apple site association file we host for further app enhancements (https://en.wikipedia.org/apple-app-site-association)

Web

• Blocking:
  • ???
• Blocked:
  • Path forward needed on a references API - https://phabricator.wikimedia.org/T125329 Invitation to join conversation.
• Updates:
  • Hopefully lazy loaded images we will begin rolling out week commencing 14th [Dependency: performance team]

Reading Infrastructure

• AuthManager is still coming! (see last week for details)
• load.php now logs warnings when the session is accessed in wmf.16. See https://logstash.wikimedia.org/#dashboard/temp/AVNcob7pO3D718AOt5cM

Technology

Analytics

• Blocking:
  • Hadoop issues with encoding have been fixed, querying is totally ok now, data has been back-filled and has been ready for re-computation since earlier this week
• Blocked: (none)
• Updates:
  • Working on goals for next quarter, currenlty mostly revolving around transitioning Erik Zachte's work to Wikistats 2.0
  • If you're using UDFs in Hive, you need to add their jars explicitly now, but this gives more flexibility to test changes to UDFs
  • Bots following the user agent policy are now detected correctly in our pageview data pipeline
  • data on unique devices estimated by the last access cookie is now publicly available, but please let us announce it first on analytics-l before spreading word of its existence (we want to make sure people know what this is)
  • The "-" article should no longer show up in the "top articles" endpoint of the pageview API, starting with today's data

Release Engineering

• Blocking:
  • Are we doing deployments week of Jerusalem Hackathon? [Sync with ops and will announce]
• Blocked:
  • None
• Updates:
  • Reminder: code freeze week of 3/21 for codfw fallover. E-mail/tech news forthcoming as well.
  • Notice: Expect an e-mail shortly, but trebuchet is now deprecated for all new service deployments going forward. New services should use scap3.

Research

Research and Data

• Blocking:
  • None that we know of.
• Blocked:
  • None.
• Updates:
  • Stopped the reader survey (at a 1:50 rate) on 2016-03-08. Research documentations are in https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour/S3-English_Large_Scale
  • No major news for the coming week.

Security

• Blocking:
  • AuthManger?
  • SVG sanitizer - is this blocking a quarterly goal?
• Blocked:
  • ???
• Updates:
  • password hash format updated
  • Lots of security patches deployed - releng, release soon (T124940)?
  • AuthManger review starting

Services

• Blocking:
  • none
• Blocked:
  • none
• Updates:
  • morelike API live - https://phabricator.wikimedia.org/T125983
    • Reading: Android wants 320px thmubs, is the rest of Reading OK with it?
  • RB cluster expansion continuing
  • Cassandra
    • switched to 2.1.13, soon testing of 2.2.5
    • start of Brotli compression in staging imminent
  • rate limiting in service-runner
    • RB endpoints - now only logging excesses, soon will start enforcing them
  • API: removing some listing endpoints

Technical Operations

• Blocking:
  • ORES - moving into a python wheel repo structure
• Blocked:
  • None
• Updates:
  • moved ocg to use a replicated redis
  • redis replicated and encrypted on the sessions
  • got <40 precise machines now
  • Please delete unwanted labs instances