Scrum of scrums/2015-12-16

2015-12-16

Reading

Web

• Read more feature live on all language Wikipedias as desktop beta feature / mobile beta. Please try it out!
• Lead image beta experiment now using WikidataPageBanner
• TechOps involvement requested: potential cache issue, unclear that it's actually MobileFrontend - https://phabricator.wikimedia.org/T121594

Mobile Content Service

• 55% of Android Beta App users use the RB based service for link previews (page summary) + page content

Android

• 2.1.136-2015-12-09 published.
  • New Wikipedia Maps.
  • Final release of 2015.

iOS

• Marching to TestFlight beta testing (with org.wikimedia.wikipedia bundle ID to ensure database upgrade coverage) soonish. Got an iOS device? Download the Wikipedia app and let Josh Minor know if yu're interested

Reading Infrastructure

• Security: ping re SessionManager security review: Is it done, or else what else do we need to do?
• Security: If you could squeeze in a look at https://gerrit.wikimedia.org/r/#/c/259066 and related patches (it's another thing needed for AuthManager that isn't "in" AuthManager itself), we'd appreciate it.
• DBA: https://phabricator.wikimedia.org/T121335 (which I think he already knows about)

Community Tech

• Community Wishlist Survey is complete - https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey/Results
  • We will be doing analysis on the top 10 proposals and pinging other teams as needed
• Progress on PageAssessments extension - now uses parser ExtensionData and JobQueue for better scalability
• Still working on Gadgets 2.0 - you can play with a test instance at http://commtech.wmflabs.org/wiki/Special:GadgetManager

Editing

Language

• Deployment of cxserver service-runner rewrite is delayed to Thursday

Collaboration

• Cross-wiki Echo notification work is mostly done.
• Also planning some UX changes to how Echo notifications work in general
• We finished populating the Flow artificial primary keys.
• Flow Nuke integration almost done

Fundraising Tech

• Investigating potential CN bug limiting banners on mobile
• Updating donation forms and fraud prevention for backup credit card processor
• Fixes for CiviCRM
• Internal dashboard updates

Research

• ORES:
  • 2 new models (etwiki, ukwiki) last week with some *substantial* improvements to the wikidata models (0.97 AUC)
  • 3 hours of downtime this morning. See incident report: https://wikitech.wikimedia.org/wiki/Incident_documentation/20151216-ores

Technology

Services

• Page summary end point live & pre-generated: https://en.wikipedia.org/api/rest_v1/?doc#!/Page_content/get_page_summary_title
• EventBus incl. MediaWiki event production ready for deploy; hardware might not be ready in time before freeze.
• mediawiki-docker-compose prototyping: https://phabricator.wikimedia.org/T92826#1866757
• includes a combined service running RESTBase, Parsoid & in the future mathoid & others: https://github.com/gwicke/mediawiki-node-services
• moved to service::node module
  • upgraded to Cassandra 2.1.12; improved performance significantly
  • Ops dependencies:
    • Node 4.2 upgrade: https://phabricator.wikimedia.org/T107762

Analytics

• Event Logging has been having trouble as we switch to TokuDB because space filled up, no data loss yet but we've been backfilling from time to time. We send notices to Analytics-l but if your numbers look weird, either ask us or rerun your report
• pageview API: blogpost, clients in python, JS, R, dev summit session, all's well that ends well
• Wikistats reports are being slowly transitioned to Hadoop, we're on track this quarter and planning to be hopefully done in two more quarters
• We're consolidating all our Phabricator work from Analytics-Backlog and Analytics-Engineering to just simply Analytics.

Security

• Reviews: Thumber almost completed, PageAssessment starting monday for CommTech
• Security release Thursday
• Not yet blocked, but will need parsing team review of T119158 soon
• Password policy updates

Release Engineering

• Blocking: (none)
• Blocked by: (none)
• Updates:
  • Scap3 refactoring and tech debt cleanup
    • https://phabricator.wikimedia.org/project/view/1449/
    • Working on common Puppet setup of scap for all services
  • Started migrating Jenkins to jessie
  • Brief experiment with Appium support in MW-Selenium
    • Sorta works but need a real test case (any interest from iOS or Android teams?)
    • dr0ptp4kt just pinged bgerstle and niedzielski on #wikimedia-mobile
  • Experimenting with JS-based end-to-end test framework
  • Gitblit redirection work in progress
    • https://phabricator.wikimedia.org/project/board/46/

Technical Operations

• Blocking: none
• Blocked on: none
• Updates:
  • udp2log decommisioning in favor of kafka complete (kudos to Andrew Otto)
  • Migration of unmaintained OpenDJ LDAP to OpenLDAP
  • restbase to service::node done
  • cxserver to service::node NOT done.

Discovery

• New Suggestion API goes live as beta feature tomorrow, https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester
• Working on language detection, PHP port of TextCat https://github.com/smalyshev/textcat - comments welcome
• Need ops help on https://phabricator.wikimedia.org/T120281
  • Akosaris is helping with it

Maps/Graphs

• Akosiaris is helping with OSM
• Vega 2.0 is stable, I manually updated many graphs to Vega 2.0, and marked others as Vega 1 so that we can switch default to 2
• SECURITY: Need Kartographer review, Vega protocols