Scrum of scrums/2018-02-14

please come prepared with status of quarterly goal dependencies

2018-02-14

Callouts

• Fundraising campaign in Sweden 2018-02-06 - 2018-03-06 and Italy 2018-02-01- 2018-02-27: https://meta.wikimedia.org/wiki/CentralNotice/Calendar
• Ops kindly requests WMDE feedback on wikidata weekly dumps expected resource usage for next 3 years https://phabricator.wikimedia.org/T181936#3895869

Audiences

Readers

iOS native app

• Blocked by:
• Blocking:
• Updates:
  • Continuing work on 5.8.0 - Synced Reading Lists ( https://phabricator.wikimedia.org/tag/ios-app-v5.8.0-manatee-on-a-skateboard/ )

Android native app

• Blocked by:
• Blocking:
• Updates:
  • Finished integration with reading list service
  • Released performance improvements to production.

Readers Web

• Blocked by:
  • RelEng/Services- trying to get deploy repository and CI setup for chromium-render (https://gerrit.wikimedia.org/r/#/q/status:open+project:mediawiki/services/chromium-render/deploy + https://gerrit.wikimedia.org/r/#/c/409115/)

• Blocking:
• Updates:
• Quarterly goal dependency update:
  • Increase learning by lowering the cost of exploration
    • Reading Web/Performance (around initial JS payload)
    • Reading Web Infrastructure summary endpoint
  • Analytics engineering support for new virtual pageview interactions metric
  • Continue improving the ways that users can download articles of interest for later consumption
    • Reading Web: Tech Ops/RelEng
    • Services to get the new chromium-pdf service deployable
    • RI in preparation for handover

Readers Infrastructure

Nothing to report this week

Maps

• Blocked by:
• Blocking:
• Updates:

Multimedia

• Updates
  • 3D live on Commons, a few bugfixes in progress before announcement on Monday
  • MediaInfo work is progressing slightly faster now
• Quarterly goal dependency update
  • Prepare backend infrastructure for structured data search
    • SDC: Search PF/Multimedia/Wikidata
  • Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and exposure
    • SDC: Research/Multimedia
  • Prepare backend infrastructure for structured data search
    • SDC: Search PF/Multimedia/Wikidata

Contributors

Community Tech

• Blocked by: none
• Blocking: none
• Updates:
  • GlobalPreferences going live on beta cluster today (thanks Security!)
  • Work on pinging from edit summary and thanking for log entries is nearing completion

Anti-Harassment Tools

• Blocked by:
• Blocking:
• Updates:

Editing

• Blocked by: None
• Blocking: None
  • Updates: Released visual diffs!

Maps

• Blocked by: Services
• Blocking: None
  • Updates:Talking with collab team about community tech work

Parsing

• Blocked by: Platform team (Kunal) on linter reviews (on critical path to let wikis identify error counts in content namespaces that they need to fix when Tidy goes away)
• Blocking: None that we know of
• Updates:
  • Participating in ATWG and annual planning
  • Heads up about upcoming changes parsing team is working on to bring Parsoid and core PHP parser output closer together. In this case, by moving PHP parser output towards Parsoid.
    • https://gerrit.wikimedia.org/r/#/c/196532/ -- WIP: Use figure and figcaption HTML5 elements for media ( https://phabricator.wikimedia.org/T51097 )
    • https://gerrit.wikimedia.org/r/#/c/370206/ -- WIP: Use custom figure-inline HTML5 element for inline media ( https://phabricator.wikimedia.org/T118520 )
    • https://gerrit.wikimedia.org/r/#/c/410362/ -- Move parsoid media styling to content (CSS to style the new markup proposed above)
    • If you are bot / gadget / or other author that depend on the particularities of image markup and tags, you might have to tweak your code to adapt to new markup when this is ready to be merged and deployed.
• Quarterly goal dependency update:
  • Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team
    • Parsing:Mediawiki PF/Services

Collaboration

• Blocked by:
• Blocking: ops on Flow dumps; Roan working with Ariel
• Updates:
  • Matt is leaving (see wmfall), last day tomorrow
  • Making progress grokking the maps stack
  • Dealing with train blockers in Flow, one caused by the comment table change

Language

• Blocked by:
• Blocking:
• Updates:

Audiences Design

• Blocked by:
• Blocking:
• Updates:
• Quarterly goal dependency update:
  • Audiences DesignStandardise our user interfaces to match user expectation of quality from our products
    • Audiences Design: Ops

UI Standardization

• Blocked by:
• Blocking:
• Updates:

Technology

Analytics

• Blocked by:
• Blocking:
• Updates:
  • Upgraded the entire Hadoop cluster to Java 8
  • Launched pageviews by country by project in Wikistats (https://stats.wikimedia.org/v2/#/hi.wikipedia.org/reading/pageviews-by-country )
  • EventLogging capsule has been moved from meta into the source code (it was always tightly coupled with the source, this makes it clearer and easier to maintain)
• Quarterly goal dependency update:
  • Improve, adjust, or create features geared at the needs identified in New Editors research project.
    • New Editors Experience:Analytics

Cloud Services

• Blocked by:
• Blocking:
• Updates:
  • labvirt1021/22 wrong NICs?
    • https://phabricator.wikimedia.org/T183937#3947621
  • Andrew making progress on deploying Horizon from source using scap3, but fighting with scap
  • Chase working on plans and diagrams for Neutron networking changes
  • Annual planning fun being had by all

Fundraising Tech

• Blocked by:
• Blocking:
• Updates:
  • Testing new CiviCRM version in staging environment
  • Deploying some CentralNotice fixes if SWATters are around: https://gerrit.wikimedia.org/r/#/c/410346/
  • Updated CentralNotice impressions EventLogging patch to not require EventLogging library, on advice of perf team: https://gerrit.wikimedia.org/r/408488
  • Would be cool to have a super-small EventLogging URL-generation module: https://phabricator.wikimedia.org/T187207
  • More work on new API for main CC processor
  • Recovering from a toggle in a banner that led many donors to unintentionally select monthly donations during the year-end campaign.

MediaWiki Platform

• Blocked by:
• Blocking:
• Updates:
• Annual Planning
  • ATWG CDP
• MCR code review
• cleanupUsersWithNoId: Done!
• TemplateStyles: enabled on Swedish Wikipedia
• Extension and ExtensionRegistry test coverage
• LuaSandbox security patches and release
• Quarterly goal dependency update:
  • Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team
    • Parsing:Mediawiki PF/Services
  • Reduce product and technical debt to modernise our tools and technologies, and to make future changes more effective and efficien
    • Parsing/Mediawiki PF
  • 1.1 It is possible to store structured data within wiki pages, in particular on media file pages on Commons. We will enable the MediaWiki storage layer to correctly store and process structured data elements within wiki pages.
    • SDC: Mediawiki PF/Wikidata

Performance

• Blocked by:
  • Research (maybe) - Gilles is looking to coordinate, reached out to Dario via email
• Blocking:
  • (gently) Analytics, trying to get rid of ZMQ crossloader, hoping to get that out this week
• Updates:
  • Testing pages on Swedish Wiki to monitor performance as TemplateStyles is rolled out
  • Still working on WebPageTest migration to Linux
  • Investigating SaveTiming spike from 2/4
  • Bunch more Thumbor bug fixes, identifying errors from logs
  • Fixes for a few issues in MediaWiki LoadBalancer, Special:Statistics
  • MW ObjectFactory has been packaged as a library, which unblocks librarization of RDBMS components and others
• Quarterly goal dependency update:
  • Increase learning by lowering the cost of exploration
    • Reading Web/Performance
      

Release Engineering

• Blocked by: None
• Blocking: None, afaik
• Updates: Nothing super interesting
• Quarterly goal dependency update:
  • Continue improving the ways that users can download articles of interest for later consumption
    • Reading Web: Tech Ops/RelEng**

Research

• Blocked by:
• Blocking:
• Updates:
• Quarterly goal dependency update:
  • Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and exposure
    • SDC: Research/Multimedia

Scoring Platform

• Blocked by:
• Blocking:
• Updates:

Search Platform

• Blocked by: none
• Blocking: none
• Updates:
  • SPARQL Client in core merged: https://phabricator.wikimedia.org/T185127
  • Deep category search keyword merged: https://phabricator.wikimedia.org/T184840
  • Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108
  • Working on Serbian analyzer for ES: https://phabricator.wikimedia.org/T183015
  • Looking into improving relevancy of cross-wiki results: https://phabricator.wikimedia.org/T185250
  • Working on Kafka poller for WDQS: https://phabricator.wikimedia.org/T185951
• Quarterly goal dependency update:
  • Prepare backend infrastructure for structured data search
    • SDC: Search PF/Multimedia/Wikidata

Security

• Blocked by:
• Blocking:
• Updates:

Services

• Blocked by:
• Blocking:
• Updates:
  • Annual planning work is going on
  • Summary switchover delays because of last-minute showstoppers popping up
  • Librdkafka upgrade
• Quarterly goal dependency update:
  • Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team
  • Parsing:Mediawiki PF/Services

Site Reliability Engineering

• Blocked by:
  • Collaboration on the usual Flow thing https://phabricator.wikimedia.org/T172025, just listing for completeness
  • WMDE (Marius) feedback on wikidata weekly dumps expected resource usage for next 3 years https://phabricator.wikimedia.org/T181936#3895869
• Blocking:
  • None
• Updates:
  • A new traffic security SRE has been hired. Valentín Gutiérrez
  • Varnish upload @ codfw traffic cluster upgrade to v5
  • ORES was moved today to the new hardware. Some minor issues left, migration deemed successful
  • Preparing to have mathoid deployed on kubernetes (our first service)
  • etherpad upgrade to 1.6.3 - security release
• Quarterly goal dependency update:
  • Continue improving the ways that users can download articles of interest for later consumption
    • Reading Web: Tech Ops/RelEng
  • Audiences DesignStandardise our user interfaces to match user expectation of quality from our products
    • Audiences Design: Ops

Wikidata

• Blocked by:
• Blocking:
  • Going to raise https://phabricator.wikimedia.org/T181936 within the team
• Updates:
  • Good progress on making constrain violation messages cachable, prerequisite before enabling them by default: https://phabricator.wikimedia.org/T179839
  • Planning dedicated secondary database tables for efficient Lexeme label lookups: https://phabricator.wikimedia.org/T184997
  • Preparing security reviews for Lexeme extension and Vue.js backend renderer: https://phabricator.wikimedia.org/T186726
• Quarterly goal dependency update:
  • 1.1 It is possible to store structured data within wiki pages, in particular on media file pages on Commons. We will enable the MediaWiki storage layer to correctly store and process structured data elements within wiki pages.
    • SDC: Mediawiki PF/Wikidata
  • Prepare backend infrastructure for structured data search
    • SDC: Search PF/Multimedia/Wikidata

German Technical Wishlist

• Not present today.