Scrum of scrums/2017-09-13

contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering

callouts

• Multimedia -> Performance: https://phabricator.wikimedia.org/T166699
• CommTech -> DBAs: range contributions ( https://phabricator.wikimedia.org/T163562 ) getting rolled out this week, please let us know if it causes problems
• Services -> Wikidata: very large jobs emitted by wikidata (up to 44Mb of serialized JSON) https://phabricator.wikimedia.org/T175316
• Contributors -> ReleaseTaggerBot has stopped tagging changes since last week https://phabricator.wikimedia.org/T175626
• Wikidata -> RewriteRule needed: https://gerrit.wikimedia.org/r/357985

Audiences

Readers

iOS native app

• Blocked by: none
• Blocking: none
• Updates:
  • Still releasing 5.6.1 with minor bug fixes ( https://phabricator.wikimedia.org/project/view/2898/ )
  • Continuing work on 5.7.0 ( https://phabricator.wikimedia.org/project/view/2899/ ) - Onboarding updates, Improved analytics

Android native app

• Blocked by: Reading List service completion/deployment
• Blocking: none
• Updates:
  • Continuing to build/refine Offline Library functionality based on initial user research feedback.
  • Testing and updating for Android Oreo (8.0)

Reading Web

• Blocked by: none
• Blocking: none
• Updates:
  • Vector print style improvements
    • Justified text and table of contents:
      • https://phabricator.wikimedia.org/T175008
      • https://phabricator.wikimedia.org/T174957
  • Working towards resolution of verifying English and German wiki Page Preview Event Logging instrumentation bucketing
    • https://phabricator.wikimedia.org/T175377
  • Building Proton (Electron PDF renderer / OCG (Offline Content Generator service) replacement using headless Chrome)
    • Assessing and updating Proton instrumentation
      • https://phabricator.wikimedia.org/T175166
    • Building out article concatenation for printing books
      • https://phabricator.wikimedia.org/T171838
    • Working towards resolution of Proton ownership and planning around OCG retirement
      • Private thread: "OCG roadmap update"
  • Project Marvin, a single page web app
    • Defining short and long-term goals for Marvin; also defining what's needed from the PCS (Page Content Service, a successor to MCS) API
      • https://phabricator.wikimedia.org/tag/marvin/
  • Fix for Related Articles not displaying on desktop Wikivoyage
    • https://phabricator.wikimedia.org/T175064

Reading Infrastructure

• Blocked by: Security (ReadingLists review https://phabricator.wikimedia.org/T174126 )
• Blocking:
• Updates:
  • OCG/Collection book concatenation patch ready, deploying for testing
  • working on RESTBase proxy for ReadingLists
  • Issues with sectioning code on some pages https://phabricator.wikimedia.org/T175305

Multimedia

• Blocked by: Performance - https://phabricator.wikimedia.org/T166699
• Blocking: N/A
• Updates: 3D work continuing, Structured Data on Commons (SDoC) user stories coming down the pipeline

Discovery

• Blocked by:
• Blocking:
• Updates:
  • Running an AB test on the search page this week
  • Converting some selenium tests from Ruby to Node

Community Tech

• Blocked by: none
• Blocking: none
• Updates:
  • Range contributions ( https://phabricator.wikimedia.org/T163562 ) getting rolled out this week
  • ACTRIAL starts this Thursday

Contributors

• ReleaseTaggerBot has stopped tagging changes since last week https://phabricator.wikimedia.org/T175626

Editing

• Blocked by: None known.
• Blocking: We need to work with Parsing to get https://gerrit.wikimedia.org/r/#/c/371074/ merged at the right time.
• Updates:
  • Working on undo support in VisualEditor and its source mode https://phabricator.wikimedia.org/T78550
  • Major internal VisualEditor re-work to merge data and meta-data nearing completion https://phabricator.wikimedia.org/T56299
  • Visual diffs work continues, currently doing some more display tweaks.

Parsing

• Blocked by:
  • https://gerrit.wikimedia.org/r/#/c/371074/ (Editing)
• Blocking:
  • https://gerrit.wikimedia.org/r/#/c/364933/ (Reading) -- being reviewed today / soon.
  • https://gerrit.wikimedia.org/r/#/c/295741/ (Editing) -- will update this week
• Updates:
  • C.Scott is porting over the html5 section id support into Parsoid (Parsoid had html5 section ids till last year - switched to html4 ids last year to match core -- now updating to html5 with legacy support again)
  • Kunal (legoktm) is running a script to lint all pages on all wikis so that linter errors are initialized on all wikis to get baseline information in place

Global Collaboration

Collaboration

• Blocked by:
• Blocking:
• Updates:
  • Mainly RCFilters. We're preparing to have the Beta feature also cover Special:Watchlist on Tuesday. We'll also start giving all Special:RecentChanges users the new functionality Tuesday, first on he.wiki, ca.wiki, fr.wiki. We'll roll out further in later weeks.
    • git #c12b2419 - Provide an opt-out preference for New Filters for RecentChanges (task T168376)
    • git #b3becd3a - Add <body> class marking whether RCFilters is enabled (task T175062)
    • git #55ba740b - RCFilters: Live update: disable peek param (task T173613)
    • git #91c2f9a6 - RCFilters: Remove getValue and use FormOptions (task T174725)
    • git #5e12ab58 - WLFilters: Respect default values (task T174725)
    • git #0725cbe9 - RCFilters: Live Update: download less data (task T173613)
    • git #a610aa28 - RCLFilters: Bring back Saved Queries
    • git #3863acef - WLFilters: hide watchlist-details during loading of the app (task T172030)
    • git #f162e47a - Align "What's this" vertically (task T165627)

UI Standardization

• Updates:
  • • Added several variables on req to WikimediaUI Base, which is now also available on Gerrit https://gerrit.wikimedia.org/r/#/projects/wikimedia-ui-base,dashboards/default
    • Aligned four further products to overhauled color palette
• Ongoing:
  • OOUI:
    • icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines
    • Preparing for v0.23.1 next week
  • Continuation on WikimediaUI Style Guide, continuing updating imagery on color section https://wikimedia.github.io/WikimediaUI-Style-Guide/visual-style_colors.html
  • Get remaining two out of 16 patches merged on aligning arbitrary, historically grown colors to WikimediaUI color palette https://phabricator.wikimedia.org/T148708
  • Make TransparencyReport fully accessible – patch still needs review https://gerrit.wikimedia.org/r/#/c/376875/

Technology

Analytics

• Blocked by: Ops (email sent, awaiting answers)
• Blocking:
• Updates:
  • We've been missing SoS for 2 weeks (booooooh)
  • We're back full-team after two successive parental leaves
  • Wikistats 2.0 frontend has made good progress (4 metrics implemented)
  • Wikistats 2.0 backend is under heavy development and moves fast (using Druid as backend)
  • EventLogging purging is continuing to run.

Fundraising Tech

• Blocked by:
• Blocking:
• Updates:
  • Still trying to figure out our database replication lag problems
  • Versioning our composer packages, adding phpcs to our CI tests
  • Looking into CentralNotice banner tracking gaps on Safari
  • various payment processor integration tweaks

Release Engineering

• Blocked by:
  • NEED update on discovery-analytics for trebuchet->scap3 https://phabricator.wikimedia.org/T129149
  • NEED someone to help test OCG when we migrate it to scap3
  • NEED, Jobrunner should be ready for deployment https://phabricator.wikimedia.org/T129148#3582888 (Krinkle? or anyone who can ensure the correct running of jobrunner post-deployment) Aaron?
• Blocking:
  • None afaik
• Updates:
• ci has been slow recently due to nodepool overloading openstack

Research

• Blocked by:
• Blocking:
• Updates:

Scoring Platform

• Blocked by: None
• Blocking:
  • Working on file handle starvation, which blocks stress tests, which prevent deployment to our new server.
• Updates:
  • Trying to integrate the revscoring "thresholds" work with RC filters.
  • Random script to measure memory use, not including copy-on-write: https://gist.github.com/Eugeny/04ccfe8accf4bc74b0ca. Is there a better tool?
  • Adam and Aaron both gone for some time this week and next.

Search Platform

• Blocked by: none
• Blocking: https://phabricator.wikimedia.org/T129149 ?
• Updates:
  • Continuing work on ML-assisted ranking
  • Preparing upgrade to Elastic 5.5 (https://phabricator.wikimedia.org/T174662 )
  • Running second A/B test for human-graded search relevance (https://phabricator.wikimedia.org/T175046 )
  • Started dumps of category tree as RDF (https://dumps.wikimedia.org/other/categoriesrdf/) - more wikis to come soon
  • Portal stats & translations updated (https://phabricator.wikimedia.org/T128546, https://phabricator.wikimedia.org/T142582 )

Services

• Blocked by: Wikidata: very large jobs emitted by wikidata (up to 44Mb of serialized JSON) https://phabricator.wikimedia.org/T175316
• Blocking: none
• Updates:
  • Started double-processing updateBetaFeaturesUserCounts job trough new EventBus pipeline
  • Testing new RESTBase storage

Technical Operations

• Blocked by: None
• Blocking: None
• Updates:
  • New Quarter goals being drafted
  • luasandbox 2.0.14 rollout, to be completed this week

Wikidata

• Same blocker as last week: https://gerrit.wikimedia.org/r/357985
• Back from WikiCon (German community conference)
• New Echo notification are deployed on all wikis
• Working on making (Wiktionary) Lexemes fully persistent: https://phabricator.wikimedia.org/T121274
• Run into a regression highlighting selector values in red: https://phabricator.wikimedia.org/T175525