Scrum of scrums/2017-10-11

2017-10-11

contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering

Callouts

• Are there any special steps to wire a MediaWiki LoggerFactory into logstash-beta? AWight is unable to find some logs, which are present on fluorine-beta.
• CommTech would appreciate help reaching consensus regarding the way preferences are handlled: https://lists.wikimedia.org/pipermail/wikitech-l/2017-October/088980.html
• DBAs have disabled Wikidata RC syndication on Commons and ruwiki, and are working on purging existing WD RC rows on those wikis.
  • This is because of performance and table size reasons. https://phabricator.wikimedia.org/T171027
  • MediaWiki 1.31.0-wmf.3 didn't go out yesterday, we are getting caught up today. https://phabricator.wikimedia.org/T174359

Audiences

Readers

iOS native app

• Blocked by: none
• Blocking: none
• Updates:
  • Continuing work on 5.7.0 ( https://phabricator.wikimedia.org/project/view/2899/ ) - Visual updates to onboarding, iOS 11 support, iPhone X support

Android native app

• Blocked by: none
• Blocking: none
• Updates:
  • On schedule to release beta of E-release this week ( https://phabricator.wikimedia.org/project/view/2839/ )
  • Will start (local) testing of Reading List service.

Discovery

• Continuing work on portal automation, resolving dependencies that fail to build on some systems

and cleaning up old code.

Maps

• Blocked by: none
• Blocking: none
• Updates:
• Got some help debugging nodejs issues
• Community is activating <mapframe> on Spanish Wikipedia

Reading Infrastructure

• ReadingList RESTBase patch under review; hoping to deploy both RESTBase and MediaWiki part to beta soon
• MCS: accounting for Parsoid changes: <figure-inline> instead of <span> for inline figures (https://phabricator.wikimedia.org/T177301)

Web

• Turning off OCG [cc. ops]
• Working on replacement for OCG

Contributors

Parsing

• cscott's ping to wikidata team: https://gerrit.wikimedia.org/r/#/c/379669/ -- Parsing is not blocked on this per se, but this is something that cscott has been contributing to as part of a longer term project. Using SoS to ping wikidata on a stalled review for him.
• Updates
  • Parsing team offsite next week -- will be absent at next week's SoS.
  • Parsoid code to use the updated Templatedata format now merged (suggested by Thiemo at Wikimania 16). It is in the beta cluster and needs testing before being deployed.
  • We are likely to merge both the <section> wrapper code as well as the <figure-inline> code at the same time so that we can bump the HTML version number once.
    • Looks like cxserver/CX has patches to handle section output. Is MCS ready to handle Parsoid's section output? C.Scott said he is going to add a patch for VE to handle this, but, if VE folks want to tackle that on your own, please do so.

Global Collaboration

Language

• Compact Language Links out of Beta deployment being scheduled next week for German Wikipedia.
• Work continue with VE-CX integration and adpatation.
• cxserver has been updated to latest master, debugging issues with Beta cxserver.

Collaboration

• New filters on RC now the default experience on all wikis except some FlaggedRevs wikis
• Looking into Watchlist/ORES slow query issues, but DBAs are also working on that

UI Standardization

• Blocked: icon feedback appreciated, see below / none
• Updates:
  • Improved whitespace usage in OOUI HTMLForms https://phabricator.wikimedia.org/T177668
  • OOUI v0.23.4 release in preparation
    • Generalized icon/indicator padding/positioning https://phabricator.wikimedia.org/T161177 & streamlined icon/indicator visibility and layout https://phabricator.wikimedia.org/T160593
• Ongoing:
  • OOUI:
    • icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432
  • Aligning arbitrary, historically grown colors across products to WikimediaUI color palette https://phabricator.wikimedia.org/T148708

Community Tech

• Blocked: not blocked, but see our callout
• Blocking: none
• Report:
  • Working on GlobalPreferences
  • Parser caches are almost populated with Unicode IDs

Technology

Analytics

• No blocker (AFAIK)
• Not blocked
• Updates:
  • Wikistats 2.0 UI is alpha-level with reading metrics, still missing edit metrics because of missing backend
  • Setting up a production data cluster for druid with public data to be the edit-data public APIs backend. We have split our Druid cluster in two, with 3 hosts each.
  • Getting Druid Grafana dashboards in place.
  • Working on processing eventlogging data in hadoop instead of MySQL
  • Still purging data in MySQL eventlogging hosts, slow going
  • Work to move metrics to prometheus continues.

Scoring Platform

• Blocked by: none
• Blocking:
  • Still working on file handle exhaustion, which prevents us from using the new ORES cluster.
• Updates:
  • Deploying "revscoring 2.0", which could cause some instability in Extension:ORES and in the ORES service.

Search Platform

• Blocked by: none
• Blocking: none
• Updates:
• Messaging fallbacks usage for search disabled (https://phabricator.wikimedia.org/T147959) - actual switch happens when we reindex
• MLR assisted results test analysis published: https://wikimedia-research.github.io/Discovery-Search-Test-InterleavedLTR/, results generally positive
• Patch for indexing wikidata statements merged, will index P31 and P279 initially after deployment
• Working on porting Selenium tests from Ruby to JS
• Working on upgrade to Elastic 5.5
• Working on indexing Wikidata descriptions and adding them to fulltext search

Services

• Blocked: none
• Blocking: none
  • Updates:
    • Deploying cassandra 3 in beta cluster
    • Starting testing Parsoid with Cassandra 3

Technical Operations

• Blocked:
  • none
• Blocking:
  • none
• Updates
  • Singapore caching DC ongoing
  • Quite a bit of puppet refactoring for varnishes
  • ganglia slowly being killed (nobody will probably even notice)

Fundraising Tech

• Still trying to figure out db lag, have some leads: https://phabricator.wikimedia.org/T173472
• Still trying to figure our missing data from kafkatee, getting some help: https://phabricator.wikimedia.org/T177331
• More CiviCRM contact de-duplication improvements
• Fixing Civi import bugs: https://phabricator.wikimedia.org/T177806, https://phabricator.wikimedia.org/T177803
• Cleaned up contact groups in Civi to improve performance
• Diagnosing apparent missing banner impressions (big chunks seem to be bot-related): https://phabricator.wikimedia.org/T176802

Wikidata

• (Hopefully) unblocked QUnit 2 update: https://phabricator.wikimedia.org/T177764
• We had to turn of RecentChanges integration on big wikis for performance reasons, looking into workarounds: https://phabricator.wikimedia.org/T177707
• Fixed a regression we believe happened because of a jQuery update: https://phabricator.wikimedia.org/T177485
• We turn JS-only components into pure NPM libraries: https://phabricator.wikimedia.org/T177087
• Constraint checks also on qualifiers and references: https://phabricator.wikimedia.org/T176863
• Added new Lua functions to the mw.wikibase library to avoid pulling entire entities: https://phabricator.wikimedia.org/T142903, https://phabricator.wikimedia.org/T173262
• Full URIs for external identifiers will soon be in the RDF export: https://phabricator.wikimedia.org/T121274
• Optimizing runtime of the wikidata.org dump generation: https://phabricator.wikimedia.org/T177486
• wb_entity_per_page table is not updated any more: https://phabricator.wikimedia.org/T140890
• wb_terms table on labs does have a term_full_entity_id column now: https://phabricator.wikimedia.org/T167114
• As usual, we continue working on the Lexeme extension for Wiktionary: https://phabricator.wikimedia.org/T168263

RelEng

• Blocked: none
• Blocking: none?
• Updates:
  • wmf.3 cut this morning, back on track today group0 and group1

Security

• Reviews:
  • Ex:FileImporter (T160982)
  • Ex:WikipediaExtracts (T149424)
  • pdfrw