Scrum of scrums/2017-09-27
2017-09-27
editcontact: https://www.mediawiki.org/wiki/Wikimedia_Engineering
Callouts
edit- Selenium Ruby framework deprecation (September) https://phabricator.wikimedia.org/phame/post/view/75/selenium_ruby_framework_deprecation_september/
- Discovery: Maps: Figuring out what to do with code no one on the team is in charge of
- New linter category coming up - html5-misnesting - which triggers when misnested tags behave different in Tidy vs. HTML5 (<span> is notably one of them).
- Eventlogging purging progressing much too slowly, it is becoming clear we cannot sustain mysql backend for EventLogging, we are prioritizing sunsetting mysql, replacing it with a better EventLogging analytics experience on the Hadoop cluster
- Please resolve comments that can be resolved in proposal of redesign of Scrum of Scrums meeting: https://docs.google.com/document/d/11gMloAKqtOJsKaDhpx_oIE8-vXvJs7pyLsDhqikaG08/edit?ts=59b0e080
Technology
editAnalytics
edit- Blockers: none
- Updates
- Working full steam to hit our goal of having a backend for wikistats 2.0 to support editing metrics (unique devices and pageviews are alredy supported on ui). It is looking like we are going to split our current druid cluster (6 hosts) into two clusters: one private, one public so restbase services connect to public cluster. The security concerns arising from this decision will probably make the goal spill into next quarter but we will have an alpha next quarter if no suprises arise.
- We have use differential for 1 quarter and we are going back to gerrit, not much value added.
- Eventlogging purging progressing much too slowly, it is becoming clear we cannot sustain mysql backend for EventLogging, we are prioritizing sunsetting mysql, replacing it with a better EventLogging analytics experience on the Hadoop cluster
- Removing outdated instrumentation of events from EventLogging that are now automagically available via eventbus, like “page create”. https://gerrit.wikimedia.org/r/#/c/379137/
Audiences
editReaders
editMultimedia
edit- Pushing out 3D to test/test2 today (Wednesday)
- Need read-only time on s4 for a schema change related to 3D (adding a new media type)
- Beginning work on MediaInfo extension for Wikibase, which will continue for some time.
iOS native app
edit- Blocked by: none
- Blocking: none
- Updates:
- Finally released 5.6.1 with minor bug fixes ( https://phabricator.wikimedia.org/project/view/2898/ )
- Continuing work on 5.7.0 ( https://phabricator.wikimedia.org/project/view/2899/ ) - Onboarding updates, Improved analytics, iOS 11 support, iPhone X support
Discovery
edit- Putting together plan for automating portal deployments
Maps
edit- nodejs 6.11 done
- Figuring out what to do with code no one on the team is in charge of
Web
edit- Turning off OCG. Investigating using chromium for printing.
Reading Infrastructure
edit- MCS: Dealing with sectioning issues before Parsoid adds <section> tags
- Reading Lists: finishing RESTBase part; MediaWiki part going through security review
Contributors
editGlobal Collaboration
editLanguage
edit- Blocked:
- Request services in help to debug: https://phabricator.wikimedia.org/T173801 This again blocks cxserver deployment(s)
- Blocking:
- Updates:
- CX-VE work continue: saving, restoring.
- cxserver now using readable and splitted registry files.
Collaboration
edit- Updates
- RCFilters:
- Features
- git #b747307a - WLFilters: Live update and View newest (task T176348)
- git #5c499174 - RCFilters: Make 'lastRevision' filter include non-rev types (task T176328)
- Bug fixes
- git #f1340739 - RCFilters: restore watch/unwatch link (task T176264)
- git #a0c00f00 - RCFilters: Make the interface not jump around while loading (task T176300)
- git #78703ae9 - RCFilters: Don't grey out results area when initializing, unless there's a default saved query (task T173533)
- Performance
- git #0005805a - RCFilters: Cache ChangesListSpecialPage::buildChangeTagList() (task T176652)
- git #32e6b77d - RCFilters: Log performance data (task T176652)
- git #20bcfec0 - RCFilters: Don't load all of OOUI
- git #8de793cb - RCFilters: Make live update polling configurable
- Features
- RCFilters:
Parsing
edit- New linter category coming up
- html5-misnesting: This triggers when misnested tags behave different in Tidy vs. HTML5 (<span> is notably one of them).
- Repeat update from last week as an FYI
- Heads up for Parsoid clients (VE, CX, Flow, MCS)
- <section> wrapping code is now out of WIP and in review ( https://gerrit.wikimedia.org/r/#/c/364933/ ) -- please test your code to make sure you can handle <section> wrappers. If necessary, you can pre-process the DOM to strip out <section> tags. Parsoid's output is guaranteed to preserve template wrapping semantics with / without <section> tags. Parsoid can also accept DOMs with / without <section> tags (for serializing back to wikitext).
- We plan to switch Parsoid DOM output to use <figure-inline> tags instead of <span> for inline images ( https://gerrit.wikimedia.org/r/#/c/370227/ ) -- please test your code to make sure you can handle the new markup. VE can handle this.
- Heads up for Parsoid clients (VE, CX, Flow, MCS)
Community Tech
edit- Blocking: none
- Blocked by: none
- Report:
- Currently populating ip_changes
- Working on GlobalPreferences
- HTML5 sections IDs are in stage 1 population
Services
edit- Not attending
- Blockers: none
- Updates
- All mobile traffic served from the new Cassandra 3 cluster exclusively
- Preparing to test Parsoid with Cassandra 3
Technical Operations
edit- Blocked:
- Flow dumps speed issue still, waiting on Collab Team T164262
- Blocking:
- None
- Updates
- Resuming Asia DC work
- Salt removal ongoing, almost fully done
- Possibly fixed a long standing varnish issue with mailbox lags
Scoring Platform
edit- Blocked by: none.
- Blocking: Still blocking ORES deployment to the new cluster.
- Updates:
- Working on the statistics-derived thresholds changes, on the MediaWiki side now. This is a blocker to us releasing the next major version of ORES. Deployment will be messy since the new code breaks v1 of the API.
Search Platform
edit- Blocked by: none
- Blocking: none
- Updates:
- Explore similar language links test concluded. Unfortunately, the result was negative - this functionality doesn't seem to be used by real users.
- Widgets are still available for users that want to experiment with them (see https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-09-18#Highlights)
- Announced turning of usage of messaging language fallbacks for analysis (https://www.mediawiki.org/wiki/Wikimedia_Discovery/Disabling_Messaging_Fallbacks_for_Language_Analysis )
- Running A/B test on ML ranking for top 18 wikis
- Chinese & Hebrew wikis were reindexed, which means they now have new analyzers enabled
- Vagrant setups now have new language plugins
- Working on porting Selenium tests from Ruby to JS
- Working on upgrade to Elastic 5.5
RelEng
edit- Blocking
- None?
- Blocked
- None
- Updates
- Selenium Ruby framework deprecation (September) https://phabricator.wikimedia.org/phame/post/view/75/selenium_ruby_framework_deprecation_september/
- "This is your friendly but penultimate warning..."
- 1.30 REL branch cut last week, 1.31-alpha/1.31.0-wmf.1 starting this week
- Selenium Ruby framework deprecation (September) https://phabricator.wikimedia.org/phame/post/view/75/selenium_ruby_framework_deprecation_september/
Fundraising Tech
edit- Quieting down error logs by fixing small bugs
- Refining fraud detection
- Handling more conflicts in CiviCRM contact deduplication
- Moving last ganglia bits over to Prometheus
- Deploying CentralNotice bug fixes