Discovery/Status updates/2019-03-04

This is the weekly update for the week starting 2019-03-04



  • We've updated Textcat with the new Russian wrong-keyboard and wrong-encoding models and now need CirrusSearch to be up to date as well [1]
  • In order to ease migration to ES6 we should migrate to 5.6.14 first (epic task is now completed) [2]
    • As part of the ES6 update, we needed to upgrade logstash and the logstash elasticsearch cluster to 5.6.14 [3]
    • We also upgraded logstash plugin to 5.6.14 as part of the prep for elasticsearch upgrade [4]
  • We've deployed & tested WikibaseCirrusSearch on beta cluster [5]
  • We've also deployed & tested WikibaseCirrusSearch on testwikidata and found things to be good! [6]
  • We found that incigna monitoring for Elasticsearch doesn't seem to notice when an out-of-memory error has happened on a node — this will be fully fixed with the next cluster restart [7]
  • There was some logspam due to deduplicating I18N messages between Wikibase and WikibaseCirrusSearch that we resolved [8]
  • A discovery was made with a WMFTimeoutException in Special:Search that was resolved by decreasing timeouts and removing our timeout hack for regexes [9]
  • An umbrella task to track patches of merge commits to activate elastic6 support code [10]
  • A ticket created way back in 2016 asked: 'the new substring search is great on mw.o. Can we have it on office wiki too?' and yes, now it is [11]

Wikidata Query ServiceEdit

  • We've created CI testing environment for Blazegraph on our Jenkins CI [12]
  • Working on hiring a contractor to help us with Blazegraph tasks [13]
  • We noticed that metrics from wdqs updater JMX should be prefixed while we were investigating something else, metrics exposed by jmx_exporter running on wdqs-updater should be prefixed with wdqs_updater_ to make it more clear what they are referring to (done!) [14]

Other Noteworthy StuffEdit

  • Read Trey's new blog post about 'the anatomy of search: a place for my stuff'