Discovery/Status updates/2019-04-08

This is the weekly update for the week starting 2019-04-08



  • New WikibaseCirrusSearch code is deployed and enabled on all wikis, including Wikidata & Commons. [1]
  • We re-indexed all Greek, Turkish, and Irish-language wikis with better handling of Greek accents [2], Turkish İ/ı [3], and Irish initial mutation [4] when lowercasing searches in quotes. [5]
  • We noticed that indices can move to readonly state when disk space is low and needs to be reset manually once disk space is available again, so we made it a cookbook [6]
  • David wrote a patch to help with an exception from CirrusSearch/Sanity/Checker (cannot fetch ids from index) [7]
  • We fixed/finalized more things with the ES6 upgrade:
    • Create checks that alerts on cirrussearch update lags [8]
    • Elasticsearch 6: the classic similarity is deprecated [9]
    • Setup elasticsearch on cloudelastic100[1-4] [10]
    • Rack/setup/install cloudelastic100[1-4].eqiad.wmnet systems [11]
  • Erik converted mjolnir from KafkaRDD to direct kafka-python usage, because KafkaRDD in pyspark only supports the 0.8 api [12]

Wikidata Query ServiceEdit

  • Deployed patch to fix an issue in Blazegraph workbench [13]
  • Working on improving cache behavior of Updater [14]