Discovery/Status updates/2018-08-20

This is the weekly update for the week starting 2018-08-20

Discussions edit

Gehel did a lot of work on getting the servers ready for ElasticSearch 6: resolving a deprecated data directory structure [1], migrating from Jessie to Stretch [2], and migrating the cirrus cluster to RAID0 [3]
Erik and Gehel fixed an issue where the mjolnir-kafka-bulk-daemon was failing on all elastic / eqiad nodes [4]
Erik also added stats collection for observability of mjolnir daemons using prometheus [5]
We had an issue where the warning message was not very informative when doing a search with greater than 10,000 results; Erik updated the warning to now read: "A warning has occured while searching: Could not retrieve results. Up to 10000 search results are supported, but results starting at 10000 were requested." [6]
When the ElasticSearch server went down recently, there were various PHP notices spamming the logs. Erik was able to fix this by better handling the null result from link count msearch [7]
Trey worked on creating and merging, a Esperanto Elasticsearch plugin using Esperanto morphological libraries [8]
Stas worked on a enabling indexing all entity-valued statements [9] which is now enabled for newly edited items and will be added to all items on next reindex [10]

Stas has fixed some bugs in categories daily RDF diffs generation [11] and wrote puppet recipies to enable auto-loading them on WDQS servers. Switch from weekly to daily loading to start on Monday 27th.

--