Wikimedia Discovery/Meetings/Search retrospective 2016-04-18

Discovery Search Retrospective

2016-04-18

Covering whatever has happened related to the team since the last retro: last retro on 2016-03-24.

Review action items from before

Mikhail: Check whether meta Discovery/Testing page is up-to-date
- Done
Dan: Should probably have an automatic task to announce each test
- Done https://meta.wikimedia.org/w/index.php?title=Discovery/Testing&diff=prev&oldid=15481896
Guillaume talk to robh to understand how procurement works
- Done (mail sent about a few of the things I learned)
Think more about velocity question: Hire more? Change process? Is it OK as is? Start doing guesstimations?
- NOT DONE (it was not assigned). Not a major problem right now, so consider for later.
Announce the past test(s)
- Carried forward as an action item, but this time assigned to a person

Have rough action plans for progress to finish new quarterly goals (but see related point below)
Everything in place for codfw rollout, we didn't block anyone. Thanks gehel! (and thanks to all the people who helped me - David, Erik, Joe...!)
Elasticsearch hardware ordered - New elasticsearch servers *should* be there tomorrow!
Weekly updates - https://www.mediawiki.org/wiki/Discovery/Status_updates
TextCat documentation - https://www.mediawiki.org/wiki/TextCat
WDQS geosearch has prototype, can be deployed as soon as Blazegraph has official release
I just used the completion suggester to edit the quarterly goals page on mediawiki.org in VisualEditor and it's awesome :-)
Minor interactions with legal went well--he has been on top of things

Action plans for quarterly goals are not yet publicly available anywhere
- Said action plans don't really exist. That might be why. ;-)
- Actually i wrote some basic stuff for es2.0 update after talking with dcausse, and Trey wrote up something (minus dates) for language switching.
  - where are those writings? Would love to read them!
  - see action item #2 :) They are in person to person emails that probably could have been to the public list
- I managed to miss both of these then. Herp. Can we broadcast them?
Could send more of our person to person emails to the public mailing list
- Or on wiki
"Cindy" (our browser test platform) complains all the time. Our browser tests are not reliable and when something breaks we tend to merge anyways because we don't notice the difference between new tests breaking and tests that intermittently break.
- Last breakage was actually bugs, we probably should pay closer attention to it. Which of course means it should be more stable too :)
- If tests all worked reliably, they could vote as part of CI
- GL: Would preconditions for tests help? e.g. detect malfunctioning/missing components before actually attempting tests
  - Perhaps in some cases
- GL: Should we reduce browser testing and increase unit tests
  - EB: Actual unit tests might help, but end-to-end might catch issues that show up in production
  - SM: Unit-testing elastic is difficult in many cases (Nik agreed)

wdqs1002 not restarting when asked nicely -> full reinstall. I learned a ton, but it took a lot of time... (it's normal for full reinstall to take a day or two, these things are not fast) (and we'll need to do it again for wdq1001 sometime soon-ish ;)
This is a draft page for the search team. I'd love some feedback, additions - https://www.mediawiki.org/wiki/Wikimedia_Discovery/Search
Ran test for phrase boost, results inconclusive. Looking into if analysis could be more targeted or if it just has too small an effect to measure.
- Re-analysis is still pending, which could change our thinking
Analysis load for search team was light this month (which was OK since we only have one analyst now)

Erik: Post action plans with rough timelines for finishing Q4 goals
Erik: Advance the conversation about improving browser tests
Chris: See if there are any past tests that weren't announced that should have been
Chris: Update Discovery page on Mw.org to reflect q4 goals
Chris: Request feedback on search team draft page