Discovery/Retrospective 2015-08-10

Review of action items from the last retro (2015-07-13)

We don't seem to have a smooth way to convert Dan's "I want X" (a.k.a. user stories) into well-defined tasks
- Kevin will facilitate improvements
  - STATUS: No explicit action, but raised awareness. Is it better now?
Maintenance/technical operational work (restarts, adding extra data centres) continues to crop up and detracts from achieving goals. Need clarity around this
- Wes will try to get us a dedicated Ops person. This is a long term plan.
  - STATUS: Discussions to hire an Ops person are in progress
- Kevin will document Who from ops is helping with each of our projects
  - STATUS: Discovery page now shows ops support for each sub-team
Experiment with Scrum
- Kevin will coordinate a discussion
  - STATUS: Email discussions have happened, and some meetings have been dropped or rescheduled. Additional process changes have been proposed, but not yet adopted
Circulate retrospective etherpad ahead of time (Kevin)
- STATUS: Was done

Items from retros before that

(For reference in case they are still problems)

Kanban process had not been communicated clearly
Concerns about level of support from ops
Single point of failure for resolving cluster problems
We are working on a lot of things slowly rather than a few things quickly
IRC norms. Should we agree and document them?
Analysis was feeling overwhelmed

What has made us happy?

Fast turnaround on A/B tests
Onboarding Mikhail has been wonderful (thanks Mikhail!)
Very fast response time from the team to the email I (Dan) sent about the prefixsearch bug which is affecting VisualEditor and other API clients.
Praise for ops: BBlack is awesome! In one day did most of the work required to make maps public, and gave us two more servers (Varnish)! Akosiaris did the initial work during Wikimania, but much harder to catch him in offline mode.
Happy about presentation and visibility cadence (raising awareness of Discovery in the foundation and community)
We got a LOT of interest and questions after Metrics seemingly just because I (Dan) was stood there
- Not only that, but the questions were about our current work rather than tangential things, so people are engaged with our objectives
Biggest decrease to the zero results rate today since we started our work... although it is within statistical variance.
Happy that analytics is built into the department
WDQS finally running on production hardware
Dan's product desires are getting translated into actionable tasks better than previously (also see "sad" section)
- One example is reverse index
Dan is very much enjoying the simplicity and lightweightness of Discovery Team's long term strategic planning
Maps are shaping up! The service is gaining in attention, we are on the final stretch of making it public.
Turnaround on the A/B infrastructure was incredible - thanks, Bernie! (wait, if Bernie Sanders wins the election will 'thanks, Bernie' be like 'thanks, Obama'? And this is clearly Oliver.)
The amount of stuff we're happy about
Trey is awesome! The analysis of API (ab)users was top-notch.

What has made us sad (or angry)?

A/B testing has been painful.
- The definition of the experiment and what was being tested did not factor Data in or involve them, causing scientific issues. +++++++ 7
- The data format was a pain to work with.
- Data questions around the A/B test weren't being given to the analysts.+++ 3
Almost nothing that should be going on to the mailing lists is
- It's either happening on IRC (the wrong platform), or not at all.
- IRC not being logged. Could be, but it's not clear that's going to help the situation.
- Proposal: If it didn't happen on the mailing list, it didn't happen. Let's stick to that. ++++ 4
  - https://lists.wikimedia.org/pipermail/wikitech-l/2015-August/082651.html follow up on maybe a better way to format?
Elasticsearch 1.7.1 upgrade was not as fast as we expected: it requires time to investigate the cause, Chase proposed to have look. ++++++6
Dan's product desires could still be translated into actionable tasks even better than currently! (also see "happy" section)
- Happiness on this topic outweighs sadness
- One example of "could be better" was getting A/B test going
Maps sometimes feels like a foster child to the "search" team
- (IRC: "#wikimania-search" -- only recently we started using the "discovery" word instead of "search") ++++++ 6
- Wes has been feeling some bureaucratic frustrations around trying to officially change us to Discovery
Some lack of clarity as to engineering priorities (e.g. rolling restarts work)
Timezone mismatch with ops causing things go slower than it could be
- Also only having fragments of various ops people [Tomasz will own this]

What has made us confused?

When is the off-site? This is getting to the point of being disruptive to meatspace.
- OFFSITE WILL BE SEPT 30 - OCT 1 - Cape Canaveral, FLA. I am sending a formal announcment this week.
  - Yay! Yay! Yay! SPACESHIPS FTW! OH MY GOD IT'S FULL OF STARS

How did this retro go? How could the next one be better?

Lots of good written feedback on the etherpad but fewer voices in the hangout
- Ask people to read their own etherpad comments aloud
Retrospective went a lot more smoothly and people were more open than previously. Very good.
Unclear who owns action items from this retro
- Kevin will make sure the voted items have owners

Action Items

A/B test process concerns [Owner: Oliver]
- The definition of the experiment and what was being tested did not factor Data in or involve them, causing scientific issues. +++++++ 7
- Data questions around the A/B test weren't being given to the analysts.+++ 3
Elasticsearch 1.7.1 upgrade was not as fast as we expected: it requires time to investigate the cause, Chase proposed to have look. ++++++6 [Owner: Erik]
Maps sometimes feels like a foster child to the "search" team (only recently we started using the "discovery" word instead of "search") ++++++ 6 [Owner: Tomasz]
- Add KPI's to dashboard
- Get people using it
- As we give talks, include all our projects
- Rename #wikimedia-search -> #wikimedia-discovery
Proposal: If it didn't happen on the mailing list, it didn't happen. Let's stick to that. ++++ 4 [Owner: TBD]
- IRC already flows to phab well; remind each other to go to email when appropriate