Discovery/Retrospective 2016-01-25

Review action items from previous retrospective

• Dan: write a goal for improving the UX of the search page on-wiki
  • DONE! Deprioritized for Q3
• Dan: Discussion of improving the relevance/sorting of results rather than just zero results rate
  • DONE, we think. Covered by current work (this was before completion suggester)
• Dan: Follow up on the common terms query A/B test
  • Not done. Internet archive folks suspect this isn’t a great solution. Dan will follow up this month.
• Moiz: Talk about whether we really can run A/B tests on the portal, since it's not subject to a deployment freeze
  • DONE. We ran a test during December.
• Mikhail: Look into listing features that affected the results set for a query (sister project to 'query categorizer UDF')
  • Dependent on a 10% project that hasn’t been top priority. May happen this week.
  • David: stopped to work on the scoring doc, I should continue

What has happened since the last retro?

The last Discovery retrospective was 2015-11-30. Notable events/accomplishments since then:

• Deb arrived
• December deployment freeze
• Dev Summit/All Hands/Discovery Days
• Quarterly review
• First portal A/B test was deployed and abandoned
• Portal A/B/C test (including relaunch of original test) was launched and was great!
• Language detection & switching yields more results (still need to check if they are relevant)
  • TextCat & limiting languages to relevant targets improved accuracy a LOT.
  • Ported TextCat to PHP
• Maps plans crystallized, maps got community traction
• Created Q3 goals

What went well?

• Portal A/B test deployment, responsiveness, analysis +1 +1 +1
• It was great to see everyone in person (esp. new folks)! Talking to people (our team and others) instead of going to various meetings/lectures was a better use of time.+1
• Meeting everyone on the team was fantastic! I just wish I hadn’t been so sick and lost my voice. :(
• Deb is awesomeeeee! +1 +1 +1 +(2e23-1) +1 (now you’ve caused an integer overflow, well done trey—Deb is that awesome!) (but now it says she’s –2147483648 awesome!) (I mean actually we could be fine, it’s only minimum maxima and maximum minima. On a specific compiler and OS she’s probably good) So, does the negative number mean I’m not awesome now? ;) No, you’re still awesome, we just need to switch to using maybe an unsigned long. We could probably rely on a double too. (We need arbitrary precision arithmetic to calculate your awesomeness.)
• Collaboration with analytics team building out our first usage of analytics infrastructure in prod
• How the portal team got back on top after the first a/b test didn’t go well.
• All dashboards and graphs that need it now have automata filtering +1
• Mikhail’s 33% time support for the annual fundraiser
• Quarterly review deck was minimal effort to create, and review itself went fine***
• Italian food with the team … plus dessert +1+1 although that chocolate lava cake really wasn’t that good in the end! ;-) (we were warned)
• Completion suggester beta feature is live and people love it! And it’s still getting better! +1+1 +1
• Interactive graphs launched

What could have gone better?

• Communication between design/engineering on portal tasks
  • New team, still working things out. Test went well, but open questions remain.
• Would like more contribution in tickets (comments, ideas, suggestions, closing out and resolving, adding in gerrit check-in’s) ********
• Working with operations to move forward our needs for analytics<->elasticsearch has gone ok, but has taken quite some time and is still not complete. Hopefully integrating Guillaume will help us do better here. *
• Guillaume starts 2/1 Yay! +1
• Use meetings to spend more time discussing ideas rather than cleaning the sprint board. Involves looking at the board more often between meetings and be aware of the tickets in progress and in review. *********
• First portal a/b test (didn’t log what we really needed to know)
• Portal A/B(/C) test is still in production (not a big problem; just a loose end) **
  • should we extend the analysis of this test? or just accept the first week’s worth of the testing?
• Elasticsearch health over the last several weeks (but it’s finally happy after shifting traffic away from eqiad) ****

=What didn't fit into either of those buckets?

• Pageviews ate ⅔ of my soul. (Pageviews were my entire job for 18 months, you’re lucky)
• Recruiting for a relevance engineer went well, process-wise, but we haven’t found anyone
  • JD has been taken down. We’ll figure out what position to post.
• EventLogging database :( :( :( *tear emoji* 😢
  • Has been unreliable--required us to backfill data

Discussion

Use meetings to spend more time discussing ideas rather than cleaning the sprint board. Involves looking at the board more often between meetings and be aware of the tickets in progress and in review. *********

• If we do a better job of fleshing out tickets, these meetings should be easier/more efficient
• Would be good for the mtg to feel less administrative/mechanical (costly in a synch mtg)
• In the last search grooming meeting, it was unanimous that the meeting was valuable.
• Since each team (and meeting within the team) is different, let’s continue this discussion within each team.

Would like more contribution in tickets (comments, ideas, suggestions, closing out and resolving, adding in gerrit check-in’s) ********

• Seems to mostly work on search side. Moving to the correct column is important.
• Just looking at any given ticket without context, it can be hard to understand the task and/or its status.
• During the meeting, Dan adds comments explaining why he is moving tasks, but those comments tend to be quite terse because they were written in a hurry (during a meeting)
• Descriptions are often under-defined. Ok as an initial placeholder, but should be refined at some point.
• Some comment threads are massive; would be great if decisions could be summarized so you don’t have to read them all.
• Writing comments during meetings is fine (necessary), since it gets consensus from everyone. Maybe take the extra minute to make it a better comment.
• Our process (non-scrum) encourages under-developed descriptions. Devs don’t have a chance to sign off on tasks being well-defined before they get dropped into the sprint. Dan prefers scrum. But also doesn’t want to flip-flop between processes.
• Portal board has a range of tasks from small to large, and it’s not clear which tasks need input from who. (e.g. a column to the left of backlog, where they need more definition, or an “in discussion” column.) Maybe have a column for tasks that urgently need answers?

Action Items

• Dan: Follow up on the common terms query A/B test
• Mikhail: Look into listing features that affected the results set for a query (sister project to 'query categorizer UDF')
• Deb: work on road map for A/B/C testing
• Deb: work on Quick Surveys
• Deb: talk with community (mxn and Edward Saperia) portal testing and how fast we can launch new things
• Moiz: Allocate UX for maps (based on priority input from Dan, which would require clearer definition of what is required)
• Tomasz: Clarify to the team the long term plan for maps
• Kevin: Consider splitting up retrospective next month (17 invitees)
• Kevin: Work w/teams to ensure that meetings (e.g. sprint planning) are as effective as possible.