Reading/Multimedia/Retrospectives/2019-01-17

2019-01-17

Previous Action Items

• Ramsey to do the ticket guidelines doc DONE

• Max to send out examples of Phab templates used by teams, how they create them, share them, etc DONE

• Amanda and Pam to create short proposal for using epics (e.g. "this is what we need, here's a way to do that"), team to comment async (e.g. "this is covered in the status quo" or "I see these risks") DONE

Special Topic

Launch Retro

What about the launch went well that we would want to repeat in the future?

• We didn't break everything, and didn't have to revert

• Lots of support from colleagues in WMDE (especially), Release Engineering, and other teams. +

• There are so many captions!  In so many languages!

• I really liked hanging out on the deploy call

• Despite the pressure we got from some sources, I feel like the team persisted in the work that needed to be done and handled it well +

• Live hacks for fixing community gadgets in person as we deploy! Whee. +

• The deploy call was much appreciated/neat to be involved in

• Community messaging plan seemed to work - little expression of being taken by surprise ... really?  (maybe? we only have anecdotal evidence, and comparisons to past software releases)

• Folks working behind the scenes to rally support from people who believe in SDC - very helpful

What about the launch could be improved for future launches?

• Some things were a bit buggy - we REALLY need a QA person on the team.  ++++++
  • +1 and also more robust automatic testing?
  • +1 I am advocating/can advocate more for this
  • +1 (Growth & Android teams do QA testing & then final design review)

• Stakeholders need to be more reasonable/informed about what's realistic re: timeframes ... ++

• As there were tech issues (out of our control), we would have benefited from have flexibility around the launch date to have more time to do design reviews/testing on both Alpha & Beta

• i’m now seeing the challenges of [design] working two releases ahead of the team. it makes it hard to both connect with the work i did last spring and do more testing. +++
  • as a result, i am building time into my schedule for tending to tickets & design reviews of implemented designs, as well as more testing (either on Alpha or Beta)

• We've put energy into streamlining the ticket structure/process and to ensure that it's effective, going forward, on my part I will be writing and tracking tickets much more, and I need us to be explicit about when and who needs signing off of tickets

• Babel doesn't work on Beta :(  A key piece of the way folks see file captions
  • Babel should work on Beta? It did before. Just tested, Babel totes works on Beta.

• There was no final design review/approval to ship--I want a clear gate +++

• Deployment call should have had WMDE in it, rather than them also helping in a side-channel.

• We should have a single, shared QA checklist for regression testing that everyone uses? +

• Multiple sources in the product team (including myself) report a need for a true production mirror so we catch things like weird content/rules/templates the community has set up

• We burned some community capital with our allies :( How can we avoid that in the future?  Do we do a check in with them before shipping?

• I want to know if our release is going to break a key volunteer tool

Topics voted up

• Some things were a bit buggy - we REALLY need a QA person on the team.  ++++++
  • +1 and also more robust automatic testing?
  • +1 I am advocating/can advocate more for this
  • +1 (Growth & Android teams do QA testing & then final design review)
  • We did ask for one, were told there are only so many, and that maybe we can use an external group that wouldn't really fit our needs
  • This is a WMF-wide issue
  • We could also have everyone on the team do QA in a more structured fashion
    • If we do this, there is a risk to let the execs off the hook. QA is a standard part of development everywhere in the world, despite accomplishing so much without it
  • MGMT has been pushing for a production env mirror to make testing easier
  • Are we convinced that a QA person would find the bugs that happened?
    • Yes
  • Major themes: get a QA person, figure out staging, find out what improvements we can do to make beta user testing productive and getting feedback
  • Feature flagging is risky because of the potential to lose eyes on issues
• i’m now seeing the challenges of [design] working two releases ahead of the team. it makes it hard to both connect with the work i did last spring and do more testing. +++
  • as a result, i am building time into my schedule for tending to tickets & design reviews of implemented designs, as well as more testing (either on Alpha or Beta)

Parking Lot

• Ramsey's ticket guidelines ought to be on a wiki page

• "Pain works"?

Action Items

• Amanda and Ramsey for meeting with MGMT: QA, staging env

• Keegan to noodle on getting community to test on  leave feedback
  • Perhaps a checklist

• Design
  • We need to be more explicit about when things need design review
  • Pam to pair with devs as they implement designs, see how that goes
  • Both of above are ad-hoc pings for now

• Ramsey: We need a QA checklist, with Design elements as a part of that

• Pam: Look into how other teams deal with regression testing with a Designer involved

• Don't lose the epics thread