Wikimedia Engineering/WMF Tech Days 2013/Product and Data Workshop
Product & Data workshop
editNotes: http://etherpad.wikimedia.org/p/data-and-product-workshop
- see the brainstorming page
- Leads
- Dario Taraborelli, Aaron Halfaker, Toby Negrin
- Schedule
- 1:30 pm - 60 minutes - 6th floor collab space
- Audience
-
- PMs and analysts: how do we incorporate data into product decisions?
- Engineers who want to collect data on new features, changes to core etc.
- Outcomes
-
- how we incorporate data into product decisions
- how to work with analysts to evaluate products
- review features currently in design/under development and the most appropriate evaluation approach
- Things to discuss
-
- Evaluating new features/products
- Product lifecycle
- Using data to inform product design vs product optimization
- Types of request: counting things vs testing hypotheses
- Controlled tests (hypothesis testing)
- When is it appropriate to run a controlled test?
- A/B testing: low hanging fruits
- A/B testing: state of the art. What can we do with the existing infrastructure?
- What are the costs associated with a controlled test?
- Descriptive usage data (counting things)
- I want to use EventLogging to collect usage data, where do I start? How do I set up a dashboard?
- Ad-hoc analysis: e.g. feature adoption by user segment
- Cohort analysis: Wikimetrics
- Other product evaluation methods
- Usability testing
- Qualitative analysis
- Evaluating new features/products