User:MSantos (WMF)/Parsoid Usage Confidence Framework
The Parsoid Usage Confidence Framework is aimed at evaluating the readiness and efficacy of enabling Parsoid on diverse wikis. This guide delineates the steps and methodologies for operating the framework to ensure thorough evaluations and informed decisions regarding Parsoid deployments.
The output of this framework should not force us to use "cold numbers" to deploy Parsoid for the sake of achieving the goal. We should use to enquiry more profound product questions.
Metrics
editThese are the current metrics we are looking for:
- Supported Extensions Coverage
- Visual diff coverage
- Linting errors (?)
Performance and missing functionalities are moving targets. Current parsoid metrics should be evaluated across the board, it's not clear what would be the value if we can differentiate per wiki so this will be our of scope in the first experiment, however these are other metrics to consider in the future:
- Qualitative Reports (?)
- Number of bugs (?)
- Parsoid metrics
- Performance (?)
- Parsoid missing functionality in affected pages
Confidence Score
editBased on the combination of the metrics we should always have an answer for enabling Parsoid Read Views for every wiki:
- Not confident at all
- Not confident
- Somewhat confident
- That means this wiki can be considered the next target and we should investigate the metrics
- Confident
- Extremely confident
With that we will be able to look at the reasons for not being confident with Parsoid and focus on its characteristics and architecture with a Product Strategy lens to mitigate the gaps that affect our confidence to deploy Parsoid for Read Views.
Goals
editTurn on parsoid on self-contained and safe places to develop the framework for "Parsoid Usage Readiness".
Immediate Goals
edit- Deploy Parsoid in controlled and safe environments to enable learning and development of the "Parsoid Usage Readiness Framework"
- Globally for specific wikis
- First candidates are Wikitech DiscussionTools and Officewiki Read Views
- Next English Wikivoyage DiscussionTools
- Later Hebrew Wikipédia DiscussionTools
- Allow users to opt-in to Parsoid in any wiki by using the ParserMigration extension
- Globally for specific wikis
Next wikis
editWiki Name | Visual Diff Coverage (%) | Supported Extensions Coverage (%) | Linting Errors | Confidence | Reason |
---|---|---|---|---|---|
Office Wiki | Confident | ||||
Wikitech DiscussionTools | Confident | ||||
English Wikivoyage | Somewhat confident | ||||
Hebrew Wikivoyage | Somewhat confident |
Performance and missing functionalities are moving targets. Current parsoid metrics should be evaluated across the board, it's not clear what would be the value if we can differentiate per wiki so this will be our of scope in the first experiment.
Open questions
edit- Decide how many features we need to turn on to avoid community discontent
- This is not about having 100% of wikitext features done, but does coverage matters?
- What will be the target metrics to get parsoid read views on testwiki?
- What is the amount of wikitext that users focus on DiscussionTools?