Parsoid/Parser Unification/Confidence Framework

The Parsoid Confidence Framework is a mechanism for transparently evaluating the readiness of Parsoid Read Views on diverse wikis. This guide describes the goals of the framework and the metrics used in order to allow insight into the decisions of the Content Transform Team to deploy Parsoid as the default wikitext parser on a given wiki.

Status

An overview and summaries of the reports are at Parsoid/Parser Unification/Confidence Framework/Reports.

Detailed results for current and planned deployments are within sub-pages:

• Wikivoyage
• Wiktionary

History

Parsoid began as part of the VisualEditor project, and has been used in production as part of VisualEditor, mobile apps, and tools like Flow and Content Translation since 2012. The Confidence Framework was developed alongside the first deployment of Parsoid Read Views in 2024, part of the Parser Unification project. As described on our updates page, our first deployments were to Wikivoyage wikis, followed by Incubator wiki and newly-created wikis. As the roll out of Parsoid Read Views progresses, the Confidence Framework will continue to be developed to reflect newly-uncovered challenges.

In particular, we plan to incorporate performance metrics into the Confidence Framework in the near future to ensure it properly reflects Parsoid readiness for large-page-view wikis.

The Confidence Framework

Although the framework is built from numeric metrics, we do not blindly set cutoffs. Instead, the metrics are used to prompt product questions and investigation when they seem atypical.

Metrics

These are the current metrics we use for decision making:

• Supported Extensions Coverage
• Visual diff coverage
• Qualitative Reports

Confidence Score

Based on the combination of the metrics we have an answer for enabling Parsoid Read Views  for every wiki:

• Not confident at all
• Not confident
• Somewhat confident
  • That means this wiki can be considered the next target and we should investigate the metrics
• Confident
• Extremely confident

With that we will be able to look at the reasons for not being confident with Parsoid and focus on its characteristics and architecture with a Product Strategy lens to mitigate the gaps that affect our confidence to deploy Parsoid for Read Views.

Processes

Communication examples

• Village pump notification message regarding Wiktionary roll out
• Email regarding Wikitech Discussion Tools
• Email regarding Wikivoyage roll outs