ORES/Issues

Warning:

The ORES infrastructure is being deprecated by the Machine Learning team, please check wikitech:ORES for more info.

ORES makes mistakes. It's not as smart as humans are. In order to make sure ORES is as useful as possible despite its limitations, documentation around its issues and mistakes is critical. This page is intended to be a central reference for reporting and discussing issues with ORES.

Edit quality

See /Edit_quality for reporting misclassifications.

Bias against anonymous editors

The edit quality models have a known bias against anonymous editors. In some ways, this is desirable since most vandalism does come from anonymous editors. However we've been able to mitigate some of the issue. See the original reports (T118982, T129624), our task for identifying the problem (T120138 and analysis. This issue was presentation by EpochFail at a mw:Wikimedia Research/Showcase (slides).

We're working to mitigate the problem by increasing signal from non-user-related features.

• T144636 -- [Epic] Implement PCFG features for editquality and draftquality
• T145812 -- Implement ~100 most important hash vector features in editquality models
• T162617 -- Use 'informals', 'badwords', etc. in Wikidata feature set

Article quality

See /Article quality for reporting misclassifications.

Draft quality

See /Draft quality for reporting misclassifications.

See also

• phabricator.wikimedia.org/tag/scoring-platform-team
• T148700 -- The tracking task for building a queryable system for tracking misclassifications