JADE/Open questions
< JADE
This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date. |
Open questions
- We need to define what type of feedback we accept from people. Free-form text, but also some quantitative number that measures level of disagreement? Prespecified, structured data and a comment from human evaluator.
- Which entities will we eventually support as thread target?
- What makes a JADE thread any different from a talk page topic?
- We need terminology for each element of our system. "target entity"? "refutation", "review", "agreement"? "comment"? "reply"?
- Since comments will likely reference ORES scores as part of a refutation, we'll want to store that relationship explicitly.
- Should this affect the automated score, automatically? After some arbitration process?
- Meta ORES human judgments should have some kind of influence on the error function we use during ML training. How is that feedback structured? Is it a strong signal? Do we respect all judgment equally? Do we need another level of human meta-meta-curation of feedback??
- What do we expect clients will do with this information?
- Will there be any automated syncing between JADE and Wikilabels?