Having rules live on-wiki (like at MediaWiki:EditCheckRules.json or the like) would help projects adjust to their needs in a lighter weight format than the abusefilter (inspired by the newcomer tasks setup)
@Xaosflux: well put and agreed. I've added what you've suggested above to T327959#8725987. This is the task where we'll need to work together to figure out how to go about implementing Edit Check's on-wiki configuration.
...as far as 2 - not sure. Actual edits with it should get identified (a revision tag would prob suffice).
Noted. It sounds like we're on the same page...
The Editing Team is planning to do as you suggested above: introducing a hidden revision tag (T324733) that will enable us (volunteers and members of the Editing Team) to review the edits (T324734) the Edit Check heuristic (T324730), as currently implemented, would identify as needing references.
It seems like edit-check is going to interfere in the publish workflow - this should be carefully considered with how it overlaps or times with ORES and AF - there are certainly edit types that could trigger all 3 of these things.
Great spot...
Do you happen to have a scenario in mind where you think what you described above could come to fruition? I'd like to make sure I have a clear image of what you're seeing in mind so that we can account for it and if possible, ensure people do not experience a deluge of potentially redundant messages.