User:Jeblad/subjective voting

Subjective voting is an idea to use w:subjective logic (w:Dempster–Shafer theory; w:probabilistic logic, w:probabilistic soft logic, w:Markov logic networks, w:subjective logic, w:evidence-based subjective logic) to make a kind of voting system for deletion candidates. The method use a set of criteria, preferably as a set of prepared claims or w:propositions. These are the "set of possibilities under consideration". Users will then cast votes on whether these claims are true or false, by choosing among other prepared claims or w:opinions. These are a "subset of the frame of discernment".

The voting process would be organized with propositions in a w:Disjunctive Normal Form (DNF). It is a DNF if and only if it is a disjunction of one or more conjunctions of one or more singular propositions. By assigning belief or disbelief to propositions, clauses will be dismissed as invalid, and hopefully only a set of valid theoremes will be left.

The propositions and the opinions may or may not point to evidence for the correctness. Usually it would be a GoodThing™ to use such evidence, but the method as such does not depend upon it. It could be interesting to use evidence as a means to add prior belief to opinions.

Outcomes and actions

Voting on Wikipedia leads to some outcome, and then someone will make an action accordingly. If the outcome is not actionable, then it usually makes no sense to vote over possible outcomes. It is although possible to formulate outcomes without actions, like a voting process whether we like or dislike w:Jar Jar Binks, but it that is expressed somewhere, then we have an action.

Most commonly on Wikipedia we vote on pages for deletion, and the two outcomes are keep and delete. Some standardized actions can be automated, and a quicklink provided for the page action. It could be necessary to adjust some page actions to facilitate all the necessary arguments, so the executing user lands on a page with all necessary data prefilled.

This type of process fits a vote for deletion and vote for move quite well, but it is not a good fit for a featured article process. That is because a featured article process is not really about voting, but an investigation to find and fix questionable parts of the article. It is possible to change the voting process so it is iterative. That is interesting, but slightly outside the scope of this text.

Necessity and sufficiency

Not all propositions are equal, some hold or imply conditions that is necessary or sufficient. A necessary condition will imply that the proposition must hold, but in itself it is not sufficient. A sufficient condition must hold, and is in itself enough to give the final outcome.

If we have a deletion process, and a final proposition is sufficient and has delete as its outcome, and it is enough belief for the proposition, then the overall outcome will be delete no matter what other propositions exists. We let the sufficient condition take precedence over the necessary condition. If we have opposing sufficient conditions, then we have a stale outcome.

If we have a deletion process, and the propositions are all necessary and has delete as its outcome, and it is enough belief for the propositions, then the overall outcome will be delete. If some propositions are necessary and has keep as its outcome, and it is enough belief for the propositions, then…

Propositions and opinions

Assume we have a proposition. Then we have (at least) two opposing opinions about this proposition, we have a belief and a disbelief. This is called a binominal opinion. It is not obvious, but it is also a third opinion for those that does not commit themselves to any of the two previous beliefs. This uncommitted belief change how much belief is necessary to reach a previously set threshold. It is not enough that the belief is just slightly bigger than the disbelief, it must be bigger that the disbelief and uncertainty combined. How much bigger depends on the different models for subjective logic.

It is tempting to make a system where the voters only mark the propositions they like, but such a system would miss the uncommitted belief, thus tend to choose a single winner on raw count.

Now assume we have two propositions. Claim (A) has a sufficient condition, and claim (B) has a necessary condition. For simplicity assume they are opposing claims. Each of those two has their own pair of opposing opinions. If the voters put too little belief in claim (A), then claim (B) might win. If the voters put enough belief into claim (A), then it does not matter what claim (B) is, claim (A) wins anyhow as (A) has a sufficient condition. See w:Necessity and sufficiency for details.

Within specific domains sets of propositions could be prepared, like the guidelines from w:Category:Wikipedia notability guidelines, and then just included on a talk page or other discussion page by referencing the set somehow. One way to do this is to make a JSON structure, and then reference the page. Another way to do it would be to make templates.

Trust and reputation

The votes don't have to be given equal weights, new editors can be given less weight than an ordinary user, and an admin even higher weight. This is interesting as on-ramping of new editors, but it can also give a wrong signal to the community.

One interpretation of non-equal weights are as trust. For example the admins are assumed to be more knowledgeable than other users, thus an admin as higher trust. During voting we can use this and give the admin a higher weight. If we give the user too much weight, then the voting process will start to fail as other users starts to distrust the outcome.

One quite interesting method for assigning trust for a specific user is to calculate w:covariance between previous votes and previous outcomes. If the user has a high covariance with outcome on his previous votes, then assign the user high trust. Note that covariance will be a w:covariance matrix for multinominal cases, with no entry for uncommitted belief. That is the user votes are compared to the outcome.

Another method is to use a w:reputation system to calculate a trust metric. The reputation could be calculated as part of another voting process, for example the votes cast during admin nomination, or a gild system. A gild system is based on users joining a thematic guild, and thus being assumed to have special knowledge about the given theme. On Wikipedia there are projects, like w:Wikipedia:WikiProject Astronomy, and the projects has members. The members within a project is a guild. On English Wikipedia there is one named guild and it is w:Wikipedia:WikiProject Guild of Copy Editors, it is somewhat different from a guild in this context.

Evidence

Subjective voting can be extended. Much of the previous assume we use subjective logic, but it could also use w:evidence-based subjective logic. To use prior knowledge of …

Interrupted voting

It is not obvious what happen when an ongoing voting is temporarily interrupted. It is probably simplest to ask those that already has cast votes to do it again, and reject the previous result.

User interface

The user interface is kept as simple as possible. Radio buttons in a row for each proposition, and the opinions as headers above the buttons. This works well as long as evidence isn't included. The action is given as part of the proposition, and the sufficiency is shown by bolding. It should probably be additional flyover help in a real interface. A real user interface could be very different.

Soccer player False Uncertain True Proposition ○ ● ○ Keep: High enough division ○ ● ○ Keep: Has news coverage

Musician False Uncertain True Proposition ○ ● ○ Keep: Exhibit at w:Rock and Roll Hall of Fame ○ ● ○ Keep: Has published works ○ ● ○ Keep: Has available works ○ ● ○ Keep: Has news coverage

Propositions should be falsifiable, that means that for deletion processes it is very difficult to formulate a condition that imply deletion. To get around this there can be a default action to delete if all propositions fails.

One way to show evidence is simply to list them below each table row.

For users without Javascript to be able to vote there must be a submit button somewhere on the page.