We have been discussing some of the challenges of patrolling "add a link" and "add an image" structured tasks with patrollers on Arabic Wikipedia, Spanish Wikipedia, and Russian Wikipedia.
We are considering several improvements suggested by community members, and I want to open up a discussion here as well:
Hello! I found this thread quite by accident ;)
The current implementation of structured tasks has several problems:
1. They are very easy to make, which allows users not to think what they are doing.
2. You can make a lot of them, partially solved by limits, but still it is impossible to reduce the limits forever.
3. There can be multiple added links in one edit, which does not make it easy to undo the wrong link.
4. There is no automatic/semi-automatic notification to a newbie that their edit has been reverted for such and such a reason.
To deal with these issues, you can do the following:
1. Create a module, but for experienced editors (500+ edits)
2. Instead of / together "add link" they will have "check link"
3. When checking, the editor gets already corrected articles, with added links.
4. He checks each link and if it is incorrect, he marks it and leaves a comment.
5. The comment is posted on the participant's talk page.
6. If there were many (60%) such reverts for one participant, then access to the tool is blocked for him.
It seems to me that this will solve problems with new editors as well and will attract experienced editors to review contributions.
Thank you very much for sharing this very interesting gadget, @NGC 54!! We have so many gadgets available, it is difficult to know all of them.
I will test it with my volunteer account.
Thank you @Iniquity for this feedback! Our team is planning to conduct some further outreach to gather ideas this week and then team members who aren't active patrollers are participating in a demo from a patroller so we can better understand frustrations related to "add a link" and "add an image" tasks. After this, we will review ideas like yours so we can start to evaluate the effort and impact of each idea. I will be sure that your ideas are represented in our discussion.
I really like this idea of creating an easier "check link" workflow for experienced editors. Do you have any thoughts as to how this would fit into the current patroller workflow? Would this be an "advanced" task available on the newcomer homepage, but only available to users with 500+ edits? Or would this a tool somehow integrated into the Recent changes feed? A special Editing gadget that was opt-in only?
Thanks again for the feedback, I appreciate your insight!
Thanks for the detailed answer :) I'm glad that you listen to our opinion!
It seems to me that initially it should be integrated into the newcomer homepage.T hen, if everything goes well, perhaps we can create a full-fledged tool (patrol) for the future.
I was still thinking about how we can fasten the check and removal of templates. See phab:T274909. That is, the priority is to display pages with a template, and then check them.
Hi @MMiller (WMF) is there a 'Newcomer task' for removing redlinks? See for example https://en.wikipedia.org/w/index.php?title=Violette_Impellizzeri&diff=1067612170&oldid=1067612034 - where the redlink is removed with the tag of a newcomer task within visual editor. In general this kind of edit is *really bad* since it removes links to articles that could exist in the future, and newbies definitely shouldn't be encouraged to do this.
Hello Mike
No, there is no such task.
Most tasks we offer are edits suggestions, where we guide newcomers to work on something with full editing capacity remaining. Here, I guess by the banner that the newcomer was asked to fix the tone of the article. To do so, the newcomers can edit the entirety of the article.
Our guidance don't tell them to remove red links, so I think this user thought it was a good idea. I see that you left a message to this user explaining why it wasn't a good idea, which is the best thing to do.
We have some other tasks where users are much more guided, and edit Wikipedia through precise tasks. See our Add a link project for more information.
Thanks for pointing this out, @Mike Peel. Putting myself in the shoes of the newcomer, I think perhaps they might have interpreted the red link as a "broken link", i.e. "links are usually blue, and this one is red, so something must be wrong". Do you know if removing redlinks is something we see less experienced do erroneously? Maybe @Sdkb may have seen something along the lines?
Thanks for the ping. This hasn't been something I've seen, although I'm not sure I see enough newcomer edits that I would've noticed it if it was happening.
To give some context, redlinks are a tricky area, because they often indicate a problem (e.g. adding a non-notable person to a list), but not always. The circumstance in which they're warranted is when there's a notable topic that should have an article but just doesn't yet. So the decision of whether a redlink should exist or not requires an understanding of notability, which is obviously a fairly advanced skill. I see even many more experienced editors removing them overzealously, sometimes citing w:WP:Write the article first. There's just this natural pressure to take something that's normally bad and easy to identify and overgeneralize to it being always bad and try to eradicate it at scale.
In terms of the beginner experience, I think it'd be a good idea for the VisualEditor to do a better job explaining what redlinks are when you click on them. For instance, in the screenshot at right, there's nothing telling an editor what the redlink means. I think it'd be good to put something just below the title, where the short description would go for a bluelink, saying perhaps Unwritten article (learn more) ("unwritten" hopefully connotes both that the article does not currently exist and that someone thinks it should exist).
One last thing to note about redlinks is that quite often, they reflect articles that do exist in other languages, and the best way to handle them is with w:Template:Interlanguage link. For instance, at the instance Mike came across, there's an article in Dutch, and I've added an interlanguage link to it. If we wanted to get fancy, we could add a "search for this in other languages" tool that'd assist with the creation of interlanguage links. But I think that's much farther off/lower priority than just helping folks understand what they are.
Thanks - good to know that this wasn't a specific newbie task, I agree with Sdkb that this is a tricky area that's best to be avoided by newbies. I also agree that VisualEditor should explain these better.
There's an AbuseFilter at enwiki that tags edits by IPs and newcomers who are removing links from Wikipedia articles. I looked at some of the recent results, and they were not bad. In most cases, it looked like the link should have been removed. In a few, it would have been ideal to replace the link rather than removing it ("Chimney Stack" vs w:en:chimney stack), but overall I don't think these were generally bad edits. Looking more statistically, about a quarter of such edits are quickly reverted.
Also, less than 10% of these edits used the visual editor. This suggests that even if the visual editor handled this better, it might not make a significant difference.
> less than 10% of these edits used the visual editor.
At en.wp, you mean?
Problems shared here can be universal. :) You certainly have wikis with an higher use of VE, and all edits made using Growth features are using VE. Users would certainly benefit some changes regarding red links there.
Hello @John Broughton @Sdkb @NickK @Nick Moyes @Galendalia @Barkeep49 @Pelagic @Czar @LittlePuppers @HLHJ and everyone else who is following along!
First results
Thank you all for helping us design and build the "add a link" feature (which now has its own separate project page). We deployed the first version of it about ten weeks ago in four wikis, and we've since expanded it to ten wikis. It's going well so far! We've recently posted the data from the first two weeks of the feature, which we used to get an initial read on whether users seem to be engaged, and whether it is resulting in valuable edits. I invite you all to take a look, and reply with any of your reactions and questions, or ideas of what to look further into.
Basically, we see newcomers doing a high volume of these edits (more than from the conventional unstructured tasks), and that these edits have a low revert rate (lower than the conventional unstructured tasks). Some users do dozens or even hundreds of these edits, with one user on Arabic Wikipedia having done over a thousand. We're not seeing anyone abuse the feature or cause runaway vandalism by, say, clicking "yes" on everything.
We're now assembling a larger dataset with more than the first two weeks of data, and we'll be posting a more in-depth analysis in the future. I want to refrain from drawing any big conclusions before then, but from this initial data, I am optimistic that the "add a link" task is valuable for getting newcomers engaged. We'll still want to look into important questions, like whether these newcomers move on to other kinds of edits, whether this task is more engaging on mobile or desktop, and where in the flow users get stuck. Those questions will help us decide how to improve "add a link", and also how to build our next structured task, "add an image" (please check that out if you haven't yet!)
Wikimania
Wikimania starts tomorrow, and I hope that anyone who is registered for it can attend our session about "add a link". We'll be doing into details on the algorithm, how we built it, and the results so far. There will also be some time for Q&A. The session will be Sunday August 15, 15:450 UTC, and the details on the session are here. Since this Wikimania is virtual, I hope many of you will be attending part of it!
@Czar -- yes, we're keeping track of data that will allow us to answer all those questions. That analysis is another level deeper than what we've done so far, and we're going to be getting to the sorts of questions you're asking in September, when we'll have more bandwidth from our team's data scientist. I'm definitely looking forward to digging into those numbers, and I'll post the results so we can discuss!
Thanks for the update! I'll take a look at the data and join the session tomorrow, but glad to hear that the initial results seem promising!
One question that comes to mind from whether these newcomers move on to other kinds of edits—is there any particular pathway for that? It'd be neat if, after a user has done a bunch of suggested edits, they're invited to check out their homepage, the task center, or even the community portal.
@Sdkb -- we haven't yet built an explicit pathway for what we're calling "leveling up", in which we might say to newcomers, "You've done 20 of these link edits, with no reverts! You may be ready to try a more difficult task, like adding an image." It's something that we're planning to do later this year as part of our broader "positive reinforcement" work (this project page is still quite bare, but we'll be expanding it).
One other thing that we did recently that's in this vein is to let the user switch out of "suggestions mode" into the visual editor. This means that if they're working on link suggestions, but they notice a copyedit that needs to be made, they can switch over and make that copyedit -- it provides them a door to discover other kinds of editing. We don't yet encourage them to make that switch -- it's something we need to think about the right way to do, but the opportunity is there for them. Please let us know any thoughts you have on how might do this well, and see you at the session on Sunday!
For anyone who wasn't able to attend the Wikimania session, the video is available to view here: https://www.youtube.com/watch?v=ar034Gha24o. Thank you to @Sdkb, who joined the discussion after and weighed in with good perspective!
We can think of several editing workflows that could be structured, along with the help of algorithms. Here are some examples. Which of these workflows do you think have the most potential to be structured? Which ones would be useful for the wiki and which ones not useful? Are there others you can think of?
I think adding a table and the aspects that go with it should be an advanced task as a lot of articles have tables some basic some advanced.
@Galendalia -- interesting. Do you know of some way to identify articles that need tables but don't have them?
Adding wikilinks is not particularly useful (also, a "link" can be either a wikilink - internal - or an external [http] link; the latter are generally undesirable, at least in the English Wikipedia; and it is helpful to distinguish between the two). Adding maintenance templates is generally not useful
Every task that is listed consists of two things - (a) changing an article, and (b) finishing the edit by publishing it (ideally, adding an edit summary). Starting out (as an editor) by making a minor change, such as fixing a typo, is a good way for editors to learn that second thing, which they will be using every single time that they edit. By contrast, adding a section involves (1) adding content (sentences), (2) adding citations, and (3) finishing by publishing.
In other words, "fix a typo" or "make a minor change" should, ideally, be the first structured task that an editor learns, because it incorporates the "finishing the edit by publishing it" micro-task. And once the editor has learned to do that micro-task, other tasks will be easier.
@John Broughton -- I think this is a good point, that every task teaches wiki skills (e.g. adding an edit summary) that are not part of the core task itself (e.g. adding wikilinks). We should keep in mind that as we structure the experience of editing, we may also be teaching other universal wiki skills and concepts. Other examples might be teaching users that their edit is immediately public (except in wikis with flagged revisions), or that they can see their edit on the history page.
I thought this was what the Wikipedia Adventure was for? It shows the basics of using WP, however, there is no obligation to go through it. If there was 3/4 of our Teahouse questions would stop coming in. Galendalia (talk) 06:50, 20 May 2020 (UTC)
@Galendalia -- good question! Our team looked at the Wikipedia Adventure (and many other attempts at onboarding newcomers), and we've learned a lot. In summary, our current theory is that a good way to help newcomers stick around Wikipedia is to help quickly have a positive editing experience. We think that if they can make a good contribution within minutes and understand its value, they will be excited and want to keep going. Whereas if they have to go through a long tutorial, they might lose patience and not stick around. So this idea, "structured tasks", is about how we can give newcomers a real editing experience, but with guardrails so that the experience is positive for them and for the wiki.
More background information: In a study on the Wikipedia Adventure, while a lot of users claimed to enjoy the experience, it unfortunately didn't statistically increase their retention, or any other important metrics. But in a study about the Teahouse, it was shown that being invited to the Teahouse does statistically increase retention. So our team took this all to mean that there is something valuable in the personal connection that happens with getting a question answered (although we know it takes a lot of time from experienced editors). That's why we decided to build the mentorship module for the newcomer homepage. And, to your point, as we deploy the mentorship module on more wikis, we are continually trying to strike the balance of giving newcomers a personal connection, while not overburdening the mentors who answer the questions.
I think that the not sticking around part is the bullying of admins and the not following the don't bite the newcomer rule. Many a time in my start and even until today, I get admins telling me what to do and what not to do as well as adding their own POV to why I should or should not be doing something. Two recent examples are last night I asked a question on IRC about BLP for clarification from someone who I thought would have the answer and their response was "You should find something else to do as you have bitten off more than you can chew as a new comer." The second was today an editor pinged me about removing the gnome and fairies tags from indefinitely blocked user pages to clean up the active user lists as it contained some 50 or so blocked users from years back to current. That editor opened an ANI against me because he/she didn't get the answer they wanted. I think if admins and other people were to stay out of the new members using their in your face routines (does not apply to all, but to some) and let normal editors be a mentor, this would work great. There are definitely cliques in the admin and sysops teams that seem out to get newbies and instead of being helpful they are rude and not helpful. When I first joined I went into IRC to the en-help channel and got chastised because I did not have a cloak nor am I at 3 months as a wikipedian. When I asked about these I was pointed to 2 links of which neither were helpful. I watched this same user in the IRC and they are rude to everyone in the tone of their messages and I even PM'd them to let them know I felt they were being hostile, not only towards me, but others as well and the response I got was "Deal with it' then I got kicked from the room. I requested a courtesy vanish on Friday last week. Before I knew it, those I have worked with on various things posting messages for me to come back and continue my contributions. So I decided to come back and again, same hostility towards me. So in short, I would recommend that the mentor's not be admins, sysops, clerks, ARBs, etc. Just normal everyday wikipedians who volunteer to take on someone. How would we define who is an experienced editor I guess would by my next question.
I was the person Galendalia asked "about BLP for clarification". They had asked for help in private message to me with a dispute resolution case they were mediating for on en-wiki. It was a particularly complex case and they had already pinged two others on-wiki for assistance with it. The "quote" that Galendalia is posting here is not an accurate quote. My response to them was actually: "It's a pretty involved situation you're asking for advice on, you may have bitten off more than you can chew right now." and "I see that you've pinged Robert McClenon and Nightenbelle, I would await their responses." As you can see, the tone of my reply is quite a bit different than the "quote" they are offering here.
They are also complaining about us asking them to not idle in the help channel until they meet the requirements for idling in the channel as specified at en:Wikipedia:IRC/wikipedia-en-help. They were repeatedly pestering numerous people about getting a WM cloak and were pretty upset that they were not getting a cloak despite not meeting the minimal criteria specified at m:IRC/Cloaks. They kept obtaining various different cloaks, trying to get past the channel rules regarding idling in -help without meeting the criteria for idling or helping. Honestly, I think I was pretty patient and polite given the level of intensity from them regarding this.
This rudeness to helpees they speak of, and this quote of "Deal with it", I do not know what they are referring to. If this is referring to me, this is entirely inaccurate and they never PMed me with anything of the sort. I'm actually very patience and polite with helpees, even ones who are difficult and/or UPE.
Frankly, I'm not appreciative of this blatant mischaracterization of my actions.
Thanks for sharing that perspective, @Galendalia. We know for a fact from research that hostility toward newcomers drives them away. Here is one of the most important papers about it, and here is another influential research project. I think it's definitely hard to improve the culture of a wiki, and I think it's great that you're trying to be a force for positivity in your work. So far, the mentors that we've recruited seem to be generally encouraging to newcomers, and I think you have a good idea that we should make sure it's clear that many people can be a mentor -- it doesn't only have to be the most experienced and involved editors on the wiki.
I can feel Galendalia’s pain. Shortly after becoming an Administrator earlier this year, I thought I’d go and try out IRC.chat as I’d never used it and thought I ought to get a feel for the place. I not only found it incomprehensible as well, but I was permanently blocked by a so-called ‘helper’ whose manner towards me was appallingly unwelcoming. There is no accountability or complaints system at IRC, so I will never ever recommend any newcomer on en-wiki to ever have go there unless major changes happen there, or unpleasant/unhelpful editors are kicked out. The person who I encountered wasn’t an admin, so unpleasant attitudes to newcomers isn’t something unique to those with extended rights. Finding mentors/helpers with the right interpersonal skills to be able to deal with inexperienced users is critically important.
I'm sorry that Nick Moyes had a bad experience, although I must say that it was somewhat self-inflicted for them. There is accountability on IRC, and there is a process for complaints and appeals. For a more complete and accurate explanation of what actually happened here, please read the thread at en:User_talk:Waggie#Your_attitude_on_IRC. I go into great detail about why this happened. I am also willing, with Nick Moyes' and Jeske's (as the other involved person here) permission, to publicly release the logs of the encounter. There was no "permanent block", bans in -help are for 24 hours by default. Secondly, as soon as they were identified to a known "good" user, I lifted the ban immediately.
Looking through the list of tasks at https://en.wikipedia.org/wiki/Wikipedia:Task_Center...
As I've mentioned at a previous stage, I still think anti-vandalism has a ton of potential to be a structured task for newcomers (it somewhat already is with WikiLoop Battlefield). Categories and copy editing both sound good. There are also some more niche tasks that could be easily structured, such as fixing links to disambiguation pages that pop up in mainspace.
@Sdkb -- I remember when you mentioned that, and @Zoozaz1 brought up WikiLoop Battlefield as an example of how reverting vandalism is like a structured task. I guess my open question is still whether newcomers would do a good job of judging vandalism, given their low wiki experience. You recommended that we check in with some Wikipedians who do a lot of edit patrolling. I can go seek some out -- is there anyone in particular who you would recommend or tag?
I'm not too sure about specific users, but I'd recommend posting at w:Wikipedia:Counter-Vandalism Unit. @Kudpung might know a better person to reach out to or have thoughts.
My issues with the rollback that everyone gets are:
1. Inexperienced
2. Not trained
3. Causes edit wars
I recommend one or all of the following:
A. IP users are not allowed to use the rollback feature B. Only the people who have graduated from the CVUA should have rollback rights (I see a lot of new users getting the right without any type of training. C. To use the rollback built in it must be a registered user with 3 months experience.
Hi @Galendalia -- thanks for thinking about this. We've been talking a lot about easy editing tasks for newcomers to do, and we wanted to hear from someone in CVU because of the idea that maybe reverting simple vandalism is something newcomers could help with. It seems like an interesting idea, because on the one hand, some vandalism is really obvious, but on the other hand, newcomers know little about Wikipedia or vandalism, and might not have the judgment required. What's your take? Could you imagine newcomers being given something like a very simple version of Huggle, and asked to revert obvious vandalism? If I'm reading your previous comment correctly, it sounds like maybe you would say it's not a good idea.
Hi @MMiller (WMF) : Even though I have been on WP just over a month, I feel the inexperience would be a major hindrance. Like I stated above, They need to complete the CVUA and be on WP for at least 3 months. This will allow new editors time to process the policies and learn from their mistakes rather than reverting a valid entry. There are sometimes subtle entries which would probably not being noticed unless you are looking for them, like no source listed in the diffs. Wait what is a diff? That is a question I see users asking a lot of.
Thanks, @Galendalia. It sounds like your general advice is that reverting vandalism takes some experience and knowledge. Got it. But it also sounds like you have an interesting story, if I may ask -- how did you find your way to reverting vandalism so soon after joining Wikipedia? What caused you to try that type of editing in the first place? What were the very first edits you did?
Honestly it seemed like the only thing I can do without having someone revert anything I did or go on a tangent about questions I asked that end up not even answering the question I posed in the first place. I pretty much do 2 things. CVU and Dispute Resolution. I also am in the process of rebooting Spoken Wikipedia as there is plenty of interest in it. That will be the 3rd thing. I’ve been trying to maintain where active user lists are maintained and I’m getting a lot of flack for that because in one instance it requires removing the tag or userbox from someone’s user page and I only did this to those who are permanently blocked. However as soon as I did it people were all over me and reported me to ANI and I’m getting nothing but crap for housekeeping.