Topic on Talk:Growth/Positive reinforcement

Positive reinforcement discussion

19
MMiller (WMF) (talkcontribs)

Hello @HLHJ @Mike Peel @Czar @John Cummings @T Cells @Pigsonthewing @Sdkb @John Broughton @Pelagic @NickK @LittlePuppers @Zoozaz1 @Nick Moyes @Galendalia @Barkeep49 @NGC 54 @ValeJappo @Ата --

I wanted to invite you all to a discussion on the feature set the Growth team is working on next. This "positive reinforcement" effort is our next big project following "add an image", which many of you helped guide. The idea behind "positive reinforcement" is that newcomer tasks (like "add a link" and "add an image") are doing a good job of helping newcomers make their first edits -- but they aren't having a clear impact on getting newcomers to come back another time to make more edits. In other words, they are helping with "activation", but not with "retention". We think the "positive reinforcement" project could increase retention by showing newcomers the impact they are having and encouraging them to continue. Also, when we discussed newcomer tasks with communities, many of you wanted to make sure that newcomers wouldn't get stuck doing easy tasks, and that they would progress to more valuable kinds of tasks. With positive reinforcement, we can encourage that progression. But we know that the wikis are not like other apps and website -- we don't want to motivate people to make bad edits just to rack up points and awards. So we definitely need community input to figure out how to motivate newcomers in the right way that fits our communities.

I also want to tell you about an important transition and introduce you to @KStoller-WMF. I'm moving into a new role at WMF as the manager over the Growth, Editing, and Web teams. Kirsten will be the new product manager for the Growth team, and we're excited about her background as a librarian and as a product manager for a website that also has an active community. Going forward, you'll mostly be interacting with Kirsten around Growth product work, but I will still be working closely with her on strategy and tying projects together across multiple teams. Thank you all for your help building the Growth team features over the previous years, and Kirsten and I are looking forward to continuing to learn from you as we keep making the wikis a better place for newcomers!

Back to positive reinforcement: it would be great if you can check out the background and some of the design ideas on the project page, and post any of your thoughts, concerns, and questions in this thread or in a new one. Here are some prompts to think about:

  • What has worked well on your wiki for motivating newcomers?
  • Which of the ideas on the project page do you think have the most promise? The least promise?
  • What could go wrong and what should we try to avoid?
John Broughton (talkcontribs)

I'm fairly certain that the WMF tried, maybe five or so years ago, thanking new contributors (on their User Talk page, I think). I believe this was done randomly, and I think the conclusion was (a bit more nuanced that this, but basically) that it made no difference. So, if my memory is right [sorry, unable to find a page to point to, regarding that effort/project], personal praise isn't a magic bullet.

KStoller-WMF (talkcontribs)

Thanks for the feedback @John Broughton! I’m not sure which study you are thinking of, but you are right, personalized praise definitely won’t be a magic bullet. But based on several past studies we think it’s reasonable to expect this idea to be a step in the right direction.  

Here are a few of studies we looked at that found that newcomers’ editing activity is increased (at least in the short term) when they receive thanks and other positive messages about their work from an experienced editor or mentor.  Here are few of the studies we looked at:

I think one of the major challenges this idea faces is that reviewing and thanking newcomers requires time from contributors. We are still brainstorming ways to surface praise-worthy edits to mentors or other contributors. Do you have any feedback? How do you think we should flag contributions that might be worthy of thanks?

Pigsonthewing (talkcontribs)

Hello Kirsten - always good to have more library folk on board! And good luck in your new role, Marshall.

We show editors a "your edit has been saved" notice each time that happens. Perhaps for new accounts, we could mae use of that? Something like "Your edit has been saved - you're doing great work - why not do another one"?

KStoller-WMF (talkcontribs)

Thanks for the warm welcome, @Pigsonthewing!  

Nice, it sounds like we are thinking along the same lines as you.  Here’s a mockup of one idea we are considering, which displays an encouraging prompt after a newcomer edit is saved. Is this similar to what you are thinking? Do you have any ideas for improving this idea?

Pigsonthewing (talkcontribs)

Generally yes, but I'm not so keen on the "1/5", "5/5", "daily goal" part. Is that goal set by the user? We shouldn't be discouraging people who might otherwise make more than five edits, nor encouraging people to make unnecessary edits just to hit an arbitrary target.

KStoller-WMF (talkcontribs)

Great, thanks for the feedback! I also have some of the same hesitations regarding the daily goal. I assume allowing users to set and customize their own goals would likely be more motivating. Customization would also add to the complexity of implementation, but it's certainly worth considering. We'll keep this in mind as we take another look at the designs.

John Broughton (talkcontribs)

As the project page says, "We believe that these reinforcements from real people would be more effective than automated ones coming from the system."

Mike Peel (talkcontribs)

A general suggestion: I think it's worth learning from other sites how they do it. I think that often involves, e.g., an email a day later saying thank you for your first contribution, have you considered coming back to do more, and here's some suggestions? This is perhaps more what newbies will expect, and less what experienced Wikimedians would want? But I'd also suggest contacting people that work actively on this (I don't), I know @WereSpielChequers has worked on this, and there's lots of on-wiki projects that would have relevant expertise (new page patrol, Teahouse, etc.) - and always good to ask new editors who're just past the first edit stage, what kept them going?

WereSpielChequers (talkcontribs)

Thanks Mike, I spent a lot of time on Quora in recent years, and there there was much more automated feedback on things like the number of people who'd read what you'd written. Obviously not worth mentioning if figures are low, but good to send people feedback if what they've done is read by a lot of people, we do this with the DYK process but could do it more widely. As far as Wikipedia is concerned, I think there is a big potential in tailored invites - people's first edits usually indicate some of their interests, and we could auto generate invites from relevant WikiProjects. Even if the WikiProject concerned is not currently active, it is likely to contain a bunch of resources and list some relevant and potentially tempting tasks.

KStoller-WMF (talkcontribs)

Thanks, @Mike Peel and @WereSpielChequers. Agreed, it's fairly standard to receive a series of welcome emails when joining a new site. We've recently started some work in that direction, and plan to closely measure the impact: Engagement emails

One of the Positive Reinforcement ideas we are discussing includes Personalized Praise which could result in an email depending on the implementation method we pursue (along with user preferences and if the new user has an email address associated with their account). I think the main challenge we will face with that project is finding ways to encourage mentors and other contributors to spend time providing praise to newcomers who make high-quality initial edits. How do you think we should flag contributions from newcomers that might be worthy of thanks?

I think there is a big potential in tailored invites - people's first edits usually indicate some of their interests, and we could auto generate invites from relevant WikiProjects.

Currently the Growth team is using the Welcome survey to provide some personalization on the newcomer homepage. But certainly there is more we could do here. I love the idea of connecting users to the topics and WikiProjects that would appeal to them. Newcomers have so much to learn and read when they get started, it's easy for them to get lost and never find the right project or task that appeals to them. I wonder if we could use the topics a newcomer selects as interests as a way to also suggest a WikiProject that relates to that topic. Is that what you were thinking?

Do you have any other feedback or concerns about the Positive Reinforcement ideas and designs?

WereSpielChequers (talkcontribs)

Hi KStoller, I was thinking in terms of the Wikiprojects tagged on talkpages of articles newbies have edited or that are relevant to categories those articles are in. This has the drawback that some tags will be completely irrelevant to the user, however transparancy could be maintained if you send hem an invite that contains links to all the relevant Wikiprojects - they can then hone in on the ones they think are relevant to them. Hi <insert name hare> based on your first ten edits, we thought you might be interested in some of the following Wikiprojects Vulcanology, mountaineering, Washington state and skiing - WikiProjects are groups of Wikipedians like you who have an interest in a certain area. OK they may have edited that article on a mountain because of the botany or abandoned radar base, but they will have picked up the principle that Wikiprojects exist and there might be ones of interest to them. As for encouraging praise of newbies, I'd add some software to the welcoming process, enable those who welcome newbies to get feedback on those newbies such as their 100th or 500th edit, with an easy opt out such as these are newbies who you welcomed and whose userpage you still watch. You could also create a page on WikiProjects that enables members to give thanks to new editors who have begun to be active in that area, though I suspect that has been covered by watchlisting etc. Perhaps a better route would be to send messages to active Wikipedians who haven't yet issued thanks or barnstars explaining how to do so now that you've done your thousandth edit, have you considered encouraging Wikipedians who you see doing good edits, either by using the thank button or giving them a barnstar? WereSpielChequers (talk) 09:07, 19 May 2022 (UTC)

KStoller-WMF (talkcontribs)

Thank you for detailed feedback, @WereSpielChequers.

I agree, there is more we could do to personalize suggestions based on what a newcomer has edited. The newcomer homepage currently suggests edits based on topics the individual has selected as interests, but certainly there is more personalization we should consider as we further onboard newcomers.

And thanks for all of the ideas related to encouraging personalized praise of newcomers. As you've noted, there are definitely many different routes we could consider. Many of the ideas we have discussed involve mentors, but I like that your ideas aren't limited to just that small (and likely already very busy) subset of contributors.

WereSpielChequers (talkcontribs)

Thanks KStoller, this business of newbies selecting topics of interest is new to me. Unless we are talking about userpages, either it didn't exist in 2007 or I've forgotten about it. Can you tell me what proportion of new editors indicate such interests? If it is the great majority then this might be a more effective route than the one I detailed. WereSpielChequers (talk) 10:32, 20 May 2022 (UTC)

KStoller-WMF (talkcontribs)

Hi @WereSpielChequers. This is a relatively new feature, so not something you would have encountered in 2007.

As part of the newcomer Homepage and newcomer tasks, we allow newcomers to select interests from about 40 topics. Here's an example screenshot. We then use that to personalize the articles / edits we suggest.

I'll have to dig into metrics to see how many newcomers complete this step of the onboarding funnel and select interests. Parsing the data is slightly complex since the percentage of new accounts which have the newcomer homepage and newcomer tasks enabled by default varies by wiki. But certainly for users who have self-selected interests, it's likely to be a fairly accurate way to understand which topics or WikiProjects they might find appealing.

If you are interested in checking out the newcomer Homepage, here's info in enabling the Homepage. And if you have any feedback, please let me know. Thanks again for the feedback about the Positive Reinforcement project!

WereSpielChequers (talkcontribs)

Thanks, that's interesting. I see a disconnect in that forty topics is too few to match with the Wikiprojects. So Sports and Biography are in the forty, but Wikiprojects function more at the level of professional wrestling or Baseball. Sports is an umbrella. WereSpielChequers (talk) 00:14, 21 May 2022 (UTC)

KStoller-WMF (talkcontribs)

You are totally right, the topics are very general, and the Wikiprojects can get very specific. It's also worth considering that the smaller and medium wikis will have fewer Wikiprojects than English. And I believe that some small wikis have no Wikiprojects, so it's perhaps not an idea that would work for all wikis.

But I still love the underlying idea: that as newcomers start advancing and trying more difficult tasks, eventually we should expose them to more advanced ways to contribute and involve themselves in the community. Learning to contribute as a newcomer feels like a labyrinth of technical, conceptual, and cultural challenges, and there's so much more we can do to help ease newcomers along the path to becoming successful contributors.

WereSpielChequers (talkcontribs)

Sorry, yes I was thinking of English. Taking Georgian as an example, it is a few years since I went to a Tbilisi meetup, but they had a monthly focus project that a lot of them would participate on. If that's still their modus operandi, for that wiki you'd likely promote the idea of a monthly topic. What seems obvious to me, is that this requires asking the various wikis what process we should promote to newbies. Currently because of our software limitations we don't have autosign on talkpages, so we have to promote the idea of signing with four tildas in all welcomes to Newbies. Fix that problem and we have a lot of communication bandwidth that could be repurposed for a message more positive than "Our software is so klutzy you have to use this symbol you've never heard of to sign your posts".

KStoller-WMF (talkcontribs)

Thanks for the feedback and example, @WereSpielChequers. Your Georgian example is in line with a concept the Growth team wants to focus on more: ensuring Growth features are extensible and customizable to better fit the unique needs of each wiki.

And you are right, manually signing talk pages is definitely not intuitive to newcomers. I know the Editing team has many Talk Page improvements in the works that should benefit newcomers. I really hope we see an improvement in new editor retention as we start to remove some of these common stumbling blocks and the Growth team works on features to ensure newcomers receive positive reinforcement along the way.

Thanks again for taking the time to chat through these ideas!

Reply to "Positive reinforcement discussion"