Growth/Growth team updates

Growth

• All features: Understanding first day (EditorJourney)
• Personalized first day (Welcome survey, Newcomer homepage, Newcomer tasks, Structured tasks)
• Focus on help desk
• Community resources (Get the tools , Resources for communities )

Help contents: Use the tools: (Help panel, Enable the Homepage, How to claim a mentee, Suggested edits)

• Latest news: Growth team updates
• Newsletters
• Contact

This page contains updates on the Growth team's work to increase new editor retention in Czech and Korean Wikipedias. The planning for this work began in June 2018. The team works on multiple projects at a time, but they will all be discussed on this page. During July, August, September, and October 2018, the team was also simultaneously working on a project to improve the Articles for Creation process and New Pages Feed in English Wikipedia, which is discussed on its own page.

Growth team updates Updates on multiple projects to increase new editor retention Group: Growth Start: 2018-07-01 Team members: Gergő Tisza (senior software engineer), Benoît Evellin (community relations specialist), Elena Tonkovidova (QA engineer), Marcella Florence (engineering manager), Roan Kattouw (lead engineer), Marshall Miller (product manager), Max Binder (coach), Kosta Harlan (software engineer), Morten Warncke-Wang (data analyst), Rita Ho (UX designer)

How to get involved

It is important that our work is grounded in the reality of the communities we hope to help. If you have thoughts or ideas around this project or our team's work, please comment on this project's talk page.

Project updates

Updates for 2020 are listed below, from this page. Please also check the archives : 2019, 2018.

Update 2020-10-18: deploying Variants C and D

• Variants C and D
  • We plan to deploy these variants of the newcomer homepage on Monday, October 19.
  • 

    Updated mockup of onboarding screen for "add a link" on mobile. User tests taught us that onboarding needed to be clearer.
    After about 5 weeks, we will analyze the data from the tests to determine which variant is helping more newcomers get farther in the suggested edits workflow. Then we'll convert all users to that best variant.
• "Add a link"
  • We are running a second set of user tests this week. Our first set tested two different mobile design concepts. With this round of tests, we'll be testing the next version of mobile designs and the first version of desktop designs. After the tests are complete, we'll finalize all designs and plan to engineer them.
  • We've detailed out the backend engineering work required, which can be seen as subtasks under this Phabricator task. The work is largely about creating a "link recommendation service", which can generated suggested links for any article and store them in an efficient manner.
• Deployments
  • Turkish Wikipedia: to be deployed this week.

Update 2020-10-13: delays on Variants C and D

• Variants C and D
  • Though we planned to deploy Variants C and D today, October 13, we have been delayed by issues with the deployment train.
  • The variants will be deployed on Monday, October 19.
  • As we transition from engineering on Variants C and D to engineering on "add a link", we'll be spending about two weeks fixing some bugs and doing some maintenance.
• "Add a link"
  • Engineering work on the backend for this project is happening in earnest. We are making a service to look up recommended links for any article, which we'll use for the frontend of the feature.
  • We are also working on the designs for the desktop version of "add a link".
• Deployments
  • Turkish Wikipedia: coming up next.

Update 2020-10-05: "add a link" user test findings

• Variants C and D
  • We will deploy these new variants on Tuesday, October 13. We're finishing the final fixes this week.
  • When deployed, all new accounts will have a 40% chance of being assigned Variant C, 40% chance of being assigned Variant D, and 20% of not having Growth features at all (this is our control group).
  • We will also be converting all existing account holders who have Growth features over to having Variant D. Users who turn on Growth features in their preferences will also have Variant D.
  • After about five weeks, we will analyze interactions to determine which variant leads to more unreverted edits, and we will convert all users to the winner.
• "Add a link"
  • We posted the detailed findings from our mobile user tests, which helped us decide to build Concept A.
  • Next, we are designing and testing for desktop, and making the next set of mobile designs.
  • Engineering for the backend of this feature has already begun with the help of the Research and Site Reliability teams: we are setting up a service for the link recommendation algorithm.
  • The Growth team's engineers will turn attention to this project fully in about two weeks.
• Deployments
  • Portuguese Wikipedia: deployed to newcomers after successful trial week.
  • Swedish Wikipedia: deployed last week.
  • Turkish Wikipedia: coming up next.

Update 2020-09-27: nearing release on Variants C and D

• Variants C and D engineering: all parts of these variants have been coded, and are now being tested and fixed. We expect to deploy in the next two weeks.
• "Add a link"
  • After analyzing our 10 user tests, we think we have a decision about whether we want to build Concept A (edits take place in the visual editor) or Concept B (edits take place in their own dedicated space) for mobile.
  • Users seemed to understand the task more easily in Concept A. This design also has other advantages, many of which are discussed by community members here.
  • We will post detailed findings from the user tests this week.
  • Next, we will design and test for desktop.
• "Add an image": last week, we evaluated a second version of the algorithm. This second version is meant to have higher accuracy, by focusing on only on images connected to the Wikidata item of an article with no image. Accuracy did improve as expected, and now we have to decide what further improvements to make before using the algorithm for a feature. That work happened on this Phabricator task.
• Deployments
  • Polish Wikipedia: deployed last week
  • Portuguese Wikipedia: deployed last week in "trial mode". People only have the features if they turn them on in preferences.
  • Swedish Wikipedia: coming up next

Update 2020-09-20: testing on Variants C and D

• Variants C and D engineering: we are still on schedule for releasing these variants in early October. This week, we're testing most of the user experience components and working on the instrumentation.
• "Add a link"
  • We're now processing the results from our user tests, and working on decisions around Concept A and Concept B. We'll post the summary of our learnings from the tests in the next week or two.
  • We've started talking with the Editing team about the engineering work required to build this feature. Since it will involve the Visual Editor, we'll need to work with them and draw on their expertise.
• "Add an image": this week, we'll evaluate a second version of the algorithm. This second version is meant to have higher accuracy, by focusing on only on images connected to the Wikidata item of an article with no image. That work is happening on this Phabricator task. This evaluation is meant to help us decide whether the algorithm could be usable in a version of an "add an image" structured task in the Android app.
• Upcoming deployments: we plan to deploy Growth features to the following wikis in the next two weeks.
  • Polish Wikipedia
  • Portuguese Wikipedia
  • Swedish Wikipedia

Update 2020-09-13: learning from user tests

• Variants C and D engineering: we are still on schedule for releasing these variants in early October. We expect them to increase the number of newcomers who try suggested edits.
• "Add a link"
  • We completed 10 user tests last week, using these prototypes of Concept A and Concept B. This week, we'll be watching the tests and using them to help decide on a final design concept.
  • An active community discussion on the design concepts for "add a link" is happening at this page. We hope community members continue to join in this week.
• "Add an image": we have been planning ahead for a potential future structured task that encourages newcomers to add images from Commons to appropriate articles. The first thing we need to figure out is whether we can have an algorithm strong enough to make good recommendations. Our evaluation of the first draft of this algorithm took place on this Phabricator task. Right now, about one third of its recommendations are good. We see clear areas for improvement and will be working on and evaluating another iteration.

Update 2020-09-06: upcoming user tests

• Variants C and D engineering: we have progressed enough in the engineering of these variants that we're starting to talk about the timeline for launching them. We expect this to happen in early October.
• Upcoming user tests for "add a link": last week, we posted new mockups of the "add a link" structured task. As we gather community thoughts about the designs, we'll also be gathering reactions from people new to editing Wikipedia through usertesting.com tests. We'll be doing 5 tests of each of the two design concepts. Here is a link to the interactive prototype of one of the concepts.

Update 2020-08-30: new mockups of "add a link"

• 

  Screen from a design concept for the "add a link" structured task
  New mockups of "add a link": back in May, we had a good community discussion about the "add a link" structured task (summary here). We then took community thoughts into account as we made new design mockups. We hope community members can check them out and react to them.
• Russian Wikipedia: we have deployed the Growth features to Russian Wikipedia, which gets about 12,000 newcomers per month. Next, we're planning a deployment to Polish Wikipedia.
• Variants C and D engineering: we are mainly engineering on this project, meant to increase how many newcomers attempt suggested edits.

Update 2020-08-24: planning for "add a link"

• Meetings on "add a link": this week, the team will have a series of meetings planning for the "add a link" structured task. We'll post in-progress mockups of our ideas during this week so that community members can see and give us advice. We're also simultaneously planning to user test the designs to get thoughts from people new to editing.
• "Add an image": although we are designing our first structured task using a link recommendation algorithm, we are also considering a future structured task that would prompt users to add an image from Commons to an article that doesn't have any images. A member of the WMF Research team is working on an algorithm to make intelligent suggestions, and we are spending some time looking through its results to see if it could become good enough to give to newcomers in a structured task. We are working in this Phabricator task.
• Variants C and D engineering: this continues to be our main focus in terms of what we're building now, even as we plan for our next projects.

Update 2020-08-17: highest newcomers task numbers

• Measurement: this past week had the highest volume of newcomer task edits (1,103 edits) and editors (193 editors) since the feature's release. We believe this is caused by both the "guidance" elements of newcomer tasks, and the addition of Persian and Hebrew Wikipedias in recent weeks. The graph of numbers of editors by week is shown here. More details and graphs on our measurements can be found on the project page.
  

  Number of editors using newcomer tasks as of 2020-08-17
• Hebrew Wikipedia: we have deployed the Growth features to Hebrew Wikipedia, which gets about 2,000 newcomers per month. Next, we're planning deployments to Russian and Polish Wikipedias.
• Upcoming meetings on "add a link": although the team's engineering is currently focused on Variants C and D, we are planning for our work next quarter on our first structured task workflow of "add a link". Next week, the team will have a series of meetings about three main areas:
  • Data analysis: to help us predict how newcomers might engage with "add a link".
  • User experience design: to develop some design ideas that we'll share with community members.
  • Engineering: to make an architectural plan of how to store and serve link recommendations.

Update 2020-08-10: deployed to Persian Wikipedia

• Persian Wikipedia: we have deployed the Growth features to Persian Wikipedia, which gets about 6,000 newcomers per month, making it the third largest wiki for Growth features. Next, we're planning deployments to Hebrew and Russian Wikipedia.
• Language question: we have added a "language question" to the welcome survey in all Wikipedias that have Growth features. This question will help us find out how many newcomers may be capable of editing with the Content Translation tool. See the question in the accompanying screenshot.
  

  Welcome survey, showing the new language question.
• Variants C and D: the team continues to work on these homepage variants, and will be engineering on it for the next several weeks.

Update 2020-07-17: working on Variants C and D

• Variant testing
  • The team has planned out how we will build Variants C and D, which are new version of the newcomer homepage that are meant to encourage more newcomers to try suggested edits.
  • We originally showed the designs in this update, but the images are included again here.
  • This will be the team's main project going forward in the coming weeks.
• Other forthcoming work
  • In this task, we will soon be deploying a question to the welcome survey asking the languages that a newcomer knows.
  • In this task, we will soon be altering the help panel in Czech Wikipedia so that questions go to mentors, not the help desk. This will be a pilot to see if newcomers have better experiences through their mentors than the help desk.

• 

  Mockup of Variant C on mobile

• 

  Mockup of Variant C on desktop

• 

  Mockup of Variant D on desktop

Update 2020-07-13: measuring suggested edits performance

• Conferences
  • Last week, we contributed a video demoing Growth features to the Celtic Knot conference.
• Suggested edits performance
  • Ever since newcomers tasks was deployed in November, we've known that the feature has "performance" issues. By "performance", we are referring to the time it takes for the feature to load on the page.
  • When newcomers visit the homepage, all the rest of the page appears immediately, and the newcomer tasks can take a couple seconds to appear. This is because queries and searches are running in the background to generate the lists of tasks.
  • This past week and next week, we are doing some work to measure exactly how long this step is taking for users around the world. We will use the data to decide if the experience is slow enough that the team should work on speeding it up.
• Variant testing
  • We have been planning for the engineering work on Variant C and D, and we are starting the work this week.

Update 2020-07-05: asking language question

• Conferences
  • Last week, we presented at the online-only Polish Wikiconference, showing the community how the Growth features work, and discussing whether they would be a fit for Polish Wikipedia. Slides are available here.
  • We will also be contributing a video to the Celtic Knot conference.
  • If your community is having a remote event, and you are interested in hearing from the Growth team, please contact us!
• Guidance
  • The work on guidance cleanup is now almost finished, and we're going to be moving on to future projects this week.
  • We continued to see elevated rates of newcomers clicking edit and saving edits on tasks, which we hope and think is attributable to guidance.
• Language question
  • In the coming year, we are considering adding a new type of suggested edit to the newcomer tasks feed: an integration with the Content Translation tool. The idea is that, for newcomers who know more than one language, we would recommend articles to translate, or sections in articles to translate.
  • To prepare for this, we are adding a question to the welcome survey that asks newcomers which languages they know. This will help us have a sense of how many newcomers might be candidates for using the Content Translation tool, and it may also provide some interesting research results.

Update 2020-06-29: guidance cleanup and next projects

• Guidance
  • Last week was our second week of cleanup around the guidance release. We made many small interface improvements, as well as a few larger ones related to making it easier to navigate.
  • We may have a couple guidance things still to finish up this week, and then we'll be moving on to next projects.
• Next projects
  • The main next project that the team will be moving onto is our next variant test: Variant C vs. D. This test will make suggested edits a much more prominent part of the newcomer homepage.
  • Before starting on that, we have a handful of interim tasks to do, including:
    • Adjustments to the welcome survey to streamline the process of deploying it to more wikis
    • Addition of a question to the welcome survey asking the languages that newcomers know. This will help us gauge the value of a future integration of suggested edits with the Content Translation tool.
    • Piloting mentorship in the help panel, to see if newcomers and experienced users have a better experience by asking mentor questions in the help panel, instead of help desk questions.

Update 2020-06-22: guidance released

• Guidance
  • We released guidance on June 15 in all wikis that have suggested edits.
  • We think we are seeing some early positive results from guidance.
    

    Number of distinct users saving a suggested edit, broken out by week and wiki. The week after the release of guidance had the most users since the suggested edits was first deployed.
    • Last week had the highest number of suggested edits completed since the feature was first deployed in 2019: 917 edits.
    • Last week also had the highest number of distinct users completing suggested edits: 139 users.
  • The team is now in the midst of spending two weeks cleaning up the feature, making small improvements, and fixing bugs. One change we're making will make it easier for users to navigate and minimize the help panel.

Update 2020-06-14: releasing guidance this week

• Guidance
  • We spent last week testing, filing, and fixing bugs for guidance.
  • The biggest challenges have been around the logic and animations for how the panel displays the right information at the right time.
  • We'll be releasing to newcomers this week, and we'll continue to fix and adjust in the coming week or two. One example is this issue, in which we're going to make it easier to toggle between the guidance and the article that the user is editing.
• Variant testing
  • After we finish working on guidance, the team will turn to our next variant test: Variant C vs. D.
  • This test will show suggested edits prominently on the newcomer homepage, and test different paths of onboarding the user to the feature.

Update 2020-06-08: guidance needs translation

• Guidance
  • The team is focused on finalizing the code for releasing guidance next week.
  • We have been testing, fixing bugs, and sorting out the edge cases around when and how the guidance should appear for newcomers.
  • We encourage everyone to start translating the many messages for the feature. To help, we've put together this translation guide, explaining where the messages fit into the feature.
• Variant testing
  • In this update, we talked about results from Variant A vs. B. We found that Variant B, which has the user skip onboarding, leads to higher engagement with the suggested edits module.
  • Even though we had those results, we had not switched newcomers over to all receiving Variant B, because we also saw evidence that the onboarding may important for helping newcomers complete edits. Variant A seems to perform about 40% better than Variant B on mobile.
  • We now have enough data to look at whether one of the variants is more associated with completing edits. The result is that Variant A leads to more newcomers completing edits on moble, while the two variants do not have differences on desktop.
  • Therefore, we will be converting all newcomers over to Variant A while we do the work in the coming weeks to build Variant C and Variant D, our next test.
  • Overall, the test showed us that while Variant B's stronger affordance of the suggested edits module increases interaction, onboarding to the module is also important. We are using both these points in the designs for Variants C and D.
    

    link=File:Wireframe_of_potential_component_of_%22structured_tasks%22_feature_in_Mediawiki_2020-05-28_1.png

Update 2020-06-01: new structured task wireframes

Wireframe from our structured task explorations, showing how a user might leave feedback on an algorithm's suggestions.

• Structured tasks
  • We have continued to have good discussion about structured tasks. So far, about 30 users have participated across four languages.
  • This week, we talked a lot about how we might automatically suggest spelling fixes in arbitrary languages. You can see our notes here from a conversation with User:Beland, who created a system to search for spelling errors in English Wikipedia.
  • Our team's designer has created new wireframes exploring some of the concepts that we think would be important for structured tasks. You can see them all here, with an example posted here.
• Guidance
  • We are planning on releasing guidance during the week June 15.
  • This week and next week are about finalizing animations, instrumentation, and testing for bugs.
  • The written content for the feature is now available for translation, and we're encouraging all wikis using Growth features to translate it in the next week if they can.

Update 2020-05-26: deployed to French Wikipedia

• Scaling
  • We deployed the Growth features to French Wikipedia on 2020-05-19. So far, there have been 95 suggested edits from 36 different users. There have also been many mentorship and help desk questions.
• Structured tasks
  • Our discussion about structured tasks is continuing. We hope community members can visit the project page for structured tasks and give their thoughts.
  • So far, we've heard from about 20 community members across many different communities, and some of the main points of discussion are:
    • Generally positive feedback -- it sounds like we are thinking on the right track.
    • It is important that newcomers have the opportunity and are encouraged to learn to edit using the traditional editors, as opposed to only learning new structured editing tools.
    • Discussion of whether copyediting or reverting vandalism can be good structured tasks for newcomers.
• Guidance
  • Our work on this project is all in code review, which means that we are making final fixes before releasing.
  • Though we have been testing as we go along, we will now start testing the whole workflow together.

Update 2020-05-17: deploying to French Wikipedia this week

• Structured tasks
  • Our discussion about structured tasks is beginning. We hope community members can visit the project page for structured tasks and give their thoughts on the questions posted on the talk page.
  • If you can translate the project page to your language, that will help us gather more community opinions.
• Guidance
  • We are finishing work on the instrumentation for guidance, which required refactoring how certain events are stored.
  • We are finishing up the animations for the help panel that will help a user navigate smoothly through the experience.
  • We are finalizing how the guiding content is displayed in the panel, including quick start tips and example sentences.
• Scaling
  • We are planning to deploy Growth features to French Wikipedia on Tuesday, May 19.
  • A discussion has started on Persian Wikipedia about trying Growth features, with all responses being positive.

Update 2020-05-11: call for discussion on "structured tasks"

• Structured tasks
  • 

    Sketch of an idea for a structured workflow for adding links to an article, geared toward helping the newcomer edit at high volume.
    We hope community members can visit the project page for structured tasks and give their thoughts on the questions posted on the talk page.
  • This is a project we're in the middle of thinking about, and we want to hear advice and reactions from community members in as many projects as we can.
  • We plan to conduct this conversation over the next five weeks (through June 18). We'll be sending the information out in our newsletter and pinging users who we think would want to take part.
  • If you can translate the project page to your language, that will help us gather more community opinions.
• Guidance
  • We are continuing to finish up engineering work on guidance.
  • Regarding our previous update about having many translatable messages, we have decided to keep all the messages, and monitor whether communities struggle to translate them all.
• Scaling
  • We are planning to deploy Growth features to French Wikipedia, which will be our first large Wikipedia. We'll start working on that deployment this week.

Update 2020-05-03: thinking about "structured tasks"

• Guidance
  • We are nearing the end of the engineering work for "guidance", currently working on the instrumentation.
  • One of our challenges is managing the many messages that will need to be translated. Since this feature guides users through completing suggested edits, it contains a substantial amount of text and examples that we wrote carefully to be helpful for newcomers. But since there are five different task types, with six or seven different steps, along with some examples sentences, and some differences between desktop and mobile -- there are about 50 messages to translate for this feature. This may be too many for communities to do easily and thoroughly. We are currently thinking about how to make this better, and welcome any ideas.
• Structured tasks
  • We mentioned this potential new project in previous updates.
  • The idea is that we think we could break down editing workflows into a series of steps that newcomers can accomplish easily, especially with the assistance of machine learning.
  • This is an exciting idea, but there are also many potential pitfalls. This new project page is in progress, and after getting some more content on there, we'll start a community discussion so that we figure out the potential for this idea.
• Piloting mentorship in help panel
  • In our previous update, we mentioned that one idea for how we could more easily scale to bigger wikis is by converting the help panel over to ask mentor questions instead of help desk questions.
  • We're going to pilot this idea in Czech Wikipedia to see how it goes, and then decide if it is an improvement that can be brought to our other wikis.
  • The details can be found in this Phabricator task.

Update 2020-04-20: continued work on guidance

• Guidance
  • We have been continuing the engineering work on guidance, with almost all the tasks now in progress.
• Top-of-funnel work
  • Technical requirements are now written for our next test, Variant C vs. D. They can be found under this Phabricator tasks.
• Scaling
  • As we prepare to deploy our features to larger Wikipedias, one important question is how we will scale the "help desk" and "mentorship features". Both these features require the involvement of experienced community members to answer questions. Large Wikipedias are expected to produce a large number of questions, and we want to make sure there are enough community members available to answer them.
  • One idea we have is to remove the "help desk" element from the help panel and to replace it with questions to mentors instead -- essentially creating one route for questions from newcomers: to their mentors. It will be easier for wikis to scale their mentorship capacity up and down than their help desk capacity. We think such a change could also enhance the newcomers' experience with suggested edits.
  • We want to try this on Czech Wikipedia in the coming weeks to see what the effects are on the volume of work given to mentors. The details are on this Phabricator task.

Update 2020-04-13: deployments to more wikis

• Scaling
  • Last week, we deployed the newcomer tasks workflow to four additional Wikipedias that already had all the Growth features apart from newcomer tasks: Ukrainian, Hungarian, Armenian, and Basque.
    

    Mockup of Variant C on desktop
  • We also deployed all Growth features to Serbian Wikipedia.
  • We deployed all Growth features except newcomer tasks to French Wiktionary.
  • Together, newcomers on these new wikis have made 33 suggested edits since last week.
• 

  Mockup of Variant D on desktop
  Top-of-funnel work
  • We finalized designs for our next variant test: Variant C vs Variant D. Variant C feature an initiated suggested edits module with optional onboarding. Variant D also features the newcomer tasks flow prominently, but immediately gives users the option to choose topics as part of mandatory onboarding. See mockups at right.
  • The full set of mockups are found here.
  • The team will work on building these and deploying this test after we complete our work on guidance.

Update 2020-04-03: first variant test results

• Guidance
  • The team continues to work on "guidance", and it is possible to follow our progress in this Phabricator task.
• 

  Results from test of Variant A vs. Variant B
  Top-of-funnel work
  • The analysis for our first variant test is now complete. It started in December, and split newcomers into these two groups:
    • Variant A: receive the original suggested edits workflow, in which they clicked a call-to-action to pass through two onboarding overlays before seeing the suggested edits module.
    • Variant B: arrive on the homepage with the suggested edits module being displayed, experiencing no onboarding overlays.
  • We received strong and significant results (click here for full results):
    • On desktop, Variant B yields double the interaction (clicking on anything in the suggested edits module), 60% more navigation (clicking on arrows to navigate to different suggested articles), and 30% more clicking on suggested articles.
    • On mobile, the variants perform the same for interaction and navigation, but Variant B leads to 15% less clicking on suggested articles.
  • Takeaways include:
    • Making the suggested edits module more prominent on the homepage gets more users to interact with it.
    • Though the overlays were not present on Variant B, they may still play an important role in giving the user context for what they're supposed to be doing with the module.
  • We will incorporate these takeaways as we design the next set of variants to test.

Update 2020-03-30: "top-of-funnel" work

• Guidance
  • In terms of engineering, the team continues to work on "guidance", and it is possible to follow our progress in this Phabricator task.
• Top-of-funnel work
  • We are now designing and planning around the project we'll be working on after releasing guidance: "top-of-funnel" work. We use this term because it is work that is meant to increase the number of newcomers who begin the "newcomer tasks conversion funnel". Right now, only about 20% of newcomers who visit their homepage interact with newcomer tasks, and we think that design changes can increase that.
  • We'll be doing this top of funnel work as a series of variant tests, which we started writing about here. Our very first variant test is almost complete, and the results will be posted next week.
  • You can look at some of the variant ideas we want to test in these mockups. They include ideas like:
    • Making the suggested edits module dominate the homepage.
    • Letting users peruse a list of task recommendations, instead of just looking at one option at a time.
    • Suggesting to newcomers who have previous edits that they edit articles they are reading.

Update 2020-03-22: slower work on "guidance"

• The Growth team, and the rest of the WMF, are adjusting our pace of work as the world deals with the COVID-19 pandemic. We'll be working more slowly and more carefully in the coming weeks, allowing time for team members to take care of their families and themselves. See this blog post for a message from the WMF's CEO about how our staff is planning to adjust.
• That said, our priority continues to be working on "guidance". The engineering work continues on that project.
• We are also continuing to plan for how we will scale Growth features to many more wikis. We will begin reaching out to potential communities in the coming weeks.

Update 2020-03-16: planning "structured tasks" project

• The Growth team is currently planning our work for the coming fiscal year, which begins in July. We plan to build on the newcomer tasks project, which has been showing promising results.
• In a previous update, we talked about evaluating a "link recommendation" algorithm. The idea is that a good algorithm could help us build a type of newcomer task that points newcomers to specific links that may need to be added -- giving them small tasks that fit their skill level, without requiring a huge amount of context.
• We have since been developing this idea, and we're now sketching out an initiative we're calling "structured tasks". In short, we think that we can break difficult wiki editing workflows into steps that are much easier for newcomers to do, and much easier to do from mobile devices. We would put these structured tasks into the newcomer tasks workflow. They could include things like adding a link, adding an image, adding a reference, etc.
• In the coming weeks, we'll post more information on wiki explaining this idea and asking for communities to react. While we think this could be a really exciting idea, it will be a lot of work, and there are many potential pitfalls. We'll need community thoughts to make sure we get it right.

Update 2020-03-10: ORES topics deployed!

• Newcomer tasks 1.1 (topic matching)
  • We deployed ORES topic models to Arabic, Vietnamese, and Czech Wikipedias on 2020-03-05. Korean Wikipedian's models had some issues that are causing its deployment to be delayed.
  • In the coming weeks, we'll be publishing some information about how other Wikimedians and developers can access these models for their own work. As an example, one way to use them is with the "articletopic" search keyword. This works by typing something like "articletopic:sports" in a Wikipedia search bar to retrieve articles that are likely to be about sports. This page and links from it show the 64 possible topics to search.
• Newcomer tasks 1.2 (guidance)
  • Engineering continues on this project.
  • This past week, we detailed exactly what user actions we'll log to understand how the feature is being used after we deploy it. The details are on this Phabricator task.

Update 2020-03-01: deploying ORES topics this week

• Newcomer tasks 1.1 (topic matching)
  • We are planning to deploy our upgrade of the topic matching algorithm this week.
  • In our evaluation of the new ORES models, we found that the new models perform substantially better than the "morelike" algorithm we have been using for topic matching since Jan 13. We evaluated the models by looking at ten articles classified by the models into each topic, and then counting how many of the articles seemed to fit well in the topic.
    • The "morelike" algorithm averaged between 5.8 and 7.5 across the topics, depending on the language.
    • The ORES models averaged between 8.0 and 9.3 across the topics, depending on the language. The also have the added benefit of more topic granularity.
• Newcomer tasks 1.2 (guidance)
  • Engineering is underway to guide users through suggested edits via the help panel.
  • This past week, we drafted the language that will explain to users how to complete the different task types (copyedit, add links, add references, update article, and expand article). The challenge was how to give the right level of detail, while not overwhelming or confusing the newcomer. We want newcomers to complete a simple edit as quickly as possible.
  • We have also been deciding which data to record to understand the way that guidance will and will not be used.
• Homepage variant testing
  

  A variant of the suggested edits module
  • We will soon be analyzing the results of our first homepage variant test, in which half of newcomers receive the suggested edits module already initiated on their homepage when they arrive.
  • As we await those results, we have designed additional homepage variants that attempt to combine the best of the variants from the first test. Those are being set up to be tested here.
  • We will also be designing and prioritizing a test that makes suggested edits the clear focus on the homepage, attempting to encourage more users to enter its workflow.

Update 2020-02-23: upcoming deployment of ORES topics

• Newcomer tasks 1.1 (topic matching)
  

  Prototype of how newcomer tasks will enable ORES topic selection
  • We are planning to deploy our upgrade of the topic matching algorithm either this coming week or the week after.
  • This upgrade will increase the number of topics available to newcomers from 27 to 64.
  • We will be altering the user interface to help newcomers navigate the larger number of topics. You can try out the upcoming interface in this prototype.
• Planning to scale to more wikis
  • Over the past 1.5 years of the Growth team, we have deployed our features to nine wikis. As we learn that the features we're building are valuable to those wikis, we are now planning to deploy to many more wikis in the coming year.
  • The reason it takes work from staff and communities to deploy Growth features is that they have required community input to configure. For instance, community members supply our team with the most important links to help pages, and also supply us with a list of mentors for the mentorship module.
  • Our team is starting to make plans about how to streamline deployments so that we can work with more wikis more quickly. We'll be posting about those plans in coming weeks.

Update 2020-02-17: evaluating algorithms

• Newcomer tasks 1.1 (topic matching)
  • Engineering continues on loading ORES topic scores into the Search API, which will enable newcomer tasks to use them, and also enable other features to draw on topics.
  • Ambassadors are now evaluating the new scores to identify any topics that perform poorly. That evaluation is happening in this Phabricator task.
• Link recommendation
  • All newcomer task suggestions are currently based on maintenance templates, a decision we made among many choices of where to source tasks to recommend.
  • Maintenance templates do not give newcomers much direction on how or where to edit in an article, and we believe they would benefit from having specific places in the article to edit.
  • We also see that many newcomers use newcomer tasks to add internal blue links to many articles.
  • Therefore, we are partnering with a researcher to attempt an algorithm that can recommend specific words or phrases in an article that should be made into blue links.
  • Ambassadors are evaluating that algorithm in this task, and we will decide within the coming weeks whether it might be good enough to incorporate into newcomer tasks.
• Increased affordance of newcomer tasks
  • Coming up, we'll be working on some ideas to encourage more newcomers to try out the newcomer tasks workflow.
  • Homepage layout: we will test a variant of the homepage in which newcomer tasks is featured prominently as the main call-to-action.
  • Entry point while reading: for newcomers who have already done a newcomer task, we will create a design that suggests they edit articles they are reading that have maintenance templates on them.

Update 2020-02-10: beginning engineering on guidance

• Newcomer tasks 1.1 (topic matching)
  • In watching user behavior after the deployment of topic matching on 2020-01-21, we see that about 75% of users who have the option to choose topics actually choose some. Though we'll need to take more time to see how topics have impacted the rate that users do suggested edits, this initial number is a good sign that users are interested in using topics.
  • The team is now primarily concentrating on upgrading the topics to use the ORES models instead of our initial "morelike" algorithm. This week, our team's ambassadors will be testing them to make sure they yield strong results.
• Newcomer tasks 1.2 (guidance)
  • We are also beginning engineering work on this part of the newcomer tasks workflow.
  • After completing user tests on the prototypes for this feature, we discovered a couple important findings:
    • Our mobile design, in which the panel "peeks" up from the bottom of the page, works.
    • Users are really interested in seeing demo videos of how to complete edits. Videos are difficult to produce, keep up-to-date, and scale, and so the team will need to talk in the future about how to address this need.
  • With that information from the user tests, we were able to create these versions of the prototypes, which are what we intend to build now:
    • Desktop
    • Mobile
  • We have created Phabricator tasks specifying the design of the guidance features.

Update 2020-01-24: topic matching deployed and new wikis involved

Topic options being displayed in Czech Wikipedia

• Newcomer tasks
  • We deployed topic matching to our four target wikis on 2020-01-21, meaning that all users with the homepage can access newcomer tasks and choose topics to narrow them.
  • We're now working on the next upgrade for topics: using the ORES models instead of our initial "morelike" algorithm. These models are expected to give better results, and we expect to begin using them in February.
  • We'll also be beginning the engineering work to add guidance to the newcomer tasks experience, so that newcomers understand how to edit the suggested articles once they arrive.
• Deploying to additional wikis
  • On 2020-01-22, we deployed all the Growth features besides newcomer tasks to Ukrainian, Hungarian, and Armenian Wikipedias. These are our next three "non-target" wikis, meaning they receive all but our latest feature.
  • If you are interested in your wiki trying out Growth features, please read about how to get started!

Update 2020-01-17: preparing to deploy to Ukrainian, Hungarian, Armenian Wikipedias

• Newcomer tasks
  • Our updated date for deploying topic matching is 2020-01-21.
  • This past week, we have been testing and fixing bugs.
• Deploying to additional wikis
  • We are preparing to deploy Growth team features to Ukrainian, Hungarian, and Armenian Wikipedias.
  • These will be our next set of "non-target" wikis, meaning that they receiving features one step after our target wikis of Czech, Korean, Arabic, and Vietnamese Wikipedias.
  • If you are interested in your wiki trying out Growth features, please read about how to get started!

Update 2020-01-14: preparing to deploy topic matching

• Newcomer tasks
  • 

    Prototype of how the help panel will "peek" from the bottom of the page to offer guidance on mobile
    V1.1 (topic matching)
    • We are almost finished building the topic matching addition to newcomer tasks, using the "morelike" algorithm. This is planned to be deployed 2020-01-14.
    • After about a month, we will replace the morelike algorithm with the higher-performing ORES model. This model will be made available through the Search API so that other Mediawiki features will be able to draw on it.
  • V1.2 (guidance)
    • After we deploy topic matching, the team will move on to engineering this final part of the newcomer tasks workflow.
    • We ran 12 user tests on desktop and mobile in December and we are currently analyzing the results to determine the final designs that we will build. These are the prototypes we tested:
      • Desktop prototype
      • Mobile prototype

Archives

• 2019
• 2018