Growth/Growth team updates

Growth

• All features: Understanding first day (EditorJourney)
• Personalized first day (Welcome survey, Newcomer homepage, Newcomer tasks, Structured tasks)
• Focus on help desk
• Community resources (Get the tools , Resources for communities )
• Mentor dashboard
• Community configuration

Help contents: Use the tools: (Help panel, Enable the Homepage, How to claim a mentee, Suggested edits)

• Latest news: Growth team updates
• Newsletters
• Data on impact
• Contact

This page contains updates on the Growth team's work to increase new editor retention in Czech and Korean Wikipedias. The planning for this work began in June 2018. The team works on multiple projects at a time, but they will all be discussed on this page. During July, August, September, and October 2018, the team was also simultaneously working on a project to improve the Articles for Creation process and New Pages Feed in English Wikipedia, which is discussed on its own page.

Growth team updates Updates on multiple projects to increase new editor retention Group: Growth Start: 2018-07-01 Team members: Gergő Tisza (senior software engineer), Benoît Evellin (community relations specialist), Elena Tonkovidova (QA engineer), Marcella Florence (engineering manager), Roan Kattouw (lead engineer), Marshall Miller (product manager), Max Binder (coach), Kosta Harlan (software engineer), Morten Warncke-Wang (data analyst), Rita Ho (UX designer)

How to get involved

It is important that our work is grounded in the reality of the communities we hope to help. If you have thoughts or ideas around this project or our team's work, please comment on this project's talk page.

Project updates

Updates for 2021 are listed below, from this page. Watch 2021 updates. ● Growth team updates : 2021, 2020, 2019, 2018.

Update 2021-05-10: "add a link" nearing deployment

• "Add a link"
  • This project is now being tested in production and is nearing release on our four pilot wikis.
  • We'll be doing final tests this week and next week, and then plan to deploy to the four wikis either during the weeks of May 24 or May 31.
  • After two weeks, we'll analyze the initial data to identify any problems or trends.
  • If things are going well after four weeks, we'll deploy to increasingly larger sets of the wikis with Growth features.
• "Add an image"
  • While the Growth team has not been working on this effort recently, the Android team has.
  • Last week, they deployed what we're calling the Android MVP, which collects image annotations from users without saving any edits.
  • Over the coming weeks, we'll learn a lot about how users like this task, whether they apply good judgment, and how well it works across languages. These learnings will help the Growth team decide whether to build such a task for the web.
• Scaling
  • English Wikipedia community members are testing out the features. They have nine mentors signed up, and so far, the testing is going well. Follow along here.
  • Wikipedias that will start receiving the features this week are: Catalan, Greek.
  • Wikipedias preparing for deployment are: Urdu, Bulgarian, and Latvian.

Update 2021-05-02: "add a link" being tested

• "Add a link"
  

  A screenshot of "add a link" feature in development in beta.
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
  • The experience is sufficiently developed that community members are welcome to try it out on Test Wikipedia. It still contains many visual bugs that we're fixing.
• Scaling
  • Wikipedias that will start receiving the features this week are: Catalan, Urdu, Greek.
  • Wikipedias preparing for deployment are: Bulgarian and Latvian.

Update 2021-04-26: community configuration

• Community configuration
  

  Community configuration editing form under development
  • The team has started a project to help more communities get set up quickly with Growth features, and to allow them to adjust the features on their own.
  • In this idea, administrators will be able to use a Special page on their wikis to adjust the configuration of the Growth features for all users. For instance, they'll be able to change which templates put articles into the newcomer tasks feed.
  • While this idea was developed for the Growth features, it has applicability for other features, and so we hope community members read through the project page and offer thoughts on its talk page.
• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
• Scaling
  • Last week, we deployed to Spanish Wikipedia, the largest Wikipedia that has Growth features (25,000 newcomers per month).
  • Wikipedias that will start receiving the features this week are: none.
  • Wikipedias preparing for deployment are: Catalan, Greek, Serbo-Croatian, Urdu, and Latvian.
• Experiment on English Wikipedia
  • We have started a discussion with the English Wikipedia community about how to deploy Growth features to that wiki.
  • We will likely deploy just to 2% of new accounts to get a sense of how things go before deploying to more users.
  • Community members are welcome to join in the discussion.

Update 2021-04-19: terminology and measurement for "add a link"

• "Add a link"
  

  Mockup of mobile version of mentor dashboard
  • We decided to use the term "machine" to refer to the link recommendation algorithm in the feature (as opposed to the term "artificial intelligence"). To learn about how we made that decision, see this section.
  • We posted our plans for measurement and experimentation with "add a link". We'll be testing the feature to see if newcomers have increased engagement with "add a link" than without it. For details, see this section.
• Mentor dashboard
  • We've posted design details and mockups for both the desktop and mobile dashboard here on the project page. See accompanying image.
  • If community members have any ideas or questions as we work to make these mockups a reality, we hope they post on the talk page.
• Scaling
  • Last week, these Wikipedias started receiving Growth features: none.
  • Wikipedias that will start receiving the features this week are: Spanish.
  • Wikipedias preparing for deployment are: Serbo-Croatian, Urdu, and Latvian.

Update 2021-04-13: engineering on "add a link"

• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
• Mentor dashboard
  • We are finishing designs for the mobile version of the dashboard. Though we believe that most mentorship currently happens from desktop users, we believe that making mentorship easy from mobile devices will encourage new kinds of people to become mentors.
• Scaling
  • Last week, these Wikipedias started receiving Growth features: Malay, Simple English, and Tamil.
  • Wikipedias that will start receiving the features this week are: Spanish.
  • Wikipedias preparing for deployment are: Serbo-Croatian, Urdu, and Latvian.

Update 2021-04-05: link recommendations algorithm remains accurate

• "Add a link"
  • Last week, we tested the link recommendation API outputs in six languages (English, French, Vietnamese, Bengali, Czech, and Arabic) to ensure that the algorithm remains accurate and to find any issues. We found an accuracy level between 70% and 90% depending on the language (we discovered fixable issues in Vietnamese that made its suggestions much lower), meaning that 70-90% of suggested links should be links, and will link to the right article. We believe this is sufficiently accurate for newcomers to be successful.
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
• Mentor dashboard
  • Engineering is now beginning, with technical architecture planning unfolding in Phabricator under this task.
  • Next, we will turn our attention to mobile designs.
• Scaling
  • Last week, no new Wikipedias started receiving Growth features.
  • Wikis that will start receiving the features this week are: Tamil, Malay, and Simple English.
  • Wikis preparing for deployment are: Serbo-Croatian and Latvian.

Update 2021-03-31: re-testing link recommendations algorithm

• "Add a link"
  • The link recommendations are soon going to be available via public API. The Growth team's "add a link" feature will use this API, but community members will also be able to use it to build their own tools using link recommendations.
  • This week, we are testing the API outputs in six languages (English, French, Vietnamese, Bengali, Czech, and Arabic) to ensure that the algorithm remains accurate and to find any issues.
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
• Mentor dashboard
  • We are finalizing designs for dashboard after receiving community thoughts, and engineering will begin soon. You can follow along with engineering under this Phabricator task.
• Scaling
  • Last week, Hindi, Japanese, Telugu, Albanian, Esperanto, and Norwegian Wikipedias started receiving Growth features.
  • Wikis that have the features available for testing are: Spanish, Tamil, Malay, and Simple English.
  • Wikis preparing for deployment are: Serbo-Croatian and Latvian.

Update 2021-03-22: mockups of mentor dashboard

• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
• Mentor dashboard
  

  A mockup showing one potential arrangement of the mentor dashboard
  • Following user interviews, we have now created an initial set of designs and posted them on the project page.
  • Community members are encouraged to look at the designs and give us their thoughts by answering the questions linked from the page. We'll also be going directly to the mentors on our pilot wikis for their thoughts.
  • In the next two weeks, we'll make final designs and begin engineering.
• Scaling
  • Last week, Indonesian and Croatian Wikipedias started receiving Growth features.
  • Wikis that have the features available for testing are: Japanese, Esperanto, Norwegian, Albanian, Hindi, Malay, and Simple English Wikipedias.

Update 2021-03-15: engineering on "add a link"

• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
  • The code is beginning to appear in Test Wikipedia, where we will be testing and fixing bugs.
• Scaling
  • Last week, Thai Wikipedia started receiving Growth features.
  • Wikis that have the features available for testing are: Indonesian, Croatian, Japanese, Esperanto, Norwegian, Albanian, and Hindi Wikipedias.
  • Wikis preparing for deployment are: Telugu Wikipedia.

Update 2021-03-07: "add a link" leading indicators

• "Add a link"
  • We are now working on a timeline that targets deployment in early April. We'll first deploy to our pilot wikis: Czech, Vietnamese, Arabic, and Bengali Wikipedias. After two weeks, we'll look at these leading indicators to see if there are any major problems. If not, then we'll deploy to the other wikis that have the Growth features:
    • Revert rate: the percentage of "add a link" edits that are reverted.
    • User rejection rate: the percentage of link suggestions for which users select "no" instead of "yes".
    • Task completion rate: the percentage of users who start the link recommendations task that also complete it.
• Scaling
  • Last week, Danish and Romanian Wikipedias started receiving Growth features.
  • Wikis that have the features available for testing are: Thai, Indonesian, and Croatian Wikipedias.
  • Wikis preparing for deployment are: Hindi, Albanian, and Esperanto Wikipedias.

Update 2021-02-28: mentor dashboard interview results

• "Add a link"
  • This week, we'll be finalizing our approach to instrumentation and experimentation for this feature.
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
• Scaling
  • Wikis that are preparing for the Growth features include Danish Wikipedia, Thai Wikipedia, Romanian Wikipedia, Indonesian Wikipedia, Albanian Wikipedia, and Croatian Wikipedia.
  • We are also preparing to discuss deployment on English Wikipedia. To weigh in on how we'll approach our largest Wikipedia, see this discussion thread.
• Mentor dashboard
  • We have completed 19 interviews with mentors to help us design this feature. See the project page to learn about the interviews and their results.
  • Next, we will be creating wireframes of possible designs and posting those for community reactions.

Update 2021-02-22: allowing communities to configure Growth features

• "Add a link"
  • We've completed writing the copy for the feature, and it is now finalized in all the specifications.
  • We have also decided what we'll be measuring and are finalizing our plan for running a controlled experiment.
• Scaling
  • Wikis that are preparing for the Growth features include Danish Wikipedia, Thai Wikipedia, Romanian Wikipedia, and Indonesian Wikipedia.
  • For these wikis (and future ones), we are deploying the Growth features in "dark mode", meaning that no users have them on by default. Then communities have two weeks (or longer, if they request) to test, discuss, translate, and configure the features to fit their wiki. If after two weeks, there are no objections, we turn the features on for newcomers.
  • To facilitate this process, the team is working on a way for communities to adjust and configure the Growth features on-wiki on their own with a simple form. This will help the Growth team to deploy to more wikis more quickly, since each one won't need individualized preparation with Growth engineers.

Update 2021-02-15: results from second variant test

• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
  • We're also continuing to write the copy for the feature and to develop the measurement plan.
• 

  Mobile results showing higher editing numbers for "Variant C"
  Variant C vs. D
  • We recently completed the analysis of our second variant test of the newcomer homepage. These tests are meant to try different arrangements of the newcomer homepage to see which ones increase newcomer engagement. Half the newcomers get one variant, and half get the other. Our first variant test was completed in June 2020, leading us to deploy "Variant A" to all users.
  • This test compared Variants C and D, both of which built on learnings from the first test. Both variants make suggested edits the most prominent part of the newcomer homepage, but differ in how users are onboarded to suggested edits.
  • We found that both Variants C and D led to higher interactions and edits than Variants A and B. Variant D led to more users editing on desktop and Variant C led to more users editing on mobile.
  • To take action on these results, we will soon be shifting all users of the desktop experience to Variant D and all users of the mobile experience to Variant C.

Update 2021-02-08: statistics and summaries for "add an image"

• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
  • We're also continuing to write the copy for the feature and to develop the measurement plan.
• "Add an image"
  • Last week, we posted three pieces of information that provide more detail on our team's and the community's thinking about this project.
  • Notes from community discussions: a summary of the points brought up by community members in five languages around this feature idea.
  • Algorithm coverage: these statistics show how many image suggestions the algorithm will be able to make.
  • Metadata coverage: these statistics show how much non-English metadata exists for image suggestions.
• Mentor dashboard
  • We're conducting interviews with mentors in multiple communities to inform the design of this feature.
  • If community members want to weigh in with their opinions, they can contact us on this talk page.

Update 2021-01-31: user test results for "add an image"

• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
  • We're also continuing to write the copy for the feature and to develop the measurement plan, which will explain which data we're measuring and how we'll set up any A/B tests.
• "Add an image"
  • In December, we ran 15 user tests with newcomers on the "add an image" idea, using this interactive prototype. The objective was to observe as many user judgments on image matches as possible, so we can understand the strength of user judgment and figure out how to design to increase it.
  • The full notes on the user tests are here. We are encouraging community members to read and react to them on the project talk page.
  • In summary, we think that these user tests confirm that we could successfully build an "add an image" feature, but it will only work if we design it right. Many of the testers understood the task well, took it seriously, and made good decisions. On the other hand, many other users were confused about the point of the task and made weak decisions -- but for those confused users, it was easy for us to see ways to improve the design to help them succeed.
  • Another important development for this project is the start of work on the Android MVP (minimum viable product): the Android app team will be building a simple image matching workflow as a "suggested edit" in the app. The MVP will not save any edits. It is only for the purposes of learning how to improve the image matching algorithm and the design, so that when the Growth team works on this idea, we can benefit from the learnings.

Update 2021-01-25: progress on "add a link"

• Team changes
  • Our team has been undergoing some changes as we gain new members and old members transition to other teams. We're expanding, which is good! But it means we'll be spending effort over the next few weeks onboarding new members and helping them get up to speed.
• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
  • The beginnings of the experience will start being available in beta wikis this week. We will let volunteers know when the pieces start being ready to try out.
  • We're now writing all the copy for the feature (the words that users will see).
  • We're also writing our measurement plan, which defines which data we'll gather and how we'll use it. This will get posted on-wiki for community members.
• "Add an image"
  • We've finished gathering our notes on the user tests that we ran around this idea. We'll be posting our findings on-wiki this week so community members can see them.
  • The Android team is going to be building an MVP (minimum viable product) around image recommendations. It's not going to make any edits to Wikipedia: it's just going to help us learn how users react to an image-related task, and give us a sense of how strong their judgment will be when they do the task. We'll link to more details on this idea once it takes shape.

Update 2021-01-18: design work on mentorship

• "Add a link"
  • We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
• "Add an image"
  • We are finishing up the final threads on the discussion page for this project. We'll summarize the state of the conversation this week, but please feel free to join in at any time.
  • We'll also be posting learnings from our 15 user tests on the image recommendation prototype.
  • On the technical side, the Platform Engineering team is scoping out a "proof of concept" API for serving image recommendations. The Growth and Android teams will be able to use this to try out building an "add an image" feature and get a sense for hw such an API would function.
• Mentorship dashboard
  

  Sketch of a potential configuration for a mentor dashboard
  • Mentorship is an important part of the Growth team feature set, and it's the one in which community members participate most, and have the most ideas and questions.
  • But because our team spends the majority of our resources on newcomer tasks, we have not been able to dedicate time to improvements for mentorship.
  • The team now has a design intern working with us for the coming months, who will be focused on mentorship features.
  • Over the last years, the most requested mentorship feature is some sort of page where mentors can see who their mentees are and monitor how they are progressing.
  • The design intern will be working on building this, using the existing project page as a starting point and the existing community thoughts on its talk page.
  • If you have opinions or ideas, please speak up on the talk page so we can take them into account.

Growth team updates : 2021, 2020, 2019, 2018.