The leading indicators of this experiment look healthy. In the last week the Community Update module Click Through Rate (CTR) remained above 1%. The Community Updates CTR isn’t too far from the CTR we see for the Homepage's Help Module.
The Community Updates CTRs vary by wiki, it seems like the CTR might be higher on updates that target more experienced editors.
Design specs for surfacing “Add a Link” are finalized and we have started some very early explorations to mockup a similar UI for surfacing the “Add an Image” task (T378776).
We are completing our experiment measurement plan and instrumentation specification(T377096).
We are completing the initial work to accommodate highlighting a fragment of text in article's read mode (T378354), and have the initial work done to show a pop-up when that highlighted text is tapped on (T376680).
Growth team Product Ambassador, Habib Mhenni, attended the Wikimedia conference in Quebec City, supported by a scholarship. As a volunteer, he assisted with event logistics and presented on tools like the Newcomer Homepage, Mentorship Module, EditCheck, and Wikimedia Commons features, emphasizing their role in supporting new editors and improving content quality. He connected with experienced volunteers, gathering insights on improving tool usability, updating outdated Help Pages, and extending mentorship to more Wikimedia projects. These interactions reinforced the importance of user-friendly tools and accurate resources for fostering community growth.
Note: This feature is being released as an A/B test, so only 50% of eligible homepage visitors will see the Community Update on their Homepage. Communities can set a minimum number of edits an account must complete to view the Update, allowing them to target campaigns more appropriately. For instance, if a Community Update is less suited for brand new accounts, it can be restricted to accounts with at least x edits.
Update 2024-10-25: Dark Mode and Flow deprecation work
We are also supporting the Editing team with creating a maintenance script to move all flow boards on a wiki to a subpage. This work should help support the effort to move wikis away from using Flow, which is no longer maintained (T371738).
Growth Engineers have started work on T376678: Surfacing Structured Tasks: internal Action API endpoint.
We decided to run this initial experiment on Spanish Wikipedia, French Wikipedia, and Egyptian Arabic Wikipedia (T377358).
English Wikipedia has discussed enabling “Add a link” and several experienced editors have expressed support for enabling, but also hesitation around suddenly enabling the task for 100% of newcomers. They suspect a sudden influx of “add a link” edits might cause unnecessary frustration for patrollers, and have therefore suggested a more gradual rollout. We will discuss this further next week (T377631).
Update 2024-10-11: Community Updates module instrumentation
We are making an improvement to how the Community Updates module’s Community Configuration to ensure the module works for a wider variety of campaigns, events, and use cases (T374237).
We are finalizing instrumentation improvements (T376833 & T374229) experiment variant bucketing (T376266)
We wrapped up the design stage of our Q1 annual plan hypothesis: Wiki Experiences 1.2.3:If we conduct user tests on two or more design prototypes introducing Structured Tasks to newcomers within/proximate to the Visual Editor, then we can quickly learn which designs will work best for new editors, while also enabling engineers to assess technical feasibility and estimate effort for each approach.
While so far object-fields had i18n messages for labels, it was left up to their sub-properties to add more context. But recently we’ve added support for adding overall context to the object as a whole, allowing for adding more high-level guidance or guidance that applies to multiple fields: T373182 Add support for overall description / help texts for objects in CommunityConfiguration
Another improvement concerns controls for simple strings to validate the max-length and min-length of the input while typing. Now you won’t be submitting a form just to learn that your title is 2 characters too long! (T372850).
The leading and trailing spaces are now pruned from strings (T373160)
With support from the Language team, we merged the first iteration of CommunityConfiguration for the Babel extension! (T328171 ). The next step is to communicate with the communities and roll it out incrementally to the wikis (T374348). We chose an opt-in way for adding CommunityConfigration support to Babel, so that 3rd-party wikis can decide for themselves whether they want to adopt it or not.
Update 2024-09-20: Community Updates module + Metrics Platform
The Community Updates homepage module is instrumented via the Metrics Platform, and associated metrics can be tracked via Superset (T371498 & T371347).
A decision has been made that for now, the Community Updates module will use GrowthExperiments for the experiment and variant assignment manager.
Growth team’s tech lead summarized the missing functionality that will bring the Metrics Platform in line with what the GrowthExperiments extension currently supports (T374471#10191299).
Allow to (force) set an experiment and experiment variant for a given user.
Allow to target only new accounts, existing accounts, existing accounts of X age
Per platform variant sampling rate
Update 2024-09-13: Metrics Platform Experimentation + Community Updates Module
The Growth team is working with the Data Products team on setting up the very first Metrics Platform experiment. Progress this week, included:
Agreeing on a variant system approach for bucketing and surfacing approaches that would inform the final version of the Metrics Platform bucketing tooling (T374471).
A proof of concept for tracking impressions of the Community Updates homepage module is merged and ready to test in Beta (T371498).
The core schema was been updated to capture the instrument name and experiment (T366802).
Update 2024-09-06: Community Configuration improvements
While so far object-fields had i18n messages for labels, it was left up to their sub-properties to add more context. But recently we’ve added support for adding overall context to the object as a whole, allowing for adding more high-level guidance or guidance that applies to multiple fields: T373182 Add support for overall description / help texts for objects in CommunityConfiguration
Another improvement concerns controls for simple strings to validate the max-length and min-length of the input while typing. Now you won’t be submitting a form just to learn that your title is 2 characters too long! (T372850). Also, leading and trailing spaces are now pruned from strings (T373160)
With support from the Language team, we merged the first iteration of CommunityConfiguration for the Babel extension! 🎉
Next step is to communicate with the communities and roll it out incrementally to the wikis (T374348).
We chose an opt-in way for adding CommunityConfigration support to Babel, so that 3rd-party wikis can decide for themselves whether they want to adopt it or not.
We've merged a dedicated control that allows selecting multiple elements from an enumeration: (T370694)
We've started to work on showing validation errors directly on input in the browser. For now, this works in particular for number controls and the above-mentioned MultiselectEnumControl
We also started work on creating an example implementation of CommunityConfiguration directly in CommunityConfiguration: (T371044 ).
The “Add a Link” structured task has been enabled on German Wikipedia. The decision was left to the German community, and earlier today, a German admin enabled the task via Community Configuration (diff).
Update 2024-08-16: Metrics Platform work & Add a Link rollout
We agreed on a stream name and are now unblocked on T370907
A Proof of Concept for tracking impressions of the Community Updates homepage module is being worked on T371498 in this sprint as part of the research spike on preparing Growth Experiments to use the Metrics Platform.
Community Updates now has Community Configuration support T367223.
We are completing second iteration designs this sprint: T371731. The upcoming design phase will focus on narrowing down to three final designs and creating interactive prototypes for testing: T372518. We will also share these design ideas with communities on MediaWiki and with our Growth pilot wikis (Spanish and Arabic Wikipedia) to gather more feedback.
This week we reviewed"Surfacing Structured Tasks" user testing feedback from Wikimania. While no clear consensus emerged on a favorite or least favorite design, several valuable insights were provided:
“I think suggesting edits for people who have never edited or are idle in editing mode can help increase participation in Wikipedia editing."
“In the first approach, using the name “structured tasks” in the message right away might be confusing for a reader...”
“Repeated discoverability seems a challenge here. How can u not annoy returning editors, have initial discoverability and also explain to ppl how to get back to those flows in subsequent edit sessions”
Other Wikimania attendee insights have been added to the follow-up design task: T372518
The "Add a Link" task if now available for Communities to enable on English Wikipedia (T370802) and German Wikipedia (T371597). Unfortunately, on German Wikipedia, a search related issue has resulted in tasks not being populated correctly within Suggested Edits (T372362). A subset of this issue has already been resolved, but there is more work necessary to ensure tasks are populated correctly.
Bug Fixes
We have released several bug fixes, including a small improvement to the styling of the Login form (T366127), some improvements to Mentorship (T371775), and several further fixes and improvements for Community Configuration (T368728, T370190, T366577).
Update 2024-08-09: Wikimania & Metrics Platform work
Growth team members presented Community Configuration: Shaping On-Wiki Functionality Together at Wikimania (slides). The session recording is available to watch on YouTube. This session provided an update on the Community Configuration project and introduced details about the upcoming features that communities will soon be able to configure. Representatives from the Moderator Tools, Editing, Web, and Campaigns teams shared their plans for utilizing Community Configuration in the future. Following these presentations, the WMF Growth team's Benoît Evellin and Martin Urbanec answered audience questions
We had a check-in meeting with the Data Platform team to clarify the expected life of an instrument and how that affects naming a stream. T370907#10044721.
Community Updates now has Community Configuration support T367223. QA work is done on beta and is awaiting deployment to testwiki.
A Proof of Concept for tracking impressions of the Community Updates homepage module is being worked on T371498 in this sprint as part of the research spike on preparing Growth Experiments to use the Metrics Platform.
Growth has started initial design explorations for various areas and moments to introduce Structured Tasks (T368188). And we are collaborating with the Editing team to refine the evolving vision of Structured Tasks and ensure alignment with the Edit Check initiative (T370542).
This sprint, we will proceed with the first iteration of designs for surfacing structured tasks (T370539).
This week the Growth team deployed the CommunityConfiguration extension to nearly all Wikipedias (T366458). Due to technical issues (T370261), German Wikipedia and a few newer wikis were temporarily excluded.Community Configuration empowers communities to tailor wiki features to their specific needs, and can help WMF teams scale features more efficiently. As part of the 23/24 WMF Wiki Experiences 1.2 Key Result focused on moderators, it’s fitting that the Moderator Tools team, with their Automoderator feature, will be the first to utilize Community Configuration.
Update 2024-07-12: How does page load time impact Homepage engagement?
We had a natural experiment that started on 6/18/2024 where an upstream change increase Homepage loadtimes (T368405). We decided to explore the data to learn how much this increase in Homepage load times changed user engagement.
To keep this task constrained, we used a very simple approach to data gathering: we used the week prior to the bug going into effect (2024-06-11 through 2024-06-17) and compared that to the first week of the bug being present (2024-06-18 through 2024-06-24). Given this simple method, I have not attempted to determine if there's a *significant* change between the weeks, but instead try to interpret the findings based on the relative differences.
Homepage visits: no substantial difference (-0.4%), but opposing trends between desktop and mobile web. Desktop decreases (-0.9%) whereas mobile web increases (+0.9%). Visits on desktop are roughly 2x that of mobile web, btw.
Homepage module impressions: meaningful decrease (-3.8%). The trends are similar on both platforms (desktop -4.4%, mobile web -2.5%), but the mobile web effect might be dampened by the slight increase in visits.
Module interactions: substantial decrease (-12.8%). The effect is larger on desktop (-14.2%), where most of our interactions happen, but still substantial on mobile web (-9.3%).
Clicks on Suggested Edits tasks: substantial decrease (-7.1%). Here the effect is larger on mobile web (-13.7%) than on desktop (-5.6%).
Completed Structured Task edits: overall no difference (-0.7%) but opposing trends between desktop and mobile. On mobile web we see a substantial decrease (-17.8%) while on desktop there's an increase (+4.5%).
Update 2024-07-05: Community Configuration scaling
We released the CommunityConfiguration extension to more Wikipedias this week and addressed one issue unique to scaling to Portuguese Wikipedia (T366629, T368121, T368275, T368310). The CommunityConfiguration extension is now available at the following Wikipedias:
Indonesian Wikipedia (idwiki)
Bengali Wikipedia (bnwiki)
Czech Wikipedia (cswiki)
Greek Wikipedia (elwiki)
Persian Wikipedia (fawiki)
French Wikipedia (frwiki)
Polish Wikipedia (plwiki)
Portuguese Wikipedia (ptwiki)
Romanian Wikipedia (rowiki)
Turkish Wikipedia (trwiki)
Swahili Wikipedia (swwiki)
Chinese Wikipedia (zhwiki)
Community Configuration will be released to all Wikipedias on Wednesday July 17 (T360571 ).
Update 2024-06-21: Community Configuration released to pilot wikis
Engineering work on the Community Updates module has started. (T365877). Instrumentation requirements will be refined in the coming week (T365889). This work will continue in the coming fiscal year under SDS2.1.3. You can read more about the Growth team's annual plan here: Growth/Annual Plan 2024-2025
Update 2024-06-07: Community Configuration beta & test Wiki release
We scoped back what is included in the MVP, so there are many further improvements (and UI polishing tasks) in the works, but the basic functionality is all testable on beta and test wikis.
The Save Changes button and any error messages will soon be part of a fixed footer to improve UX and match the Preferences UI. (T364500 & T363855)
We are preserving the local configuration settings that were originally set in Special:EditGrowthConfig. For wikis that have the new Community Configuration extension installed, the old Special:EditGrowthConfig page will now redirect to Special:CommunityConfiguration. (edited)
The Growth team's work is guided by the Wikimedia Foundation Annual Plan, and specifically the Product & Technology department's Objectives and Key Results. The Annual Plan relates to the Wikimedia Foundation's fiscal year, which starts on July 1 and ends on June 30 of the following year. The Growth team expects that about half of our time will be devoted to annual planning priorities in the coming year, while the other half will be spent on technical maintenance and other essential work to support our actively and passively maintained projects.
Wiki Experiences, Key Result 1.2: The Growth team will mainly focus on the Wiki Experiences 1.2 Key Result, which relates to increasing newcomer constructive activation. In other words, working on features that help increase the number of new account holders who edit for the first time in a productive way that isn't reverted.
Update 2024-04-26: Wikimedia Hackathon + Community Configuration
This week we worked on a variety of Community Configuration improvements. We have introduced validation warnings for Community Configuration (T362042), created a system to support developers while upgrading configuration schemas (T357532), and we are finishing up the work to migrate the rest of the Special:EditGrowthConfig settings to Special:CommunityConfiguration.
Next week, two Growth team engineers will attend Wikimedia Hackathon, and have planned the following sessions:
The “Add an Image” structured task leads to an increase in newcomer participation on the mobile web platform, particularly by making constructive (non-reverted) article edits:
The likelihood that mobile web newcomers make their first article edit (+17.0% over baseline)
The likelihood that they are retained as newcomers (+24.3% over baseline)
The number of edits they make during their first two weeks on the wiki (+21.8% over baseline)
A lower probability of the newcomers’ edits to be reverted (-3.3% over baseline).
New Growth team engineer
This week we welcomed Michael Große to the team! Michael is a Leipzig-based software developer who previously worked for WMDE on Wikidata, and before that on DokuWiki.
The Community Configuration security review is complete, and the extension is classified as low risk. (T349568). In general, the code reviewed appears to be robust in terms of security. There are no vulnerable dependencies, and no apparent vulnerabilities are present.
We will release the first part of Community Configuration to es.wikipedia.beta next week (T357766).
Our next major milestone is to migrate the remaining logic of SpecialEditGrowthConfig to the new CommunityConfiguration extension (T355317).
We are drafting initial design guidelines for Community Configuration (T350728).
Update 2024-03-15: Early planning for additional Newcomer homepage modules
Growth Product Ambassadors have translated project documentation into Arabic and Spanish and are starting community discussions about this small experimental project.
We are still in the early discovery period for this project, and considering community needs and exploring the feasibility of filtering tasks to help address knowledge gaps.
Update 2024-03-08: Non-editing Participation Research
We have published a summary of the Non-editing Participation Research conducted this fiscal year. Including a link out to the final report. The Non-editing Participation final report, in addition to painting a clearer picture of the people who make accounts “to read Wikipedia,” finds that
(a) many readers create accounts because they expect that their accounts will provide them with reading tools and features;
(b) new account holders as a group are interested in deepening their relationship with Wikipedia, and have taken a step to do so; and
(c) interviewed readers are more interested in tools and features that improve their own experience (like creating reading lists) than they are in features that rely on interaction with others (like sharing reading lists).
On the whole, many readers want more from the Wikipedia reading experience than they currently get, and these interested readers might be enticed to deepen their relationship with and participation in Wikipedia if provided some of the tools they appear to expect.
The Growth team may focus on a project that relates to Non-editing Participation next fiscal year, but our team's focus will depend upon the WMF Contributor experience and Consumer experience priorities set in the 2024-2025 Wikimedia Foundation annual plan.
Update 2024-03-01: Community Configuration progress
Growth’s designer is working to polish designs (T336271) and localize designs (T358056), so that we can start community discussions with Growth’s pilot wikis (Arabic and Spanish Wikipedia).
Update 2024-02-23: New project ideas for the Newcomer Homepage
WikiExperiences 1.3 Key Result: A 1 percentage point increase (YoY) in the portion of newly created or improved articles on high-impact topics with acceptable quality, per the “global quality score”, that are created or edited on Wikipedia, starting with underrepresented geographic regions and gender.
The initial project ideas Growth is discussing related to the WikiExperiences 1.3 Key Result:
Community Spotlight: a new module to the Newcomer homepage. This new module can be used by communities to share important community news, campaigns, events, or Wiki Projects. This module will be Community Configurable to allow for easy customization.
Update 2024-02-16: New project ideas for the Newcomer Homepage
The Growth team is drafting initial Community Configuration Guidelines (T350728) and Developer documentation (T349039). We will share an initial draft for community discussion soon.
Engineers have completed writing specifications for the Configuration store (T342752) and the editing form (T343072).
Engineers have reviewed Community Configuration use for Growth features, Automoderator, Web, and Campaigns in order to better understand use cases and which datatype and components would work for each use case. (T351545).
The initial code for the Community Configuration dashboard is written (T354390).
We have shared experiment results for the Personalized praise feature. This feature was developed for Mentors as part of the Growth team's Positive Reinforcement project. For this experiment, we focused on the effect of praise on newcomer retention and productivity. The Personalized praise feature was deployed to the Arabic, Bangla, Czech, and Spanish Wikipedias in late May 2023. We analyzed the Spanish Wikipedia separately from the other three because on the Spanish Wikipedia 50% of newcomers are randomly assigned a mentor, which means the feature is part of a controlled experiment.
On Spanish Wikipedia, although we found no significant impact on retention, we found a significant positive impact on newcomer productivity. This finding is encouraging since our preliminary analysis of mentorship found conflicting results.
As these results were not positive enough to clearly justify the time investment from Mentors, we have decided to start conversations with our Product Ambassadors and communities and consider further improvements before releasing the feature more widely. We will consider improvements related to reducing the amount of work needed by Mentors, potential design improvements, and improvements to how newcomers are selected to be displayed in the Personalized praise module.
Support conditional defaults for user properties to help address user_properties table bloat
This week, the Growth team has finished up the work of supporting conditional defaults for user properties (T354459). Growth completed 11 tasks and documented the work (https://www.mediawiki.org/wiki/Manual:$wgConditionalUserOptions). DBAs will handle dropping the old rows (T357072). This effort to support conditional defaults was requested by Wikimedia system administrators and was considered a high priority because if ignored it may have resulted in database malfunctioning in the wikis with the most registered users.
We are finishing up work on investigating form generation options for community configuration (T332849) and documenting specifications for the Community Configuration Editing form (T343072).
Update 2024-01-26: Growth + Campaigns Offsite
The Growth team has members on four continents and many time zones, but once a year we have the opportunity to spend a work week together. This year our offsite was scheduled with the Campaigns Product team.
We spent the majority of our first day together discussing our current project: Community Configuration 2.0.
Our second day was future-focused and we discussed ideas for supporting the Wiki Experiences 1.3 Key Result. The Growth team Annual Plan always included time for a secondary annual-plan related project, but we hadn't finalized plans. After reviewing the state of all WMF Product & Technology Objectives & Key Results, WMF Product leadership suggested Growth could help support the Wiki Experiences 1.3 Key Result since it is considered an understaffed and "at risk" key result. The Growth team is currently considering various ways we can extend Growth features to best support this Key Result, and we will share some initial ideas and mockups with the community soon.
Our final offsite days together were spent working directly with the Campaigns team.
Update 2024-01-12: Donor Thank You page results on English Wikipedia
One theme that emerged from volunteer feedback was around the need to recruit more editors, rather than just fundraise. After receiving support from English Wikipedia editors, we scaled the Thank you page experiment to English Wikipedia for the 2023 English fundraising campaign. Detailed analysis and interpretation of results for the first day and first week of the campaign can be found in this task: T352116, and final campaign analysis is found in this task: T352900.
The following are results for the entirety of the 2023 campaign: 2023-11-28 @ 09:00 UTC to 2024-01-01 @ 12:00 UTC.
Platform
Page views
Unique visitors
Registrations
Registration %
Activations
Activation %
Desktop
6,045
5,203
1,912
36.7%
219
11.5%
Mobile web
7,665
6,802
2,486
36.5%
222
8.9%
Totals
13,710
12,005
4,398
36.6%
441
10.0%
Although it was a small fraction of donors that clicked through to Try editing Wikipedia, this Donor Thank you page campaign led to 4,398 new accounts, and 441 of those accounts went on to constructively edit within 24 hours (a constructive edit means the edit wasn’t subsequently reverted within 48 hours).
We also calculated the revert rate of the edits made by these users, as well as what proportion of their edits came through the Suggested Edits module on the Newcomer Homepage. When it comes to the revert rate across all edits, with the knowledge that contribution amounts vary greatly between users, the revert rate is 8.9% out of <900 edits (we're not reporting specific edit counts per our Data publication guidelines). The low revert rate might be a result of the high proportion of Suggested Edits these newcomers make. Overall the proportion is 61.8% out of <900 edits. The rate is lower on desktop (53.6% out of <450 edits) than on mobile web (70.0% out of <450 edits).
As we concluded in previous experiments in other languages (Newcomer Experience Pilot Project- Thank You Pages and Thank You Banners) the donor Thank you page "Try editing Wikipedia" call to action seems like a sustainable and low-impact way to grow the number of new editors on Wikipedia.
Design: After receiving initial user research results from Community Configuration, the Growth team’s UX designer has worked on further refinements to the Community Configuration designs (T350201).
Engineering: We have started the more front-end focused work on Community Configuration. We have started to investigate form generation options for Community Configuration (T332849 ) and write specifications for the editing form (T343072).
Support conditional defaults for user properties
Growth engineering has been partially focused on tasks related to adding support for conditional defaults for user properties to help address user_properties table bloat (T354459). This work wasn’t part of the Growth team’s annual roadmap, but was escalated by Wikimedia system administrators as ignoring the issue indefinitely could result in database malfunctioning in the wikis with the most registered users.
The Growth team’s UX designer shared some further refinements to the Community Configuration designs (T350201).
We have switched to justinrainbow/json-schema for schema validation (T353230). Initially, we went with opis/json-schema for validating schemas. This proved to be problematic, because certain features we use are only available in 2.3 (T351878) and Wikilambda currently uses 1.x (T319054).
Participants from smaller wikis are enthusiastic about how “democratizing” Community Configuration is.
Participants from larger wikis trust that the restrictions/permissions that currently govern configuration will continue to do so under Community Configuration.
Participants found the Community Configuration designs to be an improvement over the existing configuration processes/systems they are familiar with.
Participant concerns about empowering too many people to make configuration changes were alleviated once participants interacted with the Community Configuration prototypes.
The explanatory messages, logo image, and page title used in the prototype can be reconsidered in light of the fact that Community Configuration will be used by many non-technical, non-native speakers of English
Support conditional defaults for user properties
Engineers have been mainly focused on tasks related to adding support for conditional defaults for user properties to help address user_properties table bloat (T354459). This work wasn’t part of the Growth team’s annual roadmap, but was escalated by Wikimedia system administrators as ignoring the issue indefinitely could result in database malfunctioning in the wikis with the most registered users.
No back icon on mobile growth home page overlay (T351097).
Section level “add an image” placeholder bug fix (T352208).
In order to support future effort to deprecate Flow, users can no longer enable the Flow beta feature on their user talk page (T248309). We also updated copy to help provide clarity (T352565).
Mentorship bug fix for Mentor whose mentees were not reassigned after quitting mentorship (T354220).
