Editor Engagement/2013 strategy planning (AFT)

This is a strategy planning space for the Article Feedback team in 2013. Our initial focus is to plan the third quarter of the Wikimedia Foundation's 2012-2013 fiscal year (January-March 2013).

See our Article Feedback Q3 planning slides, which is a companion document for this team page.

Goals

Our overall goal for 2012-2013 is to help reverse Wikipedia's editor decline by recruiting new editors and better retaining existing editors.

Our overall AFT quarterly goal for January-March 2013 is to engage more readers to use Article Feedback to contribute and register on Wikipedia.

Milestones

And here are proposed quarterly milestones for January-March 2013:

Complete Article Feedback v5 features, with limited testing
Complete, review and deploy code refactoring with a new database cluster on the English Wikipedia, as soon as ops can help us do that
Deploy AFT v5 to 100% of the English Wikipedia, using a special database cluster

Target

The WMF plan identified a target of 86,000 active editors to Wikimedia projects by end of June 2013 (from 85,000 in March 2012).

This target was based on the number of editors who make 5 edits per month for all projects except Commons. As of October 2012, WMF reported 80,000 active editors for all projects, of which 8,000 were for Commons.

Research

Some of these research metrics and findings can help inform our work this quarter.

Funnel

Here are some rough estimates of our monthly user funnel for the English Wikipedia:

Readers: ~300M
New accounts: ~150K
New editors (10+ edits): 6K
Active editors (5+ edits/month): 32K
Very active editors (100+ edits/month): 3.2K

While these are just estimates from a range of different sources, they give us a sense of scale. For example, there seems to be on the order of 9,000 readers for each active editor on the English Wikipedia. And only about one in 25 new members who register go on to become auto-confirmed editors (with 10+ edits).

These examples suggest a large untapped potential for editor engagement, in three distinct groups: readers, new members and new editors.

(Source for editor counts: Wikimedia Report Card for October 2012)

Editor Decline

We’re currently seeing a 5% y-o-y decline in active editors for the English Wikipedia (34.3K in Oct. '11 vs. 32.5k Oct. '12). We're also observing a smaller 0.5% y-o-y decline in active editors for all projects combined for the same time period, according to the Wikimedia Report Card for October 2012.

As of October 2012, WMF reported 80,000 active editors for all projects, of which 8,000 were for Commons, which could mean a net 72,000 active editors for all projects other than Commons. This suggests a wider gap than anticipated between this report and the WMF's target of 86,000 active editors by July 1, 2013.

Reversing this decline will take sustained effort on multiple fronts, from editor engagement features and experiments to visual editor, internationalization and site performance improvements. What will be our focus for E2?

New Members

One of our best opportunities for engaging new users is by inviting them to first create an account, so we can effectively communicate with them through notifications and other tools. And we are signing up on the order of 125-150k new users per month on the English Wikipedia alone, which is significant.

Yet our research suggests that about 60-65% of the new users who register end up not making any edits after they sign up.

It seems that a few well-designed notifications could invite these new users to contribute and could help convert them into editors. Otherwise, notifications would have almost no impact at all on these new members, which would be unfortunate.

Examples of such invitations could include a reminder that they can edit, suggestions of articles to improve, tips on how to edit, how to leave talk page messages, how to use the watchlist, how to send wikilove, etc.

Feedback Registrations

Our latest research suggests that Article Feedback appears effective in getting new users to contribute to Wikipedia, as summarized on the WMF blog (see full slides).

For example, 2.7 percent of readers who post feedback go on to create a new account, after being invited to sign up. And 3 percent of these new users go on to edit articles within 24 hours from signing up.

At this rate, we project that Article Feedback can generate several hundred thousand new registrations per year on the English Wikipedia, which is significant. While we cannot accurately estimate how many of these new contributors would become active editors (5 edits/month), it appears that this program can have some impact in reversing the current editor decline, if combined effectively with other engagement programs such as notifications.

Feedback Moderation

Our research suggests that up to 40% of the reader feedback is found useful by editors, but it is buried under a lot of noise, which requires some form of moderation and/or filtering. But editor moderation activity is quite low when compared to the volume of feedback posted every day. Less than 10% of feedback posted every single day receives any kind of moderation within a month, whether by readers or registered editors.

These issues could be addressed by better moderation tools and filters, both automated and manual. And many editors can’t easily find comments for articles they edit, an issue which could be addressed by making feedback more visible to editors on the article pages.

Next steps

Our current development work on AFT5 has been significantly delayed, and is at least 3 months behind the schedule proposed in WMF's Engineering Goals for 2012-13. We will want to revisit this original plan and determine more realistic milestones together in coming weeks.

For now, this document is intended to help us identify objectives we want to focus on in this quarter (Jan. - March 2013).

To get the ball rolling, here are a few questions about which we would like to discuss with you.

How can we best improve our filtering tools to automatically surface more good feedback? to remove irrelevant comments?

How can we make useful feedback more visible to editors? through a special link on article pages? with a 'post to talk page' feature?

How can we reduce the editor workload? with simpler feedback moderation tools? with better guidelines?

How can we make this tool more scaleable, so it can support millions of comments? with a special database cluster? through a sharding tool?

How can we collaborate with WMF Ops to insure a timely deployment of this tool on a special database cluster for now?

How can we socialize this project with Wikipedia editors, to promote its benefits and address any community concerns?

See our features under consideration for solutions to some of these questions, which we identified as a team last quarter.

Active Editor Target

What would be reasonable targets for our E2 editor engagement features in this quarter?

For this question, let's consider the following points:

the editor decline has increased on some projects since the current active editor target was originally established
this target is shared by many other groups besides the E2 team: what can we contribute towards this company-wide target?
we have experienced serious delays that make some of our original goals impractical

Features

Here are the key features we are now considering for this quarter:

Simpler moderation tools ('useful', 'resolved,' 'no action needed', 'inappropriate')

Better feedback page filters ('featured', 'unreviewed', 'more filters')

One-click moderation actions (instead of flyout)

Hide reader tools for editors (to reduce clutter)

Auto-archive comments over time (if not moderated)

Satisfaction ratings for feedback page (% who found what they were looking for)

See also this full list of features under consideration.

Please use this space to enter any other feature ideas for meeting our goals. They may be along the lines of any of the above research or questions, or something entirely different.