Growth/Analytics updates

Work log

When Morten, our data analyst, works on analysis of team-related things we'll do our best to make that available to the community. Some of the work is done in daily "work logs", those are linked from the box on the right. Larger pieces of work are listed and described below.

Personalized first day

The Personalized first day project deploys a survey, which we refer to as the "Welcome Survey", to new users in our target Wikipedias. We have created an "experiment plan" for that intervention. The plan describes what our goals are, outlines how we plan to measure results, and how we will act depending on what happens after deployment.

The main goal of the one-month experiment with the survey was to determine if it had an effect on whether newly registered users make an edit within 24 hours after registration. We have published an analysis of the results, where we find that the survey does not have a statistically significant impact.

We have also published an initial report of survey responses based on survey responses from the first four weeks after deployment.

Understanding first day

Our Understanding first day project aims to learn what new users are doing during the first 24 hours after registration so that we can identify opportunities for technical solutions to help them achieve their goals. We have published an initial report of EditorJourney data, where we take a quick look at some initial results from our analysis to get a high-level view of what users are doing.

Help Panel

The "Focus on help desk" project deploys a help panel to new users in our target Wikipedias. We have created an "experiment plan" for that intervention. The plan describes what our goals are, outlines how we plan to measure results, and how we will act depending on what happens after deployment.

The experiment plan page also contains an evaluation of leading indicators completed with data from a month after deployment. Initial data indicates that the help panel is behaving in a healthy way, and identifies a few potential areas for improvement.

After having run our experiment with the Help Panel for about 2.5 months in Czech and Korean Wikipedias, and a month in Vietnamese, we have published an analysis of whether the Help Panel has an effect on editor activation (new users making an edit within 24 hours after registration), and find that it does not appear to have a statistically significant impact.

Homepage

The "Newcomer Homepage" project creates a homepage where new users get pointers to get started on-wiki (e.g. tutorial for learning to edit, starting their user page, etc). Our experiment plan describes our A/B test for this homepage on our target wikis, the research questions we're seeking to answer, and more detailed measurements that we'll be taking.

The experiment plan also contains a plan for leading indicators that we measure during the first weeks after deployment. We have published a report on the Homepage leading indicators. In that report we find no particular reason to be concerned with the Homepage, many newcomers are visiting it and using its various modules!