Growth/Analytics updates/EditorJourney initial report

Group:	Growth
Start:	2018-10-01
End:	2019-01-15
Backlog:	#T205754
Lead:	Morten Warncke-Wang (data analyst)

Translate this page

As part of the Growth team's "Understanding first day" project, we deployed the EditorJourney EventLogging schema to Czech and Korean Wikipedias on November 15, 2018, around 20:00 UTC. This schema logs all page views by users up to 24 hours after registration so we can learn more about what new users are doing (with particular attention paid to privacy around reading articles). Before deploying, we published a research outline here.

The motivation for this work is the fact that most new users don't ever make an edit. It makes us ask, "What do they do after creating their account? And how can we help them find their way to editing?" More concretely, this effort endeavors to answer these questions:

What are the most common workflows that Czech and Korean new account holders go through during their first 24 hours?
What percent of those new account holders go through each of those workflows?
Which workflows do and do not tend to lead to edits?

We are particularly interested in understanding to what extent users try out but abandon editing, seek out help information, read about Wikipedia policy, or visit pages where they can interact with the community. This page is our team's initial report on results, and will be followed by more detailed analyses that address the questions in the research outline. Here are three questions addressed in this report:

From what context do newcomers create their accounts? Are they reading an article, editing an article, or doing something else? This is the location from which they begin their first session as a registered user.
What are the set of first actions taken by new users, and what percentage of users take each of those actions? In other words, what's the first thing they do after creating their account?
What percent of users do any of these within the first 24 hours after registration?
- View the Main page
- View an article
- Open an editor
- Attempt to save an edit
- Successfully save an edit
- View a Help or Project/Policy page
- View an Article Talk page
- View their own User or User Talk page
- View somebody else's User or User Talk page

The data in this report is based on views for users registered on the Czech and Korean Wikipedias between November 19 and end of day (UTC) on December 17, 2018, excluding users who already have an account on a different wiki, so called "autocreated users". Several events/views that are part of the registration process, such as the Growth Team's welcome survey, have been removed. We provide an overview of what users have been doing, and are not making claims about statistical significance nor differences between user groups or Wikipedias. Instead, we present these as preliminary findings and discuss some potential next steps these findings might suggest, and will be doing a more thorough analysis in the upcoming quarter.

Toplines

This data substantially clarifies our understanding of newcomers: many of them notice outreach, look for help content, and a majority of them open an editor. It's good news that so many are engaged and trying to succeed, and the clarity gives us opportunities to meet them where they are, so that when they go looking for something helpful, they find it.

Mobile vs. desktop: overall, most users create their account (and therefore begin their journey as an account holder) from a reading context, with only 17% of users creating their account from editing context. This is different on mobile, where 36.5% of all users find themselves in the editing context after account creation. We suspect this difference on mobile is largely driven by a full-page overlay asking users to register when they try to edit on mobile without logging in.
Welcome messages: 17.9% of Korean users visit their User Talk page immediately after creating their account. Korean Wikipedia automatically posts a welcome messages to all new User Talk pages, and this suggests that these are quite often being read immediately. Czech Wikipedia does not post such a message, and the corresponding number there is 1.0%, showing that the Korean welcome message is attracting a substantial share of newcomers. In the future, we will be able to analyze whether those newcomers follow any of the posted links.
Help pages:
- Large numbers of users view help or policy pages on their first day: 41.5% in Czech and 27.8% in Korean. This gives us a sense of how many users are trying to find help, and we will be able to look into exactly which help pages are most popular, with an eye toward improving those and seeing if they are associated with editing success.
- 8.3% of Czech users visit a help page immediately after creating their account.
Looking for a starting place: large numbers of users view their own User or User Talk page on their first day: 33.8% in Czech and 39.3% in Korean. This is much higher than expected, especially in Czech Wikipedia, where there is no automated welcome message attracting users to those pages. We hypothesize that new users are looking for some sort of "homepage", "dashboard", or "profile page" where they can orient themselves and begin their work. This finding may guide the Growth team to attach new features to the "User" page – a natural starting place for a newcomer to a website.
Editing challenges: a majority of new users open an editor on their first day – but about a quarter of them do not go on to save an edit during that time. This quantifies an important target group for interventions like the Help Panel, which gives users help while the editor is open.

Account creation context

When users create their accounts, they go through a signup process, and then are returned back to the page they were on before. If they were editing, they end up back in the editing experience. This is where their journey as a registered user begins.

Generally, users come from, and return to, a reading context after account creation – although starting from editing is much more common on mobile. The results below are largely similar to what we found in a quick analysis of account creation context back in October 2018.

Table 1: Reading or editing context
	Czech Wikipedia		Korean Wikipedia
	N	Prop.	N	Prop.
Reading	1034	82.5%	1226	81.5%
Editing	220	17.5%	278	18.5%
Total	1254	100.0%	1504	100.0%

The overall proportions are remarkably similar across the two wikis. When we split this up by whether the account was registered on the desktop site or the mobile site, the proportions shift drastically:

Table 2: Context by mobile or desktop site registration
	Czech Wikipedia				Korean Wikipedia
	Desktop		Mobile		Desktop		Mobile
	N	Prop.	N	Prop.	N	Prop.	N	Prop.
Reading	839	90.1%	195	60.4%	998	90.6%	228	56.6%
Editing	92	9.9%	128	39.6%	103	9.4%	175	43.4%
Total	931	100.0%	323	100.0%	1101	100.0%	403	100.0%

As mentioned above, a much higher proportion of accounts from mobile are registered from the editing context, and this is consistent with our previous analysis. Though there may be many reasons for this, we believe one may be that mobile users who attempt to save an edit without being logged in are presented with a full page overlay encouraging them to create an account. Similar warnings exist on desktop, but are less obtrusive and easier to ignore.

The current analysis may be affected by the welcome survey, which makes it possible for users to navigate elsewhere in the wiki instead of returning to the context they were in before account creation. In this initial investigation we have not dug deeply into how often this happens.

Account creation context namespaces

While the previous section looks at whether a user created their account from the reading or editing context, this context exists in many different namespaces. What namespace do users create their account from (and therefore return to) after account creation, and is this different depending on whether they are viewing or editing a page?

Tables 3 and 4 show that account creation primarily happens from either an article or the Main page. There are also a fair amount of accounts created from the Wikipedia namespace (where policy pages exist), as well as Special (where search results are shown). When accounts are created from an editing context, they are dominated by the article namespace.

Table 3: Namespace of account creation context, Czech Wikipedia
	Viewing a page		Editing a page
Namespace	N	Prop.	N.	Prop
0 (Article)	407	39.4%	199	90.5%
0 (Main page)	309	29.9%
1 (Talk)	3	0.3%	5	2.3%
2 (User)	32	3.1%	9	4.1%
3 (User talk)	12	1.2%	1	0.5%
4 (Wikipedia)	83	8.0%	2	0.9%
5 (Wikipedia talk)			2	0.9%
8 (MediaWiki)
10 (Template)			1	0.5%
12 (Help)	27	2.6%
14 (Category)	2	0.2%	1	0.5%
100 (Portal)
-1 (Special)	159	15.4%

Table 4: Namespace after account creation, Korean Wikipedia
	Viewing a page		Editing a page
Namespace	N	Prop.	N	Prop.
0 (Article)	521	42.5%	258	92.8%
4 (Main page)	374	30.5%	1	0.4%
1 (Talk)	1	0.1%
2 (User)	21	1.7%	11	4.0%
3 (User talk)	38	3.1%	3	1.1%
4 (Wikipedia)	97	7.9%	2	0.7%
5 (Wikipedia talk)
8 (MediaWiki)	1	0.1%
10 (Template)	1	0.1%	2	0.7%
12 (Help)
14 (Category)	3	0.2%
100 (Portal)	7	0.6%	1	0.4%
-1 (Special)	162	13.2%

Although the "Main page" is not its own namespace, it represents so many views that we have separated it out for this analysis.
It might seem impossible to have editing the Main page on Korean Wikipedia as the account creation context, because the page cannot be edited without an account. However, the URL reflects the requested editing context, whereas the actual content shows the source (wikitext) of the Main page to the user without enabling them to edit it. Therefore, if one creates an account from that context, one will be returned to editing because the process is based on the URL.

First action after account creation

Having established the context where a new user begins their journey, the next step is to look at where they go next: their first action. Since this quick analysis only uses data from the EditorJourney schema, we do not yet identify user actions like "reading a notification", or "attempting to save an edit", as those require data from other schemas. For this analysis, we were able to look at a couple important actions that shed immediate light on what new users do.

In Czech Wikipedia, 8.3% of new users visit a page in the Help namespace as their first action after account creation. This number is higher than in Korean Wikipedia, but it is currently difficult to tell how much higher, since Korean help pages are incorporated into the Wikipedia namespace.
In Korean Wikipedia, 17.9% of new users visit a User Talk page, and these are largely visiting their own user talk page. This is likely driven by the fact that Korean Wikipedia posts a welcome message to their User Talk page of all new users. Czech Wikipedia does not have an equivalent welcome message, and accordingly, the equivalent number in Czech Wikipedia is 1.0%. This tells us that the notification banner telling new users that they have a message on their talk page immediately attracts the attention of a fair number of them. In the future, we will be able to tell whether they interact with the links in the welcome message.

What do new users do on their first day?

The EditorJourney data allows us to easily look at how many new users do which activities at some point on their first day. The table below gives an overview of how many and what share of users perform certain actions within 24 hours of creating their accounts. The total group is 1,329 registrations on Czech Wikipedia and 1,663 registrations on Korean Wikipedia.

Table 9: Actions performed during the first 24 hours
	Czech Wikipedia		Korean Wikipedia
Action	N	Prop.	N	Prop.
View the Main page	609	45.8%	746	44.9%
View an article	1,019	76.7%	1,267	76.2%
Open an editor	853	64.2%	929	55.9%
Attempt to save an edit	620	46.7%	666	40.0%
Successfully save an edit	599	45.1%	665	40.0%
View a Help or Project/Policy page	552	41.5%	463	27.8%
View an Article Talk page	34	2.6%	75	4.5%
View their own User or User Talk page	449	33.8%	654	39.3%
View somebody else's User or User Talk page	110	8.3%	186	11.2%

It comes as no surprise that a large proportion of users visit the Main page, nor that reading articles is a large draw. We're also not surprised to see that a lower share of Korean newcomers open an editor than Czech because the welcome survey showed us that a higher share of Czech users created their accounts to edit, whereas more Korean users created their accounts to read.

When it comes to editing, it is worth noting firstly that a majority of newcomers open up an editor at some point on their first day, and that a minority end up saving an edit. Therefore, the proportion of users who open the editor is much larger than the proportion of users who save an edit, suggesting there might be an opportunity to help these newcomers complete their edits. Of the Czech and Korean newcomers who open an editor, about 27% of Czechs and 29% of Koreans don't save an edit. This is the group that the Growth Team is hoping to help with the Help Panel interface, which enables users who seek help while editing to get that through the community's help desk. This result further motivates that effort.

We also see a large proportion of users reading help and/or project pages during their first day, indicating that the information the community provides is important. We will be able to identify exactly which pages are most popular, and which ones are associated with editing success. This will allow us to encourage communities to improve their most important pages.

Lastly, we see the large proportion of users visiting their own User or User Talk page, even in Czech Wikipedia, where no welcome bot posts a message to attract users to that area. We hypothesize that new users click on their username at the top of their desktop screen because they are looking for a profile page or other place to orient and begin their wiki work. This gives us a big hint on where new users are going and what they're looking for, and may help us place new features and content where it will be seen.