User:Etonkovidova (WMF)/GrowthExperiments

Documentation

General info/documentation is here - Growth team projects.

Growth team projects (GrowthExperiments) are run on target wikis and the projects aim to help new users to start editing. Below is the list of participating wikis that also exist in betalabs:

Betalabs: cswiki, bnwiki, arwiki, viwiki

Production: cswiki, bnwikii, arwiki, viwiki, testwiki

General testing notes

Start testing with creating a new user account on participating wikis (see the list above, or use the updated list on - Growth team projects.

For a new account on participating wikis, GrowthExperiments projects - Welcome survey/Homepage/Help panel - might be enabled by default. However, due to variant testing (which gets adjusted based on the data analysis) the features might not be enabled by default, in this case they might be enabled manually (see below in related sections) which is also useful when testing is done for existing user accounts.

Refer to QA column on phabricator Growth Team Current sprint workboard to see which phabricator tasks need QA verification.

Pre-deployment testing

- the testing is done on betalabs env

- review QA column on Growth Team Current sprint workboard to finish QA verification before features would be deployed

Post-deployment testing

- check the GrowthExperiments features functionality on testwiki (posting Help panel questions, submitting edits can be done there)

- check logstash for errors in the following extensions- GrowthExperiments, Flow, Echo (these extensions are supported by Growth team).

Features testing

Welcome survey

Welcome survey documentation

- create a new account

- for an existing account, add Special:WelcomeSurvey?_group=exp2_target_specialpage to url to see Welcome survey UI

Homepage

Growth - Newcomer homepage

- new users should be redirected to automatically enabled Special:Homepage after Welcome Survey

- for existing accounts: enable Special:Homepage from Special:Preferences in User profile tab -> Newcomer homepage option

- enable Help panel from Special:Preferences in Editing tab -> Enable the editor help panel option

Check the presence and functionality of the following modules on Special:Homepage

Intro module
Mentorship module
Help module
Impact module
Suggested Edits (SE) module

Help panel & Guidance panel

Help panel

Growth - Help panel

New users accounts should have enabled Help panel when they click on Edit; Help panel is present in reading mode in some namespaces. For existing accounts enable the option “Enable the editor help panel” on Special:Preferences-Editing tab

On participating wikis, visit User talk or “User page - the Help panel should be present
Visit any article, click on Edit - the Help panel should be present

You can post test questions and check the whole functionality of Help panel on betalabs wikis and testwiki.

Guidance panel

Guidance panel appear for users who click on suggested article in Suggested edit module on Special:Homepage.

- Guidance panel is present in reading mode on suggested edit articles

- Guidance panel has post-edit dialog box for users who publish their edits.

Add link structured task

Documentation: Growth/Personalized first day/Structured tasks/Add a link

Mockups: Desktop - [1], Mobile - [2]

Test environment: Presently the testing can be done on beta cluster - cswiki on beta cluster. In production the feature is displayed on testwiki.

For enabling the Homepage - refer to Homepage section.

Instrumentation testing

Several schemas are associated with Newcomer experience and Homepage features - HomepageModule, HelpPanel , EditorJourney, and EditAttemptStep. The following steps are applicable to all of the schemas checking (substitute the required schema's name). For a general QA practices for instrumentation testing, refer to Testing Instrumentation.

Before testing

- read about event logging and possible updates/changed access etc - Analytics/Systems/EventLogging/TestingOnBetaCluster

- read a schema description (e.g. Schema:HomepageModule) to understand what actions to perform to trigger eventlogging events

- create a test outline (several user workflows) to make sure that all events from a schema would be recorded

In a browser (client side testing)

The testing with the console snippets can be done in beta cluster and in production.

- enable your browser's devtools and enable "persistent log" in the Console

- use the following snippet (credit to User:KHarlan (WMF)

mw.trackSubscribe('event.HomepageModule', console.log);

- perform the actions that trigger eventlogging and compare with the info documented in the relevant schema description

Server side testing

The instructions are for beta labs cluster only.

- log in and go to the directory with logs:

ssh deployment-eventlog08.deployment-prep.eqiad1.wikimedia.cloud

cd /srv/log/eventlogging

- to verify that events have been properly logged you may use grep or any other means of your choice. For example, to see which editor_interface were recorded, you may use:

grep '"schema": "HelpPanel"' all-events.log |grep -oE ".editor_interface.{0,17}"

Feature flag testing

Growth team explores new approaches to engage new users in editing. Often features deployed either as A/B testing (or testing of groups of users in differently proportional segments) or the features have a feature flag to test them in production environment before turning on for everyone (see Growth team page with the list of wikis where Growth experiments have been deployed). Using Javascript snippets (URL parameters) in the a browser Console might help to optimize testing. The following cases illustrate when either javascript snippets or URL parameters could be used :

- different variants of features (e.g. instead of creating new users every time you need to take a look at different UI layouts)

- re-test features that are presented only one one-time (or limited times) to new users (e.g. Welcome survey)

Please refer to Growth team projects page and to the Growth team for current projects testing needs.