Reading/Web/Desktop Improvements/Updates/2023-05 Zebra9 A/B test

Overview

Features

Repository

FAQ

An A/B test coming soon. Visual separation (Zebra #9) between regions

Hi everyone,

In a few weeks (as of early May 2023), we'll be performing a test. Half of logged-in desktop users will see a version that shows the regions of the page separated more clearly. This version will include more borders and gray backgrounds (preview it here). After two weeks, everyone will see the current white interface again. We will analyze the usage data for both and make a decision on next steps in July.

Feedback about the white space

Currently, Vector 2022 uses whitespace to separate the various regions of the interface. It also has a white page background (outside of the content area). We have received a lot of feedback about this. There are two areas of concern:

1. Comments about adding more visual separation between regions of the interface. For example, a boundary around both the content area and the side menus. Volunteers have been wondering if this would improve the reading experience. Mainly by:
   • Drawing more focus/attention to the content when landing on the page (rather than taking in the interface as a whole first, then focusing on the content).
   • Making it easier to stay focused on the content while reading, because the content area would be more well defined.
2. Comments about adding a gray background outside of the content area. Volunteers have been wondering if this would reduce eye strain some people are experiencing from the large white area (i.e. “glare”) on larger screens.
   • Many of you have reported that the white space is too high in contrast and that adding a darker hue such as a gray decreases this strain.
   • This may also add further focus to the content area and the table of contents.

About the test itself

To verify these hypotheses, we have set up user testing with readers. Here you can read more about the test as well as learn about the outcomes.

We also want to ensure that the new layout does not negatively affect any high-level metrics for the project. This is why the A/B test will take place. We will be looking for no statistically significant decreases in the following: pageviews, opt-out rates, edit rate, Table of Contents usage, scrolling, and page tools usage. Due to privacy and technical restrictions, we are only able to run this test with logged-in users. We will filter the results by the number of edits a given user has. Then, we will proxy logged-out users via logged-in users with 0 edits.

This test will start within two weeks and will take two full weeks. It will be performed on Wikipedias in: English, Farsi, French, Hebrew, Korean, Portuguese, Serbian, Thai, Turkish, and Vietnamese.

After two weeks, everyone will see the basic (white) version again. Next, we will check if the new layout has negative effects on those metrics. If it doesn't, then we will enable it across all users and all wikis. It won't happen before July, though.

Thank you!