Iterations after the first A/B test
See Quantitative Testing section below for more details.
In July 2021, we moved the language button to a more convenient location at the top of the page. However, the results of our A/B test indicated that the new location of the button might be difficult to discover, especially in the following use cases:
- For users that are accustomed to the previous location of the language switching functionality
- For users that tend to switch languages across multiple wikis. In the latter case, this created a situation where the language switching functionality was available in different locations depending on the wiki used and whether that wikis was a an early adopter of the Desktop Improvements.
To improve on these issues, we will iterate on the implementation of the new language button with the goal of making the button easier to find across all scenarios.
List of proposed improvements:
- Providing direction to the new location of the language button from the previous location by placing an alert in the sidebar informing users where the new location of language links is
- Making the language switching functionality more noticeable by changing on the visual styling and appearance of the button
- Providing direction to the new location of the language button from the current location using guided tours that will present users with a notice on where the new location of the language links is
- Improving main pages to make it possible to present the language switching functionality in a more prominent location
These success of changes will be measured via quantitative testing across various pilot wikis.
As far as we know there is no "standard", globally recognized language icon. The icon that we use currently in Vector, Minerva, and Timeless features a Latin "A" character and a Chinese "文" character (which means "language"). On Phabricator, there is a discussion, including some history, about the icon.
After further discussion we determined that the only other strong candidate for a language icon is a globe 🌐 . We ran a simple test on usertesting.com to try and understand if the globe icon is more or less obvious as a language icon than the one we currently use (we can call it the character icon). Out of 40, multilingual participants: 35 guessed the character icon was for switching languages, and 13 guessed the globe icon was for switching languages (people were able to guess that both icons were for languages, thus the total number of guesses is more than the number of people in the test). These results are not statistically significant, however they help inform two assumptions:
- the globe icon is probably not more obvious as a language icon than the character icon
- we should use the label "languages" whenever possible in addition to the icon (since ~13% of people did not guess the character icon was related to switching languages).
Conclusion: until further testing is done we will continue using the character icon.
First A/B test
An was performed to compare the rates of languages switching for logged-in users with the new button on top of the page (the test group) with the previous location (the control group).
The analysis of the results showed that there was an average 44.57% decrease in total clicks on language links by logged-in users on the early adopter wikis in treatment group compared with control group.
The decrease is contradicted with our hypothesis.
After further evaluation, we hypothesize that the main reason for the decrease to be a lag in feature adoption, mainly among users that are accustomed to the previous location of the language switching functionality and users that tend to switch languages across multiple wikis.
In the latter case, this created a situation where the language switching functionality was available in different locations depending on the wiki used and whether that wikis was a part of the early adopter group for the Desktop Improvements project.
Based on the results of the A/B test, the team has began iterating on the feature to improve discoverability for the user groups identified above.
- Main pages don't have page headings. As a result, by default, the language selector is placed at the bottom of the page. Communities can request to place the button at the top of the page by making a Wikimedia site request. This change requires communities to display the header and make changes to their main page. See our FAQ for more information.
- Currently, the setting for the compact language list will not be functional on the new version of the vector skin, as the language functionality introduced via the button makes this setting irrelevant. We will be removing this option for the new version of the vector skin in phab:T282149.
- Coordinates: On pages where coordinates are displayed at the top of the page, the language button would look best if the coordinates are displayed as page status indicators. Please see the main page for page status indicators for instructions.
The Language Team is working on the Universal Language Selector. Once they have finished we will be replacing the current ULS with their updated one. The updated ULS will, among other improvements, feature support for language variants.