Reading/Web/Desktop Improvements/Features/Collapsible sidebar/diq
We would like to give readers and editors the flexibility to collapse and uncollapse the left menu (the sidebar). They will gain a more focused view of the content. At the same time, the functionality will be available when needed.
By this functionality, we would like to pursue two of the main goals of our project. First, we would like to make it easier for people to focus on the content itself, whether during reading or editing. The other of these goals is to highlight important functionality such as the edit and history buttons, language switching, and search.
Feature description and requirements
The collapsible sidebar feature will introduce a button in the top left corner of the screen. Selecting this button will collapse the sidebar and hide the items within it. To open - press the button again.
If logged-in, your sidebar collapsing preference will be saved. This means that if you chose to have the sidebar collapsed, and then end your session, your sidebar will still be collapsed when you start a new session, and vice-versa.
For logged-out users, the preference will only be saved within a given browser window.
Design requirements and guidelines
We have prepared a prototype of this functionality.
-
The sidebar of Wikidata in French: uncollapsed and collapsed -
Visualisation of the collapsed sidebar on Farsi Wikipedia -
Visualisation of the collapsed sidebar on Basque Wikipedia
User testing
With editors
We performed a feedback round with a prototype of the collapsible sidebar with editors across multiple wikis. Editors were invited to explore the prototype and provide their feedback using a central notice banner. The majority of users who provided feedback liked the collapsed sidebar for personal use, and especially for the purposes of reading. Please read through our report for more details on feedback, concerns raised, and iterations.
With readers
A prototype of the sidebar was also tested with readers through a user study done in collaboration with an independent user testing firm, Hureo, based in India. During the course of the study, we studied readers unfamiliar with Wikimedia projects, as well as casual readers who use Wikimedia projects regularly. The tests were performed in English, Hindi, Telugu, Tamil and Marathi.
Readers reported a strong preference towards the collapsed version of the menu, stating that it created a better reading experience than the current version.
We will be publishing the full results of this study soon.
Quantitative testing
Since making the change, we were able to observe the patterns of behavior of anonymous and logged-in users while using the sidebar. This report represents our findings so far. We hope to use this data in future iterations of the collapsible sidebar functionality.
Study questions
When measuring the usage of the sidebar, we wanted to consider the following set of questions:
- What is the frequency that users collapse and uncollapse the sidebar
- What is the difference in interaction with sidebar links between people with the sidebar collapsed (the majority of the time) versus uncollapsed (the majority of the time). In other words, once the sidebar is collapsed, do users open it when they need to, or do they just ignore it altogether?
- What is the overall difference in interaction with sidebar links before and after the change
- In particular, what is the overall difference in interaction with the donate link?
Results
What is the frequency that users collapse and uncollapse the sidebar?
- Between July 22 and August 31, 2020, there were 28,745 unique sessions that included a click to collapse and/or uncollapse the sidebar. This represents only about 0.13% of all recorded sessions using the Vector 2022 skin and 13.67% of all Vector 2022 sessions with a click recorded.
- Since the sidebar was set as uncollapsed by default, there are more total clicks to collapse the sidebar. However, there are roughly the same average number of clicks per session to either collapse or uncollapse. Average Clicks Per Session to Collapse: 1.37; Average Clicks Per Session to Uncollapse: 1.38.
- French Wiktionary had the lowest average sidebar clicks per session (1.87) while Persian Wikipedia had the highest (2.94).
- Logged-out users have more than twice the number of collapse events compared to uncollapse events, likely because the state of the sidebar for logged out users is not preserved.
- Most sessions overall included just one-click to either collapse or uncollapse the sidebar. Logged out users have a higher frequency of one sidebar clicks per session while logged in users have a higher frequency of two clicks per session.
What are the different interaction types with the sidebar?
- For the majority of sessions (81.88%), the sidebar was collapsed and/or uncollapsed just once. For sessions where the sidebar was collapsed only once, 34.5% sessions kept the sidebar collapsed the majority of their session time and 65.4% of sessions had the sidebar uncollapsed the majority of their session time (did not collapse it until towards the end of their session).
| Session Behavior | Num_sessions | Percent of sessions with click to collapse and/or uncollapse |
|---|---|---|
| Collapse but don't uncollapse | 13475 | 46.91% |
| Uncollapse but don't collapse | 1043 | 3.63% |
| Collapse once and then uncollapse once[1] | 242 | 0.84% |
| Uncollapse once and then collapse once[1] | 26 | 0.09% |
| Both Collapse and Uncollapse under 1 second | 8733 | 30.40% |
| Collapse and/or Uncollapse Multiple times within a session | 5204 | 18.12% |
What is the overall difference in interaction with sidebar links before and after the change?
Note re approach:
Since this was deployed as opt-out to all users on the test wikis, there is not a random, equal split of users between the two Vector skins. A direct comparison cannot be made as there may be some bias. The old skin version was only available to those who explicitly opted out. There were more users who saw the new version. However, we compared the average interactions per session for each skin type. This was to control for the different number of users and confirm if there were any significant differences.
Note: Data in these comparisons may still be affected by self-selection bias between the two groups. We also reviewed data from a two-week period before and after each test wiki's deployment date to confirm if there were any significant changes.
- The Legacy Vector has a slightly higher number of total sidebar link clicks per session (4.68) compared to Vector 2022 (3.63). This is also true on a per wiki basis except for on French Wiktionary and Portuguese Wikiversity, which have a slightly higher average number of total sidebar link clicks on the new skin compared to Legacy
- Clicks to the donate link do not appear to be affected by the new skin version. There is a slightly higher number of average clicks per session on the Vector 2022 (1.11) compared to Vector 2010 (1.06).
- There was a decrease (-9.15%) in the overall total number of sidebar link interactions comparing the two weeks prior and two week following the deployment of the collapsible sidebar across all test wikis. However, this percentage is likely inflated due to seasonality effects (previous year's data was not available using webrequest to complete a year over year analysis).
Full report
The full report is available in this Jupyter Notebook.
Discussion and next steps
Based on the results above, we have learned that logged-in and logged-out users have very different behavior while using the sidebar. Logged-in users tended to, overall, collapse the sidebar and leave it collapsed. As we understand it, not many of the items in the sidebar were useful to them. However, the rate of uncollapsing also indicated that users are aware that, were they to need to navigate to an item in the sidebar, that option was available to them.
Logged-out users, on the other hand, tended to collapse and uncollapse as needed. This indicates that they had a greater need of the items in the sidebar and that their behavior was more task-based, rather than indicating an overall pattern.
Based on the above findings, we took the following actions:
- Making the sidebar collapsed by default for logged-out users
- Keeping the sidebar uncollapsed by default for logged-in users
We decided to continue to monitor the above patterns, in particular the decreases in clicks to links in the sidebar. We will work on it again based on future findings.
Release plan
We began deploying the first iteration in May 2020 to Office Wiki and Test Wiki, and plan to continue to our early adopter wikis in following months. See our main Features page for more details.