通过此功能，我们希望实现项目的两个主要目标。 首先，我们希望使人们在阅读或编辑期间更轻松地专注于内容本身。 其次便是强调重要功能，例如编辑和历史记录按钮、语言切换和搜索。
We performed a feedback round with a prototype of the collapsible sidebar with editors across multiple wikis. Editors were invited to explore the prototype and provide their feedback using a central notice banner. The majority of users who provided feedback liked the collapsed sidebar for personal use, and especially for the purposes of reading. Please read through our report for more details on feedback, concerns raised, and iterations.
A prototype of the sidebar was also tested with readers through a user study done in collaboration with an independent user testing firm, Hureo, based in India. During the course of the study, we studied readers unfamiliar with Wikimedia projects, as well as casual readers who use Wikimedia projects regularly. The tests were performed in English, Hindi, Telugu, Tamil and Marathi.
Readers reported a strong preference towards the collapsed version of the menu, stating that it created a better reading experience than the current version.
We will be publishing the full results of this study soon.
Since making the change, we were able to observe the patterns of behavior of anonymous and logged-in users while using the sidebar.
This report represents our findings so far.
We hope to use this data in future iterations of the collapsible sidebar functionality.
When measuring the usage of the sidebar, we wanted to consider the following set of questions:
- What is the frequency that users collapse and uncollapse the sidebar
- What is the difference in interaction with sidebar links between people with the sidebar collapsed (the majority of the time) versus uncollapsed (the majority of the time). In other words, once the sidebar is collapsed, do users open it when they need to, or do they just ignore it altogether?
- What is the overall difference in interaction with sidebar links before and after the change
- In particular, what is the overall difference in interaction with the donate link?
- Between July 22 and August 31, 2020, there were 28,745 unique sessions that included a click to collapse and/or uncollapse the sidebar. This represents only about 0.13% of all recorded sessions using the Vector 2022 skin and 13.67% of all Vector 2022 sessions with a click recorded.
- Since the sidebar was set as uncollapsed by default, there are more total clicks to collapse the sidebar. However, there are roughly the same average number of clicks per session to either collapse or uncollapse. Average Clicks Per Session to Collapse: 1.37; Average Clicks Per Session to Uncollapse: 1.38.
- French Wiktionary had the lowest average sidebar clicks per session (1.87) while Persian Wikipedia had the highest (2.94).
- Logged-out users have more than twice the number of collapse events compared to uncollapse events, likely because the state of the sidebar for logged out users is not preserved.
- Most sessions overall included just one-click to either collapse or uncollapse the sidebar. Logged out users have a higher frequency of one sidebar clicks per session while logged in users have a higher frequency of two clicks per session.
- For the majority of sessions (81.88%), the sidebar was collapsed and/or uncollapsed just once. For sessions where the sidebar was collapsed only once, 34.5% sessions kept the sidebar collapsed the majority of their session time and 65.4% of sessions had the sidebar uncollapsed the majority of their session time (did not collapse it until towards the end of their session).
|Session Behavior||Num_sessions||Percent of sessions with click to collapse and/or uncollapse|
|Collapse but don't uncollapse||13475||46.91%|
|Uncollapse but don't collapse||1043||3.63%|
|Collapse once and then uncollapse once||242||0.84%|
|Uncollapse once and then collapse once||26||0.09%|
|Both Collapse and Uncollapse under 1 second||8733||30.40%|
|Collapse and/or Uncollapse Multiple times within a session||5204||18.12%|
- Clicks were more than than 1 second apart
Note re approach:
Since this was deployed as opt-out to all users on the test wikis, there is not a random, equal split of users between the two Vector skins.
A direct comparison cannot be made as there may be some bias. The old skin version was only available to those who explicitly opted out. There were more users who saw the new version. However, we compared the average interactions per session for each skin type. This was to control for the different number of users and confirm if there were any significant differences.
Note: Data in these comparisons may still be affected by self-selection bias between the two groups. We also reviewed data from a two-week period before and after each test wiki's deployment date to confirm if there were any significant changes.
- The Legacy Vector has a slightly higher number of total sidebar link clicks per session (4.68) compared to Vector 2022 (3.63). This is also true on a per wiki basis except for on French Wiktionary and Portuguese Wikiversity, which have a slightly higher average number of total sidebar link clicks on the new skin compared to Legacy
- Clicks to the donate link do not appear to be affected by the new skin version. There is a slightly higher number of average clicks per session on the Vector 2022 (1.11) compared to Vector 2010 (1.06).
- There was a decrease (-9.15%) in the overall total number of sidebar link interactions comparing the two weeks prior and two week following the deployment of the collapsible sidebar across all test wikis. However, this percentage is likely inflated due to seasonality effects (previous year's data was not available using webrequest to complete a year over year analysis).
The full report is available in this Jupyter Notebook.
Discussion and next steps
Based on the results above, we have learned that logged-in and logged-out users have very different behavior while using the sidebar. Logged-in users tended to, overall, collapse the sidebar and leave it collapsed. As we understand it, not many of the items in the sidebar were useful to them.
However, the rate of uncollapsing also indicated that users are aware that, were they to need to navigate to an item in the sidebar, that option was available to them.
Logged-out users, on the other hand, tended to collapse and uncollapse as needed.
This indicates that they had a greater need of the items in the sidebar and that their behavior was more task-based, rather than indicating an overall pattern.
Based on the above findings, we took the following actions:
- Making the sidebar collapsed by default for logged-out users
- Keeping the sidebar uncollapsed by default for logged-in users
We decided to continue to monitor the above patterns, in particular the decreases in clicks to links in the sidebar.
We will work on it again based on future findings.