Flow/Research
This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on any information on this page. |
- See more recent research at Flow/Moderated Testing, November, 2014: talk pages and Flow.
Research on talk discussions has been a long-running process.
New Users edit
Flow prototype testing edit
- Flow UX workshop, October 15, 2013
- Usability testing, November 23, 2013
Test Videos edit
The Foundation conducted several user tests to get a glimpse into the problems facing new users when they are exposed to what are relatively simple (and common) scenarios with user talk pages.
Tests have been annotated, calling out specific issues that users face and behaviors that they expect.
Raw data and links to view individual tests may be found at User Test Data.
Takeaways edit
- None of the tested users were able to intuitively grasp anything about the use of User_talk pages.
- On average, it takes new users around 15 minutes to grasp the basics of user talk messaging.
- Users generally have negative impressions of using wikitext for communication ("it's like math"). It is extensively confusing and somewhat demoralizing.
- Users have expectations that the discussion system will be significantly more modern and be much more user friendly. To wit:
- Users have difficulty locating their new messages and talk pages.
- Signatures and colon-indentation are especially confusing.
- Users expect an obvious mechanism for replying to comments. They do not understand section edit links.
- Users do not generally understand how to watch for replies (or if this will even happen).
- Users have an expectation that those they engage with will be notified in a sane manner.
- Users expect a way to leave messages for other users from User pages (not just User_talk).
- Talkback templates are completely confusing and non-intuitive.
- Users are surprised that they can edit other people's messages.
- Users have difficulty understanding that conversations can exist in multiple locations.
Note that the scenarios that these users are sent through - and the talk pages they end up on - are typical experiences for new users.
Experienced Users edit
Feedback during first release edit
Below are charts showing the sentiment of feedback in various feature areas that users gave during the early "sandbox" release period. This feedback was gathered from Flow-enabled pages on the ee-flow test wiki and mediawiki.org during November-December 2013; it was qualitatively hand-coded for feature area and sentiment.
Note: this is a very small sample (only 28 unique users total gave feedback during this period), so the results should not be considered representative of all experienced users.
Quora Question edit
User:Jorm (WMF) asked on Quora: What do you find confusing about Wikipedia's discussion systems?. The full text of the answers can be found at Experienced user responses.
What do you find confusing about Wikipedia's discussion systems? edit
Not everyone is aware that there actually *is* a discussion system on Wikipedia. There are two basic types: discussions about articles and then per-user discussion spaces (these are called "Talk pages").
I am currently working on a design proposal to build a better discussion system. I know what I think is confusing about Talk pages (there's a big list) but I want to hear what other people think, too, so that I can fold issues I may have missed into the design.
Experienced User Suggestions edit
These are suggestions from experienced users for improving the user-to-user discussion system. This list is by no means exhaustive, and is the result of a limited set of answers.
Bugs/Issues edit
- Too much wikitext:
- Signatures should be automatic
- Indenting is tedious
- Managing these errors by inexperienced users adds workload to experienced ones
- Wikitext is "too free"
- The label "talk" is counterintuitive.
- It doesn't stand out enough that there is a conversation space.
- No way to watch specific threads; must watch entire page.
- Archiving is completely broken in many ways.
- Lack of permalinks is especially bad
- Context and history in conversations is typically lost
- Where do replies go? My talk page or someone elses?
- Lack of notification when someone mentions you in a discussion or replies to a discussion you are involved in
- Timestamps can be difficult to follow or clutterish
- Thread/topic management is an issue.
- To easy to distort conversations through refactoring
- Talkback templates/notices add clutter
- On-wiki processes (such as suggesting merges or deletions) are complex and difficult to follow
Feature Requests edit
- A desire for context/annotations
- Namechecking/mentions (@Username)
- Better way of linking to diffs
- Native support for quoting
- Profile photos or avatars. Discussions can be hard to follow and lack empathy
- A way to "upvote" comments or indicate simple gratitude or support
Past research on user talk edit
"Ignored" new users edit
Since 2006, new users who are "ignored" (e.g., do not receive any talk page messages) are more likely to keep editing than users who do receive a talk page message. This suggests that the current way we communicate to new users is doing more harm than good.
However, for new users whose edits are reverted, not sending a warning is much more likely to make the user go away forever than sending a template warning. 41% of new users will continue editing after being sent a template warning for a test edit, versus 9% who will continue editing if reverted and sent no warning.
First messages to new users edit
Since 2006, English Wikipedia has seen an explosion of semi-automated tools and bots that send welcome and warning messages to new users. By 2011, 80% of first messages to new users were templates sent by a tool or a bot, and 65% were warning templates.
The impact of user talk templates on new users edit
About 50% of registered and unregistered users check their talk pages within 24 hours of receiving a message. The majority of users whose edits are reverted and who receive a warning never return to edit again. Changing the tone of template warnings has a small but significant effect on short-term retention, but not long-term retention.
New user help requests on user talk edit
Based on a sample of new user edits from 2011, the majority of new editors ask for help on user talk pages – their own or another editor's.