VisualEditor/Portal/Why/User Test Data
As part of our preparations for the launch we ran a group of users from UserTesting.com through a set of tasks, first using the VisualEditor and then using Wikimarkup. Some of them had tried to edit in the past; many were total newcomers. The script for the tests is available (in English).
Using the source editor
edit"This automatically looks confusing and overwhelming."
"There was tons of foreign jargon amidst the content and it looked nothing like how the real page looked."
"I don't want to have to learn a new language just to edit Wikipedia."
When presented with the source editor, users tended to have the same set of problems. Many of these centred around identifying what they were expected to change; with so much markup, they found it difficult to identify things in the markup view that matched what they'd seen when reading the rendered page. Users were also worried by the clutter of the editing interface, particularly the mass of buttons at the bottom of the "save page" window.
Users struggled to understand the wikimarkup found in a moderately-sized article; when they managed to identify bits, it was almost entirely from comparing their memory of the rendered page to individual words, and looking at the formatting around those words (for example, noting that all of the headers had equals signs, and thus determining that equals signs made headers work). With one exception, every user found the source editor intimidating and would opt not to use it.
Using VisualEditor
edit"I've given up in the past because it seemed too confusing. With [VisualEditor] it seemed like anyone can figure it out."
"[VisualEditor] feels more like editing a word document and isn't as intimidating as the [source editor] which feels more like editing code."
Several problems were raised with VisualEditor. Many users found adding links to be confusing, something we have noted and are evaluating, and (as known) VisualEditor was slow to load for some testers.
Several other (now fixed) bugs, such as problems with saving the page, also frustrated users. However, all-but-one of the testers concluded that they preferred using VisualEditor to using the source editor, one of them noting that "[with VisualEditor] I would be more likely to make edits. That interface was a lot easier to understand and I had more confidence that the changes I was making were the changes that I wanted to make. I also like that I had an opportunity to review the changes and note them."