Usability testing citations dialogue on VE, brand new editors (May 2015)

Goal

Goal of this research is to test the usability of the iterated citations dialogue before we start A/B testing Visual Editor. Once testing is done, we will share results here.

Findings

All tasks tested in this usability study passed the test.

✓ Creating a new subsection 7 of 10 succeed

✓ Adding a citation 9 of 10 succeed

✓ Editing a citation 9 of 10 succeed

✓ Adding a second citation 8 of 10 succeed

✓ Finalizing/saving edits 6 of 10 succeed

✓ Understanding ‘automatic’, ‘manual’, and ‘re-use’ cite tab 10 of 10 succeed

Citations: Observations

Sometimes people think all you have to do to commit a citation is to press “generate”. However, most of them see “insert” and are able to actually insert the citation. Everyone eventually figured this out, but at first there was some confusion. A few people thought they would be able to edit the title of the citation before they have inserted the citation.

Saving Edits / Awareness of need to save

Awareness of the save button is low for brand new editors. Because VE is a WYSIWYG editor, as people edit, they think once they edit, changes are automatically saved (or committed). So, pressing the save button is not something they think to do. More attention needs to be brought to save button for newbies.

Recommendation: Iterate a way to have the loading progress bar point to the save button. Is it possible bring up a ghost of the tool bar before all tools load and have the progress bar load at the bottom of that and point to “save”? Point is, we need to bring awareness to the save button (and the need to save) for new editors.

New User Education

As observed in several rounds of testing (both usability and discovery) brand new and casual editors tend not to understand citations and internal and external links, and how and why to use them.

There are a few ideas for solutions to provide new editors a way to gain clarity on the above mentioned confusion. This is an important effort for us to attend to in the interest of supporting new editors being productive and confident in their creation of links, and citations.

Method

Usability testing with new editors.

Raw notes can be found here

Report in a google doc can be found here.

Links to videos on Usertesting.com:

These 4 participants were tested with a slight change in protocol, using the term ‘reference’ instead of ‘citation’ in the tasks related to the cite tool.