Wikimedia Research/Design Research/Contributors Team UX Research/2018.02-03 Compact Language Links Testing

Abstract/Background

edit

The Language team is looking to, after some time in beta, activate the compact language links feature as a default on English Wikipedia. Before doing so, I held research sessions with a couple of Wikipedians who use visual assistive technologies to address any concerns, identify possible objections, and glean any suggestions for improvements with regard to this feature.

As of February 28, 2018, this feature is now live/default on English Wikipedia.

Research Questions

edit
  • Are users aware of the list of languages?
  • Can users find articles in other language wikis?
    • Are the initially-hidden languages "visible" to the accessibility software?
    • Is the 'more' languages button visible to their accessibility software (like JAWS), and can it be pressed/accessed?
    • Is the search box accessible?
  • Do screen readers output foreign language names in a useful way?
  • Is the feature accessible without a mouse?

Methodology

edit
  • Youtube Live / Google Hangout on Air sessions
  • Test environments
    • English wikipedia (logged out) article pages for the full languages list
    • English wikipedia (logged into test account with the compact language links beta feature activated) article pages for the compact language links view

Participants

edit

Wikipedians using visual assistive technologies.

Protocol

edit

Screener

edit
  • How long have you been editing on Wikipedia or other wikiproject site? (anything is acceptable as response)
  • How many languages do you speak? (anything is acceptable)
  • Do you use visual assistive technologies in your work on the wikis and/or browsing the internet? (must be a yes answer)
  • Have you used the compact language links feature before on the wikis?

Tasks

edit
edit
  1. Before we get started, please tell us what kind of assistive technology you’re using.
  2. Alright! First, make sure you’re not logged in. Then, go to the article for ‘Shiba Inu’ on en.wikipedia.org
  3. Please find this version of the article in Japanese, and please describe in detail your process of doing so (your thoughts, what your assistive technology is telling/showing you, your actions).
  4. What do you think of the process?
edit
  1. Now, log in to the en.wikipedia.org user testing account with the credentials username: […] password: […]
  2. Go ahead and navigate to the ‘Wombat’ article on en.wiki.
  3. Once you’re at the article page, can you find this same article page in Polish? Again, please describe this process in detail as you’re going through it.
  4. How was that experience?
  5. Did you notice any similarities or differences in your experience finding the article in another language while logged in to the test account?
edit
  1. Now, let’s talk about the bottom left portion of the page. Have you used this ‘Languages’ section before?
  2. How do you feel about this section of the page?
  3. Why do you think these languages are the ones displayed?
  4. Let’s get specific about the two tasks you completed earlier.
    1. When searching for ‘Wombat’ in Polish, did your assistive technology indicate to you how you might find this?
    2. Did the software make you aware to the search functionality? Did you use it, and if so, what did you think
    3. How did your software represent this list of languages to you, the user, during the two separate tasks?
  5. Imagine if you were to complete these tasks without a mouse or trackpad. How would that experience be?
edit
  1. Now, navigate to the article page for ‘Dog’ on en.wiki.
  2. Imagine you’re fluent in Esperanto and would like to access articles in Esperanto more easily. How would you change this list shown here so that it includes Esperanto?
  3. Now, go to the article page for ‘Cattle’ on en.wiki.
  4. From here, how would you navigate to the ‘Cattle’ page in Esperanto?
  5. Again, talk through your thoughts on the ‘Languages’ section.

Final Questions

edit
  1. How do you feel about the ‘Languages’ section? Likes? Dislikes?
  2. Give us any feedback you'd like of your experiences with the section today, and the different article language search tasks you completed.
  3. Is there anything you’d like to do with regard to the section that you currently cannot?
  4. Any other comments or suggestions on what you experienced today?

Findings

edit

Participants

edit

Users both spoke 2+ languages and used JAWS (job access with speech) or NVDA (non-visual desktop access) to browse the web.

edit
  • Users use different synthesizers which can process different sets of languages; however, the languages that are not recognized by a particular synthesizer is expressed in unicode
  • A standardization would be to have a romanized version of each language next to each entry, but this could become problematic if this standardization is implemented across all language wikis
  • The two participants in the sessions were still able to identify target languages based on their understanding of the section (alphabetized) and where languages are typically located in context; e.g. ‘nihongo’ (japanese) is located between ‘neder’ (dutch) and ‘norsk’ (norwegian)
    • It is unclear how apparent this would be for a newer/less-experienced user
    • It is also important that the user is not only aware of how the language links area is laid out in this particular view, but is knowledgeable about how to use the assistive technology. Participant 2, for example, expected to search for japanese and be taken to that entry, but it was not a ‘listed’ language based on what the technology could find on that page. Depending on the participant’s awareness, an assumption could be that the article was not available in japanese or that it existed but needed to be found manually and based on an understanding of where japanese ‘nihongo’ was located in context of other languages in the list (because the synthesizer used registered japanese characters as unicode).
edit
  • Both participants were able to process the compact language list with 9 listed, and the 51 more languages button.
    • No significant issues accessing the search languages box and finding a desired language there.
  • Both indicated that the search box ‘solves’ the unicode problem; even though the language list ‘context’ is gone, the desired language can be searched in a few different ways and provides the desired result in test cases.
edit
  • Users’ assumptions about how the compact languages are generated are essentially correct; combination of more common languages and location-based language associations. Other guesses are what language categories with which users self-identify and userboxes indicating language proficiency on userpages.
  • Both users mentioned that their assistive technologies made them aware to the compact language links search box and that neither needed more than a keyboard to navigate the feature.
edit
  • Both users aren’t clear on how to add a language to the compact languages list. They both try the cog icon on the languages section, assume it is somewhere in preference, and similar.
  • One user indicates that changing the prefix in the url is an oft-used method in lieu of using language list.

Overall

edit
  • Both users have generally positive reactions to the feature, and both react positively to the ‘more languages’ button and search feature.
  • One user finds the feature useful and another is simply used to the existing interface and indicates he ‘probably wouldn’t use [compact language links]’.

Recommendations

edit
  • Adding a language to the compact list by selecting/searching a language may not directly translate to a desire to add that language to the list. A suggestion is made to base that action on a usage statistic (add to list after accessed a certain number of times).
  • The process of editing the list itself is also generally not intuitive. An alternative is to allow the user to personalize and edit the language list directly within a hover menu (preferably one integrated into the existing language section cog icon menu).
  • In the meantime, a tooltip for how to turn the feature on and off should come up in case users would prefer the full language list.
    • A tooltip to explain how to curate the compact languages would be useful as well; if changes are made to that process, update tooltip at that time.
  • Adding more ‘landmarks’ throughout wiki pages. This will allow users to more efficiently navigate through page sections and elements without keying through each one individually. An example of this would be jumping from search box directly to the first search result on the search page, instead of having to key through ‘content pages’, ‘multimedia’, etc.
  • If there are other similar lists on Wikipedia, we should aim to provide a compact version of those as well, due to the added accuracy of the search function and the fact that it minimizes cluttering on the interface.
  • If technically possible, increase users’ awareness and visual assistive technology’s find-on-page capabilities by providing additional language reference. This would mean that, as examples on english wikipedia, one could search ‘deutsch’ or ‘german’ to reliably find that entry on-page. For those without the appropriate synthesizer, searching a romanized ‘nihongo’ or ‘japanese’ would find that entry even if the synthesizer can’t interpret the characters or scripts. This would provide users who prefer the complete language list view with more information and ease-of-use.