Structured Data Across Wikimedia/MediaSearch usability

Jan 2021

Research performed by Wikimedia Foundation designer Matthew Williams

Research Goals

Background

As the Structured Data team prepares to continue improving MediaSearch and potentially making it the default search on Commons, we want to gain general insight about the difference in experience between the current Special:Search and our latest work, Special:MediaSearch.

Primary Goal

The primary goal of this usability test is to complement all other data and surveys in order to make the best decision possible in regards to what should be the default search experience and how we might improve search on Commons.

Target audience

The focus of this test was to gain insight and feedback from potential users of Special:MediaSearch who are not experienced Common users.

• # of Participants: 10
• Age: Ranged from 19 to 55
• Location: 6 From the United States, 2 from the UK, 1 from France, and 1 from India.
• Platform: Desktop
• Language: English
• None of the participants were familiar with Wikimedia Commons but all use Wikipedia
• 8 out of 10 use Google images as their main image search engine (Bing and DuckDuckGo were also brought up)
• 3 out of 10 use separate sites in addition to Google for licensing reasons (Pexel, Canva, and Getty)
• Image searching needs ranged from school and work projects/presentations to general curiosity/learning

Task Description

Each participant followed the same script. This was an unmoderated test on usertesting.com

1. General questions about their image searching behavior
2. Searching for a city that they might like to visit on special:search
3. Going through 3 filter tasks for that query
4. Switching to MediaSearch and performing the same query and filter tasks
5. Comparing and rating the two experiences

Summary of Special:Search experience

1. At least 5 participants were confused by the autosuggest as it showed exact matches at times and not general terms.
2. At least 3 participants didn’t realize they had to press search again after applying a filter
3. At least 7 participants stumbled trying to understand the difference between “search in” and “advanced search”. After looking in advanced search and understanding what it was, things went smoother but this took time.
4. At least 3 participants typed things into the namespace input that didn’t belong e.g. “jpg”
5. At least 8 participants didn’t understand the active filter chips. For example, you are required to have a sort by option but it has an X to dismiss it that doesn’t work. Lots of misclicks and general misunderstanding about what was active or not and how to clear your search.
6. At least 7 participants mentioned how many clicks it took to complete the tasks
7. At least 3 participants  mentioned that this search maybe seemed good for advanced users, lots of text accompanying the images.
8. Most participants  seemed to still need to go to the file page to read the whole description and find licensing information

Summary of Special:MediaSearch experience

1. At least 9 participants mentioned that they liked the larger images
2. At least 8 participants mentioned that it was simpler, more user friendly, and clearer to use.
3. Most mentioned something positive about having the available filters and options visible and not hidden
4. Everyone completed the filter tasks much faster, in what seemed like half the time or less
5. 4 participants didn’t realize that the license information was in the quick view (not labeled clearly enough) and went to the file page to find it
6. There was a bug on the Video tab where no results were showing and this was frustrating or confusing to most users.
7. Two of the participants wondered if we could put more information below the images, and this might be the only thing they preferred in option 1.

Relevant Quotes from participants

• “Option #2 (Special:MediaSearch) is easier for me because overall it was a very simple and fast to use site. I was able to click on things without hitting the search button...less buttons to press in total. I could see all the options easily without clicking an extra button”
• “In Option #1 (Special:Search) it was frustrating to find all of the advanced search options, I wouldn’t have expected it there but figured it out after awhile.”
• “I liked the second experience (MediaSearch) much better. The pictures were laid out and easy to see. The filters were much more accessible”
• “This is quite good, well done. I’m going to start using this, this is brilliant and I am liking this. It is way simpler and easier to find file types, way more straightforward” (Referring to MediaSearch)
• “Option #2 (MediaSearch) felt much easier to operate and it is comfortable to use. It was much simpler in style. Option 1 (special:search) was more complicated.”
• “Much better format, less space being wasted (Referring to MediaSearch) Option 1 (special:search) was harder to look at but you saw relevant information up front”
• “Option 2 (MediaSearch) was easier to use. I think I preferred the layout of the images all there and seeing all the filters more visible.”
• “Option 2 (MediaSearch) is far better from option 1 as it easy to use and perform the task very easily on a single click”
• “Option 2 (MediaSearch) is much clearer, I prefer this version because the images are bigger. There were so many details (in option 1), it was hard to see.”

• “Option 2 (MediaSearch) is much more familiar to me”

• “No mumbo jumbo next to the images” (Referring to MediaSearch)

Averaged Ratings

Participants were asked to rate both Special:Search and Special:MediaSearch from 1 (being the worst) to 5 (being the best)

The averaged ratings came out to:

Special:Search  3.5/5

Special:MediaSearch   4.5/5

Conclusion

Overall, these 10 participants preferred Special:MediaSearch to Special:Search. The most talked about advantages to MediaSearch included larger images and easier to use filters. The words “easier”, “simpler”, “cleaner” were a constant in referring to MediaSearch by nearly all participants.

Like we’ve seen in other surveys and feedback, the only issues brought up with MediaSearch were about not showing more information up front.

After an initial steep learning curve, misclicks, and confusion Special:Search seemed to do its job and generally towards the end of testing most reached a “it’s fine” attitude about it. 5-10 UI changes could eliminate a good percentage of confusion but at its core the busy-ness and domain specific concepts seemed to trip people up.

If we want to cater at all towards the huge percentage of users that end up on Commons that aren’t familiar with Commons, MediaSearch is an improvement in making them feel comfortable and confident in their searches and interactions.