Reading/Web/Projects/Mobile Page Issues/Research Results
Starting on April 6th 2018 we conducted qualitative user testing in an attempt to answer our two main research questions:
- Does the new treatment for page issues increase the awareness among readers of page issues?
- How do users feel about being informed of page issues? How does awareness of page issues affect their perception of Wikipedia?
We were additionally able to glean some information regarding some of our our additional research questions:
- How do readers form opinions about the quality and reliability of Wikipedia pages in general?
- Do readers care more about issues considered by Wikipedia to be of higher severity than issues considered to be of lower severity?
Methodology & participants
- Remote, unmoderated user tests conducted through usertesting.com
- Users looked at an Invision prototype
- Users are non-experts
- 4 test groups
| Group 1 (control) | Group 2 | Group 3 | Group 4 |
|---|---|---|---|
| - 4 participants
- 2 women, 2 men - 2 under 40, 2 over 40 - 2 US, 1 Australia, 1 Germany |
- 4 participants
- 2 women, 2 men - 2 under 40, 2 over 40 - 3 US, 1 Australia |
- 8 participants
- 4 women, 4 men - 4 under 40, 4 over 40 - 5 US, 1 Australia, 1 Canada |
- 6 participants
- 4 women, 2 men - 4 under 40, 2 over 40 - 4 US, 1 Australia, 1 India |
| Will see 3 Wikipedia pages w/o any page issue notifications | Will see 3 Wikipedia pages w/ current treatment of page issue notifications | Will see 3 Wikipedia pages w/ new treatment (A) of page issue notifications | Will see 3 Wikipedia pages w/ new treatment (B) of page issue notifications |
Findings
Top level findings are listed here. For more detail please see PDF below.
Does the new treatment for page issues increase the awareness among readers of page issues?
✓ Yes, participants shown the new treatment showed higher awareness of page issues in this test (particularly when they are in a more evaluative/critical mode)
✓ Surprisingly, participants shown treatment B did not notice it more than participants shown treatment A
How do users feel about being informed of page issues? How does awareness of page issues affect their perception of Wikipedia?
✓ Page issues make sense to readers and they understand how they work
✓ Readers care about page issues and consider them important
✓ Positive sentiments towards Wikipedia associated with readers learning about page issues
