Wikimedia Apps/Team/Android/Anti Vandalism/Protocol for asynchronous feedback

Background

The community has requested moderation tools that do not show a performance preference to a single wiki language (request can be observed in the 2022 wishlist). We would like to do this in partnership with the community and are currently gathering feedback that will shape the next iteration of designs for the 'Anti-vandalism tool'.

Participants

We are looking for participants who are experienced editors, apps users and volunteer their time patrolling on any language wikis.

Instructions

• On your computer please add your username (or if you wish to stay anonymous type in 'Anon') in the input field below where it says 'Add username'.
• Next, please create a new section by clicking on the blue 'New section' button below. A new section will be created that will automatically include the usability test questions.
• You will just need to respond to the questions in the newly created section and publish your answers.
• Alternatively, you can also send your responses to otichonova@wikimedia.org.

Please note: Your answers will be publicly viewable.

Protocol (Testing script)

Opening scenario

The patrolling tool is meant to improve and make patrolling on Android easy and accessible. This is only the first design iteration of the patrolling tool and in this test you will be interacting with a prototype or a ‘simplified model’ that is very limited in functionality. We are still working on the product so don’t worry if things are not as polished, we are mostly testing the current workflows and the content.

Tasks and questions

Before we look at the designs, I’d like to ask you just a few quick questions.

1. What wiki(s) are you part of?

2. How long have you been editing for?

3. What role(s) do you have on Wikipedia(s)?

4. Do you patrol edits?

5. Why did you start patrolling?

6. How long have you been patrolling for?

7. Which difficulties have you run into while patrolling?

8. Do you use any extensions/tools while you patrol? If yes, what type of extensions/tools do you use?

9. How often do you currently patrol via your phone? Why?

10. Do you use the Android Wikipedia app? If yes, how long have you been using it for? What do you use it for?

Discovering the tool

11. I have just sent you a link with the prototype. Please click on this link and it should open the design prototype in your browser.

12. Once you have opened the link, do you mind sharing your screen with me so that I can see how you interact with the product?

13. Looking at the screen where would you expect to find the patrolling tool?

14. Do you think the location of the patrolling tool is intuitive?

15. Please tap on ‘Vandalism patrol’ on the screen.

List of edits

16. Without tapping on anything, please describe what you see and your understanding of the elements on the page.

17. As you might have noticed this is a queue of unpatrolled edits. Is there anything on the page that you find confusing or cannot find?

18. Do you usually use filters when you patrol currently? If yes, what filters do you use?

19. There will be a filtering feature on this page. What types of filters would you like to have access to on this page?

20. Please now tap on the first edit on this screen called 'Types of volcanic eruptions'.

Content

21. Without tapping on anything, please describe what you see and your general understanding of the elements on the page.

22. What do you think about the information included at the top of the page?

23. Would you add or remove any information?

24. When you are patrolling, what actions do you use most often?

25. Are there any actions in the bottom sheet that you would remove or you think are missing?

26. Do you feel like you have access to enough information to make a quick decision about the diff you see?

Patrol the diff on the screen

27. Please review the diff on your screen (you can scroll on this page to see the full diff) and revert the edit (you can do this by tapping on the 'Revert' option in the bottom sheet).

28. What did you think of the workflow that you just completed, did you find it confusing or intuitive?

29. Would you add or change anything about the workflow?

Use template

30. Do you often use templates/warning messages when you patrol? If yes, what type of templates do you use most frequently?

31. In the first iteration of the tool we would like patrollers to create and have access to their own collection of pre-filled warning messages/templates.

32. Please tap on 'Warn' in the bottom sheet to warn the editor using the existing saved messages. Please choose the message with the subject ‘Revert edit’ from the drop down.

33. Imagine you forgot to write ‘Thanks!’ at the end of the message and you to add this by tapping on the input field (the word will get added automatically).

34. Please tap 'Save' to save this message and then choose the option 'Update existing message' and tap on 'Save' again.

35. What do you think about this workflow you just completed?

36. Would you alter anything about this workflow?

37. Do you see yourself using this feature while patrolling?

38. Do you often write on the editor's talk page when you patrol?

39. Please move on to the next edit by tapping on 'Next'.

Menus

40. You have access to a ‘more’ menu in the top right corner. What would you expect to find there?

41. Please tap on the menu (the three dot icon). Would you add any more options to this menu?

42. Would you remove any options from the menu?

43. You have access to the menu by tapping on the user name (in this case an IP editor). Please tap on the user name on the screen.

44. Would you add any more options to this menu?

45. Would you remove any options?

Upcoming version 2

46. What you saw now is the beginning of the first version of the patrolling tool on Android. We are planning to have a second version where the tool can be used by experienced editors that want to practice patrolling on their wikis. The V2 will have a reduced number of actions and tooltips that will onboard and guide people through patrolling.

47. What skill(s) do you think patrolling teaches editors?

48. What do you wish you knew early on when you started patrolling?

These are all my questions. Thank you so much for participating in the usability testing, your feedback will shape the further development of this tool. Next we are planning to finish the usability testing, gather the feedback, make changes to the designs and start development. If you wish to follow the progress of the project you can do that by keeping up with the updates on the Android project page.

Publish

Please don't forget to publish your answers by clicking on the 'Publish page' button at the top of the page.