Flow/Research/User tests

Test: User messaging 1: Talk page basics

This test was run on enwp so that testers would be able to interact with a full environment. Accounts were set up for the interaction, with talkpages and contributions populated for the test.

A second version of this test was run using a different article (Dwarf cat) and some simplified wording of the tasks, but the gist is the same.

Test

Introduction

This is a test of Wikipedia's user-to-user communication systems. In this scenario, you have recently joined Wikipedia and tried a few first edits. A few days have passed since your initial login and you are now returning to the site, curious if anyone has noticed or objected to your edits.

Remember, we're testing the interface, not you. If you're having difficulty with something, the problem is with our design. Please "think out loud" as much as possible; tell us your thought process during each task, and try to explain your general opinions as you arrive at them.

If a task takes more than five minutes or so, just move on.

Tasks

Start point: https://en.wikipedia.org/wiki/Special:UserLogin

1. Log in using the account Silver waffles. Suppose this is the account you previously created.
2. See if you can find out if you have new messages regarding your prior edits, including to an article you recall being about redemption. Where would you expect this to be found?
   Spend no more than a few minutes on this.
3. You should have found your way to this page: http://en.wikipedia.org/wiki/User_talk:Silver_waffles - If not, go there now.
   What is your impression of the messages? Do they appear to have been left by a human or automatically?
4. Now see if you can reply to the second message regarding the article you previously edited. Pretend you disagree with it and say so (you can use a reason if you want, but it's not required).
5. Suppose you're not sure Orchaen solns (the user who left the message) will know that you replied. Does there appear to be anything further you can/would need to do to make sure they get your response?
   Try to do this in whatever way makes the most sense to you.

Questions

1. Did you edit Wikipedia before this test, even once?
2. What frustrated you the most? What improvements would have made the process easier?
3. What did you like about the process, if anything?
4. Is it okay if we share this video publicly with other Wikipedia contributors? We'd love to show them what new members think.

Test: User messaging 2: More complex talk page usage

This test will also be run on enwp, using the usual assortment of templates and gadgets, so that testers would be able to interact with a full environment. Some accounts from the previous test were reused, and two new ones set up for the interaction, with talkpages and contributions populated for the test.

Test

Introduction

This is a test of Wikipedia's discussion system. In this scenario, you have recently joined Wikipedia and edited a few pages, including to the Dwarf cat article. Trying to add an image, however, you found the process too confusing and had to ask another user, Orcaen solns, for help. A few days have now passed since you last logged in, and by now you expect there will have been a response to your query.

Remember, we're testing the interface, not you. If you're having difficulty with something, the problem is with our design. Please "think out loud" as much as possible; tell us your thought process during each task, and try to explain your general opinions as you arrive at them.

If a task takes more than 5-10 minutes, just move on.

Tasks

Start point: https://en.wikipedia.org/wiki/Special:UserLogin

1. Log in using the account 'Gleaming muffins' using the password '...'. This was the username you used to make your edits and ask for help.
2. See if you can find out if Orcaen solns, the user you went to for help, responded and answered your question. Where would you expect to find the response?
   Can you find where you would have originally sent the message with the question? Note that in the scenario it has been a few days and you would not necessarily remember this anyway.
   Spend no more than a few minutes on this task.
3. You should have found your way to this page: https://en.wikipedia.org/wiki/User_talk:Orcaen_solns - if not, go there now.
   What appears to be the significance of this page?
4. Find the section 'I want to upload a picture of my cat' at the bottom. What seems to be happening here?
   Can you tell who all is involved in the discussion that started from your question? What is your impression of these users?
5. Assume Orcaen solns' advice was helpful. How would you go about thanking him? Try to do this as a response to his explanation.
6. Can you tell how the messages are threaded? Regardless of what you did in the previous step, consider how you would need to thread your own responses.
   Keeping this in mind, see if you can also reply to Doranton's question about your cat, keeping threading intact - suppose its name really is Binky and say so.
7. Now suppose you want to be sure Doranton is informed of your reply. You may have seen a blue 'talkback' message on your own talkpage (https://en.wikipedia.org/wiki/User_talk:Gleaming_muffins) from when Orcaen solns replied to your initial message, and you would like to do the same for Doranton. How would you go about doing this, or finding out how to do it?
   Try to go about this in whatever way makes sense to you.

Questions

1. Did you edit Wikipedia or another wiki before this test, even once? If so, which?
2. What frustrated you the most? What improvements would have made the process easier?
3. What did you like about the process, if anything?
4. Is it okay if we share this video publicly with other Wikipedia contributors? We'd love to show them what new members think.

Things possibly worth testing

• {{Talkback }}/ "Reply on my talkpage"
• Echo
• Finding/noticing replies
• Contacting other folks (ask for help, question, whatever)
• Getting blocked and finding a notice saying they may appeal (say for their username)

Random pile of other strange things people may encounter and/or have trouble with when using talkpages

• Notes at the tops of talkpages saying where they will respond
• Alternate accounts
• Bots
• Indenting
• Outdenting (and {{Outdent }}ing)
• Cross-project communication and notices
• Arbcom clerks
• Wikilove
• LQT, DPLforum, and similar, especially when only used on some pages (inconsistency)
• Wikisyntax
• Vandalism (talkpage is blanked: You have new messages!)
• {{Trout }}
• Other people engaging in heated arguments about a frivolous topic on an unrelated user's talkpage
• Email notifications (and lack thereof after the first)
• Transclusion and substitution of templates
• Signing posts
• A notice about username issues, please change your username
• Welcome templates
• The entire notion of a 'talkpage'