User:JKlein (WMF)/guerilla-user-testing

Quick summary

What is guerilla user testing, and why should I do it?

Martin Belam defined guerilla testing as “the art of pouncing on lone people in cafes and public spaces, and quickly filming them whilst they use a website for a couple of minutes.” While this is an accurate definition, we can expand upon it by using the term to apply to testing any product produced in service of Wikimedia, at any fidelity. The basic idea is that this is a method for testing out product ideas on real humans in an inexpensive way.

Here are a few times that you should consider performing this kind of testing:

To test: When you have a product in beta and are looking for bugs
To think: When you have a proof of concept and you want to experiment with it to see where the idea goes
To learn: When you want to gain a better understanding of how your users are currently using a product

What do I want to learn from testing?

Before showing up to an event with your product, think about who you want to test with and what you want to learn from your test.

Here are some tools that you can inform your decisions:

Personas

Personas give us a person to connect with, someone who has goals for using the product, ensuring human centered design. Typically you will have a user in mind when you are creating a tool, you use this information to think about recruiting people or meeting these user types in places that they already inhabit, such as an Edit-a-thon or IRC channel.

Journey Mapping

Map your personas to a happy path. Use journey mapping as an exercise for determining a list of requirements for your project as well as uncovering hidden issues. This will help you to consider what you to do with participants who volunteer to test and will inform your script writing

Hypothesis Definition

Defining our solutions through a hypothesis framework allows us to acknowledge that we are making assumptions and testing those assumptions through our prototypes validates our hunches. This will ensure that you know why you are prototyping and testing. With this information you can figure out if you are using the right method and/or prototype to validate your hypothesis.

Where to test

- Edit-a-thons

- wiki salons

- chat platforms

- On Wiki

- usertesting.com

Pre-event

You've decided what you want to learn from your test, so now you need to do all the prep to make sure that it happens smoothly

Collaboration

If you plan on showing up at an event, it's a good idea to connect with the event organizers.

Script writing

Write out what you want the testers to do as well as the person who is conducting the testing

Testing devices and Prototypes

Think about where you want folks to test.

- paper prototypes

- nonfunctional clickable prototypes

- In production on Wiki - on desktop, mobile, tablet

- on the web or on app

- In a test environment

Sandbox wiki pages

- how to make

- where to make

- why to make

Recording devices

- screen recording

- video

- audio

- data logging

- microphones

- headsets

Materials to advertise who you are

- business cards

- swag

- signs

At the event

Testing

- introducing yourself

- getting over the jitters

- going with the flow

Analyzing

- have a set of questions that you are trying to always get the answer to

- have a notepad for anecdotal and unrelated findings

Real-time iteration

- do you need to change your questions, framing, script ?

- do you need to massively fix the prototoype?

- can you make a tweak on the fly?

Testing with remote participants

special considerations

Post-event

Synthesizing feedback

- Write findings

Feedback Party

- Share with your team

- Work openly

Identifying next steps

-proposed solutions and/or plans for solving