Process / May 2018

How to Conduct User Testing for a Digital Project

User testing is the key to ensure we’re building something people will enjoy and actually use. This method of evaluation doesn’t need to be expensive or time consuming: small batches of regular testing are better than conducting one large, expensive test. This post will share how to effectively conduct user testing with examples from our user testing process with Mighty Writers for games we recently created for the National Museum of the American Indian.

How to Conduct User Testing for a Digital Project

Explain what you’re doing and make your participants feel comfortable Be transparent and open about the process and interview. Use a standardized protocol to introduce the testing session to your participants. Get to know them and make sure the participants understand that there is nothing on the line for them. Offer refreshments, and be thankful.

Useful Phrases
“We’re not testing you; we’re testing the system.”
“There are no right or wrong answers.”
“There is no ego involved here; we want to hear both positive and negative comments.”

Stay engaged! Acknowledge and provide feedback to your participants as they ask questions or give feedback throughout the interview, but don’t help them along or try to explain how to use the system. It’s okay if they’re not able to solve a problem or accomplish a task—try to evaluate how to improve it.

Ask for clarification If you need to know more, ask the participant. This is your time to learn what you need to make the system better, so don’t hesitate to ask questions. When working with younger audiences this is especially important. During testing for the games for the National Museum of the American Indian, students were pretty shy at first to talk and share, so we had to keep a close eye on potential things that required conversation or elaboration. One specific example was watching one student ignore the floating icons on screen because they didn’t know to click on them—without asking, we wouldn’t have been able to understand why.

Useful Phrases
“Talk me through what happened there.”
“Can you explain why you did that?”
“If you have to give this a letter grade, what would you rate it?”

Four people at a laptop completing user testing
Common User Testing Questions

How many participants should I test with? Nielsen Norman Group, a leader in the User Experience field, suggests between five to seven participants per audience group to maximize the number of problems found while keeping resource spend low.

Who do I recruit? Look to your audiences identified during discovery. We were building the NMAI games for use primarily in the high school classroom, so we wanted to work with students and teachers between 9th and 12th grade. We partnered with Mighty Writers since they work with students after school hours in a space where they were already learning and writing.

How do I recruit participants? Draft a profile of your ideal participant (who are they?). Ask other people in your organization to help you identify participants or put you in touch with groups that may be able to help. We don’t work with students, so we connected with an organization that does.

Do I need a consent form? The best practice is to have one. It helps participants feel more comfortable about the process. If you have a consent form, explain the form and have participant fill it out with you. Explain how you’ll use the information gathered. Bring two copies: one for you, signed, and one for them to take home.

Do I need an incentive or reward? Consider how much effort participants are putting into attending your testing session, and allocate a budget to give a small gift. Give something that is easy to cash in on, especially if you can give it out on the spot (like a gift card or a free coffee or donut).

Do I need to do the testing in-person? Conducting user testing in-person is generally better than remotely, but there are really good user testing tools out there for conducting testing remotely. If you’re not able to invite your participants physically, consider tools like Zoom, GoToMeeting, Skype, or Google Hangout to give them control of your screen and record them directly on your computer.

What should I take note of? Consider what information will be most useful to improving your project. If you’re looking for qualitative information, ask participants to respond to preferences, language or terminology, and look to their expressions and reactions. Quantitative data can include the time spent on a task (record start and end times), task success (are they able to complete the task successfully?), the number of errors or the amount of effort it takes to complete the task, and learnability (do they get faster or more efficient completing tasks over time?).

Are you ready to plan your user testing? Download our User Testing Preparation Checklist.

Further Resources about User Testing

User Testing Explained – The Next Web

A Beginner’s Guide to Usability Testing – UX Mastery