notch-blue

Taking the Stress Out of Test Day

By: +
July 12, 2016

Conducting user research is at the core of our process. It’s a way to uncover problems and inefficiencies in a website or application. We also conduct user research as a step in our design process to ensure that the application we’re collaborating on with our clients is performing well with the target users. Doing this always saves a lot of development time and budget.

After conducting hundreds of evaluations over the years, we’ve noticed one major theme emerge: Test day can be stressful. In this article we’ll discuss ways to bring a bit more Zen to the test day so you can think about logistics less and focus on the observations more. We strive to ensure that the research experience is great for both our clients and our participants.

Here are a few things to consider prior to your next research session.

ulab-tour

Welcome to the Crux Collaborative Usability Lab.

Have a clear intent and purpose for the research

Sometimes clients will contact us knowing their site or application has barriers to success but they can’t articulate the business problem that is occurring as a result. We encourage coming to the table with a handful of clear questions you’d like to have answered as a result of the research.

Consider questions like:

  1. Can users complete [insert primary task]?
  2. Is the navigation structure clear and intuitive?
  3. Do users understand what action to take if they want to buy [insert item]?

Having a clear intent and purpose for the research helps us formulate a solid test plan and helps us understand where there are opportunities to make recommendations that provide the most impact.

Know who your target audience is

We know that most sites and applications are designed to appeal to a wide range of users. For a usability evaluation, our goal is to talk to people who represent the target user groups.

Think about some of the primary characteristics that define your key user groups. We recommend not getting too granular with secondary and tertiary audiences as many of them will more than likely share the same traits and behavior patterns. We generally talk to 8-10 participants in a day and we start seeing the same trends surface after we talk to 5 participants.

Coming to the table with some characteristics of your primary users will help us  help us more effectively write the screening documentation and know what questions to ask. This will ensure that our recruiter will get the right people scheduled for the evaluation.

Be prepared on test day

One of the worst things that can happen is for the website or application we’re testing to be down or not functioning as expected on the day we have usability sessions scheduled. When we receive a prototype of an application and there are bugs or the development environment is taken offline for maintenance, the quality and reliability of the research can be significantly impacted. We always recommend doing a dry run with the test plan a day or two before the evaluation in order to catch any bugs or problems.

Prepare for test day by:

  1. Ensuring that dead prototype links are disabled and don’t result in 404 errors
  2. Confirming with the technical team that they will not be doing any maintenance on the application during testing days
  3. Creating and testing user logins before test day and ensuring the right functionality and content will be presented to the user
  4. If a VPN connection is being used, make sure the proper credentials have been given to the facilitator and that the VPN has been tested before hand
ulab-tour-observation

The view from our observation room.

Be present and engaged during the sessions

Watching session after session can become a bit monotonous. That is why we have a note-taking process in place to keep our clients engaged during the sessions. This process encourages clients to capture important quotes and observations that are important to them, which will help us when we write the report.

Remember, you get the most out of the evaluation if you’re actually observing it. We recommend that observers refrain from multi-tasking during the sessions, which means staying off of Facebook and only responding to emails during breaks. Try to avoid scheduling conference calls on test days if possible. We schedule breaks between sessions to help keep everyone energized and allow time to catch up on emails if necessary.

While observing the sessions, keep track of any questions you’d like to ask the participant about what they saw or experienced. After each session, we ask the participant to fill out a short questionnaire. While they do this, the facilitator checks in with the observation room to see if any thing surfaced during the session that we might want to dig a bit deeper into. Sometimes the most valuable insight comes from questions that are not on the test plan.

We hope that these tips will help you maximize the impact of your next user research day. If you have a website or application you’d like to gain some insight on or find out where there are opportunities for improvement, contact us.

By John Golden
Principal

John’s career in interactive media design began in 1995 and has spanned over two decades with a focus on developing simple, streamlined approaches for complex problems.

View John's Bio

By Katherine Block
User Experience Consultant

Katherine is a jack of all trades with nearly every aspect of the UX process. Her broad approach enables her to fill gaps between the different phases of interactive design including Information Architecture, Development, and Project Management.

View Katherine's Bio

notch-white

Let us know your name and email address and we’ll send you our newsletter.


First Name

Last Name

Email Address

 

 

Thank you!