How we conduct usability tests at Hygraph

One of the most valuable pieces of advice I've ever received is to pay attention to what users do, not just what they say. Usability testing is the perfect way to do just that. It is a powerful tool that allows teams to observe real users interact with the product and draw valuable insights from it.

At Hygraph, we conduct usability tests as part of the user testing process to identify and address problem areas early on in the development process. This article will walk you through it, along with some examples and resources to help you get started!

1. Formulating testing goals & questionsAnchor

Is usability testing the best way to find answers to your questions? Take some time to reflect on the problem you’re trying to solve (or discover). Talk to your stakeholders and team to clearly define a goal. Some starter questions can be:

• What are we trying to learn from the user tests?
• What kind of feedback are we looking for?

Let’s take an example:

This is the goal we defined for one of our internal projects: *“The aim is to understand how users approach and use nested components in Hygraph (subtext: …so that we can improve on it)”*

We wanted to observe the users as they work with this fairly new feature and have a dialogue with them as they move to different parts of the flow. Usability testing seemed the best way to do this in real-time with real users.

So next, we mapped out the scope of the testing by coming up with questions we wanted insights on:

• Does the user understand what nested components are and what they do?
• Can the user easily build them using the current flow? And so on…

The tricky part here is to keep the questions specific without being biased toward a certain ‘solution’. These questions are more contextual and differ from feature to feature.

2. Building a prototype and writing a scriptAnchor

This is the creative part of the process because there are multiple ways to build a prototype. You have the liberty to decide the extent of details and complexity you want to get into depending on the time you have. Here’s how we approach it in brief:

• Define the part/s of the flow that needs to be tested
• Build a story/scenario around the flow to be able to communicate it to the users
• Put the screens together with the story in a click-through prototype

Once the prototype is made, it’s time to write a script for the person conducting the usability tests. It’s proven very useful for us to have a script outline ready to make sure you communicate all the details properly to the user without overwhelming them. You can use the template shared above to write your script.

Some users might not be aware of how prototypes work, so it helps to mention and demonstrate what they are expected to do. Also, saying things like “any action or flow you cannot figure out is feedback to us and doesn’t reflect your skills.” can help ease their anxiety. It is important that the user feels comfortable and is able to think out loud to get the right feedback.

An additional step here is to conduct a few sessions internally to test your script as well as the prototype before taking them to real users. You will overall be more confident and have the opportunity to make the prototype better.

3. Recruiting and reaching out to usersAnchor

As you are building prototypes, you or your team members can parallelly focus on looking for the right users to test them with. The kind of feedback that is received will differ significantly from persona to persona, so it’s best to consider this carefully before inviting users.

Some organic ways to reach out to users are by posting in your Slack or Discord channels if you have an active community and want access to a diverse set of users. If you have a website or application online, you can experiment by adding in-app prompts for targeted users.

If you want to test with people who haven’t used your product before, social media platforms like Twitter or Instagram can help. So far, adding Intercom pop-ups to targeted users on our web app has gotten us the most responses.

If you are not sure of the methods above, another way to recruit users is through agencies. There are a lot of them available online that can help you find users that match your persona.

4. Running the sessionAnchor

The sessions can be in person or over a video call. You can decide what works best for you and the user when you reach out to them. We usually provide a link (Calendly or Google Calendar) to the user so they can choose the time and date of their preference.

Running the session can be fairly simple- follow the script and pay close attention to the user. Make sure to tell them to think aloud and ask questions like “What were you expecting to happen when you clicked that ——?” to get fuller insights.

Observing facial expressions and overall body language is also a critical part of running a usability test. If you spot the user feeling unsure about something while using the prototype, pause and ask questions to understand the reason behind their confused glances. Remember that we are trying to learn what the user thinks and gain a deeper insight into their behavior as they use the product. It’s not just about whether they get the flow right. The more tests you conduct, the better you get at it.

5. Documenting and analyzing the findingsAnchor

Hurray! A major chunk of the work is done, and now it’s time to write a report. Summarising extensive findings into readable data points can be a challenge. The way we approach it is by trying to answer the questions that we wrote in Step 2. Here’s an example of what your bullet points can look like:

• 7 out of 8 users clicked on the blue button to go to the next step.
• Most users were confused by the description of this modal and couldn’t figure out what to do next until prompted.

We use Notion to write reports and share them internally. You can make the report into a presentation format for better visibility and communication and since not everybody can be a part of the testing, scheduling a readout meeting for the relevant stakeholders and team members to share insights is the best way to conclude the journey.

And that’s it…Anchor

If you are a designer, you can take those insights and apply them to improve the user experience. If you are a product manager, you can prioritize which problems to tackle and how. When usability testing is done right, it can save a lot of time, money, and effort. Don’t hesitate to experiment and tweak the process to your needs. Good luck and happy testing!