Simple methods for involving key stakeholders in your usability study

A group of participants watching a remote usability study.

Who is this for?

Large government agencies and nonprofits who build digital tools for an entire community, municipality or state, must understand the needs and motivations of diverse audiences. Usability studies – or any research study where you have the chance to talk to your users in-person – are great windows into the mindsets of those you serve.

As we’ve seen at Anthro-Tech in our work with large health care organizations, transit agencies, and government: Designing solutions for large and diverse audiences requires abandoning assumptions, actively listening, and keeping an open mind toward new design solutions. Hearing directly from your users can be transformational when thinking about how to design an inclusive and straightforward digital space.

We always do our best to report back to our clients what we learn in usability studies in interesting and engaging ways. But one of the most effective methods to build empathy is through direct in-person communication. Watching someone have difficulty interacting with your service is a wholly different experience – one that sticks with you longer – than reading success rates in a report.

These types of interactions are even more powerful when your users aren’t like you. If your citywide service must work just as well for a 75-year-old retiree as it does for a digitally native teenager, you certainly don’t have a deep understanding of the lived experience of both these individuals. Usability research is a great method to get folks at your organization to step outside of their experience and capabilities and see your service from different perspectives.

Why is this necessary?

While it would be great to involve key stakeholders in every step of the research process to achieve organizational buy-in from the outset, there are a number of things competing for their time and attention. This presents a challenge.

Luckily, there are methods you can use within your organization that maximize stakeholder learning from a usability study and don’t require a huge time commitment, helping you secure buy-in and deepen understanding of your users at all levels .

Methods

1. Invite stakeholders to observe study sessions in real time

At Anthro-Tech, most of our usability sessions are around 1 hour long, which is not terribly time consuming. Observing a study as it is happening is powerful way to show - not tell - key stakeholders what is and isn’t working with a digital process.

What’s more, asking a stakeholder to observe a live usability session requires much more active listening and engagement with user problems than asking them to read a written report. We encourage no laptops and minimal phone use while observing a session. And while your stakeholders won’t be responsible for the heavy lifting of the research study, their note-taking contributions and insights will be very useful for the analysis and future design directions.

To set up their experience appropriately, book a conference room for observation for the duration of the usability study. It is important that you set up a live video feed that connects the study session room to a television in the observation room. We really like Zoom and WebEx as video conferencing software.

When setting up the video and screenshare in the study session room, make sure you minimize the video window so that the participant is not looking at themselves the entire time. That can get very distracting and make the participant self-conscious. Spend time before the study setting up all of your technology so that you have plenty of time to troubleshoot should you run into any trouble. Leave lots of post-its and sharpies in the observation room so that stakeholders can write down their learnings in real time.

Next, convey the following pieces of information to your stakeholders:

Preparation: Tell them upfront how much of their time you are asking for. Request an additional 15-30 minutes on top of however long the session is so that you can debrief and learn about their key takeaways. This will help you understand how they’re interpreting the insights and give you context as you think through the best ways to share the insights to the organization at large
Responsibilities: As an observer, stakeholders should leave their laptop at their desk, and come prepared to write down everything they find interesting. Good note-taking requires attentive listening. As they watch sessions, have them write down the interesting things they hear on a post-it one insight per post it. Upon the conclusion of each session, have them cluster similar insights on a wall in the observation room. Then ask them to spend 15-30 minutes with you to debrief what they wrote down.

During the study, write observations on Post-It notes.

Respect, respect, respect: The user is the expert in a usability session; they are never wrong. A usability study is not the time for a facilitator to explain the intent of a design or debate a user’s interpretation. If a stakeholder agrees to observe a research session, instruct them to refrain from judging the user, or categorizing them as “not our typical user” (if the study recruitment went as planned, everyone you’re interviewing should be a potential user). If they have questions, they can let you or the researcher know at the end of the session and the researcher can try to incorporate these questions in a later interview.

2. Synthesis

Another way to effectively bring stakeholders into the research fold is to invite them to a group synthesis session after the final research interview. Synthesis is a broad term used to describe methods of collaboratively processing and unearthing insights from a study.

These sessions typically take anywhere from 1-3 hours, so if your stakeholders are really busy you can stick with a 1-hour synthesis. Through a process of guided facilitation, all those involved with the study – and any curious stakeholders – can share their key takeaways and agree on a prioritized list of issues to address. For good group mojo, aim for 4-10 participants in your synthesis.

Before Synthesis: Stakeholders must first do a bit of homework. All research sessions should be video recorded (with participant consent). Ask your stakeholders to watch at least 2, hopefully more, before the synthesis. You can’t come to a synthesis session without any knowledge of the research, otherwise you’re left out while other people share exciting learnings. As far as materials go, bring lots of different colored post-its and markers. If possible, synthesize in a room with a whiteboard.

During Synthesis: You’ve made time on the calendar, you’ve sent invitations and instructions, and you have your stockpile of post-its. But how do you actually synthesize your research? Synthesis can take many forms, but here’s one possible agenda for a 90-minute session.

Participant Introductions (20 min): You or someone who was actively involved in the entire research process should make small index cards with a participant identifier and key stats about them. To kick off the synthesis, share these out loud with the group. Encourage those who were present for specific interviews or watched the video recordings before the synthesis to chime in with any additional pieces of information.
Confirmations / Aha’s (30 min): Take a whiteboard marker and write “Confirmation” and “Aha!” in giant letters on the board. Instruct everyone to spend 5 minutes silently writing any of their expectations that were validated in the research. These are Confirmations. Write one statement per post-it. Have each person share their post-its while placing them on the whiteboard. Group similar confirmations together. Next, do the same for Aha’s – anything that surprised them, raised questions, or went unresolved during the usability study.
Ideation (35min): Ideation is a fancy way of saying creative brainstorming. Look at the clusters of post-its you see under “Aha!”. Take each cluster one-at-a-time and as a group, discuss ways your design can address some of the questions and unresolved issues unearthed during the research. Keep an eye on the clock, this part can get juicy.
Wrap-up (5 min): Summarize what was discussed in the synthesis, the group's biggest takeaways, and the path forward. Encourage stakeholders to stay in communication with you through the analysis and sharing of findings.