The Value in Combining In-Person and Online Research

In-person Versus online Research

Are your customers frustrated, angry, or confused after interacting with your organization in person or online? Maybe calls to your customer care team have increased after an update to your website or portal. It’s possible you’ve seen a decline in people signing up for your services. You’ve seen the signs that something isn’t working. You are ready to conduct user research to figure out what your customers need to feel satisfied with your website, application, or service.

Perhaps you’ve decided on the research method you want to use, whether it’s a survey, a usability study, a card sort, or a mix of methods.

Next, you start to think about whether it makes sense to conduct the research online or in-person. You like the idea of being able to watch users interact with your product, but won’t it be easier to get them to do the study online?

To make this decision we recommend that you consider the goals of your project, your budget constraints, and who you want to participate in the study.

When to Use In-person Research

If the goal of your project is to gain a deeper understanding of your users’ needs and motivations, an in-person study makes the most sense. With in-person studies, a researcher can ask follow-up questions that uncover why users are behaving a certain way.

There are audiences that are difficult to reach through online studies, such as those without regular access to or literacy in technology. For many projects, these groups are the most important to understand so that your product or service works for users regardless of their technical access or skill level. Including these users in your study requires an offline approach.

When to Use Online Research

If the goal of your project is to collect metrics that empower your organization to make well-informed decisions, online research methods typically work best. While it is possible to collect robust metrics using in-person methods, it is easier and more cost-effective to do this online.

When it comes to your target audience, distributing studies via social media, email, and websites allows you to reach a broader audience. You can get responses from users across the state, the nation, or even the globe without needing to use resources on travel and with nominal or no recruitment costs.

The Benefits of a Combined Approach

Depending on the goals of the project, it can be best to take a combined approach. This gives you the benefits of both in-person and online studies. Sometimes we see people hesitate to do this due to additional cost. However, running both an in-person and online study can help you make the most of your research budget. The questions and tasks created for an in-person study can be reused in an online study, and vice versa. By running both versions of the study, you’ll make the most of the time and money spent on study design.

Are you wondering what a combined approach might look like? Here are two examples of our recent projects.
 

Case Study: Online and In-person Tree Test

As part of a website redesign for a large public utility, we conducted a tree test to evaluate the design of a new information architecture, which is the way that website content is structured and labeled. Participants in the study were given tasks to complete using the draft website information architecture. The goal of this study was to see if participants could find what they were looking for in planned locations or if changes would be necessary.

Since the new website could be used by any of the city’s residents, it was important to reach a diverse mix of people. We wanted to ensure the website was usable for people who don’t speak English fluently and those with low tech literacy. If we could create a website structure that worked for them, we knew that it would be likely to work for everyone else.

We started by distributing the study online, as we needed robust metrics to be confident that we were placing information in the right place. Then, we conducted an intercept study at a public library branch that we knew had high populations of patrons who were non-native English speakers and groups with low tech literacy. We gathered the metrics we needed and incorporated as many perspectives as possible.

Case Study: Online and In-person Usability Study

Ahead of a redesign of their website, a Los Angeles area music non-profit wanted to identify usability issues and collect baseline metrics to measure improvements against.

We conducted an online usability study for the non-profit. Participants in the study completed tasks on the website in their own time while research software collected data including:

• Whether users could navigate to the correct pages
• How long it took users to get to the correct page
• Comments they had about their experience.

These metrics were used to identify where the most critical issues existed with the website.

Next, our team conducted an in-person usability study to understand why certain website tasks were difficult for users. This allowed stakeholders to observe their users interacting with the website and empathize with their struggles. The combined results from these studies provided both baseline metrics and recommendations on prioritizing website improvements.

Every research approach has its benefits and drawbacks. When thinking about what approach makes the most sense for your organization, determine what you’d like to learn and who you’d like to take part in the study. Don’t forget that if one type of research won’t help you gather all the data you need you have the option of combining methods.

Want to conduct tailored user research to address your organization’s challenge? Get in touch.