The Biggest Mistake You Can Make During User Interviews - ProductFTW #81

TL;DR

• The biggest mistake in user interviews is walking in with a conclusion and treating the conversation as validation instead of discovery.
• "Would you use this feature?" is the wrong question because users are bad at predicting their own future behavior and tend to say yes to sound polite or aspirational.
• The worst outcome of a bad interview isn't learning nothing, it's learning the wrong thing and feeling confident about it

A couple of weeks ago, in You Don't Have a Roadmap Problem. You Have a Problem-Definition Problem I talked about how it's not uncommon for a company or founder to know what they want to build and who might pay for it, but they can't tell you what problem the feature actually solves for the user. 

So how do you figure out the problem?

Our generic advice is usually "go talk to users." User interviews are part art and part science. After observing and coaching on conducting the best interviews, I’ve noticed a common pattern that leads to the worst outcomes for companies and users. 

The Pattern

The goal of the interview is to come out the other side with a clearer picture of where the real friction is in your user's life. Sometimes that confirms the problem you suspected, and other times it surfaces a problem you didn't know existed. Sometimes it tells you the problem you were going to solve isn't actually that painful, and you just saved yourself six months of building the wrong thing.

The biggest mistake I see is that the person or team running user interviews isn’t actually conducting the interviews to learn about their users. Instead, they’ve already decided what they want to build, and they are just looking for validation, so the questions they ask are the ones you'd expect from someone looking for a yes:

• "We're thinking about building a feature that does X. Would you use it?"
• "If we added the ability to do Y, would that be valuable to you?"

Some users say yes, and that gets logged as validation. Other users say no, and that gets explained away as if they didn’t get it, they’re not the target customer after all, or they’re an edge case. Maybe all of those are true, but it doesn’t matter because the team walked into the conversation with a conclusion and walked out with the same conclusion. There’s no new insight or real learning, just a paper trail of “we talked to users.” 

Why This Doesn't Work

When you go into an interview looking for confirmation, no answer can actually change your mind. You've already decided what counts as signal and what counts as noise before the conversation starts.

Even when teams are genuinely open to learning, these types of interviews don’t yield many useful insights because users are bad at predicting their own future behavior. Ask someone whether they'd use a hypothetical feature, and they'll almost always say yes, especially if you sound excited about it. This can be for a number of reasons, including that they're being polite, they're pattern-matching to what you seem to want to hear, or they're aspirationally imagining a version of themselves who might use the feature being built, even if that doesn’t match their actual behavior. None of that translates into adoption when a feature ships.

What To Ask Instead

The interviews that surface real insight focus on how someone actually does their job (or whatever activity your product helps them with), what frustrates them, where they lose time, and what workarounds they have invented to deal with the gaps.

Some questions that get there:

• "Walk me through the last time you had to do X. What did you actually do, step by step?"
• "What's the most frustrating part of your week?"
• "Where do you feel like you're wasting time?"
• "What have you tried to fix this? What worked? What didn't?"
• "If this problem went away tomorrow, what would change for you?"

Notice what's missing? None of these mention your product, your roadmap, or any feature you are considering. They are entirely about the user's reality. You are not selling, you are not validating, you are just listening for pain points.

When Feature Questions Are Okay

Feature questions belong in the conversation only after you've defined the problem and have a specific solution to test, and the framing should be something more like "how would this fit into your workflow" not "would you use it."

The reason the framing matters is that you've now anchored the conversation in their problem instead of your idea. They can give you a useful answer instead of a polite one because they actually have the context. Better still, hand them a prototype and watch them try to use it. Observing behavior beats stated preference every time.

What You're Actually After

The biggest mistake isn't running bad interviews; it's walking into them with a conclusion already in mind. The worst outcome isn’t that you learn nothing, it’s that you learn the wrong thing and feel good about it. 

Better questions help, but the real fix is changing what you're trying to get out of the interview in the first place. You’re not there to test an idea. You're there to understand your users' problems. Listen for what frustrates them, not for openings to mention your feature. Try to better understand where their problems are being underserved. You'll know it worked when you walk out with a problem you weren't expecting, or with the same problem described more sharply than you described before the interview.

About ProductFTW

ProductFTW is a weekly newsletter about product management, with a focus on real-life experiences in startups. We want to help product leaders be successful by giving realistic approaches that aren’t for giant tech companies. We know you don’t have a full-time product designer on each team. We know your software probably hasn’t been used by millions of people worldwide–yet. We’re here to bridge the content gap from building your product and team to scaling it.

Part of the Product Requirements Field Guide — ProductFTW's collected essays on the six phases of writing requirements, from problem definition to launch.