Interviewing Users - Steve Portigal

Last Updated: 15 September 2022


What’s the book for

Focused on field researchers going out to do usability studies. This is a big industry and covers topics as broad as how people use software to what they eat for breakfast. If you want to be a professional field researcher this book will help you.

The focus is on formal, structured in-person user research (rather than remote).

It’s the opposite of the Mom Test which focuses on opportunistic, informal research.

3 highlights

  1. "Great interviewers leverage their natural style of interacting with people but make deliberate, specific choices about what to say, when to say it, how to say it, and when to say nothing. Doing this well is hard and takes years of practice."
  2. "Developing insights about users doesn’t always have to be a milestone in a product development process. Insights can be an organizational asset that is assembled quarterly (or whenever) to feed into all aspects of product development, marketing, and so on."
  3. Great practical details for field research studies, e.g.If you're going to use your phone to record audio, make sure you know what will happen if you receive a call or textIf you're using a video recorder, buy a bigger batteryHow to place participants and the camera/microphone to get good quality audio/video
  4. If you're going to use your phone to record audio, make sure you know what will happen if you receive a call or text
  5. If you're using a video recorder, buy a bigger battery
  6. How to place participants and the camera/microphone to get good quality audio/video

How to use this cheat sheet

The goal of this cheat sheet is to give you great value for time spent reading.

The book
48,324 words
This cheat sheet 4,237 words

In less than 10% of the time spent reading, I hope to give you 90% of the insights from the book. If you’re a fast reader of English, you should get through this cheat sheet in under 20 minutes.

To do that, I’ve trimmed the fat wherever I can: I didn’t reiterate the light-hearted anecdotes and I didn’t crack any funny one-liners. That makes this a dry read 😬 but that’s the trade-off - dry, but to the point and fast. Get a coffee, ready your highlighter and brace yourself.

Use it as a tool to extract the best info, and if you need more detail or have questions this cheat sheet will tell you where in the book you can find that info.

I found chapters 2, 4 and 6 to have the most valuable info - I’ve put a ⭐️ next to these.

Chapter Notes

Chapter 1 - The Importance of Interviewing in Design

Tom’s top line

Interviewing users is a tool in your toolbox.

It is a weird, unnatural interaction.
The goal of interviewing is to get the real answers. To do that you’ll need to navel gaze a little bit on how to interview well."


"A lot of important information gets left behind. Insights don’t simply leap out at you. You need to work hard and dig for them, which takes planning and deliberation"

Interviewing has other names: “user research, site visits, contextual research, in-depth interviews, contextual-inquiry, design research, and ethnography, to name a few.”

Key steps in the process:

  • Deeply studying people, ideally in their context
  • Exploring not only their behaviors but also the meaning behind those behaviors
  • Making sense of the data using inference, interpretation, analysis, and synthesis
  • Using those insights to point toward a design, service, product, or other solution

Often the pain of the problem is less annoying than the effort to solve it (aka satisficing). Examples:

  • the unfiled MP3s scattered on the computer desktop
  • ill-fitting tupperware lids
  • tangled, too-short cables connecting products

Why interview?

  • As a way to identify new opportunities, before you know what could be designed
  • To refine design hypotheses, when you have some ideas about what will be designed
  • To redesign and relaunch existing products and services, when you have history in the marketplace

Pros/cons of interviews

Interviewing isn’t the right approach for every problem.

  • It favors depth over sample size (therefore it’s not a source for statistically significant data)
  • each interview will be unique, so it's hard to objectively tally data points across the sample.
  • We try to interview in context, but it’s not fully naturalistic (vs web analytics for example).
  • Interviews are not good at predicting future behavior, especially future purchase intent or uncovering price expectations

Strangeness of interviewing

"interviewing is not a social conversation. Falling back on your social defaults is going to get you into trouble!"More similar to other professions using "verbal inquiry to succeed" (e.g. police interrogator, lawyer, reference librarian)

Interviewing builds empathy internally in organisations - both for the customer and for your colleagues.

⭐️ Chapter 2 - A Framework for Interviewing

Tom’s top line

Building rapport is the key to a good interview.
Just being a good listener and curious gets you a good way there.Don’t talk about yourself unless it helps achieve your research agenda.


"I approach the interviews with a sense of what I can only call a bland curiosity.”

“As the researcher, it’s my responsibility to find out what’s going on; I’m not invested in a particular outcome. Even more … I’m not fully invested in a specific set of answers"

"I don’t know yet what I’m curious about"

Tactics

"Transitional rituals" to get in the interview headspace

Before starting interviews:

  • Brain dump to make space in your headJot on post-its: Assumptions, expectations, closely-held beliefs, perspectives, and hypothesesAnonymise the input so people don't feel like they're being checked as right or wrongDon’t try to find consensus, just try to get all the implicit ideas outside into an "external, neutral space"
  • Jot on post-its: Assumptions, expectations, closely-held beliefs, perspectives, and hypotheses
  • Anonymise the input so people don't feel like they're being checked as right or wrong
  • Don’t try to find consensus, just try to get all the implicit ideas outside into an "external, neutral space"
  • Right before entering the room, consciously state the purpose (e.g. "learn about Paul and how he uses his smartphone")

Embrace how other People see the World

  • Do the interview in context - on their turf, rather than on your turf
  • Ask questions even when you think you already know the answeryou want the detailsyou could actually be wrongthis helps position the candidate as the expert
  • you want the details
  • you could actually be wrong
  • this helps position the candidate as the expert
  • Make sure you're not distracted (eat, pee, drink etc before. Silence your phone.)

Building Rapport

"The rapport is what makes for great interviews. You won’t leave every interview walking on a cloud, but getting to that state with your interviewee is something to strive for"

"creating that connection falls to you, the interviewer"

"Some of my best interviews have been with people who are visibly uncomfortable or disinterested at the outset."

"Listening is the most effective way you can build rapport. It’s how you demonstrate tangibly to your participants that what they have to say is important to you."

  • Questions should follow on from the last answer. Don’t switch topics randomly.
  • When you need to change the topic something like “Earlier, you told us that...” or “I want to go back to something else you said...”
  • If you want to shift completely, signal that: “Great. Now I’d like to move on to a totally different topic.”

Be selective when talking about yourself

  • Sharing a common experience/trait/preference ("me too") is only helpful to either: build rapport or encourage the participant to continue/expand.
  • It can backfire by implying you're more interested in talking about yourself rather than listening
  • The interview process can be a good tool for building rapport: By asking questions, listening intently and following up (showing you're listening), that can help to loosen people up.

"Stories are where the richest insights lie, and your objective is to get to this point in every interview."

"There’s often a point in the interview where the exchange shifts from a back-and-forth of question-and-answer, question-and-answer to a question-story setup"

Because the interview will inherently be a weird experience, leverage the constructed nature of your shared experience:

  • You are empowered to ask silly-seeming detailed questions about the mundane because you are joined together in this uncommon interaction.
  • Frame some of your questions with phrases such as “What I want to learn today is...” as an explicit reminder that you have different roles in this shared, unnatural experience.

Chapter 3 - Getting Ready to Conduct Your Interviews

Tom’s top line

If you’re researching on behalf of a client, getting on the same page is priority #1
Don’t just look at the “end user” - there might be other people in the workflow that you need to talk to

If you’re doing research for a client, getting on the same page for the goals is a challenge. You should be prepared to keep adjusting this right up until you deliver your results.

Things to find out from your client:

  • History with the organization and the research topic
  • Current beliefs about the customer and the proposed solution
  • Barriers to be mindful of (e.g. organizational barriers)
  • Business objectives for the project, specific questions the research should answer
  • Concerns around the methodology
  • Review other material (previous research reports, existing products, in-development prototypes)

Share a summary of the questions/goals to be answered with the client (and get feedback) before continuing.

Selecting participants

  • Look at several parts of a transaction; even if we are designing only one part of the experience, we can gain a deeper understanding by looking at it from multiple points of view. For example - enterprise software might have several decision makers but only one user.
  • Have a mix of the specific and the descriptive (e.g. “people that use reddit daily” and “people that love learning online”)
  • Base criteria more on behavior than attitude (e.g “reads every day” or “has written more than 100 articles in the last 12 months” vs “is a dramatic storyteller”).

There’s lots more info about writing a protocol/script in the book, so if these points are interesting then jump to this chapter.

⭐️ Chapter 4 - More Than Just Asking Questions

Tom’s top line

Pay attention when you get an answer to a question. What question did the participant actually answer?
There is a variety of exercises you can do with a participant outside of simply asking questions to get better information.

Unfortunately the book doesn’t have a foolproof recipe for making questions good ☹️ but it does offer some frameworks and exercises to help you.

What are you asking, and what are they telling you?

Questions might be confusing/disguised/not literally what you've asked:

Asking a user "What is your process for updating your playlists?" is secretly asking some other questions:

  • “How do you feel about the process for updating playlists?” (“Oh, it’s easy, all I do is....”)
  • “What are the key steps you can recall in the process for updating playlists?” (”Just click the add-to-playlist button and it’s done”)

"Can you show me how you update your playlists?" or "Can you show me how I can update my playlist?" will get you an answer to how they update their playlist (as well as their emotional response).

  • Puts them in the position of expert ✅
  • Works for more "general" process that is consistent across users, rather than a unique workflow to the individual

To get better context, join the participant before, during and after the activity. Rather than "show me how you update your playlists", "I'd like to join you for a listening session".

Role playing

Helps when a participant is struggling to remember the details or explain why something was a joy/frustrating.

e.g. “I’ll pretend to be the potential customer, and you can go through the sales process with me”.

Also good to ask "What's the ideal way this should work", and then after ask "and how is it actually different from that?"

Participatory design

Have the user build up the design from a more blank state to what they think it should be.

You don't have to implement their solutions (e.g. a physical handle), but instead observe the problems they're solving (being able to move the device around easily).

Tools

Mapping

Mapping takes something abstract (e.g. process, relationships) and makes it easier to see and discuss.

It also gives you an artifact at the end of the interview, to help with your own analysis.

Reactions to concepts

This is putting “concepts” in front of the participant to use as a conversation aid (prototypes, mockups, storyboard, wireframe, …).

The concepts don't have to be an actual solution. Your focus shouldn’t be “do they love this solution", instead the focus should be “what is important in a solution for this user”.

The concept can be a generic solution to the problem, with one single change that you’re interested in testing (e.g. we’ve made a standard looking customer dashboard, but the main menu is in the bottom right corner now - how do participants respond to that?)

Don't start with "here's something I've been working on". They’ll tell you it’s great.

Start with an ask for general feedback "what do you think?"

  • Their first answers will be the things they react most strongly to - "the concerns and delights that they express unprompted are critical"
  • Other prompts"We're here to get your feedback""Here’s a whole bunch of early ideas that I was asked to show you”“I’ll be curious to hear what you think of this one”“Our clients are exploring some possible ideas"
  • "We're here to get your feedback"
  • "Here’s a whole bunch of early ideas that I was asked to show you”
  • “I’ll be curious to hear what you think of this one”
  • “Our clients are exploring some possible ideas"

If there are questions from the participant ("How much will it cost?"), don’t answer. You're not the expert in an interview, they are. Use it as a chance to learn about them:

  • “What would you expect it to be?”
  • “Is that important to you?”

Other prompts

There are lots of tools to help the participant get the right thoughts going on in their head, so you can extract them

  • Casual card sort (put cards in front of participants, ask the participant to organise them. Pay attention to the stories and notes as they get discussed. Observe how concepts are grouped or ranked)
  • Images that resonate ("select the images that depict your ideal online experience with the brand")
  • Homework (bring 3 examples of when you used the product to the interview)
  • Diary study

Chapter 5 - Key Stages of the Interview

Tom’s top line

Be aware of and utilise the different stages an interview will go through so you can operate differently in the different stages

There are stages that an interview goes through. Knowing about them will

  • help you to recognise what's going on
  • allow you to prepare tactically for the different phases

Having 2 people interview is good (one to lead, one to backup). Having more than 2 makes the participant less open. There are some tips in this chapter for working with a novice as the 2nd interviewer.

Use open ended questions:

  • Less good: “What are three things you liked about using the bus?”
  • Good: “Can you tell me about your experience using the bus?”

(We don’t know that they liked anything about their experience on the bus!)

Stages

  1. Crossing the threshold (arriving on site)
  2. Restating objectives (getting everyone on the same page)
  3. Kick-off question (breaking the ice)
  4. Accept the awkwardness (it’s awkward, deal with it and don’t try and make it unawkward)
  5. The tipping point (switching from question-answer to question-story)
  6. Reflection and projection (moving from targeted questions to higher-level “who are you” questions)
  7. The soft close (the interview can continue even once you’ve all started walking to the door)

Doorknob Phenomenon

Be aware of the “doorknob phenomenon” (Physicians and therapists know it): crucial information is revealed just as the patient is about to depart.

Consider keeping your recording device on even if it’s packed up. Even as you are heading to the door, the interview may resume, at the participant’s initiation.

⭐️ Chapter 6 - How to Ask Questions

Tom’s top line

⚠️ This is the most important chapter of the book. ⚠️
These are the tools to make you successful: Preparation, silence a palette of different question types and humble curiosity.


The participant needs to feel comfortable telling you what is true for them, not what they think is true for you.

That can be hard. Here are some ways to increase the chances of getting it right.

Field guide

Prepare a plan/protocol/field-guide to structure the interview, in advance. Use it as a reference, not a script.

It should include any info you might want to reference:

  • The high level research goal
  • The specific questions you want to ask
  • Any background info on the participant

Ask short questions. Don't extend the question with possible answers.

e.g. “What did you have for breakfast yesterday... was it toast or juice?”

Use the silence

Making space for someone else to talk doesn’t come naturally:

"We work in a society that judges us primarily by our own contributions, rather than the way we allow others to make theirs."

The silence is awkward. To help deal with it “slowly repeat 'allow silence' as many times as it takes.”

"After they've answered, continue to be silent. People speak in paragraphs, and they want your permission to go on to the next paragraph."

  • Don't say something because it's true or the participant is wrong.
  • Only say something because it helps the research process.

A Palette of Question Types

Ask about

Ask for

  • Sequence. “Describe a typical workday. What do you do when you first sit down at your station? What do you do next?”
  • Quantity. “How many files would you delete when that happens?”
  • Specific examples. “What was the last movie you streamed?” Compare that question to “What movies do you stream?”. It’s easier to answer the specific questions.
  • Exceptions. “Can you tell me about a time when a customer had a problem with an order?”
  • The complete list. “What are all the different apps you have installed on your smartphone?” No one can answer this fully to start with, so you’ll have to follow up: “What else?”
  • Relationships. “How do you work with new vendors?” Great when you don’t even know enough to ask a specific question. Better to start general than to assume you know something with a question that’s too specific.
  • Organizational structure. “Who do the people in that department report to?”

Probe deeper.

Find what hasn’t been said.

  • Clarification. “When you refer to ‘that,’ you are talking about the newest server, right?”
  • Code words/native language. “Why do you call it the bat cave?”
  • Emotional cues. “Why do you laugh when you mention Best Buy?”
  • Why. “I’ve tried to get my boss to adopt this format, but she just won’t do it....” “Why do you think she hasn’t?”
  • Probe delicately. “You mentioned a difficult situation that changed your usage. Can you tell me what that situation was?”
  • Probe without presuming. “Some people have very negative feelings about the current government, while others don’t. What is your take?” instead of “What do you think about our government?” or “Do you like what the government is doing lately?” This gives the participant room to relate to some people and to reject some people, rather than identifying and tying themselves to a specific identity.
  • Explain to an outsider. “Let’s say that I’ve just arrived here from another decade, how would you explain to me the difference between smartphones and tablets?”
  • Teach another. “If you had to ask your daughter to operate your system, how would you explain it to her?”

Generate contrast

Uncover mental models

  • Compare processes. “What’s the difference between sending your response by fax, mail, or email?”
  • Compare to others. “Do the other coaches also do it that way?”
  • Compare across time. “How have your family photo activities changed in the past five years? How do you think they will be different five years from now?” The future prediction won’t be accurate. That’s OK. It’s meant to help the participant think outside of incremental change and tell you what they would ideally like.

The winding interview

We generally think of things as step-by-step, linear. An interview is not linear: It has lots of potential branches.

New questions will pop into your head. The participant will tangent. You have to keep these branching opportunities in mind and make decisions about which forks to take.

Here are some tips for when there’s too many branches:

  • wait until it comes up again
  • make a note to remember
  • prioritise where to go based on research objectives (some paths are the focus, others aren't)
  • prioritise on what is the best follow up (stay on the same thread instead of jumping, so you keep building rapport)

Don’t correct the participant

Keep the participant as the expert. Don't correct them.

When a participant mispronounced TiVo, Steve also chose to mispronounce it - "I referred to it as the participant did - as Tye-vo. But later, when the client paying for the research asked some of his own questions, he pronounced TiVo correctly as ‘tee-vo.’"

This causes the participant to be pushed out of the role of expert.

"You may hear and see apparent contradictions. People may tell you they value cleanliness and then open a bedroom door to reveal piles of dirty clothes on the floor"

Seeing contradictions should make you think “oh I wonder what I don’t understand here?”. Do not fall into the trap of thinking “hey you just contradicted yourself!”

  • What do they mean by clean?
  • Is it more important to say you're clean to everyone, than to actually be clean?

Don't make questions pass/fail

"As an exercise, imagine asking someone if they know what a USB is. You might even try this out loud. First, ask in a gentle, curious fashion. Next, ask in a judgmental critical tone."

It’s easy for the participant to start assessing what the right answer is, rather than telling you what is right for them.

If You Have to Fix Something, Wait until the End

You are interviewing to learn from them, not to teach them. If you really want to help them wait until the interview is concluded.

Chapter 7 - Documenting the Interview

Tom’s top line

Record your interviews.
Always focus on user behaviours rather than your opinions.
Take time after an interview to review and write a short summary.

Documentation is the data you’ll need later on. “The bottom line is that you should be recording your interviews"

Differentiate between behaviours from the participant and your assumptions that you note during the interview.

"Works 14hr/day for 10yrs" vs "He's a workaholic" (fact vs opinion)

During the interview the interaction with the participant is more important than your documentation (that’s why you’re recording). If you’re taking notes during the interview:

  • Make sure you're still making good eye contact
  • Don't ask participants to wait while you catch up
  • Don't make it obvious what is interesting for you through your intense writing

"you must absolutely allow time for debriefing after each interview … The longer you wait (say, until the next day or the next week), the less you will remember, and the more jumbled up the different interviews will become."

"As soon as possible after an interview, I write a rapid top-of-mind version of the session.”

  • Don’t try and capture the details (that’s why you recorded)
  • Do try and create a story that helps paint the picture of the user’s perspective

Chapter 8 - Optimizing the Interview

Tom’s top line

Understanding what makes an interview go wrong can help to avoid bad interviews.

Common Interview Problems

  • the Participant Is Reticent
  • the Participant Isn’t the Right Kind of User
  • the Participant Won’t Stop Talking
  • You Feel Uncomfortable or Unsafe

Some things that make an interview less than ideal:

  • You and your participant are not in the same location.
  • You meet your participant in a neutral location, out of his context.
  • You have only a short amount of time for the interview.
  • You are interviewing people as “professionals” rather than as “consumers.”
  • You have multiple participants in a single interview.

Chapter 9 - Making an Impact with Your Research

Tom’s top line

After you have interview data you need to analyse it and then synthesise it.
Through this process you should create several outputs - high level themes, your early impressions as stories (aka a Topline) and finally the formal guidance.

"Working with research data is a combination of analysis, or breaking larger pieces into smaller ones … and synthesis … (for example, building themes, implications, and opportunities)."

"You should compile a starter set with anywhere from 5 to 15 thematic areas that build on these debriefs and highlights and address the research objectives. You may also have some clues about new patterns"

"A Topline is based on early impressions, not formal analysis of data. This is a chance to share stories and initial insights from the fieldwork; to discuss what jumped out at us and list questions we still have."

"After the topline report comes the more formal data processing stage. This is important because this step is where you will uncover significant new insights that go well beyond the topline."

Where to get it

If this has inspired you to dive deeper, you can get it here:

https://portigal.com/Books/interviewing-users/

Thanks

Thanks for reading, I’m glad you made it all the way to the end!

If you’ve got any suggestions on how to make it better, I’d love to hear from you on twitter: www.twitter.com/iamtomelliot

(or if you just want to tell me you liked it, I’m into that too)

Get in Touch