How to shine when it feels like you’re drowning in great research findings

At the start of a project we often run a phase of discovery research where we aim to learn as much as we possibly can to understand the users. And then as we move on our research narrows and we focus more on testing usability of prototypes through to fully developed services. As design and development ramps up for these, researchers need to rapidly turnaround findings and recommendations to avoid becoming a blocker. We need to narrow our focus of findings to help the team, but at the same time we don’t want to stop learning. But even with a tightly planned research session, our users will surprise us and turn up a new situation or richer understanding of a user need that we’ve not considered before so we’ve got more findings come out than we planned for.

It might seem a small problem, and a nice one in the realm of others, but it is one that can be tricky to deal with.
A fellow researcher recently told me of the effect it was having on their team. They were being pushed to analyse and quickly share findings about a prototype, but there was so much rich data coming out of the sessions that they were starting to feel overwhelmed. They felt there was too much to deal with. This was slowing them down and, as they tried to focus on the findings about the prototype they worried they would be losing all those other interesting insights.

One way to approach to help resolve this problem is to use a decision tree within the analysis to guide a way through the findings and quickly decide how to deal with them.

A few years ago I developed a findings decision tree like this, as a training tool for junior researchers to help them learn how to analyse large amounts of data and shape how they reported their findings. The intent was to guide researchers through the questions to support their training and enable them to pick up the skills needed to evaluate the evidence for a strong finding and learn to critically think about underlying causes of an issue.

The decision tree did help the junior researchers but what surprised us was that is also became useful for more experienced researchers. It helped them because it was like a checklist through their rough findings and that improved their efficiency. It also helped with collaborative work because it was an objective process they could point to explain their thinking and decisions.
To be honest this didn’t come as too much of a surprise to me, as my inspiration was the incredible value of checklists described by Atul Gwande in his book, The Checklist Manifesto. If you’ve not read it then I highly recommend it.

A decision tree for drowning in research findings

Here’s a draft decision tree. I’ve assumed the team is working in an iterative agile approach looking to add key findings in regular show and tell deliveries, but it could be adapted to any working pattern.

  1. Does it help answer an immediate research question (e.g. about the prototype)?
    • Yes – put it in the show and tell deck
    • No – carry on
  2. Is it about a user need?
    • Yes – carry on
    • No – jump to Q 5
  3. Does it highlight a user need that’s not already identified?
    • Yes – create a user need list or add it to the list
    • No – carry on
  4. Does it add a new view on an existing user need?
    • Yes – add a note to the user need
    • No – carry on
  5. Does it highlight a pain point with the service that’s not already identified?
    • Yes – create an other pain points list or add it to an existing list
    • No – carry on
  6. Will it affect the business or uptake of the service?
    • Yes – consider adding it to show and tell deck
    • No – carry on
  7. Does it add positive feedback for the service
    • Yes – add it to show and tell deck, we all love a bit of positive feedback!
    • No – carry on
  8. Does it help understand a group of users?
    • Yes – carry on
    • No – go to Q 11
  9. Has the group been not previously identified?
    • Yes – create a user group list or add it to an existing list, and consider adding it to show and tell deck
    • No – carry on
  10. Does it add new information on existing groups
    • Yes – add it to the user groups list
    • No – carry on
  11. Will it help another related service?
    • Yes – create an other service list or add it to an existing list for them
    • No – carry on (only one more to go in this example!)
  12. Will it help explain the context or help you tell a story about the users?
    • Yes – consider how you can weave it into your show and tell presentation and the final delivery
    • No – let it go, and make a cup of tea, you deserve it!

As you see with the flow, it assumes you can create lists of user needs, user groups, pain points, other services. These become the repositories for all those interesting insights that pour out of the research but are not answers to the immediate research question. They then solve the problem of what to do with all the interesting insights (it doesn’t solve what you might do with them after, that’s another topic).

The decision tree also helps solve the problem of becoming overwhelmed with a lot of rich findings. The structured way of thinking about the findings the tree allows researchers to rapidly decide what to do with this, and this, and this, and frees up time to discuss only the ones that are really challenging to place.

For some, I anticipate that a decision tree or checklist approach might feel too constrained and rigid or perhaps unnecessary. So let me argue the benefits as I see them.

Using a structured approach offloads a lot of cognitive effort and frees up your time and brain. That’s good for you as a researcher because you need that time and brain power to focus on doing the research and figuring out how to present it to the team and stakeholders for maximum impact so they will act upon it.
Using a structured approach also helps you make the most of any research you can do. Often we are limited in the number of participants we can see in any development phase. Using a structured approach gives you more time and allows you to demonstrate that you have saturated your analysis from the sessions you held and you got the full value out of the research. It’s also a true ethical response to your participant’s time.
Stepping back from the actual research, a decision tree or checklist helps you make your ways of working transparent and understandable. Printed out, it becomes an easy guide for everyone to follow in group research and analysis sessions, enabling everyone to add value right away. Shared with stakeholders, it is a simple visual artefact that demonstrates your attention to detail and commitment to transparency. Providing transparency like this helps us work with everyone and helps them understand and champion user research, which is never a bad thing.

So next time you’re immersed in some research and findings are coming a-plenty, why not try out a decision tree approach? If you’re starting to feel overwhelmed then you’re not able to deliver your best for the team. A simple tool like a decision tree can free up some head space, help engage your stakeholders and make you shine.

Leave a Comment