Help! I'm drowning in findings!
One of my fellow researchers wanted help with a recent project. They were running usability sessions to test out prototype designs that were a result of an earlier discovery research phase. So they were focused on design testing, but in the analysis they realised they were learning more about their user needs and their situations. In fact there was so much new information that they were feeling overwhelmed and not sure how to organise their analysis to support the design team needs and to synthesise the new user insights into the existing knowledge all in a timely manner.
At the time, I mucked in to help them out and it ended well, but I later reflected on how we could make this easier in the future. One approach could be to use a decision tree to guide through analysis and decide how to deal with findings.
How a decision tree can help
In another role, I had developed a findings decision tree to help train junior researchers to analyse large amounts of data and develop actionable findings. The decision tree helped guide researchers through questions to help them critically think and evaluate evidence, think about the underlying causes of issues and identify robust findings.
Experienced researchers also found the decision tree useful because it gave them a checklist to work through rough findings. It improved their efficiency and helped with collaborative analysis because it provided an objective process they could use to frame discussions. Check out the benefits of using checklists in Atul Gwande's book, The Checklist Manifesto. If you’ve not read it then I recommend it.
An adapted decision tree for researchers drowning in research findings
My original decision tree was specific to a situation but the concept is generalisable. I have drafted out a more general one below that could be used by researchers in agile teams.
1. Does it help answer an immediate research question (e.g. about the prototype)?
Yes – put it in the show and tell deck
No – carry on
2. Is it about a user need?
Yes – carry on
No – jump to Q 5
3. Does it highlight a user need that’s not already identified?
Yes – create a user need list or add it to the list
No – carry on
4. Does it add a new view on an existing user need?
Yes – add a note to the user need
No – carry on
5. Does it highlight a pain point with the service that’s not already identified?
Yes – create an other pain points list or add it to an existing list
No – carry on
6. Will it affect the business or uptake of the service?
Yes – consider adding it to show and tell deck
No – carry on
7. Does it add positive feedback for the service
Yes – add it to show and tell deck, we all love a bit of positive feedback!
No – carry on
8. Does it help understand a group of users?
Yes – carry on
No – go to Q 11
9. Has the group been not previously identified?
Yes – create a user group list or add it to an existing list, and consider adding it to show and tell deck
No – carry on
10. Does it add new information on existing groups
Yes – add it to the user groups list
No – carry on
11. Will it help another related service?
Yes – create an other service list or add it to an existing list for them
No – carry on (only one more to go in this example!)
12. Will it help explain the context or help you tell a story about the users?
Yes – consider how you can weave it into your show and tell presentation and the final delivery
No – let it go, and make a cup of tea, you deserve it!
The outputs are lists of user needs, user groups, and pain points that supplement your existing knowledge. They capture the learnings that are outside the focus of the design research so that nothing is lost from the time spent with the participants.
Benefits of using a decision tree
A decision tree gives you a structure approach to reviewing the content from research sessions.
**A structured approach offloads a lot of cognitive effort**. This is great for your brain because it frees up space and energy to think about your insights and everything else you need to do.
**A simple documented approach makes your processes more transparent and understandable**. This supports working in the open with your team and helps them understand critical thinking processes that researchers apply to decide how robust a finding might be.
**The decision tree is a tool that can help team collaboration during analysis.** Part of the user research role is to engage the team and draw on their individual expertise and insights to develop rounded insights from the research. The decision tree can help those who are less familiar with analysis techniques and for those that have experience, bring everyone onto the same page to help the team work together.
Using the decision tree
I've used the decision tree in a couple of different ways this year. In one project I had to combine discovery and design research in the same sessions and then super-rapidly analyse and provide all the findings and insights. It simple questions helped me focus and deliver all the outputs on time.
In another project I was asked to pull together a summary of previous research which had been done over several phases by different researchers. Each phase had been approached differently and each researcher documented their work in different ways so I needed an objective and structured approach to bring it all together. I used an adapted decision tree to ensure I reviewed each piece of work in the same way so that future researchers could understand my decision processes and follow an audit trail back to the original work.
I'd be interested to hear from other researchers who use similar tools and techniques to support their analysis. How does it work for you? And what situations has it helped with?