Everyone wants to build a better product - but what’s the best way to learn from customers and users who are sharing their opinions and behavior patterns across a multitude of channels? Educating yourself about the two main types of community data along with how to collect, understand, and present each is a great place to start.
Know your data types
Emotional by nature, qualitative data is largely made up of what users and customers are saying. While incredibly valuable, all qualitative samples must be carefully considered - particularly when shared with teams and stakeholders. Reason being that a small sample of verbal or written feedback can be easily taken out of context while failing to represent your community as a whole.
I encountered this kind of problem with negative qualitative feedback while working on community engagement at JustAnswer.com, a platform where experts give customers paid answers online. There, we continually released new features and tools designed to aid the experts in providing spectacular service. If we went by qualitative feedback only, we would have stopped what we were working on because the internal expert community seemed to be in a dismal state, as shown by heated commentary in the company’s expert forum! However, in context we knew that only a small percentage of active experts actually spent time visiting and posting there.
Positive qualitative feedback taken out of context can also be a problem, as it has the power to send a team chasing ‘shiny objects’ based on a single individual or small group of users’ experience or opinion. Since you never want qualitative feedback to be a distraction, it’s best shared with anecdotes to maintain focus on roadmaps and goals. This is especially important when working to establish product-market fit.
As you can see, qualitative data is strongest when used in early stages of exploration as it’s valuable for uncovering small but important pain points, inspiring creative thinking, and even introducing brand new ideas.
Quite black and white, quantitative data is often favored for making quick, effective and data-driven decisions or validating ‘gut’ guesses. This data is made up of numbers rather than words and showcases how your community interacts with your product on a larger scale.
As you might imagine, the issue with quantitative data is exactly its benefit: a lack of color. While some things require logic, building a product for people often requires a deeper or more broad insight. Ideas, potential uses, edge cases, and problems affecting user segments are not always (or accurately) represented by the majority in numbers.
If you’re a community manager or someone who works in customer support, you’re likely to encounter a scenario like this when advocating for the user or customer. While quantitative data may point strongly toward trends or make an ‘obvious’ decision easy, pieces of qualitative feedback you’ve captured can be critical in sharing the whole story.
Be thoughtful in your research
“Collecting data” might sound like a drab or methodical task, but it doesn’t have to be! The human aspect of qualitative feedback lends itself nicely to conversation and engagement. Consider trying some of the following ideas:
- Conducting a focus group
- Setting up phone or video calls with users that meet your criteria
- Monitoring social conversations (consider using Hootsuite or community forum software)
- Sending out a survey
I spent the majority of my first two weeks at payments startup Tilt on the phone with various groups of users. Not only did this give me a good idea of what the product was being used for, but Tilters’ questions also provided me with immediate insight into where we could improve. Thanks to these conversations, I was able to put together a process (survey, more phone calls) to collect more specific feedback as we explored tailoring the messaging and content on our site subpages. We later confirmed the necessity of the changes by looking at quantitative data which showed us which pages actually saw the most traffic, and how that was truly impacting a new user’s experience with understanding the product. It all lined up!
Since quantitative data is cut and dry, you’ll find the numbers you need in one or more of these places:
- Analytic tools
- User tracking metrics
- Test results (A/B tests are a great example of quick and effective quantitative data!)
- Growth experiments
- NPS data (good for a monthly community pulse check and to validate hunches about happiness)
If you’re not confident about your analytical chops, I suggest teaming up with someone at work who is or taking an online class or two. Even a small bit of learning can help improve your working knowledge immensely!
Avoid bias the best you can
From selecting users to talk to or track to moderation, bias can quietly creep up into your data collection process. Karina van Schaardenburg, a User Researcher at FourSquare, recently authored a great post about qualitative data collection in which she wisely states, “There isn’t an unbiased solution, leaving me to choose the bias I am most comfortable living with.”
As Karina writes, acknowledging that bias is present and doing your best to remove anything that may skew the broad picture will help you share the most accurate representation of your community and their activities. Some of her top tips include thoughtfully selecting your method and user groups (this includes who not to talk to as well), along with carefully considering the questions you ask. For best results, she advises avoiding time-based questions, questions where the answer is obvious, and those that are too narrow early on in the process.
Successfully present your findings
It’s hopefully become obvious that combining both types of data is the best way to produce a solid look at what’s happening with your community and how they use the product. But before you start sharing anything, consider first who you’re presenting your findings to.
Is the audience a single person or an entire team? Are your listeners technical, analytical, or creative? Are there business-focused stakeholders, executives, or owners in the mix?
Once you think about the composition of your audience, you’ll be able to paint the right kind of picture for their knowledge and needs. Take time to anticipate which questions your listeners or readers will be most inclined to ask, and arm yourself with truthful responses. When including qualitative feedback examples, do your best to have quantitative data to back it up.
Remember that data and feedback are never perfect, and there’s always the possibility to find more. It’s your job to complete the most thoughtful research possible through the most effective methods, remove bias, and present your findings in the way that speaks to you most strongly. Once you do this successfully, your community and users will thank you for it with strong loyalty, improved retention rates, and even more feedback that you can continue to use to build your product and company.