Cognitive biases are, as a part of human nature, frustrating hurdles of conducting unbiased market research
Working in the product management team, you already know how to conduct a market research step by step. (If not, check How to do market research.) Analyzing and visualizing data are no longer the problems for you. However, as you experienced more, you realized the biggest challenge is to collect unbiased data in market research. Because biases can easily bend the objective truth, leading you to incorrect conclusions about your product.
Biased samples can appear in both qualitative and quantitative research. To minimize biases, you need to know what and how biases could appear. Here are 10 major biases throughout the market research process.
Before research – selecting samples
- Sampling Bias occurs when certain groups are omitted or samples are selected for convenience. For example, relying on existing customers could result in overly favorable answers to questions compared to a random sample.
Solution: Plan and organize the sample collecting process to make sure samples are random and representative.
- Self-Selection Bias results from individuals selecting themselves into a group on a voluntary basis. One common case is in mail surveys. People with extreme (either positive or negative) opinions tend to respond, while those with moderate rating are indifferent to reply.
Solution: Keep your questionnaire short & easy, send reminders, offer incentives to respond, and make respondents aware their information is confidential.
During research – asking and answering questions
- Confirmation Bias occurs when searching for information to validate your hypothesis. For example, If you believe the price is the most important factor, you may ask “How important was the price in your purchasing decision?” while respondents might not have this factor in mind at first.
Solution: Ask open-ended questions, prompt them with possible answers and ask more specific questions later in the interview.
- Question-Order Bias happens because one question can influence answers to subsequent questions. Let’s try. Q1.What is your favorite thing about elephants? Keep your answer in mind and move on to Q2. Name an animal that is grey. Are you thinking of an elephant?
Solution: Ask general questions before specific, unaided before aided and positive before negative.
- Memorable Response Bias is the tendency of only remembering the extremely positive and extremely negative feedback. It can twist how researchers interpret results while most customer or prospect experience is somewhere in between.
Solution: Keep detailed notes or recordings during the experiment or observation.
- Social Desirability Bias is the tendency of answering questions in a way that they think will be accepted and liked. Some people report inaccurately on sensitive or personal topics to portray the best image of themselves.
Solution: Communicate and implement anonymous random model surveys, keep the purpose of the survey vague, word questions.
- Habituation is the tendency of providing the same answers to questions that are worded in similar ways because paying attention takes energy. You can catch the signs of fatigue, such as mentioning that the questions seem repetitive or start giving similar responses across multiple questions.
Solution: Keep the engagement conversational and vary the question wording.
After research – perceiving, analyzing and reporting results
- Cultural Bias is the tendency to assume that everyone sees the world the same way and hold similar values. You may attach a positive value to the answer you subconsciously believe is “correct.”
Solution: Best to conduct and analyze research through a partner who can straddle any potential cultural divides or at least show unconditional positive regard for any answer a respondent might make.
- Knowledge Bias appears because people sometimes prefer familiar options over options that are objectively better. Coca-Cola had blind taste tests on their New Coke, resulting in customers’ preference on New Coke. However, it turned out people actually went for the more familiar option.
Solution: Keep this bias in mind during objective product testing. Since brand familiarity and loyalty play a big role on the purchase decision, dig deeper into respondents’ motivations behind likelihood to buy or switch brands.
- Irrational Escalation may show up when you’ve produced a product and put it up for sale and all of a sudden the new research shows customers may not be interested in the product. Rather than accepting you made a mistake and cutting your losses in time, you convince yourself that the research is wrong, and you just need to push your product harder.
Solution: Be open to what the results are telling you. Ask yourself: if you haven’t already launched the product, with the information you’re now receiving, would you still launch?
Biases are inevitable, but keep striving, be open-minded
It is almost impossible to completely eliminate all the biases. But never stop striving for bias-free market research. Other than watching out for cognitive biases through interviews and surveys, you can ask participants and peers to have some reviews and check for alternative explanations why you got your data. Since data serves as a foundation of research analysis, the more objective and accurate research you conduct, the more helpful the market research could be to lead your business onto the right path.