How to Conduct Unbiased Market Research for Your Product

Cognitive biases are, as a part of human nature, frustrating hurdles of conducting unbiased market research

Working in the product management team, you already know how to conduct a market research step by step. (If not, check How to do market research.) Analyzing and visualizing data are no longer the problems for you. However, as you experienced more, you realized the biggest challenge is to collect unbiased data in market research. Because biases can easily bend the objective truth, leading you to incorrect conclusions about your product.

Biased samples can appear in both qualitative and quantitative research. To minimize biases, you need to know what and how biases could appear. Here are 10 major biases throughout the market research process.

Before research – selecting samples

Researchers

  • Sampling Bias occurs when certain groups are omitted or samples are selected for convenience. For example, relying on existing customers could result in overly favorable answers to questions compared to a random sample.

Unbiased SolutionSolution: Plan and organize the sample collecting process to make sure samples are random and representative.

Respondents

  • Self-Selection Bias results from individuals selecting themselves into a group on a voluntary basis. One common case is in mail surveys. People with extreme (either positive or negative) opinions tend to respond, while those with moderate rating are indifferent to reply.

Unbiased SolutionSolution: Keep your questionnaire short & easy, send reminders, offer incentives to respond, and make respondents aware their information is confidential.

During research – asking and answering questions

Researchers

  • Confirmation Bias occurs when searching for information to validate your hypothesis. For example, If you believe the price is the most important factor, you may ask “How important was the price in your purchasing decision?” while respondents might not have this factor in mind at first.

Unbiased SolutionSolution: Ask open-ended questions, prompt them with possible answers and ask more specific questions later in the interview.  

  • Question-Order Bias happens because one question can influence answers to subsequent questions. Let’s try. Q1.What is your favorite thing about elephants? Keep your answer in mind and move on to Q2. Name an animal that is grey. Are you thinking of an elephant?

Unbiased SolutionSolution: Ask general questions before specific, unaided before aided and positive before negative.

  • Memorable Response Bias is the tendency of only remembering the extremely positive and extremely negative feedback. It can twist how researchers interpret results while most customer or prospect experience is somewhere in between.

Unbiased SolutionSolution: Keep detailed notes or recordings during the experiment or observation.

Respondents

  • Social Desirability Bias is the tendency of answering questions in a way that they think will be accepted and liked. Some people report inaccurately on sensitive or personal topics to portray the best image of themselves.

Unbiased SolutionSolution: Communicate and implement anonymous random model surveys, keep the purpose of the survey vague, word questions.

  • Habituation is the tendency of providing the same answers to questions that are worded in similar ways because paying attention takes energy. You can catch the signs of fatigue, such as mentioning that the questions seem repetitive or start giving similar responses across multiple questions.

Unbiased SolutionSolution: Keep the engagement conversational and vary the question wording.

After research – perceiving, analyzing and reporting results

Researchers

  • Cultural Bias is the tendency to assume that everyone sees the world the same way and hold similar values. You may attach a positive value to the answer you subconsciously believe is “correct.”

Unbiased SolutionSolution: Best to conduct and analyze research through a partner who can straddle any potential cultural divides or at least show unconditional positive regard for any answer a respondent might make.

  • Knowledge Bias appears because people sometimes prefer familiar options over options that are objectively better. Coca-Cola had blind taste tests on their New Coke, resulting in customers’ preference on New Coke. However, it turned out people actually went for the more familiar option.

Unbiased SolutionSolution: Keep this bias in mind during objective product testing. Since brand familiarity and loyalty play a big role on the purchase decision, dig deeper into respondents’ motivations behind likelihood to buy or switch brands.

  • Irrational Escalation may show up when you’ve produced a product and put it up for sale and all of a sudden the new research shows customers may not be interested in the product. Rather than accepting you made a mistake and cutting your losses in time, you convince yourself that the research is wrong, and you just need to push your product harder.

Unbiased SolutionSolution: Be open to what the results are telling you. Ask yourself: if you haven’t already launched the product, with the information you’re now receiving, would you still launch?

Biases are inevitable, but keep striving, be open-minded

It is almost impossible to completely eliminate all the biases. But never stop striving for bias-free market research. Other than watching out for cognitive biases through interviews and surveys, you can ask participants and peers to have some reviews and check for alternative explanations why you got your data. Since data serves as a foundation of research analysis, the more objective and accurate research you conduct, the more helpful the market research could be to lead your business onto the right path.

Resources

https://www.forbes.com/sites/jaysondemers/2015/03/26/5-cognitive-biases-to-avoid-in-your-market-research/#1c3cfc464aee

https://www.djsresearch.co.uk/glossary/item/Biased-Market-Research-Sample

https://labs.openviewpartners.com/4-distortive-market-research-biases-to-avoid/#.WxgBw1MvzLY

https://www.surveygizmo.com/resources/blog/4-ways-to-eliminate-bias-in-market-research/

Yuyan (Fiona) Mao

About Yuyan (Fiona) Mao

Hi! I'm Yuyan (Fiona) Mao. I'm currently pursuing my Master's degrees in International Business and Business Analytics from Hult International Business School and received my Bachelor in Economics from University of Maryland. I care about efficiency, profitability and feasibility. I believe in data and communication. So I'm excited to share my knowledge and keep learning at the Data School by Chartio.