Web Books

Avoid Misrepresenting Data

Written by: Matt David
Reviewed by: Blake Barnhill

There is so much data you have access to within a company. Communicating accurate insights from company data is challenging. This book covers the mental biases and common mistakes that people make when analyzing data. It then provides guidance on how to prevent and avoid these costly errors.

Cognitive Biases

• Confirmation Bias 

  Confirmation bias negatively affects the accuracy of your analysis. Learn how to detect Confirmation Bias and how to avoid this cognitive bias.

• Selection Bias 

  Selection bias negatively affects your analysis. Learn how to detect Selection Bias and how to avoid this cognitive bias.

• Survivorship Bias 

  Survivorship bias negatively affects your analysis. Learn how to detect Survivorship Bias and how to avoid this cognitive bias.

Analysis Mistakes

• Statistic vs Distribution 

  Distributions provide much more nuance to the data than a statistic does. Learn how to query to get distributions and then interpret them.

• Overall vs Groups 

  Overall statistics in data can be misleading because there may be distinct groups within the data that have very different statistics. Learn to avoid this analysis mistake.

• Metric vs Metrics 

  A single Metric can be misleading. Learn how to use multiple Metrics to avoid misleading yourself and others.

• Trends 

  Trend in data can be misleading depending on what time frame you are looking at it. Learn how to accurately interpret trends.

• Comparing to Historical Averages (In progress) 

  Historical Averages can be misleading because they hide trends in the data. Learn to identify misleading historical statistics.

• Relative vs Absolute Change 

  Relative and Absolute changes can bias your interpretation of data you are analyzing. Learn to interpret them correctly.

Experiment Design

• Predicting Outcomes 

  Predicting Outcomes prevents cognitive biases from affecting how your interpretation of the results of an experiment. Learn more.

• Define Experiment Parameters 

  Defining Experiment Parameters improves analysis and increases trust in results that are shared in an organization.

• Review Outcomes 

  Review Outcomes of Feature Releases to evaluate their impact and to create institutional knowledge. Learn how to interpret results accurately.

Extras

• Increase Ecommerce Sales with Metrics 

  Learn why using a single metric prevents you from being able to increase ecommerce sales. See how to use multiple metrics to get more sales.