Web Books

Cloud Data Management

Written by: Dave Fowler, Matt David, Tim Miller, Tracy Chow, Jaime Flores-Lovo, Aaron Aihini, Kostas Pardalis
Reviewed by: Twange Kasoma

Learn how to manage your data stack and set up processes to get the most out of your data in your organization. We will cover best practices for when you are still querying production all the way up to setting up data marts for different business lines in your organization.

Download book

Introduction

• Introduction - The 4 Stages of Data Sophistication 

  Learn how Data Governance practices change as the level of data sophistication changes.

• About this Book 

  Information about who this book is for, who it's not, how we wrote it, disclaimers, influences and how to contribute.

• What Stage are You at? 

  Figure out what level of data sophistication your team is at.

Stage 1 - Source

• Starting with Source Data 

  Learn how to analyze data from applications, production databases, and financial records.

• Source Data Connections 

  Learn how to configure your database to analyze source data effectively.

• Source Data Best Practices 

  Learn how to manage queries, model in a BI tool, and use drag and drop query interfaces.

Stage 2 - Lake

• Why Build a Data Lake 

  Learn why you should build a data lake to improve analytics at your company.

• What Engine to Use For a Data Lake 

  Learn the pros and cons of the modern database options like Snowflake, Redshift and BigQuery to build your Data Warehouse on.

• Extract and Load a Lake 

  Learn how to extract and load data sources like SalesForce, Hubspot, Marketo, etc into a single source.

• Data Lake Security 

  Learn best practices for ensuring data security on a Data Lake database.

• Data Lake Maintenance 

  Learn best practices for data lake maintenance. Handle Data Source updates and improve performance.

Stage 3 - Warehouse

• Why Build a Data Warehouse 

  Data inside of Data Lakes is challenging to work with, because it is messy and not optimized for ad hoc querying. Data in a Data Warehouse is clean, simple, and easy to use.

• Data Warehouse Architecture 

  Learn why you should build a Single Source of Truth in your Data Warehouse. Overcome common obstacles and empower your colleagues

• Data Warehouse Security 

  Learn how to secure sensitive data on your database and BI platform.

• Data Warehouse Implementation 

  Learn how to setup a Data Warehouse. Model and transform data to make it easy to analyze.

• Defining a Data Governor 

  Data Governors maintain the database so that is remains valuable to an organization. This involves security, education, and modeling.

• Data Warehouse Maintenance 

  Learn the best practices to maintain a Data Warehouse. Learn how to add new data, remove deprecated data, and optimize for performance..

Stage 4 - Mart

• Democratized or Centralized - choosing your workflow (In progress) 

  Choosing whether data should be completely run by a centralized team, or efforts should be made into enable others in the organization to work with data is a big choice for companies, with great consequences on both sides.

• Why Build Data Marts 

  Learn the best reasons to build a data mart on top of your data warehouse

• Data Mart Implementation 

  Learn the best practices for building a Data Mart

• Data Mart Maintenance 

  Learn Best Practices for Maintaining a a Data Mart, such as handling errors and incorporating new data sources.

Extras

• Evaluating Data Stack Technologies 

  Learn the various functions a Data Stack needs to perform in order to select the correct data tools to take raw data and turn it into insight.

• ETL vs ELT 

  Learn why you should use an ELT over a ETL process for your Data Lake

• Doing more with your Data Mart 

  Learn why you should use a tool like census to push your data back to your data sources

• Acknowledgments & Contributions 

  This is a community driven book - with contributions from many different people and organizations. Help keep it relevant and continually improving.