Cloud Data Management
Reviewed by: Twange Kasoma
Learn how to manage your data stack and set up processes to get the most out of your data in your organization. We will cover best practices for when you are still querying production all the way up to setting up data marts for different business lines in your organization.
Introduction - The 4 Stages of Data Sophistication
Learn how Data Governance practices change as the level of data sophistication changes.
About this Book
Information about who this book is for, who it's not, how we wrote it, disclaimers, influences and how to contribute.
What Stage are You at?
Figure out what level of data sophistication your team is at.
Starting with Source Data
Learn how to analyze data from applications, production databases, and financial records.
Source Data Connections
Learn how to configure your database to analyze source data effectively.
Source Data Best Practices
Learn how to manage queries, model in a BI tool, and use drag and drop query interfaces.
Stage 1 - Source
Why Build a Data Lake
Learn why you should build a data lake to improve analytics at your company.
What Engine to Use For a Data Lake
Learn the pros and cons of the modern database options like Snowflake, Redshift and BigQuery to build your Data Warehouse on.
Extract and Load a Lake
Learn how to extract and load data sources like SalesForce, Hubspot, Marketo, etc into a single source.
Data Lake Security
Learn best practices for ensuring data security on a Data Lake database.
Data Lake Maintenance
Learn best practices for data lake maintenance. Handle Data Source updates and improve performance.
Stage 2 - Lake
Why Build a Data Warehouse
Data inside of Data Lakes is challenging to work with, because it is messy and not optimized for ad hoc querying. Data in a Data Warehouse is clean, simple, and easy to use.
Data Warehouse Architecture
Learn why you should build a Single Source of Truth in your Data Warehouse. Overcome common obstacles and empower your colleagues
Data Warehouse Security
Learn how to secure sensitive data on your database and BI platform.
Data Warehouse Implementation
Learn how to setup a Data Warehouse. Model and transform data to make it easy to analyze.
Defining a Data Governor
Data Governors maintain the database so that is remains valuable to an organization. This involves security, education, and modeling.
Data Warehouse Maintenance
Learn the best practices to maintain a Data Warehouse. Learn how to add new data, remove deprecated data, and optimize for performance..
Stage 3 - Warehouse
Democratized or Centralized - choosing your workflow (In progress)
Choosing whether data should be completely run by a centralized team, or efforts should be made into enable others in the organization to work with data is a big choice for companies, with great consequences on both sides.
Why Build Data Marts
Learn the best reasons to build a data mart on top of your data warehouse
Data Mart Implementation
Learn the best practices for building a Data Mart
Data Mart Maintenance
Learn Best Practices for Maintaining a a Data Mart, such as handling errors and incorporating new data sources.
Stage 4 - Mart
Evaluating Data Stack Technologies
Learn the various functions a Data Stack needs to perform in order to select the correct data tools to take raw data and turn it into insight.
ETL vs ELT
Learn why you should use an ELT over a ETL process for your Data Lake
Doing more with your Data Mart
Learn why you should use a tool like census to push your data back to your data sources
Acknowledgments & Contributions
This is a community driven book - with contributions from many different people and organizations. Help keep it relevant and continually improving.