Alex Gorelik | The Enterprise Big Data Lake. Delivering the Promise of Big Data and Data Science (2019) [PDF] [EN]

The Enterprise Big Data Lake

Preface

1. Introduction to Data Lakes
Data Lake Maturity
Data Puddles
Data Ponds
Creating a Successful Data Lake
The Right Platform
The Right Data
The Right Interface
The Data Swamp
Roadmap to Data Lake Success
Standing Up a Data Lake
Organizing the Data Lake
Setting Up the Data Lake for Self-Service
Data Lake Architectures
Data Lakes in the Public Cloud
Logical Data Lakes
Conclusion

2. Historical Perspective
The Drive for Self-Service Data—The Birth of Databases
The Analytics Imperative—The Birth of Data Warehousing
The Data Warehouse Ecosystem
Storing and Querying the Data
Loading the Data—Data Integration Tools
Organizing and Managing the Data
Consuming the Data
Conclusion

3. Introduction to Big Data and Data Science
Hadoop Leads the Historic Shift to Big Data
The Hadoop File System
How Processing and Storage Interact in a MapReduce Job
Schema on Read
Hadoop Projects
Data Science
What Should Your Analytics Organization Focus On?
Machine Learning
Explainability
Change Management
Conclusion

4. Starting a Data Lake
The What and Why of Hadoop
Preventing Proliferation of Data Puddles
Taking Advantage of Big Data
Leading with Data Science
Strategy 1: Offload Existing Functionality
Strategy 2: Data Lakes for New Projects
Strategy 3: Establish a Central Point of Governance
Which Way Is Right for You?
Conclusion

5. From Data Ponds/Big Data Warehouses to Data Lakes
Essential Functions of a Data Warehouse
Dimensional Modeling for Analytics
Integrating Data from Disparate Sources
Preserving History Using Slowly Changing Dimensions
Limitations of the Data Warehouse as a Historical Repository
Moving to a Data Pond
Keeping History in a Data Pond
Implementing Slowly Changing Dimensions in a Data Pond
Growing Data Ponds into a Data Lake—Loading Data That's Not in the Data Warehouse
Raw Data
External Data
Internet of Things (IoT) and Other Streaming Data
Real-Time Data Lakes
The Lambda Architecture
Data Transformations
Target Systems
Data Warehouses
Operational Data Stores
Real-Time Applications and Data Products
Conclusion

6. Optimizing for Self-Service
The Beginnings of Self-Service
Business Analysts
Finding and Understanding Data—Documenting the Enterprise
Establishing Trust
Provisioning
Preparing Data for Analysis
Data Wrangling in the Data Lake
Situating Data Preparation in Hadoop
Common Use Cases for Data Preparation
Analyzing and Visualizing
The New World of Self-Service Business Intelligence
The New Analytic Workflow
Gatekeepers to Shopkeepers
Governing Self-Service
Conclusion

7. Architecting the Data Lake
Organizing the Data Lake
Landing or Raw Zone
Gold Zone
Work Zone
Sensitive Zone
Multiple Data Lakes
Advantages of Keeping Data Lakes Separate
Advantages of Merging the Data Lakes
Cloud Data Lakes
Virtual Data Lakes
Data Federation
Big Data Virtualization
Eliminating Redundancy
Conclusion

8. Cataloging the Data Lake
Organizing the Data
Technical Metadata
Business Metadata
Tagging
Automated Cataloging
Logical Data Management
Sensitive Data Management and Access Control
Data Quality
Relating Disparate Data
Establishing Lineage
Data Provisioning
Tools for Building a Catalog
Tool Comparison
The Data Ocean
Conclusion

9. Governing Data Access
Authorization or Access Control
Tag-Based Data Access Policies
Deidentifying Sensitive Data
Data Sovereignty and Regulatory Compliance
Self-Service Access Management
Provisioning Data
Conclusion

10. Industry-Specific Perspectives
Big Data in Financial Services
Consumers, Digitization, and Data Are Changing Finance as We Know It
Saving the Bank
New Opportunities Offered by New Data
Key Processes in Making Use of the Data Lake
Value Added by Data Lakes in Financial Services
Data Lakes in the Insurance Industry
Smart Cities
Big Data in Medicine

Index