Data Lake

A data lake differs from a data warehouse in its purposes.

A data warehouse is a database that’s optimized to analyze relational data from a line of business applications and transactional systems. The data structure and schema are defined to optimize for fast queries to be used for reporting and analytical purposes. This data is cleaned, organized, and formatted to act as a single data source.

A data lake stores relational data from business applications as well, but it also stores non-relational data from internet of things devices, social media, and mobile apps. The data structure is undefined at the time of capture. All of this data is stored without regard for organization or potential uses in the future. Machine learning, real-time analytics, full text search, big data analytics, queries, and other types of analytics can be performed with the data stored in a data lake.

Data lakes offer data from multiple sources in a short period of time, giving users the ability to collaborate and analyze data in multiple ways. This can improve customer interactions, research and development, and operational efficiencies by informing the decision-making process.

Because data lakes lack structure, it’s easy to make changes to queries and models. They are flexible and easily configured for the task at hand. It’s much more time-consuming to alter the structure of a data warehouse.

Both data lakes and data warehouses offer access to all users, but data scientists are usually the ones using a data lake, as they have the skills needed for deep analysis. Data warehouses are used by specific users looking to report and extract particular insights (usually analytics) from available data. Many data scientists find the structure too restrictive for their purposes.

Data warehouses are more mature than data lakes, giving them more security. Data stored in a data lake is centralized, and the central location makes auditing and compliance easier.

Many organizations use both data lakes and data warehouses to enable diverse capabilities to inform business processes. For organizations with an established data warehouse, a data lake can address many of the restrictions within a data warehouse.