This is an automated email from the ASF dual-hosted git repository.
vinothchandar pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 9d3d1352c336 docs(blog): explainer wave 2: streaming data lake, CDC,
lakehouse comparisons (#19270)
9d3d1352c336 is described below
commit 9d3d1352c3360de42d35d1aba305ea2ca1e842fa
Author: vinoth chandar <[email protected]>
AuthorDate: Tue Jul 21 16:18:28 2026 -0700
docs(blog): explainer wave 2: streaming data lake, CDC, lakehouse
comparisons (#19270)
---
.../2021-07-21-streaming-data-lake-platform.md | 4 +
.../blog/2024-07-11-what-is-a-data-lakehouse.md | 4 +
website/blog/2024-07-30-data-lake-cdc.md | 4 +
.../2026-07-21-what-is-a-streaming-data-lake.md | 117 ++++++++++++++++++
.../blog/2026-07-22-what-is-cdc-on-a-data-lake.md | 100 ++++++++++++++++
...-23-lakehouse-vs-data-warehouse-vs-data-lake.md | 133 +++++++++++++++++++++
...26-07-24-open-table-format-vs-data-lakehouse.md | 96 +++++++++++++++
website/static/llms.txt | 4 +
8 files changed, 462 insertions(+)
diff --git a/website/blog/2021-07-21-streaming-data-lake-platform.md
b/website/blog/2021-07-21-streaming-data-lake-platform.md
index 9db7ac21fb03..84a00a1b33fc 100644
--- a/website/blog/2021-07-21-streaming-data-lake-platform.md
+++ b/website/blog/2021-07-21-streaming-data-lake-platform.md
@@ -28,6 +28,10 @@ But first, we needed to tackle the basics - transactions and
mutability - on the
This is going to be a rather long post, but we will do our best to make it
worth your time. Let’s roll.
+:::info Related reading
+For a 2026 explainer companion to this essay, see [What is a Streaming Data
Lake?](/blog/2026/07/21/what-is-a-streaming-data-lake)
+:::
+
## Data Lake Platform
We have noticed that, Hudi is sometimes positioned as a “[table
format](https://cloud.google.com/blog/products/data-analytics/getting-started-with-new-table-formats-on-dataproc)”
or “transactional layer”. While this is not incorrect, this does not do full
justice to all that Hudi has to offer.
diff --git a/website/blog/2024-07-11-what-is-a-data-lakehouse.md
b/website/blog/2024-07-11-what-is-a-data-lakehouse.md
index dd97b28c2454..8aa832bc5e3b 100644
--- a/website/blog/2024-07-11-what-is-a-data-lakehouse.md
+++ b/website/blog/2024-07-11-what-is-a-data-lakehouse.md
@@ -21,6 +21,10 @@ A data lakehouse is a hybrid data architecture that combines
the best attributes
The lakehouse architecture supports the management of various data types, such
as structured, semi-structured, and unstructured, and caters to a wide range of
use cases, including business intelligence, machine learning, and real-time
streaming. This flexibility enables businesses to move away from the
traditional two-tier architecture—using warehouses for relational workloads and
data lakes for machine learning and advanced analytics. As a result,
organizations can reduce operational co [...]
+:::info Related reading
+Newer companion explainers to this guide: [What is an Open Table
Format?](/blog/2026/07/14/what-is-an-open-table-format), [Data Lakehouse vs
Data Warehouse vs Data
Lake](/blog/2026/07/23/lakehouse-vs-data-warehouse-vs-data-lake), and [Open
Table Format vs Data
Lakehouse](/blog/2026/07/24/open-table-format-vs-data-lakehouse).
+:::
+
## The Evolution of Data Storage Solutions: How did we go from Warehouses to
Lakes to Lakehouses?
Historically, organizations have been investing in building centralized and
scalable data architectures to enable more data access and to support different
types of analytical workloads. As demand for these workloads has grown, data
architectures have evolved to address the complex needs of modern data
processing and storage.
diff --git a/website/blog/2024-07-30-data-lake-cdc.md
b/website/blog/2024-07-30-data-lake-cdc.md
index 9954f120611f..5ae4291dc843 100644
--- a/website/blog/2024-07-30-data-lake-cdc.md
+++ b/website/blog/2024-07-30-data-lake-cdc.md
@@ -14,6 +14,10 @@ tags:
Data lake change data capture (CDC) is a technique that continuously
identifies and captures changes—inserts, updates, and deletes—from source
databases and applies them to a data lake, keeping lake tables fresh and
consistent with operational systems. Combining CDC with data lakes addresses
several challenges commonly faced by ETL pipelines delivering data from
transactional databases to analytical systems: maintaining data freshness,
ensuring consistency, and improving efficiency in da [...]
+:::info Related reading
+For a beginner-friendly 2026 companion to this deep dive, see [What is CDC on
a Data Lake?](/blog/2026/07/22/what-is-cdc-on-a-data-lake)
+:::
+
## Introduction
In data management, two concepts have garnered significant attention: data
lakes and change data capture (CDC).
### Data Lake
diff --git a/website/blog/2026-07-21-what-is-a-streaming-data-lake.md
b/website/blog/2026-07-21-what-is-a-streaming-data-lake.md
new file mode 100644
index 000000000000..368943724e9b
--- /dev/null
+++ b/website/blog/2026-07-21-what-is-a-streaming-data-lake.md
@@ -0,0 +1,117 @@
+---
+title: "What is a Streaming Data Lake?"
+excerpt: "Defines the streaming data lake: a data lake that ingests and serves
data continuously with minute-level freshness, and the storage building blocks
that make it possible."
+description: "What a streaming data lake is, how it differs from batch data
lakes and Kafka + warehouse stacks, and the storage building blocks that make
it work."
+authors: [vinoth-chandar]
+category: how-to
+image: /assets/images/blog/hudi_streaming.png
+tags:
+- streaming
+- data lakehouse
+- mor
+- cdc
+- incremental processing
+---
+
+A streaming data lake is a data lake that ingests and serves data continuously
— with minute-level freshness — instead of through periodic batch loads.
Batch-era data lakes refresh on a schedule, typically hourly or daily, which
was acceptable when the lake mostly fed reports and offline model training.
Today, operational analytics, change data capture (CDC) from transactional
databases, and AI applications all want much fresher data — and the traditional
answers, standing up a separate [...]
+
+<!--truncate-->
+
+This post is the definitional companion to the original 2021 essay, [Apache
Hudi - The Data Lake Platform](/blog/2021/07/21/streaming-data-lake-platform),
which first articulated this vision in depth. Here we focus on what the term
means in 2026, what it takes to build one, and how it relates to the data
lakehouse.
+
+## From Batch Data Lakes to Streaming Data Lakes
+
+A batch data lake and a streaming data lake can look identical on the surface
— Parquet files on object storage, queried by Spark, Trino, or Presto. The
difference lies in three operating assumptions.
+
+**Ingestion cadence.** A batch lake receives data in large, scheduled loads: a
nightly dump, an hourly partition. A streaming data lake receives data
continuously, committing every few minutes. This is not just a smaller batch
interval. Committing hundreds of times a day instead of once means the storage
layer must absorb many small writes without accumulating thousands of tiny
files, must keep table metadata cheap to update on every commit, and must
reconcile new data against old data c [...]
+
+**Mutation support.** Batch lakes were built around append-only, immutable
files, and rewrote whole partitions to "update" anything. Streaming sources are
dominated by mutation: CDC streams from databases are sequences of inserts,
updates, and deletes keyed by primary key; late-arriving events revise records
that were already written. A streaming data lake must treat record-level
upserts and deletes as first-class write operations, applied continuously.
+
+**Read patterns.** Batch consumers scan the latest partition after the nightly
load lands. Streaming consumers need more: dashboards want the freshest
consistent snapshot, downstream pipelines want to ask "what changed since I
last ran?" and process only that delta. A streaming data lake serves both the
classic full-table scan and this incremental, change-oriented consumption.
+
+None of these can be bolted onto a batch lake with a faster scheduler. They
require rethinking storage itself.
+
+
+<p align = "center">Figure: The incremental stack, alongside batch and stream
processing</p>
+
+## The Required Building Blocks
+
+Five capabilities separate a lake that can operate in streaming mode from one
that cannot.
+
+**Continuous upserts, backed by indexes.** Applying a CDC stream means
locating, for every incoming record, the file that holds the previous version
of that key — millions of times per commit. Doing this by scanning data is
hopeless; a streaming data lake needs indexes that map record keys to file
locations so upserts and deletes touch only the files that actually change.
This is why indexing sits in the write path of Apache Hudi rather than being
only a query-side optimization.
+
+**Merge-on-Read storage.** With frequent commits, rewriting columnar base
files on every write (Copy-on-Write) multiplies write amplification: a handful
of changed rows can force rewriting an entire file, hundreds of times a day.
The [Merge-on-Read (MOR) table type](/docs/table_types) instead absorbs changes
into compact delta log files and merges them with base files at query time or
during compaction. Write cost then scales with the amount of change, not with
the size of accumulated st [...]
+
+
+<p align = "center">Figure: A Merge-on-Read table absorbing frequent commits
into delta logs</p>
+
+**Asynchronous table services.** Deferring merge work is only viable if
something reliably performs it — without stopping ingestion. A streaming data
lake needs built-in table services: compaction to fold delta logs into new base
files, clustering to reorganize data layout for query performance, and cleaning
to reclaim old file versions. Crucially, these must run asynchronously,
concurrently with the ingestion stream. If maintenance requires pausing the
writer, the lake is not really str [...]
+
+**Change streams.** Once tables update continuously, downstream pipelines
should not recompute from full scans. A streaming data lake exposes
record-level change streams — the ability to query exactly the records that
changed between two points in time — so that derived tables can be maintained
incrementally. This is what turns a single fast table into an end-to-end
incremental architecture, where each layer consumes the changes of the previous
one, much like chained stream processing jo [...]
+
+**Concurrency control.** Continuous writers, background services, and readers
all operate on the same table at once. Readers need snapshot isolation so
queries always see a consistent, committed view. Writers and table services
need to proceed without blocking one another — an optimistic model that simply
fails one side wastes enormous resources when a long-running compaction
overlaps a stream that commits every minute. Streaming-first designs favor
non-blocking, MVCC-style coordination [...]
+
+## The Write Path: Streaming Ingestion
+
+In practice, the write path of a streaming data lake is a stream processing
job that terminates in lake storage. [Apache Flink can write continuously into
Hudi tables](/docs/flink-quick-start-guide), applying CDC streams from Kafka or
Debezium as upserts; Spark Structured Streaming and Hudi's own streaming
ingestion utilities do the same. The writer commits at a chosen interval —
commonly every one to five minutes — and each commit is atomic: queries see all
of it or none of it.
+
+Two streaming realities shape this path. First, **ordering**: events for the
same key can arrive out of order, so the writer resolves collisions
deterministically using an ordering field (typically event time), ensuring an
older update never overwrites a newer one regardless of arrival order. Second,
**late data**: an event may arrive hours or days after the period it belongs
to. In a batch lake this forces re-running jobs over old partitions; in a
streaming data lake the index locates t [...]
+
+## The Read Path: Snapshot, Read-Optimized, and Incremental Queries
+
+A streaming data lake gives readers an explicit choice of freshness versus
cost, expressed as [three query types](/docs/table_types):
+
+- **Snapshot queries** merge base files with delta logs at read time,
returning the freshest committed state — data written minutes ago. This is the
near-real-time analytics path.
+- **Read-optimized queries** read only compacted base files, giving pure
columnar scan performance at the price of bounded staleness (data freshness
tracks the compaction schedule). Latency-tolerant reporting can run here at
lower cost.
+- **Incremental queries** return the records that changed between two commit
times, powering the incremental pipelines described above.
+
+The same table serves all three, from the same single copy of data, through
standard engines. There is no separate "speed layer" holding fresh data and
"batch layer" holding history to reconcile — the failure mode that made Lambda
architectures notoriously hard to operate.
+
+## Streaming Data Lake vs. Kafka + Warehouse
+
+The common alternative for fresh analytics is to keep data in Kafka and
continuously load it into a cloud warehouse. It is worth being honest about the
trade-offs in both directions.
+
+**Where Kafka + warehouse wins.** Kafka delivers events in milliseconds to
seconds, and a stream processor reading Kafka can react at that speed — a
streaming data lake, committing every few minutes to object storage, cannot. If
the use case is sub-second serving, alerting, or event-driven applications,
Kafka (plus a stream processor or real-time OLAP store) is the right tool.
Warehouses also offer a highly polished, fully managed SQL experience.
+
+**Where the streaming data lake wins.** Retention and scale economics. Keeping
weeks or months of history in Kafka is expensive, and Kafka is not queryable
with analytical SQL — so data ends up copied into a warehouse anyway, where
storage and compute are metered at warehouse prices and the data lands in a
proprietary format usable only by that warehouse's engine. A streaming data
lake stores one copy on cheap object storage in open formats, retains history
indefinitely, serves large sca [...]
+
+The two are complementary rather than exclusive: a common pattern keeps Kafka
as the transport at the edge, with the streaming data lake as the durable,
queryable system of record a few minutes behind it.
+
+## How This Connects to the Lakehouse
+
+A [data lakehouse](/blog/2024/07/11/what-is-a-data-lakehouse) brings warehouse
capabilities — transactions, schema management, mutability — to data lake
storage. A streaming data lake is best understood as **a lakehouse operated in
streaming mode**: the same open storage and transactional foundation, but with
continuous writes instead of scheduled loads, change streams instead of full
refreshes, and table services that run concurrently with ingestion instead of
in maintenance windows. Ev [...]
+
+This is where Apache Hudi's history matters. Hudi was created at Uber in 2016
precisely for this problem — applying massive upsert streams to petabyte-scale
tables on lake storage with minute-level freshness — and its architecture
follows from that origin: pluggable indexing in the write path, Merge-on-Read
storage, a suite of built-in asynchronous table services, and incremental
queries as a core primitive. The [2021
essay](/blog/2021/07/21/streaming-data-lake-platform) laid out this "s [...]
+
+## Conclusion
+
+A streaming data lake is not a faster batch lake or a bigger Kafka cluster. It
is a data lake whose storage layer is designed for continuous change: upserts
resolved through indexes, writes absorbed by Merge-on-Read, maintenance
performed asynchronously by table services, changes exposed as first-class
streams, and all of it coordinated safely under concurrent access. In return,
it delivers minute-level data freshness on open formats and cheap storage,
serving snapshot, read-optimized, a [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {
+ question: 'What is a streaming data lake?',
+ answer: 'A streaming data lake is a data lake that ingests and serves data
continuously, with minute-level freshness, instead of through periodic batch
loads. It keeps data in open formats on cloud object storage while supporting
continuous upserts, incremental change streams, and background table
maintenance that runs without stopping ingestion.'
+ },
+ {
+ question: 'What is the difference between a streaming data lake and a data
lakehouse?',
+ answer: 'A data lakehouse brings warehouse capabilities like transactions
and mutability to data lake storage, while a streaming data lake is a lakehouse
operated in streaming mode. It adds continuous minute-level ingestion,
record-level change streams for incremental consumers, and asynchronous table
services that run alongside live writers. Every streaming data lake is a
lakehouse, but many lakehouses are only operated as batch systems.'
+ },
+ {
+ question: 'Can a data lake handle real-time data?',
+ answer: 'Yes, within limits. A streaming data lake built on storage like
Apache Hudi can commit continuously and make data queryable within minutes,
which covers most operational analytics and CDC replication needs. For
sub-second latency, such as event-driven applications or instant alerting, a
stream processor or real-time OLAP system reading from Kafka is still the
better fit.'
+ },
+ {
+ question: 'Do I still need Kafka with a streaming data lake?',
+ answer: 'Usually yes, but in a narrower role. Kafka remains an excellent
transport for moving events from producers to consumers in seconds, and it
commonly feeds the streaming data lake. What changes is retention and
analytics: instead of keeping long histories in Kafka or copying everything
into a warehouse, the lake becomes the durable, cheaply stored, SQL-queryable
system of record a few minutes behind the stream.'
+ },
+ {
+ question: 'Why is Merge-on-Read important for streaming ingestion?',
+ answer: 'Merge-on-Read absorbs updates and deletes into small delta log
files instead of rewriting large columnar files on every commit, so write cost
scales with the amount of change rather than the size of the table. That is
what makes committing every few minutes economical. A background compaction
service later folds the logs into optimized base files without pausing
ingestion.'
+ },
+ {
+ question: 'How fresh can data in a streaming data lake be?',
+ answer: 'Typically within a few minutes of the source event, determined by
the commit interval of the streaming writer. Snapshot queries see every
committed change immediately, while read-optimized queries trade some freshness
for faster scans by reading only compacted files. The freshness floor comes
from object storage and columnar file economics, which favor batching writes
into small intervals rather than per-event commits.'
+ }
+]} />
diff --git a/website/blog/2026-07-22-what-is-cdc-on-a-data-lake.md
b/website/blog/2026-07-22-what-is-cdc-on-a-data-lake.md
new file mode 100644
index 000000000000..54da1c9ead9a
--- /dev/null
+++ b/website/blog/2026-07-22-what-is-cdc-on-a-data-lake.md
@@ -0,0 +1,100 @@
+---
+title: "What is CDC on a Data Lake?"
+excerpt: "A beginner-friendly explainer of change data capture (CDC) on a data
lake: what it is, why it needs a table layer with upserts, and how a
Debezium-to-Hudi pipeline fits together."
+description: "Learn what change data capture (CDC) on a data lake is, why it
requires upserts and deletes, and how tools like Debezium and Apache Hudi make
it work."
+authors: [sivabalan]
+category: how-to
+image: /assets/images/blog/data-lake-cdc/hudi-cdc.jpg
+tags:
+- cdc
+- streaming
+- debezium
+- data lakehouse
+- beginner
+---
+
+Change data capture (CDC) on a data lake is the practice of continuously
replicating inserts, updates and deletes from operational databases into tables
on cloud storage, so that analytics, machine learning and downstream pipelines
always work against a fresh, queryable copy of the source data. Teams adopt it
for three main reasons: it offloads heavy analytical queries from
transaction-processing (OLTP) databases that were never built for them, it
consolidates data from many operational [...]
+
+If you already know the basics and want a deeper treatment of capture
mechanisms and architecture trade-offs, read the companion deep dive,
[Understanding Data Lake Change Data Capture](/blog/2024/07/30/data-lake-cdc),
after this one.
+
+<!--truncate-->
+
+## What is change data capture?
+
+Every operational database is constantly changing: orders are placed, profiles
are edited, rows are deleted. Change data capture is any technique that
identifies those row-level changes as they happen and delivers them, in order,
to another system. Instead of periodically copying a whole table, CDC ships
only the delta — each event typically carrying the operation type (insert,
update or delete), the affected row, and often its before and after state.
+
+There are two broad families of capture:
+
+- **Query-based capture** polls the source tables for changes, usually by
filtering on a `last_updated` timestamp column. It is simple to set up but has
real gaps: it misses hard deletes entirely (a deleted row simply stops
appearing in query results), it puts recurring query load on the production
database, and it requires the schema to cooperate.
+- **Log-based capture** reads the database's own transaction log — the binlog
in MySQL, the write-ahead log (WAL) in PostgreSQL — which records every
committed change in commit order. Because it taps a log the database writes
anyway, it captures every insert, update and delete with minimal impact on the
source, and it is the approach virtually all serious CDC deployments use today.
+
+
+
+[Debezium](https://debezium.io/) is the most widely used open source log-based
CDC tool. It runs as a Kafka Connect connector, tails the binlog or WAL, and
publishes one change event per row modification to a Kafka topic per table,
typically alongside a schema registry that tracks the shape of those events.
Cloud services such as AWS DMS and various managed connectors play the same
role. Whatever the tool, the output is the same in spirit: an ordered stream of
row-level change events tha [...]
+
+## Why the data lake is the natural CDC destination
+
+The destination question — where should that change stream land? — has an
economic answer and an architectural one.
+
+Economically, cloud object storage (Amazon S3, Google Cloud Storage, Azure
Blob Storage) is the cheapest place to keep large volumes of data, and it
scales without capacity planning. Replicating dozens or hundreds of operational
tables, with full history, is affordable there in a way it is not in a
warehouse whose storage and compute pricing were designed for curated,
aggregated data.
+
+Architecturally, data stored in open file formats like Apache Parquet on
object storage is not locked to any one engine. The same replicated tables can
serve Spark for ETL, Trino or Presto for interactive SQL, and Python-based ML
frameworks — without making another copy per engine. That "one copy, many
engines" property is the core promise of the [data
lakehouse](/blog/2024/07/11/what-is-a-data-lakehouse), and a continuously
replicated mirror of your operational databases is one of the m [...]
+
+But there is a fundamental mismatch. A CDC stream is, by definition, mostly
*mutations* — updates and deletes to existing rows. A plain data lake is a pile
of immutable files: you can add new files, but you cannot change a row inside
one. Without something extra, teams historically resorted to painful
workarounds — appending every change event and forcing every query to
deduplicate, or periodically rewriting entire partitions to fold changes in.
Both are slow, expensive and error-prone. [...]
+
+## What applying CDC requires from the table layer
+
+To turn a change stream into a correct, queryable table on object storage, the
layer above the raw files needs a specific set of capabilities:
+
+- **Upserts and deletes.** Each change event must be applied against the
existing table: an update replaces the current version of a row identified by
its primary key, and a delete removes it. Doing this efficiently means quickly
finding which file holds a given key — Hudi does so with pluggable
[indexes](/docs/indexes), including a record-level index, so it does not have
to scan the table on every write.
+- **Ordering and late events.** Change events can arrive out of order,
especially after retries or repartitioning in the transport layer. If an older
update is applied after a newer one, the table silently travels back in time.
The table layer must merge records by an ordering field — the source database's
log sequence number or position — so the correct final state wins regardless of
arrival order. Hudi supports this natively through event-time ordered merging
via its [record merger](/d [...]
+- **Schema evolution.** Source tables change: columns get added, types get
widened. A CDC pipeline that fails on every upstream `ALTER TABLE` is not
operable. The table layer needs to evolve the target [schema
safely](/docs/schema_evolution), usually in coordination with a schema registry.
+- **Transactional guarantees.** Writers apply changes continuously while
queries run concurrently. ACID transactions ensure readers never see a
half-applied batch, and failed writes can be rolled back cleanly rather than
leaving corrupt partial files behind.
+
+One more design choice matters for CDC specifically: how the table absorbs
writes. Copy-on-write tables rewrite whole data files for every batch of
updates, which is simple but costly when updates are frequent. Merge-on-read
tables, by contrast, log changes as compact delta files and compact them into
columnar base files asynchronously — which is why
[merge-on-read](/docs/table_types) is generally the recommended table type for
update-heavy CDC ingestion.
+
+## The two sides of CDC on a lakehouse
+
+It is worth separating two roles that often get blurred together, because a
lakehouse can — and usually should — play both.
+
+**Consuming CDC** is the direction discussed so far: the lakehouse table is
the *destination*, continuously applying a change stream from an operational
database so it stays an up-to-date mirror.
+
+**Producing change streams** is the reverse: the lakehouse table acts as a
*source* of changes for whatever sits downstream. Once your bronze tables
absorb CDC, the next question is how the silver and gold tables built on top of
them stay fresh. If each downstream job re-reads the whole upstream table, you
have merely moved the batch-processing problem one hop over. Hudi addresses
this with incremental queries — a consumer can ask a table for "everything that
changed since my last checkp [...]
+
+## A reference architecture: MySQL/Postgres to a Hudi table
+
+Here is what a typical end-to-end CDC pipeline onto a data lake looks like,
using widely deployed open source components:
+
+
+
+1. **Source database.** MySQL or PostgreSQL, configured to expose its change
log (binlog or logical-replication WAL).
+2. **Capture: Debezium on Kafka Connect.** A Debezium connector tails the log
and writes one change event per row modification into a Kafka topic per table,
registering event schemas with a schema registry (Confluent or Apicurio).
+3. **Apply: Hudi Streamer or Flink.** [Hudi
Streamer](/docs/hoodie_streaming_ingestion) is a self-contained ingestion
utility that ships with Hudi and includes a Debezium source out of the box:
running in continuous mode, it reads the change events from Kafka and upserts
them into a Hudi table on cloud storage, using the table's record key (the
source primary key) to route updates and the log position to order them. Teams
standardized on Flink can use Hudi's Flink integration for the sam [...]
+4. **Bronze table.** The result is a "raw" replicated table — typically
merge-on-read — that mirrors the source, kept fresh within minutes, registered
in a catalog (Glue, Hive Metastore, etc.) and queryable from Spark, Trino,
Presto and other engines.
+5. **Incremental ETL downstream.** Silver and gold tables subscribe to the
bronze table via incremental queries, pulling and transforming only the records
that changed since their last run.
+
+Getting an existing table's history in before streaming begins — the bootstrap
— is handled either by letting Debezium take an initial consistent snapshot or
by bulk-loading the table directly (for example over JDBC) and then starting
the change stream from the matching checkpoint.
+
+## Common pitfalls, and how the table layer absorbs them
+
+A few failure modes recur in almost every CDC-on-the-lake deployment. Knowing
them up front is most of the battle:
+
+- **Silently dropped deletes.** Query-based capture cannot see hard deletes,
and some pipelines ignore delete events on the sink side. The mirror then
diverges from the source, one deleted row at a time — a correctness and
compliance problem (GDPR erasure requests must propagate). The fix is log-based
capture plus a sink that applies deletes as first-class operations, as Hudi's
Debezium source does by honoring the delete marker in each event.
+- **Out-of-order events.** Retries and parallelism can deliver an old version
of a row after a newer one. A sink that blindly takes the last arrival will
regress data. Merging by the source log position, rather than arrival time,
makes the pipeline immune to this.
+- **Small files.** Frequent micro-batches naturally produce many tiny files,
and query performance on object storage degrades sharply as file counts
explode. Hudi manages [file sizing](/docs/file_sizing) during writes and runs
compaction and clustering as background table services, so ingestion frequency
does not have to be traded against query speed.
+- **Backfills colliding with the stream.** Sooner or later you must reload
history — a missed window, a schema fix — while live ingestion continues.
Without transactions, a backfill job and a streaming writer stepping on each
other corrupt the table. A transactional table layer with concurrency control
lets the backfill run alongside the stream safely.
+
+## Conclusion
+
+CDC on a data lake pairs the most valuable data most organizations have — the
live state of their operational databases — with the cheapest, most open place
to analyze it. The capture side is a solved problem thanks to log-based tools
like Debezium; the hard part has always been the destination, because applying
a stream of updates and deletes to immutable cloud storage requires upserts,
ordering, schema evolution and transactions that raw file storage lacks. That
is precisely the gap tr [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is the difference between CDC and ETL?', answer: 'ETL is
the general process of extracting data from sources, transforming it and
loading it into a target system, traditionally as periodic bulk jobs that
reprocess full tables. CDC is a specific extraction technique that captures
only the row-level inserts, updates and deletes as they happen in the source
database. In practice CDC often serves as the extract step of a continuous,
incremental ETL pipeline, replacing full [...]
+ {question: 'Can you do CDC without Kafka?', answer: 'Yes. Kafka is a popular
transport for change events, but it is not required. Debezium can run as a
standalone server that writes to other messaging systems, cloud services like
AWS DMS can deliver changes directly to object storage, and ingestion tools can
pull changes from a database over JDBC. Kafka earns its place in larger
deployments because it buffers events durably and lets multiple consumers
replay the same stream independently.'},
+ {question: 'How are deletes handled in CDC on a data lake?', answer:
'Log-based CDC tools emit an explicit delete event when a row is removed from
the source database. A transactional table layer such as Apache Hudi applies
that event by locating the record via its key and removing it from the table,
so the lake copy stays consistent with the source. This matters for correctness
and for compliance, since regulations like GDPR require deletions to propagate
to downstream copies.'},
+ {question: 'What is the difference between log-based and query-based CDC?',
answer: 'Query-based CDC repeatedly queries source tables for rows with a
recent last-updated timestamp, which is simple but misses hard deletes and adds
load to the production database. Log-based CDC reads the database\'s own
transaction log, such as the MySQL binlog or PostgreSQL WAL, capturing every
insert, update and delete in commit order with minimal source impact. Log-based
capture is the standard choice [...]
+ {question: 'Why can\'t a plain data lake handle CDC?', answer: 'A plain data
lake stores immutable files on object storage, so there is no built-in way to
update or delete an individual record, which is exactly what most CDC events
require. Teams either appended all events and deduplicated at query time or
rewrote whole partitions, both slow and error-prone. Transactional table layers
like Apache Hudi add upserts, deletes, indexing and ACID guarantees on top of
the same storage, making [...]
+ {question: 'How fresh can data be with CDC on a data lake?', answer:
'End-to-end latency from a database commit to a queryable row on the lake is
typically measured in minutes with continuously running ingestion.
Merge-on-read tables help by logging changes cheaply on write and compacting
them in the background, instead of rewriting large files on every batch. That
is near real-time for analytics, though not a replacement for the millisecond
latencies of an operational database.'},
+]} />
diff --git
a/website/blog/2026-07-23-lakehouse-vs-data-warehouse-vs-data-lake.md
b/website/blog/2026-07-23-lakehouse-vs-data-warehouse-vs-data-lake.md
new file mode 100644
index 000000000000..ca6ce87cbac5
--- /dev/null
+++ b/website/blog/2026-07-23-lakehouse-vs-data-warehouse-vs-data-lake.md
@@ -0,0 +1,133 @@
+---
+title: "Data Lakehouse vs Data Warehouse vs Data Lake: What's the Difference?"
+excerpt: "A practical three-way comparison of data warehouses, data lakes, and
data lakehouses — strengths, trade-offs, a side-by-side table, and guidance on
when each architecture is the right choice."
+description: "Data warehouse vs data lake vs data lakehouse compared: costs,
transactions, performance, openness, and when to choose each architecture."
+authors: [vinoth-chandar]
+category: how-to
+image: /assets/images/blog/dlh_1200.png
+tags:
+- data lakehouse
+- data warehouse
+- comparison
+- open architecture
+---
+
+A **data warehouse** is a proprietary, structured, SQL-first analytical
database: you load cleaned, modeled data into it and get fast, governed BI
queries out. A **data lake** is cheap, open storage — typically cloud object
stores holding raw files of any shape, from CSVs to images — that scales
practically without limit but offers few database guarantees. A **data
lakehouse** is the data lake upgraded with database capabilities — ACID
transactions, upserts and deletes, indexing, schema [...]
+
+Which one you should build on depends on what your data looks like, who
queries it, and how much you value keeping your storage open. If all you need
is dashboards over modest volumes of structured data, a warehouse remains the
path of least resistance. If you are juggling large volumes, unstructured data,
machine learning, streaming ingestion, or multiple query engines, the calculus
shifts quickly toward a lakehouse. The rest of this post walks through each
architecture honestly — stren [...]
+
+## The Data Warehouse: Fast, Governed, and Proprietary
+
+Data warehouses have been the workhorse of analytics for decades, from
on-premise appliances like Teradata to cloud-native services like
[Snowflake](https://www.snowflake.com), [Google
BigQuery](https://cloud.google.com/bigquery), and [Amazon
Redshift](https://aws.amazon.com/redshift/). The model is simple: extract data
from operational systems, transform it into well-defined relational schemas,
and load it into a system purpose-built for analytical SQL.
+
+The strengths are real and should not be understated:
+
+- **Query performance.** Warehouses control the full stack — storage layout,
statistics, caching, query optimization — and are ruthlessly tuned for
low-latency SQL. For interactive BI dashboards over structured data, they set
the bar.
+- **Governance and management.** Mature access controls, auditing, data
masking, and workload management come built in. Schemas are enforced on write,
so the data inside a warehouse tends to be clean by construction.
+- **Simplicity.** A cloud warehouse is a single managed product. There is one
system to secure, one SQL dialect to learn, one vendor to call.
+
+The costs show up as the footprint grows:
+
+- **Lock-in.** Warehouse storage formats are proprietary. Your data lives
inside the vendor's system, readable only through the vendor's compute.
Migrating out — or even letting a second engine read the same tables — means
exporting and copying.
+- **ELT copies.** Because the warehouse only speaks structured SQL, data must
be copied into it, and often copied again into marts and extracts for different
teams. Each copy adds pipeline complexity, staleness, and cost.
+- **Price at scale.** Coupled compute and proprietary storage are priced at a
premium. At tens or hundreds of terabytes with heavy transformation workloads,
warehouse bills routinely become one of the largest line items in a data
platform budget.
+- **Poor fit for ML and unstructured data.** Training a model on images, logs,
or raw event streams inside a warehouse ranges from awkward to impossible. Data
science teams typically end up exporting data back out — another copy.
+
+## The Data Lake: Cheap, Open, and Unreliable
+
+Data lakes emerged from the Hadoop era and matured on cloud object storage —
[Amazon S3](https://aws.amazon.com/s3/), [Google Cloud
Storage](https://cloud.google.com/storage), [Azure Data Lake
Storage](https://azure.microsoft.com/en-us/products/storage/data-lake-storage).
The idea inverts the warehouse: instead of modeling data before loading it,
land everything raw and cheap, and decide the schema when you read it.
+
+The strengths:
+
+- **Cost.** Object storage costs a small fraction of warehouse storage, with
durability and availability handled by the cloud provider. Storage and compute
scale independently — you pay for queries only when you run them.
+- **Openness.** Data sits in open file formats like [Apache
Parquet](https://parquet.apache.org) and [Apache ORC](https://orc.apache.org).
Any engine — Apache Spark, Apache Flink, Trino, Presto, Ray, or a Python script
— can read the same files. There is no vendor gatekeeping access to your own
data.
+- **Any data shape.** Structured tables, semi-structured JSON, free text,
images, audio, model checkpoints — a lake stores all of it, which is exactly
why ML and data science teams gravitate to lakes.
+
+The weaknesses are equally well documented:
+
+- **No transactions.** A plain data lake is just files in directories. A
failed job can leave partial writes visible to readers; two concurrent writers
can corrupt each other. There is no atomic commit, no isolation, no rollback.
+- **No upserts or deletes.** Object stores are append-oriented. Updating a
single record — say, for a GDPR deletion request or a late-arriving correction
— traditionally meant rewriting entire partitions.
+- **Data swamps.** Without schema enforcement, quality controls, or
table-level metadata, lakes decay into "data swamps": directories of files
nobody trusts and nobody dares delete.
+- **Slow BI.** Query engines scanning raw files lack the indexes, statistics,
and optimized layouts a warehouse maintains, so interactive dashboard
performance over a plain lake generally disappoints.
+
+## The Data Lakehouse: The Lake, Upgraded to a Database
+
+The lakehouse closes the gap between the two by adding a transactional
metadata layer — an [open table
format](/blog/2026/07/14/what-is-an-open-table-format) — on top of the open
files already sitting in the lake. Projects like [Apache
Hudi](https://hudi.apache.org), [Apache Iceberg](https://iceberg.apache.org),
and [Delta Lake](https://delta.io) define tables (not just files) over Parquet
and ORC, and bring the database capabilities the lake was missing:
+
+- **ACID transactions** with concurrency control, so writers commit atomically
and readers never see partial results.
+- **Upserts and deletes**, made efficient by indexing mechanisms that locate
the records to change without rewriting whole partitions — the capability that
first motivated Hudi's creation at Uber.
+- **Schema enforcement and evolution**, so tables reject malformed writes and
can add or change columns without full rewrites.
+- **Performance features** — indexing, statistics, clustering, compaction,
time travel — that let query engines approach warehouse-class performance on
lake storage.
+
+Two properties follow that neither predecessor offers. First, **unified batch
and streaming**: because the table layer supports fast upserts and incremental
pulls, the same table can be written by a streaming pipeline and read by both a
dashboard and a nightly batch job, eliminating the classic lambda-architecture
split. Second, **one copy of data, many engines**: the table is defined in an
open format on open storage, so Spark, Flink, Trino, Presto, StarRocks, Daft,
and warehouse engine [...]
+
+
+<p align = "center">Figure: Data Lakehouse Architecture — open table formats
and a storage engine layered over cloud object storage, serving multiple
compute engines</p>
+
+For a deeper treatment of the architecture itself — the storage, file-format,
table-format, catalog, and compute layers — see [What is a Data Lakehouse & How
does it Work?](/blog/2024/07/11/what-is-a-data-lakehouse)
+
+The honest caveat: a lakehouse is assembled from components rather than bought
as one product, so it demands more engineering judgment than swiping a credit
card at a warehouse vendor. Managed lakehouse platforms narrow that gap, but
the trade-off between operational simplicity and openness has not disappeared
entirely.
+
+## Side-by-Side Comparison
+
+| Dimension | Data Warehouse | Data Lake | Data Lakehouse |
+|---|---|---|---|
+| **Storage cost** | High — proprietary storage at premium pricing | Low —
commodity cloud object storage | Low — same object storage as the lake |
+| **Data types** | Structured (some semi-structured support) | Structured,
semi-structured, unstructured | Structured, semi-structured, unstructured |
+| **Transactions** | Full ACID | None | ACID via table formats (Hudi, Iceberg,
Delta Lake) |
+| **Updates / deletes** | Native SQL DML | Rewrite files manually | Native
upserts and deletes, index-accelerated |
+| **Query performance** | Excellent for SQL/BI | Poor to moderate (raw file
scans) | Good to excellent with indexing, clustering, compaction |
+| **Openness / lock-in** | Proprietary format, single vendor engine | Fully
open files, any engine | Open table formats, any engine |
+| **Streaming support** | Micro-batch loading, often at extra cost |
Append-only file drops | Native streaming ingestion with incremental processing
|
+| **ML support** | Weak — data must be exported | Strong — raw data access for
any framework | Strong — same open data, plus reliable managed tables |
+| **Governance / quality** | Mature, built in | Do-it-yourself, frequently
neglected | Schema enforcement, time travel, auditing via table layer |
+| **Typical tools** | Snowflake, BigQuery, Redshift | S3 / GCS / ADLS + Spark,
Presto | Hudi / Iceberg / Delta Lake + Spark, Flink, Trino, StarRocks |
+
+## When Is Each the Right Choice?
+
+**Choose a data warehouse** when your data is modest in volume, almost
entirely structured, and your consumers are BI analysts writing SQL. A small
team running dashboards over a few terabytes will likely ship faster and
operate more cheaply on a managed warehouse than by assembling a lakehouse
stack. Lock-in is a real cost, but at small scale it may be a cost worth paying
for simplicity.
+
+**Choose a plain data lake** when you primarily need a cheap landing zone and
archive — raw event dumps, ML training corpora, cold retention — and you do not
need transactional tables or fast SQL over that data. Many lakehouses begin
life as exactly this.
+
+**Choose a lakehouse** when one or more of these is true: data volumes are
growing into the tens of terabytes and beyond and warehouse bills sting; you
need machine learning and BI on the same data; you ingest streaming or
frequently-updated data (CDC from operational databases is the canonical case);
you need record-level updates and deletes for compliance; or you want multiple
engines — and the freedom to change vendors — over a single copy of data. These
are the workloads the architec [...]
+
+The choice is also not exclusive. A common and pragmatic pattern is a
lakehouse as the system of record with a warehouse serving a subset of curated,
high-concurrency dashboards — increasingly by querying the lakehouse tables in
place rather than holding a copy.
+
+## The Convergence Trend
+
+The sharp lines between these categories are blurring, and they are blurring
*toward* the lakehouse. Warehouses are opening up: Snowflake, BigQuery, and
Redshift all now query and, in varying degrees, write open table formats on
customer-owned object storage, an acknowledgment that customers refuse to keep
their primary copy of data in a proprietary format. Meanwhile, lakes are
getting more database-like: table formats keep absorbing capabilities that were
once warehouse differentiators [...]
+
+Follow both trends to their end point and you arrive at the same place: open
data on cheap storage, database-grade table management in an open layer above
it, and interchangeable compute engines on top. That meeting point is the
lakehouse. The remaining competition is less about *whether* to structure data
this way and more about which table formats, catalogs, and engines earn a place
in the stack.
+
+## Conclusion
+
+Warehouses deliver performance and governance at the price of lock-in and
copies; lakes deliver cost and openness at the price of reliability; the
lakehouse keeps the lake's economics and openness while restoring the database
guarantees — transactions, upserts, indexing, schema management — that make
data trustworthy and fast to query. For small, purely structured BI workloads,
a warehouse is still a fine and simple answer. For everything pushing past that
— scale, streaming, ML, multipl [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {
+ question: 'Is a lakehouse cheaper than a warehouse?',
+ answer: 'Usually, at scale. A lakehouse stores data on commodity cloud
object storage, which costs a small fraction of proprietary warehouse storage,
and it avoids maintaining multiple ELT copies of the same data. At small
volumes the difference narrows, and a managed warehouse can be cheaper in
engineering time, so total cost depends on scale and team.',
+ },
+ {
+ question: 'Can a lakehouse replace a data warehouse?',
+ answer: 'For many organizations, yes. Open table formats bring
transactions, upserts, indexing, and schema management to lake storage, and
engines like Trino, StarRocks, and Spark deliver strong SQL performance on
those tables. Some teams still keep a warehouse for very high-concurrency,
low-latency dashboards, often querying the lakehouse tables directly instead of
holding a separate copy.',
+ },
+ {
+ question: 'Is Snowflake a data lakehouse?',
+ answer: 'Snowflake is a cloud data warehouse at its core, storing data in
a proprietary format managed by its own engine. It has added lakehouse-style
capabilities, notably support for Apache Iceberg tables on customer-owned
object storage. Whether a Snowflake deployment qualifies as a lakehouse depends
on whether the primary data copy lives in an open format that other engines can
access.',
+ },
+ {
+ question: 'What is the main difference between a data lake and a data
lakehouse?',
+ answer: 'A data lake is raw files on cheap object storage with no database
guarantees, while a lakehouse adds an open table format layer, such as Apache
Hudi, Apache Iceberg, or Delta Lake, on top of those files. That layer provides
ACID transactions, record-level updates and deletes, indexing, and schema
enforcement, turning unreliable file collections into managed tables.',
+ },
+ {
+ question: 'Do I need Hadoop to build a data lake or lakehouse?',
+ answer: 'No. Modern lakes and lakehouses are built on cloud object stores
like Amazon S3, Google Cloud Storage, or Azure Data Lake Storage, with engines
such as Apache Spark, Apache Flink, and Trino running against them. Hadoop HDFS
is still supported by these tools but is no longer required or typical for new
deployments.',
+ },
+ {
+ question: 'Which lakehouse table format should I choose?',
+ answer: 'Apache Hudi, Apache Iceberg, and Delta Lake all provide ACID
transactions and schema evolution on open storage, and all are
production-proven. Hudi stands out for update-heavy and streaming workloads
thanks to its indexing and incremental processing, while the best fit overall
depends on your engines, workloads, and ecosystem. Interoperability tools like
Apache XTable also let one table be read in multiple formats.',
+ },
+]} />
diff --git a/website/blog/2026-07-24-open-table-format-vs-data-lakehouse.md
b/website/blog/2026-07-24-open-table-format-vs-data-lakehouse.md
new file mode 100644
index 000000000000..6f4619902170
--- /dev/null
+++ b/website/blog/2026-07-24-open-table-format-vs-data-lakehouse.md
@@ -0,0 +1,96 @@
+---
+title: "Open Table Format vs Data Lakehouse: What's the Difference?"
+excerpt: "Untangles two constantly conflated terms: the open table format is
one layer of the stack; the data lakehouse is the whole architecture built
around it."
+description: "An open table format is a component; a data lakehouse is an
architecture. This post walks the lakehouse stack layer by layer and corrects
common conflations."
+authors: [vinoth-chandar]
+category: how-to
+image: /assets/images/blog/hudistack/hstck_new.png
+tags:
+- table format
+- data lakehouse
+- open architecture
+- architecture
+---
+
+An open table format is a component; a data lakehouse is an architecture. The
table format is the metadata layer that turns files on object storage into
transactional tables — the lakehouse is the complete system built around it:
storage, table format, catalog, table services and query engines. The two terms
get used interchangeably in vendor marketing and conference talks, and the
conflation causes real confusion: teams "adopt a lakehouse" by picking a table
format and then discover tha [...]
+
+This post untangles the two terms. We will walk the lakehouse stack from the
bottom up, pin down exactly what a table format contributes at its layer, spell
out what a table format alone does *not* give you, and correct the most common
conflations along the way.
+
+## The Lakehouse Stack, Layer by Layer
+
+A [data lakehouse](/blog/2024/07/11/what-is-a-data-lakehouse) is best
understood as a stack of open, modular layers. Each layer has one job, and the
table format is exactly one of them. From the bottom up:
+
+**Cloud object storage.** Amazon S3, Azure Blob Storage, Google Cloud Storage.
This layer stores bytes durably and cheaply at effectively unlimited scale, and
decouples storage cost from compute cost. It knows nothing about tables — it
serves GETs, PUTs and LISTs.
+
+**Open file formats.** [Apache Parquet](https://parquet.apache.org) and
[Apache ORC](https://orc.apache.org) define how records are encoded *inside a
single file*: columnar layout, compression, encodings, and per-file statistics.
A file format makes one file efficient to scan; it has no concept of the other
files around it.
+
+**The open table format.** [Apache Hudi](https://hudi.apache.org), [Apache
Iceberg](https://iceberg.apache.org), and [Delta Lake](https://delta.io) live
here. A table format is versioned metadata over many files: an ordered log of
commits (in Hudi, the [timeline](/docs/timeline)), a record of which files
constitute the table at each version, the schema and its evolution history, and
file-level statistics for pruning. This is the layer that makes a pile of
Parquet behave like a database t [...]
+
+**Storage engine and table services.** Metadata alone does not keep a table
healthy. Something has to compact log files into base files, cluster data for
query locality, size small files, clean up old versions, and build and maintain
indexes. In some stacks this is the query engine's job or a vendor service; in
Hudi it is a built-in layer of table services.
+
+**Catalogs and metadata services.** A catalog (Hive Metastore, AWS Glue,
Apache Polaris, Unity Catalog) tracks which tables exist, their schemas, and
where each table's metadata lives — plus, increasingly, access control and
lineage. It is what lets an engine resolve `orders` to something it can plan a
scan against.
+
+**Query and compute engines.** Apache Spark, Apache Flink, Trino, Presto, plus
warehouses and Python-native engines. Engines do the actual reading, writing
and computing, speaking to tables through the table format's specification and
finding them through the catalog.
+
+
+<p align = "center">Figure: The layered data lakehouse architecture</p>
+
+Assemble all of these — storage, file format, table format, table services,
catalog, engines — and *that* is a data lakehouse. The table format is the
load-bearing middle of the stack, arguably the layer that made the architecture
possible at all. But it is one layer, the way a storage engine is one layer of
a database, not the database.
+
+## What a Table Format Alone Does Not Give You
+
+Here is where the conflation bites in practice. A table format specification
tells you what a valid table looks like on storage. It does not, by itself,
operate anything. Adopt a bare table format and you still own:
+
+- **Table maintenance.** Someone has to run compaction, clustering, cleaning
of old file versions, and file sizing — continuously, for every table,
coordinated with in-flight writers. Skip it and small files and stale metadata
quietly erode query performance and inflate storage bills.
+- **Ingestion.** The format defines how commits land; it does not build the
pipelines that pull from Kafka, CDC streams, or upstream databases, handle
schema drift, deduplicate, and checkpoint.
+- **Catalog sync and discovery.** Tables must be registered and kept current
in one or more catalogs so every engine can find them.
+- **Access control and governance.** The format stores data; deciding who can
read which columns of it is a platform concern layered above.
+- **Monitoring and operations.** Commit failures, clustering backlogs,
concurrency conflicts — someone has to watch them.
+
+This is the "platform gap" between a format and a lakehouse, and it is exactly
where projects differ in scope. Apache Iceberg deliberately scopes itself to
the specification and libraries, leaving maintenance to engines and vendors.
Hudi deliberately scopes itself wider: built-in table services (compaction,
clustering, cleaning, indexing) that run inline or asynchronously without
external orchestration, ingestion utilities like [Hudi
Streamer](/docs/hoodie_streaming_ingestion) for pullin [...]
+
+
+<p align = "center">Figure: The Apache Hudi stack — a table format plus table
services and platform components</p>
+
+## Common Conflations, Corrected
+
+A few statements you will hear regularly, each slightly wrong in an
instructive way:
+
+**"Iceberg is a lakehouse."** No — Apache Iceberg is a table format a
lakehouse can use. An Iceberg table sitting in S3 with no catalog, no
maintenance jobs and no engines wired up is not a lakehouse; it is
well-organized metadata. The same is true of Hudi and Delta Lake at the format
layer: the lakehouse is what you get after you assemble the rest of the stack
around the tables.
+
+**"Databricks' lakehouse is the lakehouse."** Databricks popularized the term
and built an excellent platform around Delta Lake, but the lakehouse is an open
architectural pattern, not one vendor's product. Uber was running Hudi in what
we would now call a lakehouse architecture in 2016–2017, and today you can
assemble a lakehouse from entirely open components — or buy one from any of
several vendors — on any of the three major formats.
+
+**"Choosing a table format means choosing a vendor."** The formats are openly
governed (Hudi and Iceberg at the ASF, Delta Lake at the Linux Foundation), and
every major engine reads more than one of them. Vendors have preferred formats,
but the specification belongs to the community — that openness is the entire
point of the "open" in open table format.
+
+**"Parquet is our table format."** A file format is not a table format.
Parquet defines the bytes inside one file; a table format defines how thousands
of Parquet files plus a transaction log behave as a single versioned table.
They are adjacent layers of the stack, not alternatives.
+
+For where the lakehouse itself sits relative to the architectures it evolved
from, see the companion comparison of [lakehouse vs data warehouse vs data
lake](/blog/2026/07/23/lakehouse-vs-data-warehouse-vs-data-lake).
+
+## The Layers Interoperate — You Are Not Locked In
+
+Because the layers are modular and the specifications open, the mapping
between them is many-to-many. Spark, Flink and Trino each read and write all
three major formats. A single lakehouse can, and often does, contain tables in
more than one format, discovered through the same catalog and queried by the
same engines.
+
+Interoperability now extends to the format layer itself. [Apache
XTable](https://xtable.apache.org) (incubating) translates table metadata
between Hudi, Iceberg and Delta Lake — in any direction — without copying or
rewriting data files, either as a one-time conversion or as continuous
incremental sync. A table written with Hudi's ingestion and indexing
capabilities can be exposed as an Iceberg table to a service that only reads
Iceberg. The practical consequence is that the table format [...]
+
+## Practical Guidance: Architecture First, Then Format
+
+The right order of decisions falls straight out of the layering. First design
the architecture: what workloads (BI, ML, streaming, ad-hoc SQL) must the
system serve? Which engines? What ingestion latency? Who operates table
maintenance — your team, built-in services, or a vendor? What are the
governance requirements? That exercise defines the lakehouse you need.
+
+Only then choose the table format — as a component decision driven by workload
fit. Mutable, incremental, streaming-heavy ingestion favors Hudi's write path,
indexing and [table types](/docs/table_types) (Copy-on-Write vs Merge-on-Read
let you tune the write/read trade-off per table). Engine-neutral batch
analytics with broad catalog support is Iceberg's design center; deep Spark and
Databricks integration is Delta's. If the operational gap matters — no team to
babysit compaction jobs — [...]
+
+Teams that pick a format first and call it a lakehouse strategy usually end up
rediscovering the rest of the stack the hard way, one missing layer at a time.
+
+## Conclusion
+
+Keep the two terms at their own altitudes and the confusion disappears. The
open table format is a layer: versioned, transactional metadata that turns
files on object storage into tables. The data lakehouse is the architecture:
that layer plus storage, file formats, table services, catalogs and engines,
assembled into a system that does warehouse-grade work on open data. A table
format cannot be a lakehouse any more than a storage engine can be a database —
but no lakehouse stands withou [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is the difference between an open table format and a data
lakehouse?', answer: 'An open table format is a single component: the metadata
layer that turns files on object storage into transactional tables with ACID
commits, schema evolution and time travel. A data lakehouse is the complete
architecture built around it, combining object storage, open file formats, the
table format, table services, catalogs and query engines into one system.'},
+ {question: 'Is Apache Iceberg a data lakehouse?', answer: 'No. Apache
Iceberg is an open table format — one layer of a lakehouse architecture. A
lakehouse using Iceberg still needs object storage, a catalog, query engines,
and services to maintain the tables. Iceberg is a table format a lakehouse can
use, not a lakehouse itself.'},
+ {question: 'Is Delta Lake a table format or a lakehouse?', answer: 'Delta
Lake is an open table format governed by the Linux Foundation. Databricks
builds its lakehouse platform on top of Delta Lake, which is why the two are
often conflated, but Delta Lake itself is the table layer, and the lakehouse is
the surrounding platform of storage, catalogs, engines and services.'},
+ {question: 'Do I need a table format to build a lakehouse?', answer: 'Yes,
in practice. The table format is what provides atomic commits, snapshot
isolation, updates and deletes, and versioning over files in object storage.
Without one you have a data lake of raw files, not a lakehouse. Apache Hudi,
Apache Iceberg and Delta Lake are the major open options.'},
+ {question: 'Is Apache Hudi a table format or a lakehouse platform?', answer:
'Both, by design. Hudi includes an open table format built around its timeline
and file layout, but the project also ships built-in table services like
compaction, clustering and cleaning, ingestion utilities such as Hudi Streamer,
and a multi-modal indexing subsystem. That puts Hudi closer to a lakehouse
platform than a bare format specification.'},
+ {question: 'Does choosing a table format lock me into one vendor?', answer:
'No. The major table formats are openly governed, and engines like Apache
Spark, Apache Flink and Trino support all of them. Apache XTable can also
translate table metadata between Hudi, Iceberg and Delta Lake without rewriting
data files, so the format choice is a per-workload decision rather than a
permanent commitment.'},
+]} />
diff --git a/website/static/llms.txt b/website/static/llms.txt
index 72ac2077ebd7..33fc4c0fc4bd 100644
--- a/website/static/llms.txt
+++ b/website/static/llms.txt
@@ -43,6 +43,10 @@ It can also synchronize your data to half dozen data
catalogs to keep table cons
- [What is Upsert on a Data
Lake?](https://hudi.apache.org/blog/2026/07/15/what-is-upsert-on-a-data-lake):
Record-level updates and deletes on immutable cloud storage, and how Hudi makes
them fast.
- [What is Incremental ETL on a Data
Lake?](https://hudi.apache.org/blog/2026/07/16/what-is-incremental-etl-on-a-data-lake):
Processing only the data that changed instead of recomputing whole tables.
- [What is ACID on a Data
Lake?](https://hudi.apache.org/blog/2026/07/17/what-is-acid-on-a-data-lake):
What atomicity, consistency, isolation and durability mean when your database
is a bucket of files.
+- [What is a Streaming Data
Lake?](https://hudi.apache.org/blog/2026/07/21/what-is-a-streaming-data-lake):
Minute-level data freshness on the lake with incremental pipelines instead of
batch reprocessing.
+- [What is CDC on a Data
Lake?](https://hudi.apache.org/blog/2026/07/22/what-is-cdc-on-a-data-lake):
Change data capture from operational databases into lakehouse tables, explained
from first principles.
+- [Lakehouse vs Data Warehouse vs Data
Lake](https://hudi.apache.org/blog/2026/07/23/lakehouse-vs-data-warehouse-vs-data-lake):
The three architectures compared — strengths, costs and when to use each.
+- [Open Table Format vs Data
Lakehouse](https://hudi.apache.org/blog/2026/07/24/open-table-format-vs-data-lakehouse):
Untangling the table layer from the architecture, and where file formats,
catalogs and engines fit.
## Blog — Migration guides
