Hi Kui Yuan, Thanks for the +1, and for surfacing the conversion step explicitly. You are right: even with this FLIP, the conversion is a one-time stop-and-pick-offset for the existing writer. The win is that everything after that point stays in the catalog and evolves through the standard MT path.
I will kick off the [VOTE] thread shortly. Cheers, Ramin On Fri, May 29, 2026 at 3:55 AM Kui Yuan <[email protected]> wrote: > Hi, Ramin > > Thanks for your reply. > > Going back to your example — if there is an existing CatalogTable > orders_summary with a corresponding streaming job writing into it, > converting it to a CatalogMaterializedTable would still require > stopping the streaming job first and choosing an appropriate starting > consumption offset during the conversion. But this is a one-time step. > > As Materialized Table capabilities continue to mature, we expect users > to migrate more use cases onto MT. Supporting the conversion from > CatalogTable to MaterializedTable will make that migration > considerably smoother for users. So I agree with your proposal. > > +1 from me. > > Best, > Kui.Yuan > > Ramin Gharib <[email protected]> 于2026年5月26日周二 23:29写道： > > > > Hi Kui Yuan, > > > > Thanks for the questions. I added Section 4 "Use Cases" to the FLIP > > covering both points. Short answers below; full walkthrough in the FLIP. > > > > 1. Typical scenario: a Flink user maintains a regular table populated by > > CTAS, e.g. > > CREATE TABLE orders_summary AS > > SELECT ... FROM orders; > > > > In streaming this spawns a long-running statement writing into > > `orders_summary`. Adding a column today is a multi-step, non-atomic > chore: > > find the writer (catalog has no link), savepoint-stop it, ALTER the > schema, > > resubmit the INSERT with manual offset handling, roll back by hand on > > failure. > > With this FLIP, the same change is one DDL: > > > > CREATE OR ALTER MATERIALIZED TABLE orders_summary > > [START_MODE=RESUME_OR_FROM_BEGINNING] > > AS SELECT ..., price FROM orders; > > > > Defining query becomes part of the catalog object. Schema, query, and > > refresh evolve together through the existing ALTER MT path (FLIP-492, > > FLIP-557). See Section 4.1 for the full step-by-step. > > > > 2. Reverse direction (MT to Table) intentionally out of scope. Two > reasons: > > > > First, the use case is asymmetric. We expect users to move toward MTs as > > capabilities mature, not back. Conversion is also gated behind an > explicit > > opt-in flag, so users don't slide into MT by accident. > > > > Second, the MT lifecycle already gives users the escape hatch a downgrade > > would: `ALTER MATERIALIZED TABLE ... SUSPEND` stops the managed refresh > > while preserving the catalog object, schema, and storage, and lets users > > write to it with plain `INSERT INTO`. See Section 4.2. > > > > If a real use case for a true MT-to-Table conversion emerges, it is a > > natural follow-up FLIP. > > > > Cheers, > > Ramin > > > > On Mon, May 25, 2026 at 10:02 AM Kui Yuan <[email protected]> wrote: > > > > > Hi Ramin, > > > > > > Thank you for for driving this topic. > > > > > > I have a couple of questions about this proposal： > > > 1. Could we elaborate a bit more on when users would typically need > this > > > feature? It will be helpful to have a few concrete scenarios > illustrating. > > > 2. Since the proposal supports converting a CatalogTable to a > > > CatalogMaterializedTable, should we also consider the reverse > direction? > > > > > > I'd appreciate your thoughts on this. > > > > > > Kui.Yuan > > > > > > Ramin Gharib <[email protected]> 于2026年5月13日周三 18:57写道： > > > > > > > Hi everyone, > > > > > > > > I'd like to start a discussion on FLIP-578: In-place Table to > > > Materialized > > > > Table conversion. [1] > > > > Flink's catalog DDL has no way to change a table's kind. Converting a > > > > regular CREATE TABLE to a materialized table today requires creating > a > > > new > > > > MT under a different name, repointing downstream consumers, and > dropping > > > > the old one — breaking references, doubling storage during cutover, > and > > > > offering no atomic rename. > > > > > > > > The FLIP proposes to allow CREATE OR ALTER MATERIALIZED TABLE > > > > <existing_table> AS <query> against an existing regular table, > converting > > > > it in place. Identity and storage are preserved, and the conversion > > > > surfaces as a distinct Operation subtype. > > > > > > > > Highlights: > > > > 1. Gated by a new opt-in config option, > > > > table.materialized-table.conversion-from-table.enabled (default > false). > > > > Existing workflows are unchanged. > > > > 2. A new Catalog method convertTableToMaterializedTable(...) is added > > > with > > > > a default that throws UnsupportedOperationException, so existing > catalogs > > > > remain source- and binary-compatible. > > > > 3. Validation reuses the existing ALTER MATERIALIZED TABLE machinery > > > > (schema reconciliation, options merge, freshness / refresh-mode). The > > > > CREATE OR ALTER keyword (FLIP-546 [2]) is idempotent and inherits > > > FLIP-492 > > > > query evolution for free. > > > > > > > > Looking forward to your feedback. > > > > > > > > Best, > > > > Ramin Gharib [1] > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-578%3A+In-place+Table+to+Materialized+Table+conversion > > > > [2] > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-546%3A+Introduce+CREATE+OR+ALTER+for+Materialized+Tables > > > > > > > >
