Hi Zhe, Thanks for the detailed questions. They highlight several limitations and operational assumptions that would benefit from clearer documentation. Here are my thoughts on each point.
**1. Write availability during migration** Yes, the initial version intentionally uses an offline migration approach that requires a maintenance window. The write outage covers the entire initialization process, including Bulk Load, initial snapshot registration, validation, and datalake enablement. Its duration depends on the amount of data being loaded and the available resources. I will clarify this in the FIP. Your understanding of the partition scope is also correct. Validation compares the table-level Paimon snapshot ID, so writes to unselected partitions that advance the snapshot would also cause validation to fail. The requirement to stop native Paimon writers therefore applies to the entire source table. The baseline-load-plus-incremental-catch-up approach you outlined is a useful direction for future work. It would require defining incremental replay, ordering, deduplication, delete handling, writer fencing, and the exact handoff boundary. Online migration is outside the scope of the initial version. The FIP should also describe recovery from incomplete initialization. As discussed in the earlier recovery question, the proposed initial approach is to resolve any remaining Bulk Load work before dropping the Fluss table created by the promotion and retrying, while preserving the original Paimon table and its data. A timeout alone should not trigger cleanup, since the underlying work may have succeeded. **2. Partially initialized regular partitions** By default, a regular partitioned primary-key table will load all existing partitions from the source snapshot. Explicit partition selection is intended for cases where users already know which existing partitions may receive subsequent writes. For example, loading only recent partitions can reduce both Bulk Load time and the required maintenance window. When choosing this optimization, users must ensure that unloaded existing partitions do not receive writes through Fluss. Fluss primary-key point lookups are also unsupported for those partitions. The initial version relies on this operational prerequisite and does not provide additional checks to reject such requests automatically. I agree that an accidental write to an unloaded partition could encounter missing historical KV state and produce incorrect update, delete, or changelog behavior. I will make this risk and the responsibility explicit in the FIP. Users who cannot guarantee the access scope should use the default initialization of all existing partitions. The current procedure also does not define a way to initialize additional partitions by repeating `CALL`; an existing Fluss table causes the procedure to skip Bulk Load. Loading the remaining partitions later would require a separately defined extension. **3. Unknown Paimon properties** I have also spent quite some time considering this tradeoff. Internally, we have many Paimon tables with custom options, including application metadata. Rejecting every unknown property would prevent many of these tables from using the feature directly. Requiring users to acknowledge each property or maintain an allowlist would also add operational overhead. Because application metadata and behavior-related options share the same property map, an unrecognized key alone does not tell us which category it belongs to. The current proposal validates recognized options against the supported mappings and rejects known incompatibilities. Unknown properties remain in the original Paimon table, but they do not define additional behavior in Fluss. Given these limitations and the usability cost, I think retaining the current default of allowing unknown properties is a reasonable choice for the initial version. This still leaves a compatibility risk when an unknown property controls behavior that the source table depends on. One improvement would be to explicitly report which properties were not validated and document the supported Paimon versions and mappings. This would make the validation coverage visible without requiring an additional acknowledgement step for every table. Successful promotion should not be interpreted as a compatibility guarantee for arbitrary custom features or behavior introduced by newer Paimon versions. Best regards, Yuxia ----- 原始邮件 ----- 发件人: "yuxia" <[email protected]> 收件人: "dev" <[email protected]> 发送时间: 星期五, 2026年 9 月 11日 下午 1:41:51 主题: Re: FIP-25: Support In-Place Promotion of Existing Paimon Tables to Datalake-Enabled Fluss Tables Hi, Liting. Thanks for reviewing FIP-25 and raising these questions. They highlight two areas that need clearer explanation: how callers observe a promotion that takes a long time, and how they recover when initialization is only partially completed. > Your first question: could the caller time out while promotion continues, and > should the procedure > return a promotion operation ID for status tracking? The reason for keeping the procedure synchronous is to provide a clear completion guarantee: when `CALL` returns successfully, Bulk Load, initial snapshot registration, and datalake enablement have all completed, and the table is ready for use through Fluss. For Flink SQL Gateway's REST endpoint, SQL submission already returns an `OperationHandle`, which clients use to query status and fetch results. The Flink SQL Client also sends session heartbeats independently while waiting. Therefore, synchronous execution inside the procedure does not require keeping a single HTTP request open for the entire Bulk Load duration. See the [SQL Gateway REST documentation][1]. A separate asynchronous promotion API would require us to define how promotion stages are tracked, how the remaining steps continue after returning to the caller, and how that execution recovers across failures. For the initial version, I would prefer to retain the synchronous procedure and reuse the Gateway's existing SQL operation handling. A client disconnect alone does not establish that promotion failed, and an interrupted procedure may already have completed some steps. I agree that these cases need clarification, together with the recovery and retry semantics in your second point. > Your second question: if table creation succeeds but Bulk Load fails, how can > a retry distinguish > incomplete initialization from a successfully initialized table and avoid > enabling datalake > prematurely? For a primary-key table requiring Bulk Load, the proposal registers the initial lake snapshot only after the required loading has completed successfully. When enabling datalake, `alterTable()` validates the current Paimon snapshot against the snapshot registered in Fluss. If initialization has not reached snapshot registration, this validation rejects enablement. Finding an existing Fluss table and skipping Bulk Load would therefore not bypass this check. However, you are right that the FIP does not currently explain how users recover from incomplete initialization. For the initial version, I propose a manual recovery path for incomplete initialization: first stop any remaining Bulk Load job and confirm that its transactions have reached a terminal state, then drop the Fluss table created by the promotion and rerun the procedure. This recovery path must preserve the original Paimon table and its data. A timeout or cancellation alone should not trigger cleanup; users should first establish the actual outcome. If Bulk Load and initial snapshot registration have both completed, and only datalake enablement failed, users can address the failure and repeat the procedure. The existing snapshot and schema checks would then determine whether enablement can proceed, without reloading the data. I will clarify these cases in the FIP, including the scope of idempotent retries. Persisting additional promotion metadata to support automatic resumption of incomplete initialization is a useful enhancement, but I would prefer to defer that capability beyond the initial version. [1]: https://nightlies.apache.org/flink/flink-docs-release-2.2/docs/dev/table/sql-gateway/rest/#overview-of-sql-processing Best regards, Yuxia ----- 原始邮件 ----- 发件人: "Zhe Wang" <[email protected]> 收件人: "dev" <[email protected]> 发送时间: 星期四, 2026年 9 月 10日 上午 12:36:48 主题: Re: FIP-25: Support In-Place Promotion of Existing Paimon Tables to Datalake-Enabled Fluss Tables Hi yuxia, Thanks for the detailed proposal. I have a few additional questions, mainly around production migration and data correctness. 1. Write availability during migration The current proposal requires users to stop native Paimon writers before invoking the procedure, while new writes through Fluss can only start after the Bulk Load, snapshot registration, validation, and datalake enablement have all completed. For a large primary-key table, the Bulk Load may take a long time, so the business write outage could potentially last for hours. Since the final validation compares the table-level Paimon snapshot ID, it also seems that writes to partitions outside the selected Bulk Load scope would advance the snapshot and cause validation to fail. Is an offline, maintenance-window-style migration an intentional limitation of this FIP? For production migration, would it be possible to separate the long-running baseline load from the final write ownership cutover? One possible flow would be: 1. capture a baseline Paimon snapshot; 2. Bulk Load that snapshot while Paimon writers continue running; 3. catch up changes committed after the baseline; 4. stop or fence the Paimon writers briefly; 5. apply the remaining changes, verify consistency, and enable Fluss; 6. switch application writes to Fluss. This would require defining the incremental replay, ordering, deduplication, fencing, rollback, and exact handoff boundary. If online migration is outside the scope of FIP-25, I think it would still be useful to document the expected write outage and the recommended recovery procedure when a long-running migration fails. 2. Enforcement for partially initialized regular partitions For a regular partitioned primary-key table, the proposal allows users to Bulk Load only selected partitions. It states that unloaded partitions do not support Fluss lookups or writes and that users must ensure no writes are issued to them. How is the set of successfully initialized partitions persisted and exposed after the procedure completes? Should Fluss reject lookup and write requests to an unloaded partition, instead of relying only on an operational requirement? Without such enforcement, an accidental write to an unloaded partition may treat the historical KV state as empty and produce incorrect upsert, delete, or changelog behavior. It would also be helpful to clarify whether and how the remaining partitions can be initialized later. 3. Handling of unknown Paimon properties The compatibility section explains that properties unknown to the Paimon version bundled with Fluss are allowed, because they may be application metadata. It also acknowledges that such a property may actually control behavior introduced by a newer Paimon version, in which case promotion could succeed even though the resulting Fluss semantics are different. For a correctness-sensitive operation such as in-place promotion, is silently allowing these properties the right default? Would it be safer to report all unknown properties and require an explicit acknowledgement or allowlist before proceeding? A strict mode that rejects unknown properties by default could also make the compatibility guarantee easier for users to understand. Best regards, Zhe Wang Liting Liu (litiliu) via dev <[email protected]> 于2026年9月9日周三 16:52写道： > Hi Yuxia and Fluss community, > > Thanks for sharing FIP-25. It is a very interesting proposal. While > reading it and comparing it with the current `apache/main` branch, I had a > few questions that may be worth clarifying. > > First, the proposal says that `CALL sys.enable_fluss_on_lake_table(...)` > waits for the `load_lake_data_to_fluss` Bulk Load job, snapshot > registration, and datalake enablement to complete before returning. > > For a large Paimon table, this operation could take a considerable amount > of time. Depending on the SQL Gateway, client, or HTTP/proxy configuration, > the caller might time out while the underlying operation is still running. > In that situation, it may be difficult for the caller to determine the > final outcome. > > Would it make sense for the procedure to return a promotion operation ID > and provide a separate way to query its status? Alternatively, it would be > helpful to document how timeout and cancellation are expected to work. > > Second, I was wondering about failure recovery and retry behavior. > > If Fluss table creation succeeds but the Bulk Load job fails, the proposal > says that the table remains present with datalake disabled. On a subsequent > call, however, the existing table may cause the procedure to skip table > creation and Bulk Load. It would be useful to clarify how the procedure can > distinguish a successfully initialized table from one whose Bulk Load > failed or is still incomplete. > > Could the design consider persisting some information such as the > following to make retry and recovery easier to understand? > > - promotion operation ID; > - source Paimon snapshot ID; > - selected partitions; > - Bulk Load transaction ID. > > It may also be helpful to describe the expected retry, abort, timeout, and > cancellation semantics, and to make sure that a retry cannot enable > datalake before the Bulk Load result has been successfully verified. > > Best regards, > > Liting Liu > > From: yuxia <[email protected]> > Date: Tuesday, September 8, 2026 at 15:23 > To: dev <[email protected]> > Subject: [DISCUSS] FIP-25: Support In-Place Promotion of Existing Paimon > Tables to Datalake-Enabled Fluss Tables > > Hi Fluss community, > > I'd like to start a discussion on FIP-25: Support In-Place Promotion of > Existing Paimon Tables to Datalake-Enabled Fluss Tables[1]. > > This FIP proposes promoting an existing Paimon table to a datalake-enabled > Fluss table in place through: > > CALL sys.enable_fluss_on_lake_table(...); > > "In-place" means reusing the existing Paimon table and historical data > without creating a replacement table or rewriting the data to Paimon. Fluss > creates the corresponding metadata, initializes historical primary-key > state through Bulk Load when necessary, and validates schema and snapshot > consistency before enabling datalake. > > Feedback and suggestions are welcome. > > [1]: > https://cwiki.apache.org/confluence/spaces/FLUSS/pages/406623572/FIP-25+Support+In-Place+Promotion+of+Existing+Paimon+Tables+to+Datalake-Enabled+Fluss+Tables > > Best regards, > Yuxia >
