Hello Yang Guo, thank you for the FIP! I read it together with issue #2362 [1] and its companion design doc [2], which proposes sys.lake_tiering_status along similar "system view" lines, and also looked at PR #3186 [3], which appears to implement this FIP's framework. A few notes, mostly aimed at reconciling the two efforts:
1) Coordinator-only serving isn't stated in the FIP text. Tiering status lives only in the coordinator's memory, so a view over it needs scanSystemView() to be coordinator-only. PR [3] already does this (RpcServiceBase rejects the call on TabletService, only CoordinatorService implements it), but the FIP itself never says this is a requirement or a supported pattern. #2362 [1][2] seems to have been designed without knowledge of this - it defines its own separate RPC (ListTieringStatuses on CoordinatorGateway) instead of extending SystemViewProvider. Worth stating the coordinator-only case explicitly in the FIP, and reconciling the two designs so lake_tiering_status becomes one more SystemViewProvider rather than a parallel mechanism. 2) Small naming inconsistency: the tiering view is called sys.lake_tiering_status in Motivation and Public Interface, but lake_tiering_service in "What's Next?". 3) The permissions section (Public Interface) reads as a blanket deny for external reads when the authorizer is disabled. The actual per-type split (views always readable, persistent tables "depends on the table") only appears later, in the guard pseudocode. Worth stating the split up front where permissions are first introduced. 4) SystemViewProvider.scanRows() takes no caller identity, and the guard model gives every view unconditional external READ. Fine for tablet_servers/buckets, but lake_tiering_status (per #2362 [2]) wants per-row filtering by DESCRIBE on the underlying table. Suggest adding a VIEW ResourceType for a coarse per-view ACL gate, plus passing caller context into scanRows() (or filtering RPC-side), so a provider can do row-level filtering where it's needed. 5) Proposed Changes fully specifies server-side and Flink-side work, but never mentions fluss-client changes, even though the Public Interface examples depend on them. PR [3] already touches FlussConnection.getTable(), FlussAdmin.getTableInfo(), and FlussTable (newScan/newLookup/newAppend/newUpsert). Worth its own explicit subsection. 6) Compatibility/Migration Plan is thin for a change that adds two RPCs and branches core client behavior. It asserts compatibility in one line without covering the old-client/new-server and new-client/old-server matrix - e.g. what an old client does with an unrecognized table_kind field, or what happens if it attempts a write/lookup against a system view. 7) Rejected Alternatives is currently empty. For a design that makes real choices (view vs. persistent split, a dedicated scan RPC instead of the standard read path, coordinator-only serving), it'd help to record what else was considered. Happy to help work through any of this, especially getting #2362 [1][2] aligned with the framework in PR [3] rather than parallel to it. [1] https://github.com/apache/fluss/issues/2362 [2] https://docs.google.com/document/d/1eJbRCwzAbeJLA97zQQ0I3JM1jerBXXhq69Dn8r4xWV0/edit?tab=t.0#heading=h.j7m79e2nnoxy [3] https://github.com/apache/fluss/pull/3186 Best, Lorenzo On Wed, Apr 22, 2026 at 3:13 PM Yang Guo <[email protected]> wrote: > Hi community, > > I'd like to start a discussion on a new feature proposal: introducing > "system tables" to Fluss.[1] > > System tables are tables managed by the cluster itself that reflect > internal cluster state and bookkeeping information. This concept is now > well established across streaming and storage systems, like Kafka internal > topics, PostgreSQL System Catalog[2] and System Views[3], StarRocks > information_schema[4] and so on. It helps those systems in many aspects > including metadata management, operations, and maintenance. > > This proposal is to introduce system tables into Fluss, like other > systems already did. It may benefit our project in three key ways: > 1. Durable storage for system-critical metadata — preparing for more > features. such as consumer group offsets and capture DDL change events, > persisted through Fluss's own log storage (LogTablet / KvTablet). > 2. Operational visibility — exposing cluster runtime state (tablet server > registrations, bucket ISR, tiering status) via standard SQL query > interfaces for operators and downstream applications. > 3. Future ZooKeeper migration — laying the groundwork to move > ZooKeeper-held metadata into Fluss itself, reducing external dependencies. > > For more details, the full draft is available here: FIP-39 > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-39%3A+Introduce+System+Tables > > I'd really look forward to any of your thoughts and suggestions. Any > feedback is welcome. > > Links: > [1] FIP-39 Introduce System Tables: > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-39%3A+Introduce+System+Tables > [2] PostgresSQL System Catalog doc: > https://www.postgresql.org/docs/current/catalogs.html > [3] PostgresSQL System View doc: > https://www.postgresql.org/docs/current/views.html > [4] StarRocks Information Schema doc: > https://docs.starrocks.io/docs/sql-reference/information_schema/ > > Best Regards, > Yang Guo > -- Lorenzo Affetti Senior Software Engineer @ Flink Team Ververica <http://www.ververica.com>
