github-actions[bot] commented on code in PR #65867:
URL: https://github.com/apache/doris/pull/65867#discussion_r3655531095
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/PaimonSysExternalTable.java:
##########
@@ -114,17 +117,37 @@ private static long generateSysTableId(long
sourceTableId, String sysTableType)
/**
* Returns the Paimon system table instance (e.g., snapshots, binlog).
* Note: system tables currently ignore snapshot semantics.
+ *
+ * <p>The wrapper is built over the cached base table, exactly the way the
Paimon catalog builds
+ * it ({@code CatalogUtils#createSystemTable}). Loading it from the
catalog instead would give
+ * this wrapper its own schema generation, independent of the cached base
table: the FE would
+ * then plan on one generation while {@code PaimonScanNode} serializes a
system table rebuilt
+ * over the other one, and a system table whose row type follows the base
schema
+ * ({@code $audit_log}, {@code $binlog}, {@code $ro}) could reach the BE
without the columns the
+ * FE asked for.
*/
public Table getSysPaimonTable() {
if (paimonSysTable == null) {
synchronized (this) {
if (paimonSysTable == null) {
- PaimonExternalCatalog catalog = (PaimonExternalCatalog)
getCatalog();
- paimonSysTable = catalog.getPaimonTable(
- sourceTable.getOrBuildNameMapping(),
- "main", // branch
- sysTableType // queryType: snapshots, binlog, etc.
- );
+ sourceTable.makeSureInitialized();
+ // System tables ignore snapshot semantics, so the empty
snapshot resolves the base table.
+ Table baseTable =
sourceTable.getPaimonTable(Optional.empty());
+ Table sysTable = baseTable instanceof FileStoreTable
+ ? SystemTableLoader.load(sysTableType,
(FileStoreTable) baseTable)
Review Comment:
[P1] Keep the fallback pair visible when constructing `$ro`
With file-based Paimon privileges enabled, the cached base reaches this line
as `PrivilegedFileStoreTable(FallbackReadFileStoreTable(main, fallback))`.
Paimon 1.3.1's `ReadOptimizedTable#newScan` creates a two-branch
`FallbackReadScan` only when its immediate wrapped object is a
`FallbackReadFileStoreTable`; with the privilege decorator outside it takes the
ordinary path instead. That path calls `newSnapshotReader()` through the
decorator, and the fallback wrapper's inherited delegated reader sees only the
main branch, so fallback-only partitions never produce FE splits for `tbl$ro`.
Rebuilding a bare fallback pair for BE later cannot recover splits that FE
omitted. This is earlier than the existing serialization-time fallback thread.
Please construct the read-optimized wrapper with the fallback pair visible
while preserving the FE privilege check, and cover a privileged fallback table
whose fallback branch has a unique partition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
