hubgeter commented on code in PR #65851:
URL: https://github.com/apache/doris/pull/65851#discussion_r3669015139
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergTransaction.java:
##########
@@ -145,6 +148,10 @@ public void beginInsert(ExternalTable dorisTable,
Optional<InsertCommandContext>
+ " is a tag, not a branch. Tags
cannot be targets for producing snapshots");
}
}
+ if (writeSchemaContext.isPresent()) {
+ writeSchemaContext.get().validateCurrentSchema(
+ table, insertCtx.isStaticPartitionOverwrite());
Review Comment:
Fixed in 6c6ac29105d. Every INSERT OVERWRITE now requires the active
partition spec to remain identical to the statement-pinned spec, both during
finish-time refresh and at the final metadata commit boundary. Added a real
Iceberg transaction test for partitioned P1 -> unpartitioned P2 drift covering
both empty and non-empty dynamic overwrite results; neither path commits a
snapshot.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergTransaction.java:
##########
@@ -145,6 +148,10 @@ public void beginInsert(ExternalTable dorisTable,
Optional<InsertCommandContext>
+ " is a tag, not a branch. Tags
cannot be targets for producing snapshots");
}
}
+ if (writeSchemaContext.isPresent()) {
+ writeSchemaContext.get().validateCurrentSchema(
Review Comment:
Fixed in 6c6ac29105d. Doris now creates the Iceberg transaction over a
validating TableOperations wrapper. Each final commit attempt validates the
pinned schema/branch/spec against the exact TableMetadata base immediately
before delegating the CAS; if Iceberg refreshes and replays pending updates
after a concurrent commit, the replay attempt validates the refreshed base
before it can commit. Added a real transaction test that stages an append,
concurrently adds a required column, and verifies commitTransaction rejects it
without committing a snapshot.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##########
@@ -505,22 +524,63 @@ private List<String> getOrderedPathPartitionKeys() {
}
public void createScanRangeLocations() throws UserException {
- super.createScanRangeLocations();
+ Schema scanSchema = getQuerySchema();
+ Set<Integer> equalityDeleteFieldIds = Collections.emptySet();
+ if (!isSystemTable) {
+ equalityDeleteFieldIds = getEqualityDeleteFieldIdsForScan();
Review Comment:
Fixed in 6c6ac29105d. Scan planning now evaluates the exact snapshot-summary
COUNT(*) eligibility before the equality-delete preflight. When the summary is
usable (including total-equality-deletes=0), it caches that decision for
isBatchMode and returns an empty equality-field set without creating or
exhausting a file scan. The new test verifies TableScan.snapshot() and
planFiles() are never called and the summary is loaded only once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
