github-actions[bot] commented on code in PR #67361:
URL: https://github.com/apache/doris/pull/67361#discussion_r4244477462
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/mvcc/PluginDrivenMvccExternalTable.java:
##########
@@ -324,25 +327,47 @@ private void listPartitions(ConnectorMetadata metadata,
ConnectorSession session
toListPartitionItem(partitionName, types,
part.getOrderedPartitionValues(),
part.getPartitionValueNullFlags()));
} catch (Exception e) {
- LOG.warn("toListPartitionItem failed, partitionColumns: {},
partitionName: {}",
- partitionColumns, partitionName, e);
+ // Counted here and reported once below. Both skip shapes are
per-TABLE conditions that
+ // repeat on EVERY snapshot load, i.e. on every query planned
against the table: a spec
+ // whose values are not representable in the partition
columns' types (an iceberg spec
+ // mixing an identity field with a month()/bucket() transform,
say - the connector
+ // supplies the transform result, keyed under the source
column) fails for every
+ // partition, every time. A WARN with a stack trace per
partition is then thousands of
+ // log lines per query, enough to fill an FE log volume; the
per-partition detail stays
+ // available at DEBUG.
+ skipped++;
+ if (firstSkippedCause == null) {
+ firstSkippedName = partitionName;
+ firstSkippedCause = e;
+ }
Review Comment:
[P2] Prevent the inner TIMESTAMPTZ parser from logging each skipped
partition. An Iceberg table with timestamp-with-zone ts,
enable.mapping.timestamp_tz=true, and a non-RANGE spec such as month(ts),
identity(region) reaches this LIST fallback. Iceberg supplies the numeric month
result for ts; PartitionKey.createListPartitionKeyWithTypes calls
getDateTimeLiteral, whose catch logs LOG.warn(..., e) for every failed value
before control reaches this catch. Large listings still emit one WARN stack
trace per partition plus the new summary, leaving the log-volume problem this
PR targets. Suppress or downgrade that inner warning for expected item-build
failures while preserving the exception and aggregate diagnostic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
