raghav-reglobe opened a new pull request, #64648:
URL: https://github.com/apache/doris/pull/64648
## Proposed changes
`IcebergScanNode.getCountFromSnapshot()` reads `total-equality-deletes`,
`total-position-deletes` and `total-records` from the Iceberg snapshot
summary
and calls `.equals()` / `Long.parseLong()` directly on the `Map.get()`
results.
An Iceberg snapshot summary is **not guaranteed** to carry these `total-*`
counters — snapshots written by compaction/replace (and some writers) may
omit
them. When a counter is absent, `SELECT COUNT(*)` throws:
```
java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because
the return value of "java.util.Map.get(Object)" is null
at
org.apache.doris.datasource.iceberg.source.IcebergScanNode.getCountFromSnapshot(IcebergScanNode.java:1154)
at
org.apache.doris.datasource.iceberg.source.IcebergScanNode.isBatchMode(...)
at
org.apache.doris.datasource.FileQueryScanNode.createScanRangeLocations(...)
```
`SELECT *` on the same table works (it scans); only `COUNT(*)` fails (it
takes
the metadata-count shortcut). Reproducible on current master.
### Fix
Extract the summary parsing into a pure static
`getCountFromSummary(Map<String, String> summary, boolean
ignoreDanglingDelete)`
that null-checks the counters and **falls back to a normal scan** (`return
-1`,
the method's existing "cannot push down count" signal) when any required
counter is absent. Behaviour is otherwise unchanged.
> A similar unguarded access exists in `IcebergUtils`
> (`Long.parseLong(summary.get(TOTAL_RECORDS)) -
Long.parseLong(summary.get(TOTAL_POSITION_DELETES))`).
> Happy to guard it in this PR or a follow-up — let me know your preference.
## Release note
Fix a NullPointerException on `SELECT COUNT(*)` over an Iceberg table whose
latest snapshot summary omits the `total-*` counters (e.g. snapshots produced
by compaction/replace).
## Check List
- [x] New feature / bug fix has unit test
(`IcebergCountPushDownTest`: missing-counter, no-delete, equality-delete
and
position-delete cases)
- [x] Behaviour-preserving for complete summaries; only adds a null-safe
fall-back to scan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
