Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/21557 )
Change subject: IMPALA-13088, IMPALA-13109: Use RoaringBitmap instead of sorted vector of int64s ...................................................................... Patch Set 10: (4 comments) Thanks for the comments! http://gerrit.cloudera.org:8080/#/c/21557/9//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21557/9//COMMIT_MSG@68 PS9, Line 68: > Some of the columns are too long. Done http://gerrit.cloudera.org:8080/#/c/21557/9/be/src/exec/iceberg-delete-builder.h File be/src/exec/iceberg-delete-builder.h: http://gerrit.cloudera.org:8080/#/c/21557/9/be/src/exec/iceberg-delete-builder.h@119 PS9, Line 119: RoaringBitmap64 > As we discussed offline, memory used by roaring bitmap could be tracked the That's a great idea. Would you mind creating a Jira ticket for this? http://gerrit.cloudera.org:8080/#/c/21557/9/fe/src/main/java/org/apache/impala/planner/IcebergDeleteNode.java File fe/src/main/java/org/apache/impala/planner/IcebergDeleteNode.java: http://gerrit.cloudera.org:8080/#/c/21557/9/fe/src/main/java/org/apache/impala/planner/IcebergDeleteNode.java@177 PS9, Line 177: onditions.checkState(fragment_.getN > Is it possible for these to be unknown? If not, then this if could be repla IcebergScanPlanner should always set these stats. http://gerrit.cloudera.org:8080/#/c/21557/9/fe/src/main/java/org/apache/impala/planner/IcebergDeleteNode.java@201 PS9, Line 201: // Let's calculate with some overhead per roaring bitmap. > Can you move the 1.5 to a constant? I updated the calculations to have 512 bytes overhead per roaring bitmap. Also increased bytes per position to 2. I also don't want to overestimate with a too big margin as that would limit Impala too much to execute the query. If there are few rows per file, then the whole roaring bitmap might consume a few hundred bytes of untracked memory at most, so it's probably won't be too critical anyway. -- To view, visit http://gerrit.cloudera.org:8080/21557 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib769965d094149e99c43e0044914d9ecccc76107 Gerrit-Change-Number: 21557 Gerrit-PatchSet: 10 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 11 Jul 2024 16:59:31 +0000 Gerrit-HasComments: Yes
