ji chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/23613 )
Change subject: IMPALA-14092 Part2: Support querying of paimon data table via JNI ...................................................................... Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/23613/8/be/src/exec/paimon/paimon-jni-scan-node.cc File be/src/exec/paimon/paimon-jni-scan-node.cc: http://gerrit.cloudera.org:8080/#/c/23613/8/be/src/exec/paimon/paimon-jni-scan-node.cc@145 PS8, Line 145: OffheapTrackFree(); : OffheapTrackAllocation(offheap_consumed_bytes); > Can this be implemented as increment rather than release and reclaim? The r Q:This code seems to allocate Arrow batch first then increase MemTracker. Usually, it is the other way around. Can we find out the size of next arrow batch before pulling it from GetNextBatchDirect? A: Not possible, since the actual bytes used by arrow is unknown, until rows of data is actually written to offheap memory. Q:Can this be implemented as increment rather than release and reclaim? The reason is, MemTracker might not able to reclaim if the portion of memory that it just freed is claimed by other PlanNode. A: According to my understanding, MemTracker is created for each paimon scan-node object, it is not shared for PlanNode,so MemTrack allocation&free is independent of each other, please correct me if I'm wrong. -- To view, visit http://gerrit.cloudera.org:8080/23613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie679a89a8cc21d52b583422336b9f747bdf37384 Gerrit-Change-Number: 23613 Gerrit-PatchSet: 8 Gerrit-Owner: ji chen <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: ji chen <[email protected]> Gerrit-Comment-Date: Wed, 19 Nov 2025 14:55:52 +0000 Gerrit-HasComments: Yes
