Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/22062 )
Change subject: IMPALA-13487: Add profile counters for memory allocation ...................................................................... Patch Set 21: (1 comment) http://gerrit.cloudera.org:8080/#/c/22062/20//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/22062/20//COMMIT_MSG@12 PS20, Line 12: The following counters are added: > I strange pattern I see here is that we have frees but no allocs: Some memory chunks are not allocated by the mem-pool of fragment level RowBatches. E.g. ScratchTupleBatch::FinalizeTupleTransfer() and ScratchTupleBatch::ReleaseResources() transfers mempool chunks to the output RowBatch. Another example is uncompressed Parquet data pages are allocated by data_page_pool_ of ParquetColumnChunkReader and attached to the output RowBatch: https://github.com/apache/impala/blob/1f473f365a211242cd865e7c5c896263a7e0ad68/be/src/exec/parquet/parquet-column-chunk-reader.cc#L77 So in this example, we only see free() used in the output RowBatch, i.e. the fragment level RowBatch. allocate() is done in other places. -- To view, visit http://gerrit.cloudera.org:8080/22062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I982315d96e6de20a3616f3bd2a2b4866d1ff4710 Gerrit-Change-Number: 22062 Gerrit-PatchSet: 21 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Xuebin Su <x...@cloudera.com> Gerrit-Comment-Date: Sat, 12 Apr 2025 04:15:58 +0000 Gerrit-HasComments: Yes
