Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22094 )
Change subject: IMPALA-13533: Calcite CTE backend ...................................................................... Patch Set 27: (1 comment) http://gerrit.cloudera.org:8080/#/c/22094/25/be/src/runtime/local-exchanger.cc File be/src/runtime/local-exchanger.cc: http://gerrit.cloudera.org:8080/#/c/22094/25/be/src/runtime/local-exchanger.cc@44 PS25, Line 44: cell->consumers_left--; > This code is based on https://github.com/StarRocks/starrocks/blob/3.3.4/be/ The main challenge this avoids is: 1. a consumer reads the last available cell 2. same consumer tries to read another cell but there are none; if we're not checking cell->next, then we do cell->consumers_left--; progress_[consumer_index]=cell->next (nullptr) 3. producer pushes a new cell Now if progress_[consumer_index]==nullptr, we don't know whether that consumer has read the cell we just pushed unless Push iterates over progress_ to see if any are nullptr and updates them with the new cell. We essentially wind up trading off memory (keeping a cell that's been read until the next cell is pushed) for computation (iterating progress_ every time we push a new cell). -- To view, visit http://gerrit.cloudera.org:8080/22094 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48f16d495d4b37be97e6a913f0eb5b94d70e199a Gerrit-Change-Number: 22094 Gerrit-PatchSet: 27 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Comment-Date: Wed, 17 Dec 2025 23:06:00 +0000 Gerrit-HasComments: Yes
