Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22091 )
Change subject: IMPALA-13531: Calcite CTE frontend ...................................................................... Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/22091/8/fe/src/main/java/org/apache/impala/planner/CTEBufferNode.java File fe/src/main/java/org/apache/impala/planner/CTEBufferNode.java: http://gerrit.cloudera.org:8080/#/c/22091/8/fe/src/main/java/org/apache/impala/planner/CTEBufferNode.java@33 PS8, Line 33: public class CTEBufferNode extends PlanNode { > the diff is push or pull? I'd overlooked that implementation. The design goals are different, although there may be arguments to combine them. My understanding is MultiDataSink receive data pushed to them, and dispatch to multiple child data sinks (with usually different expressions transforming or filtering them). This is a synchronous operation, and I believe it all happens within a fragment. CTEBufferNode stores in-memory, with the option to spill to disk. That allows it to be asynchronous, but either uses more memory (when in-memory) or is slower (when spilled to disk). It also expects to be readable by other fragments. -- To view, visit http://gerrit.cloudera.org:8080/22091 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id0840c0859d2fe25628d799a18d302cee1eb36e8 Gerrit-Change-Number: 22091 Gerrit-PatchSet: 8 Gerrit-Owner: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Anonymous Coward (816) Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Steve Carlin <scar...@cloudera.com> Gerrit-Comment-Date: Wed, 16 Apr 2025 17:52:32 +0000 Gerrit-HasComments: Yes
