andygrove commented on PR #15654:
URL: https://github.com/apache/datafusion/pull/15654#issuecomment-2794236777
I tested this PR with Comet. Here are the most relevant configs for Comet
related to this testing:
```
$SPARK_HOME/bin/spark-submit \
--master $SPARK_MASTER \
--jars $COMET_JAR \
--driver-class-path $COMET_JAR \
--conf spark.driver.memory=8G \
--conf spark.executor.instances=1 \
--conf spark.executor.cores=8 \
--conf spark.cores.max=8 \
--conf spark.executor.memory=8g \
--conf spark.memory.offHeap.enabled=true \
--conf spark.memory.offHeap.size=1g \
--conf spark.executorEnv.COMET_WORKER_THREADS=32 \
--conf spark.executorEnv.COMET_MAX_BLOCKING_THREADS=80 \
```
With Comet main branch, TPC-H q4 never completes due to deadlock. With the
changes in this PR, the query completes with good performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
