andygrove opened a new issue, #2088:
URL: https://github.com/apache/datafusion-comet/issues/2088
### Describe the bug
```
[Executor task launch worker for task 1.0 in stage 116.0 (TID 6951)] ERROR
org.apache.spark.sql.comet.execution.shuffle.CometBypassMergeSortShuffleWriter
- Error while deleting file
/private/var/folders/d2/b93h6k7174ddqxltrxgb51040000gn/T/blockmgr-fd858ada-1024-4003-b3b3-5750d5863fdc/1d/temp_shuffle_3ba6142e-cc64-41fd-929b-3a3277a740fd
[Executor task launch worker for task 1.0 in stage 116.0 (TID 6951)] INFO
org.apache.spark.executor.Executor - Executor killed task 1.0 in stage 116.0
(TID 6951), reason: Stage cancelled: Job aborted due to stage failure: Task 0
in stage 116.0 failed 1 times, most recent failure: Lost task 0.0 in stage
116.0 (TID 6950) (172.24.0.48 executor driver): java.lang.AssertionError:
Called freePage() on a memory block that has already been freed
at
org.apache.spark.memory.TaskMemoryManager.freePage(TaskMemoryManager.java:361)
at
org.apache.spark.memory.MemoryConsumer.freePage(MemoryConsumer.java:130)
at
org.apache.spark.shuffle.comet.CometUnifiedShuffleMemoryAllocator.free(CometUnifiedShuffleMemoryAllocator.java:61)
at
org.apache.spark.sql.comet.execution.shuffle.SpillWriter.freeMemory(SpillWriter.java:222)
at
org.apache.spark.sql.comet.execution.shuffle.CometDiskBlockWriter.doSpill(CometDiskBlockWriter.java:234)
at
org.apache.spark.sql.comet.execution.shuffle.CometDiskBlockWriter$ArrowIPCWriter.spill(CometDiskBlockWriter.java:454)
at
org.apache.spark.sql.comet.execution.shuffle.SpillWriter.acquireNewPageIfNecessary(SpillWriter.java:141)
at
org.apache.spark.sql.comet.execution.shuffle.CometDiskBlockWriter.insertRow(CometDiskBlockWriter.java:284)
at
org.apache.spark.sql.comet.execution.shuffle.CometBypassMergeSortShuffleWriter.write(CometBypassMergeSortShuffleWriter.java:231)
at
org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
```
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
