andygrove commented on PR #1773: URL: https://github.com/apache/datafusion-comet/pull/1773#issuecomment-2906822147
> I have rebased this branch onto the latest main. The [ubuntu-latest/java 17-spark-4.0/java](https://github.com/apache/datafusion-comet/actions/runs/15222051464/job/42819140331?pr=1773#logs) test slowed down then failed. I'm not sure if it is transient or not. > > ``` > Sat, 24 May 2025 02:45:30 GMT - columnar shuffle on map [float] (3 seconds, 751 milliseconds) > Sat, 24 May 2025 02:45:34 GMT - columnar shuffle on map [double] (3 seconds, 776 milliseconds) > Sat, 24 May 2025 02:45:42 GMT - columnar shuffle on map [date] (7 seconds, 957 milliseconds) > Sat, 24 May 2025 02:54:54 GMT - columnar shuffle on map [timestamp] (9 minutes, 11 seconds) > Sat, 24 May 2025 03:05:03 GMT [INFO] ------------------------------------------------------------------------ > Sat, 24 May 2025 03:05:03 GMT [INFO] Reactor Summary for Comet Project Parent POM 0.9.0-SNAPSHOT: > ``` Yes, this is a transient issue and we have not figured out a root cause. It did start happening just after we changed the DataFusion dependency to be a git dependency on that latest code, soi that is one clue. We do not understand why this only happens on the Spark 4 builds. Here are the timings for the above test from the Spark 3.5 run: ``` 2025-05-24T02:13:42.8902668Z - columnar shuffle on map [bool] (4 seconds, 787 milliseconds) 2025-05-24T02:13:47.7676309Z - columnar shuffle on map [byte] (4 seconds, 877 milliseconds) 2025-05-24T02:13:52.3613358Z - columnar shuffle on map [short] (4 seconds, 593 milliseconds) 2025-05-24T02:13:56.8987095Z - columnar shuffle on map [int] (4 seconds, 537 milliseconds) 2025-05-24T02:14:01.6694624Z - columnar shuffle on map [long] (4 seconds, 770 milliseconds) 2025-05-24T02:14:06.2930589Z - columnar shuffle on map [float] (4 seconds, 623 milliseconds) 2025-05-24T02:14:10.9593270Z - columnar shuffle on map [double] (4 seconds, 666 milliseconds) 2025-05-24T02:14:15.4833575Z - columnar shuffle on map [date] (4 seconds, 524 milliseconds) 2025-05-24T02:14:20.0771823Z - columnar shuffle on map [timestamp] (4 seconds, 594 milliseconds) 2025-05-24T02:14:24.7802633Z - columnar shuffle on map [decimal] (4 seconds, 703 milliseconds) 2025-05-24T02:14:29.4309000Z - columnar shuffle on map [string] (4 seconds, 651 milliseconds) 2025-05-24T02:14:33.6948025Z - columnar shuffle on map [binary] (4 seconds, 264 milliseconds) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
