parthchandra opened a new issue, #692: URL: https://github.com/apache/datafusion-comet/issues/692
### Describe the bug Comet `BatchReader` uses a private method in Spark `TaskMetrics.externalAccums`. In the latest codebase of Spark a fix for SPARK-48791 has changed the signature of this method which causes Comet to fail with ``` org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 2) (localhost executor driver): java.lang.NoSuchMethodError: 'scala.collection.mutable.Buffer org.apache.spark.executor.TaskMetrics.externalAccums()' ``` ### Steps to reproduce Build Spark from latest code base and tru to run the `HiveParquetSuite` with Comet enabled. ### Expected behavior The test should pass ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
