I am not sure if the problem persists in 3.x. On Thu, Feb 18, 2021 at 12:14 PM Dongjoon Hyun <dongjoon.h...@gmail.com> wrote:
> Thank you for sharing, Tyson. > > Spark 2.4.4 looks too old to me. Do you think it will occur at 3.x? > > Bests, > Dongjoon. > > > On Thu, Feb 18, 2021 at 11:07 AM Tyson <tcon...@gmail.com> wrote: > >> We observed an interesting stack trace that I'd like to share with you. >> The logging level is WARN, but it appears to be causing task failures. >> Please let me know if anyone has any insights. It appears to be a integer >> overflow issue from looking at the code in Spark 2.4.4 >> >> WARN TaskSetManager [task-result-getter-0]: Lost task 3175.0 in stage >> 518.0 (TID 186951, executor 150): java.lang.NegativeArraySizeException >> >> at >> org.apache.spark.sql.catalyst.expressions.UnsafeRow.getBinary(UnsafeRow.java:438) >> at >> org.apache.spark.sql.catalyst.expressions.UnsafeRow.getDecimal(UnsafeRow.java:414) >> at >> org.apache.spark.sql.catalyst.expressions.JoinedRow.getDecimal(JoinedRow.scala:95) >> at >> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.writeFields_0_4$(Unknown >> Source) >> at >> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown >> Source) >> at >> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown >> Source) >> at >> org.apache.spark.sql.execution.joins.HashJoin$$anonfun$join$1.apply(HashJoin.scala:218) >> at >> org.apache.spark.sql.execution.joins.HashJoin$$anonfun$join$1.apply(HashJoin.scala:216) >> at scala.collection.Iterator$$anon$11.next(Iterator.scala:410) >> at >> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage5.processNext(Unknown >> Source) >> at >> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) >> at >> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$13$$anon$1.hasNext(WholeStageCodegenExec.scala:636) >> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409) >> at >> org.apache.spark.shuffle.sort.UnsafeShuffleWriter.write(UnsafeShuffleWriter.java:187) >> at >> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99) >> at >> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55) >> at org.apache.spark.scheduler.Task.run(Task.scala:123) >> at >> org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) >> at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) >> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> >>
