I try to enable Tungsten with Spark SQL and set below 3 parameters, but i found the Spark SQL always hang below point. So could you please point me what's the potential cause ? I'd appreciate any input. spark.shuffle.manager=tungsten-sort spark.sql.codegen=true spark.sql.unsafe.enabled=true
15/07/31 15:19:46 INFO scheduler.TaskSetManager: Starting task 110.0 in stage 131.0 (TID 280, bignode3, PROCESS_LOCAL, 1446 bytes) 15/07/31 15:19:46 INFO scheduler.TaskSetManager: Starting task 111.0 in stage 131.0 (TID 281, bignode2, PROCESS_LOCAL, 1446 bytes) 15/07/31 15:19:46 INFO storage.BlockManagerInfo: Added broadcast_132_piece0 in memory on bignode3:38948 (size: 7.4 KB, free: 1766.4 MB) 15/07/31 15:19:46 INFO storage.BlockManagerInfo: Added broadcast_132_piece0 in memory on bignode3:57341 (size: 7.4 KB, free: 1766.4 MB) 15/07/31 15:19:46 INFO storage.BlockManagerInfo: Added broadcast_132_piece0 in memory on bignode1:33229 (size: 7.4 KB, free: 1766.4 MB) 15/07/31 15:19:46 INFO storage.BlockManagerInfo: Added broadcast_132_piece0 in memory on bignode1:42261 (size: 7.4 KB, free: 1766.4 MB) 15/07/31 15:19:46 INFO storage.BlockManagerInfo: Added broadcast_132_piece0 in memory on bignode2:44033 (size: 7.4 KB, free: 1766.4 MB) 15/07/31 15:19:46 INFO storage.BlockManagerInfo: Added broadcast_132_piece0 in memory on bignode2:42863 (size: 7.4 KB, free: 1766.4 MB) 15/07/31 15:19:46 INFO storage.BlockManagerInfo: Added broadcast_132_piece0 in memory on bignode4:58639 (size: 7.4 KB, free: 1766.4 MB) 15/07/31 15:19:46 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to bignode3:46462 15/07/31 15:19:46 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 71847 bytes 15/07/31 15:19:46 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to bignode3:38803 15/07/31 15:19:46 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to bignode1:35241 15/07/31 15:19:46 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to bignode1:48323 15/07/31 15:19:46 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to bignode2:56697 15/07/31 15:19:46 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to bignode4:55810 15/07/31 15:19:46 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to bignode2:37386 -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Came-across-Spark-SQL-hang-issue-with-Spark-1-5-Tungsten-feature-tp13537.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
