My guess is that your executor already crashed, due to OOM?. You should check
the executor log, it may tell you more information.
Yong
From: Rohit Verma
Sent: Thursday, March 9, 2017 4:41 AM
To: user
Subject: Spark failing while persisting sorted columns.
Hi
Hi all,
Please help me with below scenario.
While writing below query on large dataset (rowCount=100,000,000) using below
query
// there are other instance of below job submitting to spark in multithreaded
app.
final Dataset df = spark.read().parquet(tablePath);
// df storage is hdfs is 5.64
