Is it possible to jstack the executors and see where they are hanging?
On Thu, Mar 26, 2015 at 2:02 PM, Jon Chase wrote:
> Spark 1.3.0 on YARN (Amazon EMR), cluster of 10 m3.2xlarge (8cpu, 30GB),
> executor memory 20GB, driver memory 10GB
>
> I'm using Spark SQL, mainly via spark-shell, to query
Spark 1.3.0 on YARN (Amazon EMR), cluster of 10 m3.2xlarge (8cpu, 30GB),
executor memory 20GB, driver memory 10GB
I'm using Spark SQL, mainly via spark-shell, to query 15GB of data spread
out over roughly 2,000 Parquet files and my queries frequently hang. Simple
queries like "select count(*) from
