IgniteContext does not create any executors, Spark does. And frankly I don't know why you have so many, I was not able to reproduce it myself.
As for large number of tasks, only assumption I can make is that your cache doesn't actually have any data, which forces Spark to scan ALL partitions to find 5 elements, and default number of partitions is 1024. However, in this case issue should've gone away when you decreased number of partitions. May be you can provide a self contained project that can be ran to reproduce the issue? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-behaving-strange-with-Spark-SharedRDD-in-AWS-EMR-Yarn-Client-Mode-tp16011p16065.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
