Hello,
I was reading Spark 2.4.0 release docs and I'd like to find out more
about barrier execution mode.
In particular I'd like to know what happens when number of partitions
exceeds number of nodes (which I think is allowed, Spark tuning doc
mentions that)?
Does Spark guarantee that all tasks process all partitions
simultaneously? If not then how does barrier mode handle partitions that
are waiting to be processed?
If there are partitions waiting to be processed then I don't think it's
possible to send all data from given stage to a DL process, even when
using barrier mode?
Thanks a lot,
Joe
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
