I have spark cluster on two data centers each. Cluster on spark cluster B is 6 times slower than cluster A. I ran the same job on both cluster and time difference is of 6 times. I used the same config and using spark 2.3.3. I checked that on spark UI it displays the slaves nodes but when i check under Executor tab i saw all the nodes there but do not see active tasks while task status is active. Please help me to find the root cause.
Thanks Amit