Shuffle Spill Memory and Shuffle Spill Disk

Hello, I am running TeraSort on 100GB of data. The final metrics I am getting on Shuffle Spill are: Shuffle Spill(Memory): 122.5 GB Shuffle Spill(Disk): 3.4 GB What's the difference and relation between these two metrics? Does these mean 122.5 GB was s

Re: Shuffle Spill Memory and Shuffle Spill Disk

It looks this is not the right place for this question, I have send the question to user group. thank you, bijay On Mon, Mar 23, 2015 at 2:25 PM, Bijay Pathak wrote: > Hello, > > I am running TeraSort <https://github.com/ehiggs/spark-terasort> on > 100GB of data. The