Re: Java : Testing RDD aggregateByKey

Hi Pedro, > Anyway, maybe the behavior is weird, I could expect that repartition to zero was not allowed or at least warned instead of just discarting all the data . Interesting... scala> spark.version res3: String = 3.1.2 scala> spark.range(5).repartition(0) java.lang.IllegalArgumentException:

Re: Is memory-only no-disk Spark possible?

Hi Bobby, What a great summary of what happens behind the scenes! Enjoyed every sentence! "The default shuffle implementation will always write out to disk." <-- that's what I wasn't sure about the most. Thanks again! /me On digging deeper... Pozdrawiam, Jacek Laskowski https://about.me/Ja