Re: MemoryStore: Not enough space to cache broadcast_N in memory

2016-01-27 Thread Akhil Das
Did you try enabling spark.memory.useLegacyMode and upping spark.storage.memoryFraction? Thanks Best Regards On Fri, Jan 22, 2016 at 3:40 AM, Arun Luthra wrote: > WARN MemoryStore: Not enough space to cache broadcast_4 in memory! > (computed 60.2 MB so far) > WARN MemoryStore: Persisting block

MemoryStore: Not enough space to cache broadcast_N in memory

2016-01-21 Thread Arun Luthra
WARN MemoryStore: Not enough space to cache broadcast_4 in memory! (computed 60.2 MB so far) WARN MemoryStore: Persisting block broadcast_4 to disk instead. Can I increase the memory allocation for broadcast variables? I have a few broadcast variables that I create with sc.broadcast() . Are thes