Broadcast variables need to fit entirely in memory - so that's a
pretty good litmus test for whether or not to broadcast a smaller
dataset or turn it into an RDD.
On Fri, May 2, 2014 at 7:50 AM, Prashant Sharma wrote:
> I had like to be corrected on this but I am just trying to say small enough
>
I had like to be corrected on this but I am just trying to say small enough
of the order of few 100 MBs. Imagine the size gets shipped to all nodes, it
can be a GB but not GBs and then depends on the network too.
Prashant Sharma
On Fri, May 2, 2014 at 6:42 PM, Diana Carroll wrote:
> Anyone hav
