Assume your block size is 128MB.
Thanks
Best Regards
On Mon, May 4, 2015 at 2:38 PM, ayan guha wrote:
> Hi
>
> How do you figure out 500gig~3900 partitions? I am trying to do the math.
> If I assume 64mb block size then 1G~16 blocks and 500g~8000 blocks. If we
> assume split and block sizes are
Hi
How do you figure out 500gig~3900 partitions? I am trying to do the math.
If I assume 64mb block size then 1G~16 blocks and 500g~8000 blocks. If we
assume split and block sizes are same, shouldn't we end up with 8k
partitions?
On 4 May 2015 17:49, "Akhil Das" wrote:
> 500GB of data will have
500GB of data will have nearly 3900 partitions and if you can have nearly
that many number of cores and around 500GB of memory then things will be
lightening fast. :)
Thanks
Best Regards
On Sun, May 3, 2015 at 12:49 PM, sherine ahmed
wrote:
> I need to use spark to upload a 500 GB data from had
