f partitions should approximately equal to total_coreNumber.
>>
>> Liquan
>>
>> On Mon, Sep 29, 2014 at 2:01 PM, anny9699 wrote:
>>
>>> Hi,
>>>
>>> I read the past posts about partition number, but am still a little
>>> confused
>
-- Forwarded message --
From: Liquan Pei
Date: Mon, Sep 29, 2014 at 2:12 PM
Subject: Re: about partition number
To: anny9699
The number of cores available in your cluster determines the number of
tasks that can be run concurrently. If your data is evenly partitioned,
the
hould be a good starting place. Then
you can try different values and see how it affects your performance.
On Mon, Sep 29, 2014 at 5:01 PM, anny9699 wrote:
> Hi,
>
> I read the past posts about partition number, but am still a little
> confused
> about partitioning strategy.
>
>
Hi,
I read the past posts about partition number, but am still a little confused
about partitioning strategy.
I have a cluster with 8 works and 2 cores for each work. Is it true that the
optimal partition number should be 2-4 * total_coreNumber or should
approximately equal to total_coreNumber
