Depends on the data volume that you are operating on.
Thanks
Best Regards
On Mon, Sep 28, 2015 at 5:12 PM, Saurav Sinha
wrote:
> Hi Akhil,
>
> My job is creating 47 stages in one cycle and it is running every hour.
> Can you please suggest me what is optimum numbers of stages in spark job.
>
>
Hi Akhil,
My job is creating 47 stages in one cycle and it is running every hour. Can
you please suggest me what is optimum numbers of stages in spark job.
How can we reduce numbers of stages in spark job.
Thanks,
Saurav Sinha
On Mon, Sep 28, 2015 at 3:23 PM, Saurav Sinha
wrote:
> Hi Akhil,
>
Hi Akhil,
Can you please explaine to me how increasing number of partition (which is
thing is worker nodes) will help.
As issue is that my master is getting OOM.
Thanks,
Saurav Sinha
On Mon, Sep 28, 2015 at 2:32 PM, Akhil Das
wrote:
> This behavior totally depends on the job that you are doin
This behavior totally depends on the job that you are doing. Usually
increasing the # of partitions will sort out this issue. It would be good
if you can paste the code snippet or explain what type of operations that
you are doing.
Thanks
Best Regards
On Mon, Sep 28, 2015 at 11:37 AM, Saurav Sinh
