Re: Identify the performance bottleneck from hardware prospective

2015-03-05 Thread Julaiti Alafate
nds at the beginning, then drop close > to 0"... Do you know more? > > thanks, > Mitch Gusat (IBM research) > > On Tue, Feb 17, 2015 at 11:20 AM, Julaiti Alafate > wrote: > > > Hi there, > > > > I am trying to scale up the data size that my application

Re: Identify the performance bottleneck from hardware prospective

2015-02-17 Thread Julaiti Alafate
the length of the sentence. It is about 63 billions. Thanks, Julaiti On Tue, Feb 17, 2015 at 2:44 AM, Arush Kharbanda wrote: > Hi > > How big is your dataset? > > Thanks > Arush > > On Tue, Feb 17, 2015 at 4:06 PM, Julaiti Alafate > wrote: > >> Thank you ver

Re: Identify the performance bottleneck from hardware prospective

2015-02-17 Thread Julaiti Alafate
ork will be a bottleneck if data is not available locally on one of > the worker and hence it has to collect it from others, which is a lot of > Serialization and data transfer across your cluster. > > Thanks > Best Regards > > On Tue, Feb 17, 2015 at 11:20 AM, Julaiti Ala

Identify the performance bottleneck from hardware prospective

2015-02-16 Thread Julaiti Alafate
Hi there, I am trying to scale up the data size that my application is handling. This application is running on a cluster with 16 slave nodes. Each slave node has 60GB memory. It is running in standalone mode. The data is coming from HDFS that also in same local network. In order to have an under