Re: Performance and Latency Chart for Flink

2016-09-19 Thread amir bahmanyari
ptions.  So I am going incremental with that value. Have it set at 8192 (twice as much as before 4096).Thanks From: Greg Hogan To: dev@flink.apache.org; amir bahmanyari Sent: Monday, September 19, 2016 1:28 PM Subject: Re: Performance and Latency Chart for Flink My thought would

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
> > From: Greg Hogan > To: dev@flink.apache.org; amir bahmanyari > Sent: Monday, September 19, 2016 12:43 PM > Subject: Re: Performance and Latency Chart for Flink > > You will need to add the configuration parameters to your flink-conf.yaml. > I believe the intent is tha

Re: Performance and Latency Chart for Flink

2016-09-19 Thread amir bahmanyari
To: dev@flink.apache.org; amir bahmanyari Sent: Monday, September 19, 2016 12:43 PM Subject: Re: Performance and Latency Chart for Flink You will need to add the configuration parameters to your flink-conf.yaml. I believe the intent is that all configuration parameters should be listed

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
ink 1.2-SNAPSHOT Documentation: Configuration >>> | | >>> >>>| >>> >>> | >>> >>> >>> >>> 4096 = (16x16)x4x4 where 16 is number of tasks per TM, 4 is # of TMs & 4 >>> is there in the formula.What would

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
i Missing something here?I still need your > help to set the accurate number for my >- taskmanager.network.numberOfBuffers = 4096. > > Thanks for your response Greg.Amir- From: amir bahmanyari < > amirto...@yahoo.com> > To: "dev@flink.apache.org" > Sen

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Chesnay Schepler
o true & will give it another shot.Thanks Greg From: Greg Hogan To: dev@flink.apache.org; amir bahmanyari Sent: Monday, September 19, 2016 8:29 AM Subject: Re: Performance and Latency Chart for Flink Hi Amir, You may see improved performance setting "taskmanager.memory.p

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
ager.memory.preallocate" to true & will give it > another shot.Thanks Greg > > From: Greg Hogan > To: dev@flink.apache.org; amir bahmanyari > Sent: Monday, September 19, 2016 8:29 AM > Subject: Re: Performance and Latency Chart for Flink > > Hi Amir, > &

Re: Performance and Latency Chart for Flink

2016-09-19 Thread amir bahmanyari
flink.apache.org" Sent: Monday, September 19, 2016 10:34 AM Subject: Re: Performance and Latency Chart for Flink Hi Greg,I used this guideline to calculate  "taskmanager.network.numberOfBuffers":Apache Flink 1.2-SNAPSHOT Documentation: Configuration | | | | ||

Re: Performance and Latency Chart for Flink

2016-09-19 Thread amir bahmanyari
r bahmanyari Sent: Monday, September 19, 2016 8:29 AM Subject: Re: Performance and Latency Chart for Flink Hi Amir, You may see improved performance setting "taskmanager.memory.preallocate: true" in order to use off-heap memory. Also, your number of buffers looks

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
t anyways.If you have > suggestions to make improvements in the above case, I am all ears & greatly > appreciate it.Cheers,Amir- > > From: "Chawla,Sumit" > To: dev@flink.apache.org; amir bahmanyari > Sent: Sunday, September 18, 2016 2:07 PM > Subject: Re: P

Re: Performance and Latency Chart for Flink

2016-09-18 Thread amir bahmanyari
s://www.datatorrent.com/blog/blog-implementing-linear-ro >> ad-benchmark-in-apex/ >> >> >>      From: Fabian Hueske >>  To: "dev@flink.apache.org" >>  Sent: Friday, September 16, 2016 12:31 AM >>  Subject: Re: Performance and Latency Chart for Flink

Re: Performance and Latency Chart for Flink

2016-09-18 Thread Chawla,Sumit
nting-linear-ro >> ad-benchmark-in-apex/ >> >> >> From: Fabian Hueske >> To: "dev@flink.apache.org" >> Sent: Friday, September 16, 2016 12:31 AM >> Subject: Re: Performance and Latency Chart for Flink >> >> Hi, >> >> I am not

Re: Performance and Latency Chart for Flink

2016-09-18 Thread Chawla,Sumit
linear- > road-benchmark-in-apex/ > > > From: Fabian Hueske > To: "dev@flink.apache.org" > Sent: Friday, September 16, 2016 12:31 AM > Subject: Re: Performance and Latency Chart for Flink > > Hi, > > I am not aware of periodic performance runs for t

Re: Performance and Latency Chart for Flink

2016-09-16 Thread amir bahmanyari
://www.datatorrent.com/blog/blog-implementing-linear-road-benchmark-in-apex/ From: Fabian Hueske To: "dev@flink.apache.org" Sent: Friday, September 16, 2016 12:31 AM Subject: Re: Performance and Latency Chart for Flink Hi, I am not aware of periodic performance runs for the Flink r

Re: Performance and Latency Chart for Flink

2016-09-16 Thread Timo Walther
September 16, 2016 12:31 AM Subject: Re: Performance and Latency Chart for Flink Hi, I am not aware of periodic performance runs for the Flink releases. I know a few benchmarks which have been published at different points in time like [1], [2], and [3] (you'll probably find more).

Re: Performance and Latency Chart for Flink

2016-09-16 Thread amir bahmanyari
- From: Fabian Hueske To: "dev@flink.apache.org" Sent: Friday, September 16, 2016 12:31 AM Subject: Re: Performance and Latency Chart for Flink Hi, I am not aware of periodic performance runs for the Flink releases. I know a few benchmarks which have been published at d

Re: Performance and Latency Chart for Flink

2016-09-16 Thread Fabian Hueske
Hi, I am not aware of periodic performance runs for the Flink releases. I know a few benchmarks which have been published at different points in time like [1], [2], and [3] (you'll probably find more). In general, fair benchmarks that compare different systems (if there is such thing) are very di