Hi Rishi, 1. Yes. Ignite datastreamer uses datastreamer-pool (from 2.0 version) with size equals to number of available processors. 2. In case of datastreamer parallelism is set to 1, streamer will not send next batch until it get ack from previous one. With default settings, streamer can send up to 16 batches per node without waiting for ack. 3. Seems, Timer-0 thread is a bottleneck.
On Mon, Jul 3, 2017 at 9:06 PM, rishi007bansod <rishi007ban...@gmail.com> wrote: > Hi Andrey, > Attached is the code we have used for bench marking. Is there any > tuning > that we can apply to get better performance out of ignite single instance > further? > Also we have attached logs taken from our tool where we varied > datastreamer parallelism from 1 to 16(default). In this case it is observed > that, > (1) ignite by default creates threadpool of size 56 and datastreamer uses > threads among this threadool depending upon parallelism set(is it > correct??, > correct me if I am wrong) > (2) Also when datastreamer parallelism is set to 1, it is observed that > while loop thread(Timer-0) goes into waiting state after some interval(here > we get 30k rate)(Why is this happening? why rate here is limited to 30k > instead of 80K(80k is rate in case of default parallelism)) > (3) Whereas in case of default parallelism(i.e. 16) while loop > thread(Timer-0) is continuously in running state(here we get 80k rate > only). > But in this case public thread pools of data streamers are waiting most of > the time, is this the reason for less throughput? > > <http://apache-ignite-users.70518.x6.nabble.com/file/ > n14276/SingleThread.png> > > > <http://apache-ignite-users.70518.x6.nabble.com/file/ > n14276/DefaultThreads.png> > > code.java > <http://apache-ignite-users.70518.x6.nabble.com/file/n14276/code.java> > > Thanks > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Ignite-Data-Streamer-Performance-is- > not-improving-with-increase-in-threads-tp14151p14276.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > -- Best regards, Andrey V. Mashenkov
