To achieve the best data streaming performance, you should aim at highest utilization of resources on data nodes. There is no silver bullet for data streamer tuning, unfortunately. Try changing parameters and see, how they affect the utilization and overall performance.
For me default data streamer parameters usually work fine. Your value of perNodeBufferSize looks too big. It's measured in records, not bytes. By default it's 512. Denis пт, 28 дек. 2018 г. в 14:32, ashishb888 <ashish.bhosle...@gmail.com>: > I am using below settings: > allowOverwrite: false > nodeParallelOperations: 1 > autoFlushFrequency: 10 > perNodeBufferSize: 5000000 > > > My records size is around 2000 bytes. And see the > "grid-data-loader-flusher" > thread stats as below: > > Thread Count Average Longest Duration > grid-data-loader-flusher-#100 38 4,737,793.579 30,427,862 > 180,036,156 > > What would be the best configurations for Data streamer? > > Thanks > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
