I evaluated performance by looking at the number of input records processed over 10min and 30mins duration.
Thanks, R On Wed, Jan 8, 2020 at 2:21 AM Congxian Qiu <qcx978132...@gmail.com> wrote: > If you want to figure out the performance problem, maybe async-profile[1] > can be helpful > [1] https://github.com/jvm-profiling-tools/async-profiler > Best, > Congxian > > > William C <will...@uubeta.com> 于2020年1月8日周三 上午11:37写道： > >> Hallo >> >> on 2020/1/8 11:31, RKandoji wrote: >> > I'm running my job on a EC2 instance with 32 cores and according to the >> > documentation I tried to use as many task slots the number of cores, >> > numOfTaskSlots=32 and parallelism=32. But I noticed that >> the performance >> > is slightly degrading when I'm using 32 task slots. Performance seems >> > better at 26 task slots than >26 task slots. So I was trying to >> > understand if additional CPU cores are being utilized by checkpointing >> > or any other async (or background operations, in the process I was >> > trying to verify if the checkpointing is async. >> >> How did you evaluate the performance? >> It's may due to busy IO or thread competition or something else similiar. >> You'd better dig into more details via logging. >> >> Regards. >> >
