Sorry for not providing the context earlier. I'm running my job on a EC2 instance with 32 cores and according to the documentation I tried to use as many task slots the number of cores, numOfTaskSlots=32 and parallelism=32. But I noticed that the performance is slightly degrading when I'm using 32 task slots. Performance seems better at 26 task slots than >26 task slots. So I was trying to understand if additional CPU cores are being utilized by checkpointing or any other async (or background operations, in the process I was trying to verify if the checkpointing is async.
Thanks, RKandoji On Tue, Jan 7, 2020 at 10:16 PM Congxian Qiu <qcx978132...@gmail.com> wrote: > Hi > > RocksDB supports Incremental and full snapshot, both are async, do you > want to verify whether it's incremental or full snapshot? but I don't know > an easy to get this information currently > > Best, > Congxian > > > Zhijiang <wangzhijiang...@aliyun.com> 于2020年1月8日周三 上午10:56写道： > >> The log way is simple for tracing and you can also grep some keywords to >> find your requirement messages to avoid skimming through the whole large >> logs. >> I am not quite sure what's your specific motivation for doing this. >> Besides the log way, you can also monitor the thread stack for confirming >> whether it is happening, but maybe it is not very convenient. >> Another possible way is via the checkpoint metrics which would record the >> sync/async duration time, maybe it can also satisfy your requirements. >> >> Best, >> Zhijiang >> >> ------------------------------------------------------------------ >> From:RKandoji <rkand...@gmail.com> >> Send Time:2020 Jan. 8 (Wed.) 10:23 >> To:William C <will...@uubeta.com> >> Cc:user <user@flink.apache.org> >> Subject:Re: How to verify if checkpoints are asynchronous or sync >> >> Thanks for the reply. >> I will check and enable debug logs specifically for the class that >> contains this log. >> But in general logs are already too huge and I'm trying to suppress some >> of them, so wondering if there is any other way? >> >> Thanks, >> RKandoji >> >> >> On Tue, Jan 7, 2020 at 7:50 PM William C <will...@uubeta.com> wrote: >> Can you enable debug log to check with that? >> >> regards. >> >> on 2020/1/8 6:36, RKandoji wrote: >> > But I'm curious if there is way to verify if the checkpoints are >> > happening asynchronously or synchronously. >> > >> >> >>
