Depending on the datatypes you are using, seeing 3x more CPU usage seems realistic. Serialization can be quite expensive. See also: https://flink.apache.org/news/2020/04/15/flink-serialization-tuning-vol-1.html Maybe it makes sense to optimize there a bit.
On Tue, Jun 15, 2021 at 5:23 PM JING ZHANG <beyond1...@gmail.com> wrote: > Hi Padarn, > After switch stateBackend from filesystem to rocksdb, all reads/writes > from/to backend have to go through de-/serialization to retrieve/store the > state objects, this may cause more cpu cost. > But I'm not sure it is the main reason leads to 3x CPU cost in your job. > To find out the reason, we need more profile on CPU cost, such as Flame > Graphs. BTW, starting with Flink 1.13, Flame Graphs are natively supported > in Flink[1]. > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/debugging/flame_graphs/ > > Best, > JING ZHANG > > Padarn Wilson <pad...@gmail.com> 于2021年6月15日周二 下午5:05写道： > >> Hi all, >> >> We have a job that we just enabled rocksdb on (instead of file backend), >> and see that the CPU usage is almost 3x greater on (we had to increase >> taskmanagers 3x to get it to run. >> >> I don't really understand this, is there something we can look at to >> understand why CPU use is so high? Our state mostly consists of aggregation >> windows. >> >> Cheers, >> Padarn >> >
