Re: State size Vs keys number perfromance

Hi In the last email, I just wanted to express that the overall state size(and the access pattern, but I assume that the access pattern is the same between the two states) affects the final performance (which has to do with RocksDB's architecture), and if you use MapState and ValueState to end up

Re: State size Vs keys number perfromance

Thanks Congxian Qiu, I'm aware about your second point. In Value state I will keep String or very simple POJO, without any collections inside. I didn't get your third point, could you clarify it please? "disk read/write is somewhat about the whole state size" Actually what I will keep in Value s

Re: State size Vs keys number perfromance

Hi I'll give some information from my side: 1. The performance for RocksDB is mainly related to the (de)serialization and disk read/write. 2. MapState just need to (de)serialize the single mapkey/mapvalue when read/write state, ValueState need to (de)serialize the whole state when read/write the st

State size Vs keys number perfromance

Hi, I would to ask about what has more memory footprint and what could be more efficient regarding less keys with bigger keyState vs many keys with smaller keyState For this use case I'm using RocksDB StateBackend and state TTL is, well.. infinitive. So I'm keeping the state forever in Flink. Th