Hi Adrian,
Thank you for the additional information!
One reason to have a single folder is that Streams also stores metadata
that refers to all state stores in the state directory. That could be
changed if we have a good reason.
If you have a good idea to solve this issue, please feel free t
Hi Bruno,
Thanks a lot for your answer.
I have tried to tune store by store to the best of my ability, and indeed I
have managed to improve considerably. We even changed the disk to a much
faster one. But it's still not enough.
Yes we can try dividing the application up into sub applications to
Hi Adrian,
I am afraid that you cannot set the state directory for a single state
store to a different directory than all other stores.
Maybe the following blog post can help you debug and solve your issue:
https://www.confluent.io/blog/how-to-tune-rocksdb-kafka-streams-state-stores-performan
Hi there,
My kafka streams topology has one store that is particularly busy, that
alongside other stores in the same topology is exhausting I/O which leads
to write stalls and increased latency.
The amount of compaction that this store does with regards to others is
about 3/4 times more, so we we
