how to unsubscribe?
On Sat, Apr 16, 2022 at 3:21 AM Jai Patel <jai.pa...@cloudkitchens.com> wrote: > Hi Nico, > > Wanted to close the loop here. We did end up find a number of problems in > our code: > 1. Our operator was slow. It was iterating over several large Protobufs > in a MapState then filtering it down to 1. We were able to identify that > one up-front and significantly improve the runtime of the operator. > 2. We did increase the total memory and raised the managed memory fraction > from 40% to 70%. > > Both solutions were needed in order to bring us the stability we were > looking for. Since then, everything has been running great. > > Thanks. > Jai > > On Thu, Feb 24, 2022 at 2:42 AM Nicolaus Weidner < > nicolaus.weid...@ververica.com> wrote: > >> Hi Jai, >> >> Do writes to ValueStates/MapStates have a direct on churn of the Flink >>> State or is the data buffered in between? >>> >> >> Writes to keyed state go directly to RocksDB. So there shouldn't be any >> memory issues with buffers overflowing or similar. In general, more memory >> should increase performance (larger cache sizes before having to write to >> disk), but less memory shouldn't cause crashes. >> >> Since the errors you encountered are not that specific, can you provide >> full logs surrounding such incidents? There is not much to go on without >> further info. >> >> Best, >> Nico >> >>>
