would it be possible to
>>>> create so many operator states? Did you configure some parameters wrongly?
>>>>
>>>>
>>>> [1]
>>>> https://github.com/apache/beam/blob/4fc924a8193bb9495c6b7ba755ced576bb8a35d5/runners/flink/src/main/java/org/ap
b9495c6b7ba755ced576bb8a35d5/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L95
>>>
>>> Best
>>> Yun Tang
>>> --
>>> *From:* Stephen Patel
>&g
5/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L95
>>
>> Best
>> Yun Tang
>> --------------
>> *From:* Stephen Patel
>> *Sent:* Thursday, April 16, 2020 22:30
>&g
: Re: Streaming Job eventually begins failing during checkpointing
Correction. I've actually found a place where it potentially might be creating
a new operator state per checkpoint:
https://github.com/apache/beam/blob/4fc924a8193bb9495c6b7ba755ced576bb8a35d5/runners/flink/src/main/jav
> *To:* Yun Tang
> *Cc:* user@flink.apache.org
> *Subject:* Re: Streaming Job eventually begins failing during
> checkpointing
>
> Correction. I've actually found a place where it potentially might be
> creating a new operator state per checkpoint:
>
> https://git
Correction. I've actually found a place where it potentially might be
creating a new operator state per checkpoint:
https://github.com/apache/beam/blob/4fc924a8193bb9495c6b7ba755ced576bb8a35d5/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/Buff
I can't say that I ever call that directly. The beam library that I'm
using does call it in a couple places:
https://github.com/apache/beam/blob/v2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedSourceWrapper.java#L422-L429
But it seems t
Hi Stephen
This is not related with RocksDB but with default on-heap operator state
backend. From your exception stack trace, you have created too many operator
states (more than 32767).
How do you call context.getOperatorStateStore().getListState or
context.getOperatorStateStore().getBroadcas
