Hi, It should be absolutely fine to use multiple state objects. I am not aware of any limits to that. A minimal, reproducible example would definitely be helpful. For those kind of exceptions, I'd look into the serializers you use. Other than that I cannot think of an obvious reason for that kind of exceptions.
Best, Dawid On 08/10/2020 12:12, Colletta, Edward wrote: > > Using Flink 1.9.2, Java, FsStateBackend. Running Session cluster on > EC2 instances. > > > > I have a KeyedProcessFunction that is causing a segmentation fault, > crashing the flink task manager. The seems to be caused by using 3 > State variables in the operator. The crash happens consistently after > some load is processed. > > This is the second time I have encountered this. The first time I > had 3 ValueState variables, this time I had 2 ValueState variables and > a MapState variable. Both times the error was alleviated by removing > one of the state variables. > > This time I replaced the 2 valueState variables with a Tuple2 of the > types of the individual variables. I can try to put together a > minimal example, but I was wondering if anyone has encountered this > problem. > > > > Are there any documented limits of the number of state variables 1 > operator can use? > > > > For background the reason I use multiple state variables is the > operator is processing 2 types of inputs, Left and Right. When Left > is received it is put it into a PriorityQueue. When the Right type is > received I put that into a ring buffer. > > I replaced the PriorityQueue with a queue of Ids and MapState to hold > the elements. So I have Left stored in a queue ValueState variable > and MapState variable, and Right is stored in the ring buffer > ValueState variable. > > > > >
signature.asc
Description: OpenPGP digital signature
