Re: ConcurrentModificationException while accessing managed keyed state

2018-06-02 Thread sihua zhou
Hi Garvit, thanks for you feedback, I see you are using the 1.4.1 with Heap state backend, and there are actually two bugs in 1.4.1 related to the kryo serializer and DefaultOperateStateBackend which may cause the ConcurrentModificationException(when checkpointing), they both have been fixed

Re: ConcurrentModificationException while accessing managed keyed state

2018-06-02 Thread aitozi
Hi ,Garvit Sharma Just read your code snippet, I think it look ok. May need your reproduce senior and exception stackstrace then. Best wishes Aitozi Garvit Sharma wrote > Hi, > > Sorry guys for the delay. I was trying to reproduce the complete error on > my local machine but could not get it

Re: ConcurrentModificationException while accessing managed keyed state

2018-06-02 Thread Garvit Sharma
Hi, Sorry guys for the delay. I was trying to reproduce the complete error on my local machine but could not get it though. I will try again with actual traffic and let you the exception Stacktrace. For now, I have the following details available to me. Flink version: 1.4.1 State backend: Heap

Re: ConcurrentModificationException while accessing managed keyed state

2018-06-02 Thread aitozi
Hi Garvit Sharma, Flink run with per parallel with a single thread. Can you show a little code about how you use the keyed state in processFunction Garvit Sharma wrote > Hi, > > I have a use case where I am keeping the keyed state in ProcessFunction. > > Key: Integer personId; > > /** > * The

Re: ConcurrentModificationException while accessing managed keyed state

2018-06-01 Thread sihua zhou
Hi Garvit, this is unexpected, could you please provide more information about this? - which flink version are you using? - what state backend are you using? - are you using the incremental checkpoint?(in case you used the rocksdb backend) - did you create the customer thread to operate the sta

ConcurrentModificationException while accessing managed keyed state

2018-06-01 Thread Garvit Sharma
Hi, I have a use case where I am keeping the keyed state in ProcessFunction. Key: Integer personId; /** * The data type stored in the state */public class PersonDetails { public long count; public long lastModified;} I have encountered a lot of ConcurrentModificationException. I tho