Re: In yarn-cluster model, mapState not support transfer value

2019-09-19 Thread shengjk1
Hi, Dian Thanks for your reminder. I saw HeapMaoState.java and TtlMapState.java, You are right. But with objectList get large, every time call "mapState.put(key,objectList);” is very influencing performance, even lead to checkpoint timeout. Now I am not have a better method to improve perf

Re: In yarn-cluster model, mapState not support transfer value

2019-09-19 Thread Dian Fu
Hi Shengjk1, You should call "mapState.put(key,objectList);" manually after calling "objectList.add(stringObjectTuple2.f1);" to write it to the state backend. This is because objectList is just a common Java list object and it will not be synced to state backend automatic when updated. I gues