回复:Kryo Serialization Issue

2018-08-28 Thread Zhijiang(wangzhijiang999)
://issues.apache.org/jira/browse/FLINK-9913 -- 发件人:Darshan Singh 发送时间:2018年8月28日(星期二) 00:16 收件人:walterddr 抄 送:user 主 题:Re: Kryo Serialization Issue Thanks, We ran into differnet errors and then realized it was OOM issue which was causing

Re: Kryo Serialization Issue

2018-08-27 Thread Darshan Singh
Thanks, We ran into differnet errors and then realized it was OOM issue which was causing different parts to be failed. Flink was buffering too much data as we were reading too fast from source. Reducing the speed fixed the issue. However, I am curious how to achieve the same with S3 apart from l

Re: Kryo Serialization Issue

2018-08-26 Thread Rong Rong
This seems to be irrelevant to the issue for KyroSerializer in recent discussions [1]. which has been fixed in 1.4.3, 1.5.0 and 1.6.0. On a quick glance, this might have been a corrupted message in your decoding, for example a malformed JSON string. -- Rong [1] https://issues.apache.org/jira/brow

Kryo Serialization Issue

2018-08-22 Thread Darshan Singh
Hi, I am using a map function on a data stream which has 1 column i.e. a json string. Map function simply uses Jackson mapper and convert the String to ObjectNode and also assign key based on one of the value in Object node. The code seems to work fine for 2-3 minutes as expected and then suddenl