Re: RewriteDataFiles & DataStream streaming read error when row level delete by equal

2021-02-23 Thread 1
Hi Ryan, I received the mail about 0.11.1,so can you help to check https://github.com/apache/iceberg/issues/2219 for it? Thx. Kryo is the default serialization in Flink, user can code only logic without manually specify the serializer, but Kryo can not de/serialization the UnmodifiableColle

Re: RewriteDataFiles & DataStream streaming read error when row level delete by equal

2021-02-19 Thread Jun Zhang
hi,ryan: Flink will automatically infer the data type. If it cannot be inferred, it will use kryo to do de/serialized. https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html#general-class-types I tested this problem and it does exist. I guess that a certain pojo

Re: RewriteDataFiles & DataStream streaming read error when row level delete by equal

2021-02-09 Thread Ryan Blue
Hi Liu, Looks like you're trying to use Kryo. Can you try using Java serialization? I wasn't aware that Kryo was an option in Flink, but we usually assume that serialization is happening with Java serialization because Kryo requires extra compatibility that is difficult to inject. On Sun, Feb 7,

RewriteDataFiles & DataStream streaming read error when row level delete by equal

2021-02-07 Thread 1
Hi, All: I'm very happy for 0.11's release, that’s great! I test the new features Immediately, like flink cdc, flink rewriteDataFiles, flink streaming read But when I write a row level delete, there is something badly. I test flink cdc(just like https://github.com/apache/iceberg/blob/master/