Re: genericRecord serialisation fail with npe when logicalType is used

2024-09-12 Thread Nicolas Paris
I was able to fix with the bellow modifications. Turns out flink can encode genericRecords to avro when the schema is provided (returns chained). Also the kryo stuff is not anymore needed then. @@ -22,14 +22,12 @@ env.fromSource(source, WatermarkStrategy.noWatermarks(),

genericRecord serialisation fail with npe when logicalType is used

2024-09-12 Thread Nicolas Paris
Hi, This simplified code fails with the bellow stacktrace. In case I remove the timestamp logicalType to a regular long in the avro schema, it works fine. Also, it is independant of the sink (also did NPE with a kafka sink). Is it needed to implement a custom kryo serializer for built-in avro logi

Re: flink kafka sink batch mode delivery guaranties limitations

2024-08-23 Thread Nicolas Paris
Thanks, missed that warning ! All right

flink kafka sink batch mode delivery guaranties limitations

2024-08-23 Thread Nicolas Paris
hi >From my tests kafka sink in exactly-once and batch runtime will never commit the transaction, leading to not honour the semantic. This is likely by design since records are ack/commited during a checkpoint, which never happens in batch mode. I am missing something or the documentation should w