RE: Kafka as source for batch job

, 2018 12:24 PM To: user@flink.apache.org<mailto:user@flink.apache.org>; Marchant, Hayden [ICG-IT] mailto:hm97...@imceu.eu.ssmb.com>> Subject: Re: Kafka as source for batch job Hi Hayden, Have you tried looking into `KeyedDeserializationSchema#isEndOfStream` [1]? I think that cou

RE: Kafka as source for batch job

? Thanks Hayden From: Tzu-Li (Gordon) Tai [mailto:tzuli...@apache.org] Sent: Thursday, February 08, 2018 8:05 PM To: user@flink.apache.org; Marchant, Hayden [ICG-IT] Subject: RE: Kafka as source for batch job Hi Marchant, Yes I agree. In general, the isEndOfStream method has a very ill-defined

RE: Kafka as source for batch job

@flink.apache.org; Marchant, Hayden [ICG-IT] Subject: Re: Kafka as source for batch job   Hi Hayden,   Have you tried looking into `KeyedDeserializationSchema#isEndOfStream` [1]? I think that could be what you are looking for. It signals the end of the stream when consuming from Kafka.   Cheers, Gordon

RE: Kafka as source for batch job

: Kafka as source for batch job Hi Hayden, Have you tried looking into `KeyedDeserializationSchema#isEndOfStream` [1]? I think that could be what you are looking for. It signals the end of the stream when consuming from Kafka. Cheers, Gordon On 8 February 2018 at 10:44:59 AM, Marchant, Hayden

Re: Kafka as source for batch job

Hi Hayden, Have you tried looking into `KeyedDeserializationSchema#isEndOfStream` [1]? I think that could be what you are looking for. It signals the end of the stream when consuming from Kafka. Cheers, Gordon On 8 February 2018 at 10:44:59 AM, Marchant, Hayden (hayden.march...@citi.com) wrote

Kafka as source for batch job

I know that traditionally Kafka is used as a source for a streaming job. In our particular case, we are looking at extracting records from a Kafka topic from a particular well-defined offset range (per partition) - i.e. from offset X to offset Y. In this case, we'd somehow want the application