Re: Re: Re: Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12)

2021-02-04 Thread Abhishek Rai
e/flink/connector/kafka/source/KafkaSourceITCase.java#L69 >> [3] >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface >> >> >> >> --Original Mail -- >> Sender:Ardhani Narasimha >> S

Re: Re: Re: Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12)

2021-01-14 Thread sagar
vient. > > Best, > Yun > > -- > Sender:Yun Gao > Date:2021/01/14 15:26:54 > Recipient:Ardhani Narasimha; sagar< > sagarban...@gmail.com> > Cc:Flink User Mail List > Theme:Re: Re: Using Kafka as bounded

Re: Re: Re: Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12)

2021-01-14 Thread Yun Gao
-- Sender:Yun Gao Date:2021/01/14 15:26:54 Recipient:Ardhani Narasimha; sagar Cc:Flink User Mail List Theme:Re: Re: Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12) Hi Sagar, I think the problem is that the legacy source implemented by extending SourceFunction are

Re: Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12)

2021-01-13 Thread sagar
Hi Ardhani, So whenever I want to run this flink job, I will call the Java API to put the data to the four different kafka topics, what data to put into kafka will be coded into those API and then once that is complete, I want to run the flink job on the available data in the kafka and perform bus

Re: Re: Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12)

2021-01-13 Thread Yun Gao
] https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface --Original Mail -- Sender:Ardhani Narasimha Send Date:Thu Jan 14 15:11:35 2021 Recipients:sagar CC:Flink User Mail List Subject:Re: Using Kafka as bounded source with DataStream

Re: Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12)

2021-01-13 Thread Ardhani Narasimha
Interesting use case. Can you please elaborate more on this. On what criteria do you want to batch? Time? Count? Or Size? On Thu, 14 Jan 2021 at 12:15 PM, sagar wrote: > Hi Team, > > I am getting the following error while running DataStream API in with > batch mode with kafka source. > I am usi

Using Kafka as bounded source with DataStream API in batch mode (Flink 1.12)

2021-01-13 Thread sagar
Hi Team, I am getting the following error while running DataStream API in with batch mode with kafka source. I am using FlinkKafkaConsumer to consume the data. Caused by: java.lang.IllegalStateException: Detected an UNBOUNDED source with the 'execution.runtime-mode' set to 'BATCH'. This combinati