Re: FLINK DATASTREAM Processing Question

2020-09-08 Thread Timo Walther
Hi Vijay, one comment to add is that the performance might suffer with multiple map() calls. For safety reason, records between chained operators are serialized and deserialized in order to strictly don't influence each other. If all functions of a pipeline are guaranteed to not modify incomi

Re: FLINK DATASTREAM Processing Question

2020-09-07 Thread Vijayendra Yadav
Thank You Dawid. Sent from my iPhone > On Sep 7, 2020, at 9:03 AM, Dawid Wysakowicz wrote: >

Re: FLINK DATASTREAM Processing Question

2020-09-07 Thread Dawid Wysakowicz
Hi, You can see the execution plan via StreamExecutionEnvironment#getExecutionPlan(). You can visualize it in[1]. You can also submit your job and check the execution plan in Web UI. As for the question which option is preferred it is very subjective. As long as in the option b) both maps are cha

FLINK DATASTREAM Processing Question

2020-09-04 Thread Vijayendra Yadav
Hi Team, I have a generic Question. Let's say I have 2 Actions to be taken on Flink DATASTREAM (Kafka). 1) Convert some data fields, and write to external Database 2) Transform #1 converted data fields in to different record format say AVRO *Here are Two approaches that are possible.* a) One Ma