Re: Kafka to spark streaming

2022-01-30 Thread Gourav Sengupta
Hi Amit, before answering your question, I am just trying to understand it. I am not exactly clear how do the Akka application, Kafka and SPARK Streaming application sit together, and what are you exactly trying to achieve? Can you please elaborate? Regards, Gourav On Fri, Jan 28, 2022 at 10:

Re: Kafka to spark streaming

2022-01-29 Thread Amit Sharma
Thanks Mich. The link you shared have two options Kafka and Socket only. Thanks Amit On Sat, Jan 29, 2022 at 3:49 AM Mich Talebzadeh wrote: > So you have a classic architecture with spark receiving events through a > kafka topic via kafka-spark-connector, do something with it and send data > o

Re: Kafka to spark streaming

2022-01-29 Thread Mich Talebzadeh
So you have a classic architecture with spark receiving events through a kafka topic via kafka-spark-connector, do something with it and send data out to the consumer. Are you using Spark structured streaming here with batch streaming? check https://spark.apache.org/docs/latest/structured-streamin