الليثي
Date:08/02/2016 02:07 (GMT+05:30)
To: Diwakar Dhanuskodi Cc:
"Yuval.Itzchakov" , user
Subject: Re: Apache Spark data locality when integrating with Kafka
Diwakar
We have our own servers. We will not use any cloud service like Amazon's
On 7 February 2016 at 18:24, Diw
iwakar .
>
>
>
> Sent from Samsung Mobile.
>
>
> Original message
> From: "Yuval.Itzchakov"
> Date:07/02/2016 19:38 (GMT+05:30)
> To: user@spark.apache.org
> Cc:
> Subject: Re: Apache Spark data locality when integrating with Kafka
Fanoos,
Where you want the solution to be deployed ?. On premise or cloud?
Regards
Diwakar .
Sent from Samsung Mobile.
Original message From: "Yuval.Itzchakov"
Date:07/02/2016 19:38 (GMT+05:30)
To: user@spark.apache.org Cc: Subject: Re:
Apache Spark dat
I would definitely try to avoid hosting Kafka and Spark on the same servers.
Kafka and Spark will be doing alot of IO between them, so you'll want to
maximize on those resources and not share them on the same server. You'll
want each Kafka broker to be on a dedicated server, as well as your spark
spark can benefit from data locality and will try to launch tasks on the
node where the kafka partition resides.
however i think in production many organizations run a dedicated kafka
cluster.
On Sat, Feb 6, 2016 at 11:27 PM, Diwakar Dhanuskodi <
diwakar.dhanusk...@gmail.com> wrote:
> Yes . To
Yes . To reduce network latency .
Sent from Samsung Mobile.
Original message From: fanooos
Date:07/02/2016 09:24 (GMT+05:30)
To: user@spark.apache.org Cc: Subject: Apache
Spark data locality when integrating with Kafka
Dears
If I will use Kafka as a streaming source
