Hope this helps,-adrian
From: "tarek.abouzei...@yahoo.com.INVALID"
Reply-To: "tarek.abouzei...@yahoo.com"
Date: Sunday, October 18, 2015 at 10:28 AM
To: Xiao Li, Akhil Das
Cc: "user@spark.apache.org"
Subject: Re: Spark handling parallel requests
hi Akhlis
its a must
m.INVALID>"
Reply-To: "tarek.abouzei...@yahoo.com<mailto:tarek.abouzei...@yahoo.com>"
Date: Sunday, October 18, 2015 at 10:28 AM
To: Xiao Li, Akhil Das
Cc: "user@spark.apache.org<mailto:user@spark.apache.org>"
Subject: Re: Spark handling parallel requests
hi Akhlis
it
Yes, there are kafka consumers/producers for almost all the languages, you
can read more over here
https://cwiki.apache.org/confluence/display/KAFKA/Clients#Clients-PHP
Here's a repo for the php version https://github.com/EVODelavega/phpkafka
Thanks
Best Regards
On Sun, Oct 18, 2015 at 12:58 PM,
hi Akhlis
its a must to push data to a socket as i am using php as a web service to push
data to socket , then spark catch the data on that socket and process it , is
there a way to push data from php to kafka directly ? -- Best Regards, --
Tarek Abouzeid
On Sunday, October 18, 2015 10
hi Xiao,1- requests are not similar at all , but they use solr and do commit
sometimes 2- no caching is required3- the throughput must be very high yeah ,
the requests are tiny but the system may receive 100 request/sec , does kafka
support listening to a socket ? -- Best Regards, -- Tarek Abou
Hi, Tarek,
It is hard to answer your question. Are these requests similar? Caching
your results or intermediate results in your applications? Or does that
mean your throughput requirement is very high? Throttling the number of
concurrent requests? ...
As Akhil said, Kafka might help in your case.
Instead of pushing your requests to the socket, why don't you push them to
a Kafka or any other message queue and use spark streaming to process them?
Thanks
Best Regards
On Mon, Oct 5, 2015 at 6:46 PM, wrote:
> Hi ,
> i am using Scala , doing a socket program to catch multiple requests at
> sa
