Thanks Mayur for your clarification.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Streaming-JSON-string-from-REST-Api-in-Spring-tp2358p2451.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Easiest is to use a queue, Kafka for example. So push your json request
string into kafka,
connect spark streaming to kafka & pull data from it & execute it.
Spark streaming will split up the jobs & pipeline the data.
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rusta
Thanks Mayur for your response.
I think I need to clarify the first part of my query. The JSON based REST
API will be called by external interfaces. These requests needs to be
processed in a streaming mode in Spark. I am not clear about the following
points
1. How can JSON request string (50 per
You can create an RDD with json credentials & then run a mapper which takes
these credentials & queries the api & stores results in another RDD.
You can pass that RDD from task to task for further computation steps.
Thr are two issues here:
1. how is number of calls /sec throttled, if you want spar
