Sorry didn't explain well. Livy seems to me similar to job server by interacting through restful api interface for job submission. The task I want to achieve is more like general restful api interface where user would provide some parameters for business operations (a bit like readText or a custom date source with embedded http). So when the spark receive those requests in e.g custom date source, the spark job can then perform normal operations such as aggregation, window functions, and so on.
Thank you for the suggestions and help. Jan 20, 2020, 00:26 by chris.t...@gmail.com: > Maybe something like Livy, otherwise roll your own REST API and have it start > a Spark job. > > On Mon, 20 Jan 2020 at 06:55, <> hamishberri...@tutanota.com> > wrote: > >> I am new to Spark. The task I want to accomplish is let client send http >> requests, then spark process that request for further operations. However >> searching Spark's website docs >> >> >> https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.package >> >> >> https://spark.apache.org/docs/latest/ >> >> I do not find any places mentioning about this. Also most of the internet >> result are more lated to spark job server. >> >> Any places I should start if I want to use Spark for such purpose? >> >> Thanks >> > > > -- > Chris >
