Re: Best practice for multi-user web controller in front of Spark

Hi, also there is Spindle which was introduced on this list some time ago. I haven't looked into it deeply, but you might gain some valuable insights from their architecture, they are also using Spark to fulfill requests coming from the web. Tobias

RE: Best practice for multi-user web controller in front of Spark

o know more details about the specifics of your application to be more specific, but hopefully you get the idea. Mohammed From: Evan R. Sparks [mailto:evan.spa...@gmail.com] Sent: Tuesday, November 11, 2014 8:54 AM To: Sonal Goyal Cc: bethesda; u...@spark.incubator.apache.org Subject: Re: Be

Re: Best practice for multi-user web controller in front of Spark

For sharing RDDs across multiple jobs - you could also have a look at Tachyon. It provides an HDFS compatible in-memory storage layer that keeps data in memory across multiple jobs/frameworks - http://tachyon-project.org/ . - On Tue, Nov 11, 2014 at 8:11 AM, Sonal Goyal wrote: > I believe the S

Re: Best practice for multi-user web controller in front of Spark

I believe the Spark Job Server by Ooyala can help you share data across multiple jobs, take a look at http://engineering.ooyala.com/blog/open-sourcing-our-spark-job-server. It seems to fit closely to what you need. Best Regards, Sonal Founder, Nube Technologies