Hi Don, Here is another REST interface for interacting with Spark from anywhere.
https://github.com/cloudera/livy <https://github.com/cloudera/livy> Here is an example to estimate PI using Spark from Python using requests library. >>> data = { ... 'code': textwrap.dedent("""\ ... val NUM_SAMPLES = 100000; ... val count = sc.parallelize(1 to NUM_SAMPLES).map { i => ... val x = Math.random(); ... val y = Math.random(); ... if (x*x + y*y < 1) 1 else 0 ... }.reduce(_ + _); ... println(\"Pi is roughly \" + 4.0 * count / NUM_SAMPLES) ... """) ... } >>> r = requests.post(statements_url, data=json.dumps(data), headers=headers) >>> pprint.pprint(r.json()) {u'id': 1, u'output': {u'data': {u'text/plain': u'Pi is roughly 3.14004\nNUM_SAMPLES: Int = 100000\ncount: Int = 78501'}, u'execution_count': 1, u'status': u'ok'}, u'state': u'available'} Guru Medasani gdm...@gmail.com > On Mar 2, 2016, at 7:47 AM, Todd Nist <tsind...@gmail.com> wrote: > > Have you looked at Apache Toree, http://toree.apache.org/ > <http://toree.apache.org/>. This was formerly the Spark-Kernel from IBM but > contributed to apache. > > https://github.com/apache/incubator-toree > <https://github.com/apache/incubator-toree> > > You can find a good overview on the spark-kernel here: > http://www.spark.tc/how-to-enable-interactive-applications-against-apache-spark/ > > <http://www.spark.tc/how-to-enable-interactive-applications-against-apache-spark/> > > Not sure if that is of value to you or not. > > HTH. > > -Todd > > On Tue, Mar 1, 2016 at 7:30 PM, Don Drake <dondr...@gmail.com > <mailto:dondr...@gmail.com>> wrote: > I'm interested in building a REST service that utilizes a Spark SQL Context > to return records from a DataFrame (or IndexedRDD?) and even add/update > records. > > This will be a simple REST API, with only a few end-points. I found this > example: > > https://github.com/alexmasselot/spark-play-activator > <https://github.com/alexmasselot/spark-play-activator> > > which looks close to what I am interested in doing. > > Are there any other ideas or options if I want to run this in a YARN cluster? > > Thanks. > > -Don > > -- > Donald Drake > Drake Consulting > http://www.drakeconsulting.com/ <http://www.drakeconsulting.com/> > https://twitter.com/dondrake <http://www.maillaunder.com/> > 800-733-2143 <tel:800-733-2143>
