Re: Livy with Spark package

Thanks and agreed :) On Thu, Aug 24, 2017 at 12:01 PM, Saisai Shao wrote: > You could set "spark.jars.packages" in `conf` field of session post API ( > https://github.com/apache/incubator-livy/blob/master/ > docs/rest-api.md#post-sessions). This is equal to --package in > spark-submit. > > BTW y

Re: Livy with Spark package

You could set "spark.jars.packages" in `conf` field of session post API ( https://github.com/apache/incubator-livy/blob/master/docs/rest-api.md#post-sessions). This is equal to --package in spark-submit. BTW you'd better ask livy question in u...@livy.incubator.apache.org. Thanks Jerry On Thu, A

Livy with Spark package

Hi I have a python program which I invoke as spark-submit --packages com.databricks:spark-avro_2.11:3.2.0 somefile.py "2017-08-23 02:00:00" and it works Now I want to submit this file using Livy. I could work out most of the stuff (like putting files to HDFS etc) but not able to understan

Re: Livy with Spark

re is a great write up on Livy at >> http://henning.kropponline.de/2016/11/06/ >> >> On 5 Dec 2016, at 14:34, Mich Talebzadeh >> wrote: >> >> Hi, >> >> Has there been any experience using Livy with Spark to share multiple >>

Re: Livy with Spark

kropponline.de/2016/11/06/ > > On 5 Dec 2016, at 14:34, Mich Talebzadeh > wrote: > > Hi, > > Has there been any experience using Livy with Spark to share multiple > Spark contexts? > > thanks > > > Dr Mich Talebzadeh > > > > LinkedIn * > http

Re: Livy with Spark

You mean share a single spark context across multiple jobs? https://github.com/spark-jobserver/spark-jobserver does the same On Mon, Dec 5, 2016 at 9:33 AM, Mich Talebzadeh wrote: > Hi, > > Has there been any experience using Livy with Spark to share multiple > Spark contexts

Re: Livy with Spark

bzadeh > wrote: > > Hi, > > Has there been any experience using Livy with Spark to share multiple > Spark contexts? > > thanks > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEWh2gB

Re: Livy with Spark

There is a great write up on Livy at http://henning.kropponline.de/2016/11/06/ On 5 Dec 2016, at 14:34, Mich Talebzadeh mailto:mich.talebza...@gmail.com>> wrote: Hi, Has there been any experience using Livy with Spark to share multiple Spark contexts? thanks Dr Mich Talebzadeh Li

Livy with Spark

Hi, Has there been any experience using Livy with Spark to share multiple Spark contexts? thanks Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV