RDD storage in spark steaming

by default ? Can it store at the memory and disk ? How can it configured ? Thanks, Abhi

Re: Priority queue in spark

yes . Each generated job can have a different priority it is like a recursive function, where in each iteration generate job will be submitted to the spark cluster based on the priority. jobs will lower priority or less than some threshold will be discarded. Thanks, Abhi On Mon, Mar 16, 2015

Re: Priority queue in spark

If i understand correctly , the above document creates pool for priority which is static in nature and has to be defined before submitting the job . .in my scenario each generated task can have different priority. Thanks, Abhi On Mon, Mar 16, 2015 at 9:48 PM, twinkle sachdeva < twinkle.sa

Priority queue in spark

feasible ? Any help is appreciated ? Thanks, Abhi

Re: Issue with yarn cluster - hangs in accepted state.

Thanks, It worked. -Abhi On Tue, Mar 3, 2015 at 5:15 PM, Tobias Pfeiffer wrote: > Hi, > > On Wed, Mar 4, 2015 at 6:20 AM, Zhan Zhang wrote: > >> Do you have enough resource in your cluster? You can check your resource >> manager to see the usage. >> > > Ye

Issue with yarn cluster - hangs in accepted state.

I am trying to run below java class with yarn cluster, but it hangs in accepted state . i don't see any error . Below is the class and command . Any help is appreciated . Thanks, Abhi bin/spark-submit --class com.mycompany.app.SimpleApp --master yarn-cluster /home/hduser/my-app-1.

unsubscribe

-- Abhi Basu

SparkSQL

I am working with CDH5.2 (Spark 1.0.0) and wondering which version of Spark comes with SparkSQL by default. Also, will SparkSQL come enabled to access the Hive Metastore? Is there an easier way to enable Hive support without have to build the code with various switches? Thanks, Abhi -- Abhi

Re: Building Desktop application for ALS-MlLib/ Training ALS

t to all waivers and other terms at the following link: > > http://www.markit.com/en/about/legal/email-disclaimer.page > > > > Please visit http://www.markit.com/en/about/contact/contact-us.page? for > > contact information on our offices worldwide. > > > > MarkitSERV Lim

Re: sbt assembly with hive

, Abhi On Fri, Dec 12, 2014 at 6:40 PM, Stephen Boesch wrote: > > > What is the proper way to build with hive from sbt? The SPARK_HIVE is > deprecated. However after running the following: > >sbt -Pyarn -Phadoop-2.3 -Phive assembly/assembly > > And then >