Re: DataFrame vs RDD

As mentioned earlier, since DataFrame is associated with schema... It makes sense to be created from sqlContext.. So ur statement holds true with that understanding.. On Wed, Mar 23, 2016 at 8:28 AM asethia wrote: > creating RDD is done via spark context where as creating Dataframe is from > sqlc

Re: DataFrame vs RDD

Vinay Kashyap On Wed, Mar 23, 2016 at 7:56 AM Jeff Zhang wrote: > Please check the offical doc > > http://spark.apache.org/docs/latest/ > > > On Wed, Mar 23, 2016 at 10:08 AM, asethia wrote: > >> Hi, >> >> I am new to Spark, would like to know any guidelin

Re: Enabling spark_shuffle service without restarting YARN Node Manager

applications should not be disturbed. That was the only concern. Thanks and regards Vinay Kashyap On Wed, Mar 16, 2016 at 4:05 PM, Saisai Shao wrote: > If you want to avoid existing job failure while restarting NM, you could > enable work preserving for NM, in this case, the restart of NM wi

Enabling spark_shuffle service without restarting YARN Node Manager

e the shuffle service. Any help would be greatly appreciated. -- *Thanks and regards* *Vinay Kashyap*

Re: Error in Spark Executors when trying to read HBase table from Spark with Kerberos enabled

Hi Guys, Any help regarding this issue..?? On Wed, Jan 13, 2016 at 6:39 PM, Vinay Kashyap wrote: > Hi all, > > I am using *Spark 1.5.1 in YARN cluster mode in CDH 5.5.* > I am trying to create an RDD by reading HBase table with kerberos enabled. > I am able to launch the sp

Error in Spark Executors when trying to read HBase table from Spark with Kerberos enabled

imilar to what I am experiencing *https://issues.apache.org/jira/browse/SPARK-12279 <https://issues.apache.org/jira/browse/SPARK-12279>* Wanted to know if the issue is the same as I am facing..?? And any workaround for the same so that I can proceed with my requirement reading from HBase table.?? -- *Thanks and regards* *Vinay Kashyap*

Support for ORC Table in Shark/Spark

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) FAILED: Execution Error, return code -101 from shark.execution.SparkTask   Any idea how to overcome this..??       Thanks and regards Vinay Kashyap

Re: Low Performance of Shark over Spark.

much difference was seen.   Thanks and regards Vinay Kashyap   From:"Yana Kadiyska" Sent:"vinay.kashyap" Date:Sat, August 9, 2014 6:56 am Subject:Re: Low Performance of Shark over Spark. Can you see where your t

Re: Low Performance of Shark over Spark.

Vinay Kashyap From:"Xiangrui Meng" Sent:vinay.kash...@socialinfra.net Cc:"user@spark.apache.org" Date:Thu, August 7, 2014 11:06 pm Subject:Re: Low Performance of Shark over Spark. > Did you cache the table? There are c

Low Performance of Shark over Spark.

ery with Shark over Spark and make it comparable to Impala..??   Thanks and regards Vinay Kashyap

Support for Percentile and Variance Aggregation functions in Spark with HiveContext

Percentile with Spark SQL..?? Or .. Are there any limitations still in Spark SQL with respect to UDFs and UDAFs in the version I am using..??     Thanks and regards Vinay Kashyap