Re: Pyspark DataFrame TypeError

2015-09-08 Thread Prabeesh K.
----+ > > >>> df.columns > ['label', 'featuers'] > ``` > > On Tue, Sep 8, 2015 at 1:45 AM, Prabeesh K. wrote: > > I am trying to run the code RandomForestClassifier example in the PySpark > > 1.4.1 documentation, > > > https:/

Re: jenkins failing on Kinesis shard limits

2015-07-24 Thread Prabeesh K.
For me https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/97/console https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38417/console On 25 July 2015 at 09:57, Patrick Wendell wrote: > I've disabled the test and filed a JIRA: > > https://issues.apache.org/ji

Re: not in gzip format

2015-04-07 Thread prabeesh k
but name just confusing On 7 April 2015 at 16:35, Sean Owen wrote: > Er, click the link? It is indeed a redirector HTML page. This is how all > Apache releases are served. > On Apr 7, 2015 8:32 AM, "prabeesh k" wrote: > >> Please check the apache mirror >> ht

not in gzip format

2015-04-07 Thread prabeesh k
Please check the apache mirror http://www.apache.org/dyn/closer.cgi/spark/spark-1.3.0/spark-1.3.0.tgz file. It is not in the gzip format.

Re: Welcoming three new committers

2015-02-03 Thread prabeesh k
Congratulations! On 4 February 2015 at 02:34, Matei Zaharia wrote: > Hi all, > > The PMC recently voted to add three new committers: Cheng Lian, Joseph > Bradley and Sean Owen. All three have been major contributors to Spark in > the past year: Cheng on Spark SQL, Joseph on MLlib, and Sean on ML

Unable to execute saveAsTextFile on multi node mesos

2014-05-31 Thread prabeesh k
Hi, scenario : Read data from HDFS and apply hive query on it and the result is written back to HDFS. Scheme creation, Querying and saveAsTextFile are working fine with following mode - local mode - mesos cluster with single node - spark cluster with multi node Schema creation and q

java.lang.OutOfMemoryError while running Shark on Mesos

2014-05-22 Thread prabeesh k
Hi, I am trying to apply inner join in shark using 64MB and 27MB files. I am able to run the following queris on Mesos - "SELECT * FROM geoLocation1 " - """ SELECT * FROM geoLocation1 WHERE country = '"US"' """ But while trying inner join as "SELECT * FROM geoLocation1 g1 INNER

Better option to use Querying in Spark

2014-05-05 Thread prabeesh k
Hi, I have seen three different ways to query data from Spark 1. Default SQL support( https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/sql/examples/HiveFromSpark.scala ) 2. Shark 3. Blink DB I would like know which one is more efficient Regard

Link not working

2014-04-21 Thread prabeesh k
For Spark-0.8.0, the download links are not working. Please update the same Regarding, prabeesh

Re: [VOTE] Release Apache Spark 0.9.1 (RC3)

2014-03-30 Thread prabeesh k
+1 tested on Ubuntu12.04 64bit On Mon, Mar 31, 2014 at 3:56 AM, Matei Zaharia wrote: > +1 tested on Mac OS X. > > Matei > > On Mar 27, 2014, at 1:32 AM, Tathagata Das > wrote: > > > Please vote on releasing the following candidate as Apache Spark version > 0.9.1 > > > > A draft of the release n

Re: [VOTE] Release Apache Spark 0.9.1 (RC3)

2014-03-29 Thread prabeesh k
has completed? On Sat, Mar 29, 2014 at 9:28 PM, Tathagata Das wrote: > Small fixes to the docs can be done after the voting has completed. This > should not determine the vote on the release candidate binaries. Please > vote as "+1" if the published artifacts and binaries ar

Re: [VOTE] Release Apache Spark 0.9.1 (RC3)

2014-03-29 Thread prabeesh k
https://github.com/apache/spark/blob/master/docs/quick-start.md in line 127. one spelling mistake found please correct it. (proogram to program) On Fri, Mar 28, 2014 at 9:58 PM, Will Benton wrote: > RC3 works with the applications I'm working on now and MLLib performance > is indeed perceptibl

Re: [DISCUSS] Scala Style for import

2014-03-13 Thread prabeesh k
r can refer to all imports in one place. * Post your thoughts. Regards, prabeesh On Thu, Mar 13, 2014 at 1:49 PM, prabeesh k wrote: > example for unblocked import > > import org.eclipse.paho.client.mqttv3.MqttClient > import org.eclipse.paho.client.mqttv3.Mq

Re: [DISCUSS] Scala Style for import

2014-03-13 Thread prabeesh k
ort ? > > Prashant Sharma > > > On Thu, Mar 13, 2014 at 1:32 PM, prabeesh k wrote: > > > Hi All, > > > > We can import packages in Scala as blocked import and unblocked import. > > > > I think blocked import is better than other. This method helps

[DISCUSS] Scala Style for import

2014-03-13 Thread prabeesh k
Hi All, We can import packages in Scala as blocked import and unblocked import. I think blocked import is better than other. This method helps to reduce LOC. But in Spark code using mixed type, It is better choose any of both. Please post your thoughts on the Scala Style for import Regards,