Re: 答复: submit spark task on yarn asynchronously via java?

Or you can use livy for submit spark jobs http://livy.io/ Linyuxin 于2016年12月26日周一 上午10:32写道： > Thanks. > > > > *发件人:* Naveen [mailto:hadoopst...@gmail.com] > *发送时间:* 2016年12月25日 0:33 > *收件人:* Linyuxin > *抄送:* user > *主题:* Re: 答复: submit spark task on yarn asynchronously via java? > > > > Hi,

答复: 答复: submit spark task on yarn asynchronously via java?

Thanks. 发件人: Naveen [mailto:hadoopst...@gmail.com] 发送时间: 2016年12月25日 0:33 收件人: Linyuxin 抄送: user 主题: Re: 答复: submit spark task on yarn asynchronously via java? Hi, Please use SparkLauncher API class and invoke the threads using async calls using Futures. Using SparkLauncher, you can mention cl

Merri Christmas and Happy New Year

Merri Christmas to all spark users. May your new year will be more "structured" with "streaming" success:)

Re: scikit-learn and mllib difference in predictions python

Hi ioanna, I'd like to help look into it. Is there a way to access your training data? 2016-12-20 17:21 GMT-08:00 ioanna : > I have an issue with an SVM model trained for binary classification using > Spark 2.0.0. > I have followed the same logic using scikit-learn and MLlib, using the > exact >

Re: spark linear regression error training dataset is empty

Hi Xiaomeng, Have you tried to confirm the DataFrame contents before fitting? like assembleddata.show() before fitting. Regards, Yuhao 2016-12-21 10:05 GMT-08:00 Xiaomeng Wan : > Hi, > > I am running linear regression on a dataframe and get the following error: > > Exception in thread "main" ja

Getting Hive Metastore error without using Hive

Hi everyone, It's my first time here, I don't know if is the right place for this question, if don't, my apologies for this. I'm trying follow the Spark SQL, DataFrames and Datasets Guide [1], but I got the error *"Unable to locate hive jars to connect to metastore. Please set spark.sql.hive.metast

Re: [Spark SQL] Task failed while writing rows

Hi, I've just read your message. Have you resolved the problem ? If not, what is the contents of /etc/hosts ? On Mon, Dec 19, 2016 at 10:09 PM, Michael Stratton < michael.strat...@komodohealth.com> wrote: > I don't think the issue is an empty partition, but it may not hurt to try > a repartitio

RE: streaming performance

Hi, I may be missing something but let’s say we would have worked with a DStream window. If we have a sliding window of 5 minutes every 1 minute then an RDD would have been generated every minute, then the RDD for the last 5 minutes would have been joined and then I would have converted them to