2016 7:54:27 PM
>> *To:* user @spark
>> *Subject:* Spark Dataframe: Save to hdfs is taking long time
>>
>> Hi,
>>
>> I am using issue while saving the dataframe back to HDFS. It's taking
>> long time to run.
>>
>> val results_dataframe = sqlCont
I am trying to save the files as Paraquet.
On Thu, Dec 15, 2016 at 10:41 PM, Felix Cheung
wrote:
> What is the format?
>
>
> --
> *From:* KhajaAsmath Mohammed
> *Sent:* Thursday, December 15, 2016 7:54:27 PM
> *To:* user @spark
> *Subject:*
What is the format?
From: KhajaAsmath Mohammed
Sent: Thursday, December 15, 2016 7:54:27 PM
To: user @spark
Subject: Spark Dataframe: Save to hdfs is taking long time
Hi,
I am using issue while saving the dataframe back to HDFS. It's taking long time
t
Hi,
I am using issue while saving the dataframe back to HDFS. It's taking long
time to run.
val results_dataframe = sqlContext.sql("select gt.*,ct.* from
PredictTempTable pt,ClusterTempTable ct,GamificationTempTable gt where
gt.vin=pt.vin and pt.cluster=ct.cluster")
results_dataframe.coalesce(num
/save-to-HDFS-tp10578p10583.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
p/test1/
> after I login to the cluster?
>
> Thanks,
> lmk
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/save-to-HDFS-tp10578p10581.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
hdfs://masteripaddress:9000/root/test-app/test1/
after I login to the cluster?
Thanks,
lmk
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/save-to-HDFS-tp10578p10581.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
gt;
> The application is executed successfully and log says that save is complete
> also. But I am not able to find the file I have saved anywhere. Is there a
> way I can access this file?
>
> Pls advice.
>
> Regards,
> lmk
>
>
>
> --
> View this message in con
ccessfully and log says that save is complete
also. But I am not able to find the file I have saved anywhere. Is there a
way I can access this file?
Pls advice.
Regards,
lmk
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/save-to-HDFS-tp10578.html
Sent from
