RE: Machine Learning in Spark 1.6 vs Spark 2.0

Thanks Rezaul… Is Spark 2.1.0 still have any issues w.r.t. stability? Regards, Ankur From: Md. Rezaul Karim [mailto:rezaul.ka...@insight-centre.org] Sent: Monday, January 09, 2017 5:02 PM To: Ankur Jain Cc: user@spark.apache.org Subject: Re: Machine Learning in Spark 1.6 vs Spark 2.0 Hello

Machine Learning in Spark 1.6 vs Spark 2.0

Hi Team, I want to start a new project with ML. But wanted to know which version of Spark is much stable and have more features w.r.t ML Please suggest your opinion... Thanks in Advance... [cid:image013.png@01D1AAE2.28F7BBF0] Thanks & Regards Ankur Jain Technical Architect - Big Data |

RE: Saving Parquet files to S3

Thanks maropu.. It worked… From: Takeshi Yamamuro [mailto:linguin@gmail.com] Sent: 10 June 2016 11:47 AM To: Ankur Jain Cc: user@spark.apache.org Subject: Re: Saving Parquet files to S3 Hi, You'd better off `setting parquet.block.size`. // maropu On Thu, Jun 9, 2016 at 7:48 AM, D

Saving Parquet files to S3

Hello Team, I want to write parquet files to AWS S3, but I want to size each file size to 1 GB. Can someone please guide me on how I can achieve the same? I am using AWS EMR with spark 1.6.1. Thanks, Ankur Information transmitted by this e-mail is proprietary to YASH Technologies and/ or its C

dataframe stat corr for multiple columns

Hello Team, In my current usecase I am loading data from CSV using spark-csv and trying to correlate all variables. As of now if we want to correlate 2 column in a dataframe df.stat.corr works great but if we want to correlate multiple columns this won't work. In case of R we can use corrplot a

RE: JavaKinesisWordCountASLYARN Example not working on EMR

:31 PM To: Ankur Jain Cc: user@spark.apache.org Subject: Re: JavaKinesisWordCountASLYARN Example not working on EMR Did you built for kineses using profile -Pkinesis-asl On Wed, Mar 25, 2015 at 7:18 PM, ankur.jain mailto:ankur.j...@yash.com>> wrote: Hi, I am trying to run a Spark on YARN p