Compute /Storage Calculation

Hi Team - Any good calculator/Excel to estimate compute and storage requirements for the new spark jobs to be developed. Capacity planning based on:- Job, Data type etc Thanks, Deepu Raj

Spark Shortcut

Hi Team, Using Spark 2.3 :paste -raw not working. Do ctrl+D after pasting the code get message //Exiting paste mode, now interpreting. Nothing happens. Please help. Thanks, Deepu

Cuesheet - spark deployment

tputStream(DFSOutputStream.java:1251) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:448) Process finished with exit code 1 Thanks and Regards Deepu Raj

Re: How to load "kafka" as a data source

Please check tools ver are same throughout. Thanks Deepu On Fri, 24 Mar 2017 14:47:11 +1100, Gaurav1809 wrote: > Hi All, > > I am running a simple command on spark-shell - like this. It's a piece of > structured streaming. > > val lines = (spark >.read

Re: Spark Source Code Configuration

Thanks Kai, I am getting the message and its stuck when I run sbt any idea:- "Set current project to spark-parent (in build file:/home/cloudera/spark/)" Details attached. Regards, Deepu Raj +61 414 707 319 On Fri, 20 Jan 2017 10:27:16 +1100, Kai Jiang wrote: Hi Deepu, Hope

Spark Source Code Configuration

Hi, Is there any article/Docs/support to set up Apache Spark source code on Eclipse/InteliJ. I have tried setting up the source code, by importing into Git & using maven. I am getting lot of compilation errors. #suggestions Regards, Deepu Raj +61 414 707

Re: Date Time Regression as Feature

Maybe u want to convert the date to a duration in form of number of hours/days and then do calculation on it On Jan 8, 2016 12:39 AM, Jorge Machado wrote: Hello all, I'm new to machine learning. I'm trying to predict some electric usage with a decision Free The data is : 2015-12-10-10:00, 120

Re: Newbie question

If the method is not final or static then u can On Jan 8, 2016 12:07 PM, yuliya Feldman wrote: Hello, I am new to Spark and have a most likely basic question - can I override a method from SparkContext? Thanks

Re: Recommendations using Spark

Use spark mlib kmeans algorithm to generate recommendations On Jan 8, 2016 12:41 PM, anjali gautam wrote: Hi, Can anybody please guide me how can we create generate recommendations for a user using spark? Regards, Anjali Gautam