General Purpose Spark Cluster Hardware Requirements?

2015-03-08 Thread Nasir Khan
HI, I am going to submit a proposal to my University to setup my Standalone Spark Cluster, what hardware should i include in my proposal? I will be Working on classification (Spark MLlib) of Data streams (Spark Streams) If some body can fill up this answers, that will be great! Thanks *Cores *=

MLlib(Logistic Regression) + Spark Streaming.

2014-12-07 Thread Nasir Khan
I am new to spark. Lets say i want to develop a machine learning model. which trained on normal method in MLlib. I want to use that model with classifier Logistic regression and predict the streaming data coming from a file or socket. Streaming data -> Logistic Regression -> binary label predicti

Re: Filtering URLs from incoming Internet traffic(Stream data). feasible with spark streaming?

2014-10-27 Thread Nasir Khan
I am kinda stuck with spark now :/ i already proposed this model in my synopsis and its already accepted :D spark is a new thing for alot of people. what alternate tool should i use now? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Filtering-URLs-from-in

Filtering URLs from incoming Internet traffic(Stream data). feasible with spark streaming?

2014-10-27 Thread Nasir Khan
According to my knowledge spark streams uses mini batches for processing, Q: Is it a good idea to use my ML trained Model on a web server for filtering purpose to classify URLs as obscene or benin. If spark streaming handle data as mini batches for processing, will this increase the network laten

Machine Learning on streaming data

2014-03-16 Thread Nasir Khan
inner Level, Kindly Explain in abit detail and if some one can direct me to some good material for me will be greats. Thanks Nasir Khan. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Machine-Learning-on-streaming-data-tp2732.html Sent from the Apache