Re: Counting distinct values for a key?

2015-07-19 Thread suyog choudhari
public static void main(String[] args) { SparkConf sparkConf = new SparkConf().setAppName("CountDistinct"); JavaSparkContext jsc = new JavaSparkContext(sparkConf); List> list = new ArrayList>(); list.add(new Tuple2("key1", "val1")); list.add(new Tuple2("key1", "val1")); list.add(new T

Re: Counting distinct values for a key?

2015-07-19 Thread suyog choudhari
heir actual duplicate counts .. hence the use of the DISTINCT keyword in > the query equivalent ). > > Thanks > Nikunj > > > On Sun, Jul 19, 2015 at 2:37 PM, suyog choudhari > wrote: > >> public static void main(String[] args) { >> >> SparkConf

Is there synchronous way to predict against model for real time data

2016-12-15 Thread suyog choudhari
Hi, I have question about, how can I real time make decision using a model I have created with Spark ML. 1. I have some data and created model using it. // Train the model val model = new LogisticRegressionWithLBFGS().setNumClasses(2).run( trainingData) 2. I believe, I can use spark stream