Re: Model abstract class in spark ml

Weird, I recompiled Spark with a similar change to Model and it seemed to work but maybe I missed a step in there. On Wed, Aug 31, 2016 at 6:33 AM, Mohit Jaggi wrote: > I think I figured it out. There is indeed "something deeper in Scala” :-) > > abstract class A { > def a: this.type > } > > cl

Re: KMeans calls takeSample() twice?

I added println at the start of function takeSample, and found it was printed only once for each run of KMeans. Thanks Yanbo On Tue, Aug 30, 2016 at 10:31 AM, Georgios Samaras < georgesamaras...@gmail.com> wrote: > Good catch Shivaram. However, the very next line states: > > // this shouldn't ha

Re: Model abstract class in spark ml

http://blog.originate.com/blog/2014/02/27/types-inside-types-in-scala/ On Wed, Aug 31, 2016 at 2:19 AM, Sean Owen wrote: > Weird, I recompiled Spark with a similar change to Model and it seemed > to work but maybe I missed a step in there. > > On Wed, Aug 31, 2016 at 6:33 AM, Mohit Jaggi wrote:

Re: KMeans calls takeSample() twice?

But as can see in this: Stackoverflow question this is simply not the case for me. Could it because your data is not big enough to reproduce? Or could it be that you are actually reproducing it, but the problem co

Questions about bucketing in Spark

Hi everyone, I am working towards making Spark's Sort Merge join in par with Hive's Sort-Merge-Bucket join to use sorted. So far I have identified these main items to be addressed: 1. Make query planner to use `sorted`ness information for sort merge join (SPARK-15453, SPARK-17271) 2. Configurable

Re: Model abstract class in spark ml

Thanks Cody. That was a good explanation! Mohit Jaggi Founder, Data Orchard LLC www.dataorchardllc.com > On Aug 31, 2016, at 7:32 AM, Cody Koeninger wrote: > > http://blog.originate.com/blog/2014/02/27/types-inside-types-in-scala/ > > On Wed, Aug 31, 2016 at 2:19 AM, Sean Owen wrote: >> We