Re: Exception in connection from worker to worker

2015-01-21 Thread Akhil Das
om 1000 to 100. Now I can run > distinct and reduceByKey on files that are 2G / 50M lines. > > Unfortunately it still doesn't scale well. > > Thanks. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Exception-in

Re: Exception in connection from worker to worker

2015-01-21 Thread vantoniuk
number of splits from 1000 to 100. Now I can run distinct and reduceByKey on files that are 2G / 50M lines. Unfortunately it still doesn't scale well. Thanks. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Exception-in-connection-from-worker-to-w