Re: Spark streaming action running the same work in parallel

2015-04-27 Thread ColinMc
I was able to get it working. Instead of using customers.flatMap to return alerts. I had to use the following: customers.foreachRDD(new Function>, Void>() { @Override public Void call(final JavaPairRDD> rdd) throws Exception { rdd.foreachPartition(new VoidFu

Re: Spark streaming action running the same work in parallel

2015-04-22 Thread Tathagata Das
Unfortunately, none of the indicated logs, etc. is visible in the mail we got through the mailing list. On Wed, Apr 22, 2015 at 10:16 AM, ColinMc wrote: > Hi, > > I'm running a unit test that keeps failing to work with the code I wrote in > Spark. > > Here is the output logs from my test that I