Re: [EXT] How do I extract a value in foreachRDD operation

Thanks Michael, Can you give me an example? I'm new to Spark On Mon, 22 Jan 2018 at 12:25 Michael Mansour wrote: > Toy, > > > > I suggest your partition your data according to date, and use the > forEachPartition function, using the partition as the bucket location. > > This would require you t

Re: [EXT] How do I extract a value in foreachRDD operation

Toy, I suggest your partition your data according to date, and use the forEachPartition function, using the partition as the bucket location. This would require you to define a custom hash partitioner function, but that is not too difficult. -- Michael Mansour Data Scientist Symantec From: Toy