Re: [EXT] How do I extract a value in foreachRDD operation

2018-01-22 Thread Toy
; > This would require you to define a custom hash partitioner function, but > that is not too difficult. > > > > -- > > Michael Mansour > > Data Scientist > > Symantec > > *From: *Toy > *Date: *Monday, January 22, 2018 at 8:19 AM > *To: *"us

Re: [EXT] How do I extract a value in foreachRDD operation

2018-01-22 Thread Michael Mansour
Date: Monday, January 22, 2018 at 8:19 AM To: "user@spark.apache.org" Subject: [EXT] How do I extract a value in foreachRDD operation Hi, We have a spark application to parse log files and save to S3 in ORC format. However, during the foreachRDD operation we need to extract a date f