Re: Batch timestamp in spark streaming

2014-12-19 Thread Gerard Maas
> How do you retrieve the current batch timestamp in spark streaming? Maybe > via BatchInfo but it does not seem to be linked to streaming context or > else... I currently have 1 minutes micro-batch and i'd like to link every > logs i process with the start of the batch a log belongs

Re: Batch timestamp in spark streaming

2014-12-19 Thread Sean Owen
et the timestamp of the batch as an argument from Spark. On Fri, Dec 19, 2014 at 11:59 AM, nelson wrote: > Hi all, > > I know the topic have been discussed before, but i couldn't find an answer > that might suits me. > > How do you retrieve the current batch timestamp in spark str

Batch timestamp in spark streaming

2014-12-19 Thread nelson
Hi all, I know the topic have been discussed before, but i couldn't find an answer that might suits me. How do you retrieve the current batch timestamp in spark streaming? Maybe via BatchInfo but it does not seem to be linked to streaming context or else... I currently have 1 minutes micro-