Re: Read Accumulator value while running

2016-01-14 Thread Mennour Rostom
Hi Daniel, Andrew Thank you for your answers, So it s not possible to read the accumulator value until the action that manipulate it finishes. it's bad, I ll think to something else. However the main important thing in my application is the ability to lunche 2 (or more) actions in parallel and con

Re: Read Accumulator value while running

2016-01-13 Thread Andrew Or
Hi Kira, As you suspected, accumulator values are only updated after the task completes. We do send accumulator updates from the executors to the driver on periodic heartbeats, but these only concern internal accumulators, not the ones created by the user. In short, I'm afraid there is not curren

Re: Read Accumulator value while running

2016-01-13 Thread Daniel Imberman
Hi Kira, I'm having some trouble understanding your question. Could you please give a code example? >From what I think you're asking there are two issues with what you're looking to do. (Please keep in mind I could be totally wrong on both of these assumptions, but this is what I've been lead t