TD's portion seems to start at 27:24: http://youtu.be/jcJq3ZalXD8?t=27m24s
On Tue Dec 16 2014 at 7:13:43 AM Gerard Maas wrote:
> Hi Jeniba,
>
> The second part of this meetup recording has a very good answer to your
> question. TD explains the current behavior and the on-going work in Spark
> S
Hi Jeniba,
The second part of this meetup recording has a very good answer to your
question. TD explains the current behavior and the on-going work in Spark
Streaming to fix HA.
https://www.youtube.com/watch?v=jcJq3ZalXD8
-kr, Gerard.
On Tue, Dec 16, 2014 at 11:32 AM, Jeniba Johnson <
jeniba.j
o
>>> offer a data acknowledging mechanism, so custom Receiver can use this
>>> acknowledgement to do checkpoint or recovery, like Storm.
>>>
>>>
>>>
>>> Besides, driver failure is another story need to be carefully
>>> considered. So currently it is
make sure no data loss in Spark Streaming, still
>> need to improve at some points J.
>>
>>
>>
>> Thanks
>>
>> Jerry
>>
>>
>>
>> *From:* Tobias Pfeiffer [mailto:t...@preferred.jp]
>> *Sent:* Tuesday, August 19, 2014 10:47 AM
>
me points J.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* Tobias Pfeiffer [mailto:t...@preferred.jp]
> *Sent:* Tuesday, August 19, 2014 10:47 AM
> *To:* Wei Liu
> *Cc:* user
> *Subject:* Re: Data loss - Spark streaming and network receiver
>
>
>
> Hi
, still need
to improve at some points ☺.
Thanks
Jerry
From: Tobias Pfeiffer [mailto:t...@preferred.jp]
Sent: Tuesday, August 19, 2014 10:47 AM
To: Wei Liu
Cc: user
Subject: Re: Data loss - Spark streaming and network receiver
Hi Wei,
On Tue, Aug 19, 2014 at 10:18 AM, Wei Liu
mailto:wei
Hi Wei,
On Tue, Aug 19, 2014 at 10:18 AM, Wei Liu
wrote:
>
> Since our application cannot tolerate losing customer data, I am wondering
> what is the best way for us to address this issue.
> 1) We are thinking writing application specific logic to address the data
> loss. To us, the problem seems
