Please see the inline comments.
From: Shushant Arora [mailto:shushantaror...@gmail.com]
Sent: Monday, July 6, 2015 8:51 PM
To: Shao, Saisai
Cc: user
Subject: Re: kafka offset commit in spark streaming 1.2
So If WAL is disabled, how developer can commit offset explicitly in spark
streaming app
.com]
> *Sent:* Monday, July 6, 2015 8:30 PM
> *To:* Shao, Saisai
> *Cc:* user
> *Subject:* Re: kafka offset commit in spark streaming 1.2
>
>
>
> And what if I disable WAL and use replication of receiver data using
> StorageLevel.MEMORY_ONLY2().
> Will it commit offset
commitment mechanism is actually a timer way, so it is
asynchronized with replication.
From: Shushant Arora [mailto:shushantaror...@gmail.com]
Sent: Monday, July 6, 2015 8:30 PM
To: Shao, Saisai
Cc: user
Subject: Re: kafka offset commit in spark streaming 1.2
And what if I disable WAL and use
t you’re setting with autocommit.enable, internally Spark
> Streaming will set it to false to turn off autocommit mechanism.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* Shushant Arora [mailto:shushantaror...@gmail.com]
> *Sent:* Monday, July 6, 2015 8:11 PM
> *To:* user
> *
autocommit mechanism.
Thanks
Jerry
From: Shushant Arora [mailto:shushantaror...@gmail.com]
Sent: Monday, July 6, 2015 8:11 PM
To: user
Subject: kafka offset commit in spark streaming 1.2
In spark streaming 1.2 , Is offset of kafka message consumed are updated in
zookeeper only after writing in WAL
In spark streaming 1.2 , Is offset of kafka message consumed are updated in
zookeeper only after writing in WAL if WAL and checkpointig are enabled or
is it depends upon kafkaparams while initialing the kafkaDstream.
Map kafkaParams = new HashMap();
kafkaParams.put("zookeeper.connect","ip:2181");
