Hi Shafaq,
"Yes" to your first question.
For second question that depends on what would you like to do when your
request's offset is not valid. This is possible, for example, if your logs
gets deleted according to retention policy and your last committed offset
in ZK is within that deleted range.
Hi,
I wanted to control the commitOffset signal to ZK from High-level kafka
consumer. This will enable to process the message by consumer and incase
of failure the offset is not moved.
If I do
auto.commit.enable= false
and then use commitallOffset Api of high-level consumer, is that way to
a
That I can manage. Thanks so much.
On Tue, Nov 5, 2013 at 6:46 AM, Neha Narkhede wrote:
> Yes, it will commit offsets only for the partitions that the consumer owns.
> But over time, the set of partitions that a consumer owns can change.
>
> Thanks,
> Neha
> On Nov 5, 2013 12:17 AM, "Vadim Keyli
Yes, it will commit offsets only for the partitions that the consumer owns.
But over time, the set of partitions that a consumer owns can change.
Thanks,
Neha
On Nov 5, 2013 12:17 AM, "Vadim Keylis" wrote:
> I am using creating Consumer.createJavaConsumerConnector(kafka 0.8) for
> each topic/p
I am using creating Consumer.createJavaConsumerConnector(kafka 0.8) for
each topic/partition. Would it be safe to assume that commit offset will
apply only to stream/partition managed by that connector?
Thanks,
Vadim
On Mon, Nov 4, 2013 at 8:43 PM, Neha Narkhede wrote:
> You need to set "auto
Hi,
I wanted to control the commitOffset signal to ZK from High-level kafka
consumer. This will enable to process the message by consumer and incase
of failure the offset is not moved.
If I do
auto.commit.enable= false
and then use commitallOffset Api of high-level consumer, is that way to
a
Thanks so much Neha. That did the trick. Thanks so much.
On Mon, Nov 4, 2013 at 8:43 PM, Neha Narkhede wrote:
> You need to set "auto.offset.reset"="smallest". By default, the consumer
> will start consuming the latest messages.
>
> Thanks,
> Neha
>
>
> On Mon, Nov 4, 2013 at 4:38 PM, Guozhang W
You need to set "auto.offset.reset"="smallest". By default, the consumer
will start consuming the latest messages.
Thanks,
Neha
On Mon, Nov 4, 2013 at 4:38 PM, Guozhang Wang wrote:
> And exceptions you saw from the broker end, in server log?
>
> Guozhang
>
>
> On Mon, Nov 4, 2013 at 4:27 PM, V
And exceptions you saw from the broker end, in server log?
Guozhang
On Mon, Nov 4, 2013 at 4:27 PM, Vadim Keylis wrote:
> Thanks for confirming, but that not behavior I observe. My consumer does
> not commit data to kafka. It get messages sent to kafka. Once restarted I
> should of gotten mes
Thanks for confirming, but that not behavior I observe. My consumer does
not commit data to kafka. It get messages sent to kafka. Once restarted I
should of gotten messages that previously received by consumer, but on
contrarily I got none. Logs confirm the initial offset been as -1. What am
I doi
That is correct. If auto.commit.enable is set to faulse, the offsets will
not be committed at all unless the consumer calls the commit function
explicitly.
Guozhang
On Mon, Nov 4, 2013 at 2:42 PM, Vadim Keylis wrote:
> Good afternoon. I was under impression if auto commit set to false then
>
Good afternoon. I was under impression if auto commit set to false then
once consumer is restarted then logs would be replayed from the beginning.
Is that correct?
Thanks,
Vadim
12 matches
Mail list logo