Finding out root cause is definitely something we should do and I hope the
new Java consumer API will not have such issues anymore. Here are my
observation of the issue: the zk retry was caused by session timeout. In
theory the ephemeral node should be removed after the session is terminated
by zk
That would solve this. But it looks like a work around. We need to check
why this happens exactly and get to the root cause. What do you think?
Getting to the root cause of this might be really useful.
Thanks,
Mayuresh
On Sun, Jul 12, 2015 at 8:45 PM, tao xiao wrote:
> Restart the consumers do
Restart the consumers does fix the issue. But since the zk retry is wrapped
in an infinite loop it doesn't give a chance to consumer to respond it
until some one notices and restarts. Why I suggest to have a maximum retry
policy is if max retry is reached it can invoke a customer handler which I
ca
Can you confirm that you are not actually seeing the messages on the
lagging broker ?
Because if the Max Lag is 0 it should mean that consumer has read the
offsets till log end offset of the broker.
Thanks,
Mayuresh
On Fri, Jul 10, 2015 at 8:29 PM, Allen Wang
wrote:
> We have two applications
Storing topic metadata in kafka brokers can be done in the same way as the
offsets topic in kafka I think. It would be a good point for discussion.
Thanks,
Mayuresh
On Sun, Jul 12, 2015 at 8:25 PM, Jiangjie Qin
wrote:
> Hi Stevo,
>
> Kafka does not support customized topic metadata. What we a
Bouncing the consumers should solve this issue in most cases.
Thanks,
Mayuresh
On Sun, Jul 12, 2015 at 8:21 PM, Jiangjie Qin
wrote:
> Hi Tao,
>
> We see this error from time to time but did not think of this as a big
> issue. Any reason it bothers you much?
> I¹m not sure if throwing exception
Hi Tao,
We see this error from time to time but did not think of this as a big
issue. Any reason it bothers you much?
I¹m not sure if throwing exception to user on this exception is a good
handling or not. What are user supposed to do in that case other than
retry?
Thanks,
Jiangjie (Becket) Qin
Yes, we are going to use new consumer after it is ready.
Jiangjie (Becket) Qin
On 7/12/15, 8:21 PM, "tao xiao" wrote:
>Hi team,
>
>The trunk code of mirror maker now uses the old consumer API, Is there any
>plan to use new Java consumer api in mirror maker?
Hi Stevo,
Kafka does not support customized topic metadata. What we are doing now is
having topic metadata store in a separate zookeeper structure. But I agree
this is probably a good candidate feature to consider.
Thanks,
Jiangjie (Becket) Qin
On 7/12/15, 4:16 PM, "Stevo Slavić" wrote:
>Hell
Hi team,
The trunk code of mirror maker now uses the old consumer API, Is there any
plan to use new Java consumer api in mirror maker?
We saw the error again in our cluster. Anyone has the same issue before?
On Fri, 10 Jul 2015 at 13:26 tao xiao wrote:
> Bump the thread. Any help would be appreciated.
>
> On Wed, 8 Jul 2015 at 20:09 tao xiao wrote:
>
>> Additional info
>> Kafka version: 0.8.2.1
>> zookeeper: 3.4.6
>>
>> On We
Hello Apache Kafka Community,
Is it possible to store and retrieve additional custom topic metadata along
with existing Kafka managed ones, using some Kafka API? If not would it be
a problem (e.g. for Kafka broker or some client APIs) if I was to
store/retrieve additional custom topic metadata usi
Your payload is so small that I suspect it's an encoding issue. Is your
producer set to expect a byte array and you're passing a string? Or vice
versa?
On Sat, Jul 11, 2015 at 11:08 PM, David Montgomery <
davidmontgom...@gmail.com> wrote:
> I cant send this s simple payload using python.
>
>
> On 10. jul. 2015, at 23.03, Jay Kreps wrote:
>
> If I recall correctly, setting log.retention.ms and log.retention.bytes to
> -1 disables both.
Thanks!
>
> On Fri, Jul 10, 2015 at 1:55 PM, Daniel Schierbeck <
> daniel.schierb...@gmail.com> wrote:
>
>>
>>> On 10. jul. 2015, at 15.16, Shay
14 matches
Mail list logo
