Thanks Jiangjie for your help
On Sat, Feb 14, 2015 at 5:59 AM, Joel Koshy wrote:
> Thanks for looking into that!
>
> On Fri, Feb 13, 2015 at 05:31:39AM +, Jiangjie Qin wrote:
> > I think this is the offset checker bug.
> > The offset checker will
> > 1. first check if the offset exists in of
Thanks for looking into that!
On Fri, Feb 13, 2015 at 05:31:39AM +, Jiangjie Qin wrote:
> I think this is the offset checker bug.
> The offset checker will
> 1. first check if the offset exists in offset topic on broker or not.
> 2. If it is on broker then it will just return that offset.
> 3.
I created KAFKA-1951 to improve the offset checker to let it print offsets
stored in both zookeeper and Kafka.
I don¹t think we have a way to wipe out offset topic in Kafka for a
consumer.
‹Jiangjie (Becket) Qin
On 2/12/15, 9:59 PM, "tao xiao" wrote:
>Thanks for the explanation. It there a way
Thanks for the explanation. It there a way that I can wipe out the offset
stored in kafka so that the checker can continue to work again?
On Fri, Feb 13, 2015 at 1:31 PM, Jiangjie Qin
wrote:
> I think this is the offset checker bug.
> The offset checker will
> 1. first check if the offset exists
I think this is the offset checker bug.
The offset checker will
1. first check if the offset exists in offset topic on broker or not.
2. If it is on broker then it will just return that offset.
3. Otherwise it goes to zookeeper.
So the problem you saw was actually following this logic.
After dual
I used the one shipped with 0.8.2. It is pretty straightforward to
reproduce the issue.
Here are the steps to reproduce:
1. I have a consumer using high level consumer API with initial settings
offsets.storage=kafka and dual.commit.enabled=false.
2. After consuming messages for a while shutdown th
That is weird. Are you by any chance running an older version of the
offset checker? Is this straightforward to reproduce?
On Fri, Feb 13, 2015 at 09:57:31AM +0800, tao xiao wrote:
> Joel,
>
> No, the metric was not increasing. It was 0 all the time.
>
> On Fri, Feb 13, 2015 at 12:18 AM, Joel Ko
Joel,
No, the metric was not increasing. It was 0 all the time.
On Fri, Feb 13, 2015 at 12:18 AM, Joel Koshy wrote:
> Actually I meant to say check that is not increasing.
>
> On Thu, Feb 12, 2015 at 08:15:01AM -0800, Joel Koshy wrote:
> > Possibly a bug - can you also look at the MaxLag mbean
Actually I meant to say check that is not increasing.
On Thu, Feb 12, 2015 at 08:15:01AM -0800, Joel Koshy wrote:
> Possibly a bug - can you also look at the MaxLag mbean in the consumer
> to verify that the maxlag is zero?
>
> On Thu, Feb 12, 2015 at 11:24:42PM +0800, tao xiao wrote:
> > Hi Joel
Possibly a bug - can you also look at the MaxLag mbean in the consumer
to verify that the maxlag is zero?
On Thu, Feb 12, 2015 at 11:24:42PM +0800, tao xiao wrote:
> Hi Joel,
>
> When I set dual.commit.enabled=true the count value of both metrics got
> increased. After I set offsets.storage=zooke
Hi Joel,
When I set dual.commit.enabled=true the count value of both metrics got
increased. After I set offsets.storage=zookeeper only ZooKeeperCommitsPerSec
changed but not KafkaCommitsPerSec. I think this is expected as kafka
offset storage was turned off.
But when I looked up the consumer lag
There are mbeans named KafkaCommitsPerSec and ZooKeeperCommitsPerSec -
can you look those up and see what they report?
On Thu, Feb 12, 2015 at 07:32:39PM +0800, tao xiao wrote:
> Hi team,
>
> I was trying to migrate my consumer offset from kafka to zookeeper.
>
> Here is the original settings of
Hi team,
I was trying to migrate my consumer offset from kafka to zookeeper.
Here is the original settings of my consumer
props.put("offsets.storage", "kafka");
props.put("dual.commit.enabled", "false");
Here is the steps
1. set dual.commit.enabled=true
2. restart my consumer and monitor offse
13 matches
Mail list logo
