Re: Unexpected mirror maker behavior with new consumer

2016-02-25 Thread Luke Steensen
I've opened https://issues.apache.org/jira/browse/KAFKA-3285 to track the issue. I will test again with metadata.max.age.ms in mind, that makes sense. Thanks! On Wed, Feb 24, 2016 at 8:22 PM, tao xiao wrote: > One more additional note to new consumer. The new topic will not be picked > up by n

Re: Unexpected mirror maker behavior with new consumer

2016-02-24 Thread tao xiao
One more additional note to new consumer. The new topic will not be picked up by new consumer immediately. It takes as long as metadata.max.age.ms to refresh metadata and pick up topics that match the pattern at the time of check. On Thu, 25 Feb 2016 at 08:48 Jason Gustafson wrote: > Hey Luke, >

Re: Unexpected mirror maker behavior with new consumer

2016-02-24 Thread Jason Gustafson
Hey Luke, I took a look at the code and it does look like the whitelist argument is handled differently between the old and new consumers. For the new consumer, we just treat it as a raw regular expression, but the old consumer does some preprocessing. We should probably do the preprocessing in bo

Unexpected mirror maker behavior with new consumer

2016-02-24 Thread Luke Steensen
Hello, I've been experimenting with mirror maker and am a bit confused about some behavior I'm seeing. I have two simple single broker clusters setup locally, source and target. The source cluster has a topic foo with a few message in it. When I run mirror maker with --whitelist '*' (as noted in