I've opened https://issues.apache.org/jira/browse/KAFKA-3285 to track the issue.
I will test again with metadata.max.age.ms in mind, that makes sense. Thanks! On Wed, Feb 24, 2016 at 8:22 PM, tao xiao <xiaotao...@gmail.com> wrote: > One more additional note to new consumer. The new topic will not be picked > up by new consumer immediately. It takes as long as metadata.max.age.ms to > refresh metadata and pick up topics that match the pattern at the time of > check. > > On Thu, 25 Feb 2016 at 08:48 Jason Gustafson <ja...@confluent.io> wrote: > > > Hey Luke, > > > > I took a look at the code and it does look like the whitelist argument is > > handled differently between the old and new consumers. For the new > > consumer, we just treat it as a raw regular expression, but the old > > consumer does some preprocessing. We should probably do the preprocessing > > in both cases for consistency. Would you mind opening a JIRA? > > > > Thanks, > > Jason > > > > On Wed, Feb 24, 2016 at 4:17 PM, Luke Steensen < > > luke.steen...@braintreepayments.com> wrote: > > > > > Hello, > > > > > > I've been experimenting with mirror maker and am a bit confused about > > some > > > behavior I'm seeing. > > > > > > I have two simple single broker clusters setup locally, source and > > target. > > > The source cluster has a topic foo with a few message in it. > > > > > > When I run mirror maker with --whitelist '*' (as noted in the > > > documentation), nothing is written to the target cluster. Switching > '.*' > > > makes no difference. > > > > > > With --whitelist '\w*', everything is mirrored, including > > > __consumer_offsets. > > > > > > With --whitelist foo, the foo topic is mirrored correctly from source > to > > > target. > > > > > > With --whitelist 'foo|bar' (where foo already exists and bar does not > > yet), > > > foo is mirrored correctly, but bar is ignored when it gets created. > > > Restarting mirror maker with the same command causes it to pick up on > > bar. > > > > > > I assume there is some outdated documentation around old vs new > consumer > > in > > > mirror maker, but it's still not obvious how to automatically mirror > all > > > non-internal topics. Any pointers would be appreciated. > > > > > > Thanks, > > > Luke > > > > > >
