The invalid ACL error is an error that is passed back from Zookeeper. What version of Zookeeper are you using, and have you set up ACLs within it? I'm not able to see this on our ZK (3.4.6 with no ACLs).
-Todd On Fri, Jun 12, 2015 at 9:34 AM, Roger Hoover <roger.hoo...@gmail.com> wrote: > Hi, > > I was trying to give burrow a try and got a ZK error "invalid ACL > specified". Any suggestions on what's going wrong? > > 1434044348908673512 [Critical] Cannot get ZK notifier lock: zk: invalid ACL > specified > > > Here's my config: > > > [general] > > logdir=log > > logconfig=logging.cfg > > pidfile=burrow.pid > > client-id=burrow-lagchecker > > group-blacklist=^(console-consumer-|python-kafka-consumer-).*$ > > > [zookeeper] > > hostname=host1 > > port=2181 > > timeout=6 > > lock-path=/burrow/notifier > > > [kafka "sit"] > > broker=host1 > > broker=host2 > > broker=host3 > > broker=host4 > > broker-port=9092 > > zookeeper=host1 > > zookeeper=host2 > > zookeeper=host3 > > zookeeper-port=2181 > > zookeeper-path=/ > > offsets-topic=__consumer_offsets > > > [tickers] > > broker-offsets=60 > > > [lagcheck] > > intervals=10 > > expire-group=604800 > > > [httpserver] > > server=on > > port=7000 > > On Tue, Jun 9, 2015 at 12:34 PM, Todd Palino <tpal...@gmail.com> wrote: > > > For mirror maker and our audit application, we've been using > > Kafka-committed offsets for some time now. We've got a few other > consumers > > who are using it, but we haven't actively worked on moving the bulk of > them > > over. It's been less critical since we put the ZK transaction logs on > SSD. > > > > And yeah, this is specific for kafka-committed offsets. I'm looking at > some > > options for handling Zookeeper as well, but since our goal with this was > to > > monitor our own infrastructure applications and move forwards, it hasn't > > gotten a lot of my attention yet. > > > > -Todd > > > > > > On Tue, Jun 9, 2015 at 11:53 AM, Jason Rosenberg <j...@squareup.com> > wrote: > > > > > Hi Todd, > > > > > > Thanks for open sourcing this, I'm excited to take a look. > > > > > > It looks like it's specific to offsets stored in kafka (and not > > zookeeper) > > > correct? I assume by that that LinkedIn is using the kafka storage now > > in > > > production? > > > > > > Jason > > > > > > On Thu, Jun 4, 2015 at 9:43 PM, Todd Palino <tpal...@gmail.com> wrote: > > > > > > > I am very happy to introduce Burrow, an application to provide Kafka > > > > consumer status as a service. Burrow is different than just a "lag > > > > checker": > > > > > > > > * Multiple Kafka cluster support - Burrow supports any number of > Kafka > > > > clusters in a single instance. You can also run multiple copies of > > Burrow > > > > in parallel and only one of them will send out notifications. > > > > > > > > * All consumers, all partitions - If the consumer is committing > offsets > > > to > > > > Kafka (not Zookeeper), it will be available in Burrow automatically. > > > Every > > > > partition it consumes will be monitored simultaneously, avoiding the > > trap > > > > of just watching the worst partition (MaxLag) or spot checking > > individual > > > > topics. > > > > > > > > * Status can be checked via HTTP request - There's an internal HTTP > > > server > > > > that provides topic and consumer lists, can give you the latest > offsets > > > for > > > > a topic either from the brokers or from the consumer, and lets you > > check > > > > consumer status. > > > > > > > > * Continuously monitor groups with output via email or a call to an > > > > external HTTP endpoint - Configure emails to send for bad groups, > > checked > > > > continuously. Or you can have Burrow call an HTTP endpoint into > another > > > > system for handling alerts. > > > > > > > > * No thresholds - Status is determined over a sliding window and does > > not > > > > rely on a fixed limit. When a consumer is checked, it has a status > > > > indicator that tells whether it is OK, a warning, or an error, and > the > > > > partitions that caused it to be bad are provided. > > > > > > > > > > > > Burrow was created to address specific problems that LinkedIn has > with > > > > monitoring consumers, in particular wildcard consumers like mirror > > makers > > > > and our audit consumers. Instead of checking offsets for specific > > > consumers > > > > periodically, it monitors the stream of all committed offsets > > > > (__consumer_offsets) and continually calculates lag over a sliding > > > window. > > > > > > > > We welcome all feedback, comments, and contributors. This project is > > very > > > > much under active development for us (we're using it in some of our > > > > environments now, and working on getting it running everywhere to > > replace > > > > our previous monitoring system). > > > > > > > > Burrow is written in Go, published under the Apache License, and > hosted > > > on > > > > GitHub at: > > > > https://github.com/linkedin/Burrow > > > > > > > > Documentation is on the GitHub wiki at: > > > > https://github.com/linkedin/Burrow/wiki > > > > > > > > -Todd > > > > > > > > > >
