Just to add to Jagasish's suggestion - you can configure
MetricsSnapshotRecorder to output the metrics into a stream and read them
from there if it works better for you.
Boris.
On Mon, Nov 16, 2015 at 12:32 PM, Jordan Shaw wrote:
> Michael,
> I should have added if your using Burrow in the cont
Michael,
I should have added if your using Burrow in the context of samza consumers
it probably won't work because samza does it's own offset tracking (see
checkpoint topics). The messages-behind-high-watermark is probably your
best bet if you just want something out of the box and don't care about
Thanks Jagadish! I'll look further into this.
Jordan, I tested Burrow with 0.8.3-SNAPSHOT and set it to read consumer
offsets from zookeeper because I assumed that it's the default Kafka config
for commiting offsets. Will try again with Burrow set to read from
__consumer_offsets.
Thanks
On Mon,
Michael,
It depends on how you define lag.
1) If you define lag as the total number of messages behind then burrow is
a good tool as long as all your infrastructure is on 0.8.2, it basically
works by inspecting the __consumer_offsets topic which was introduced in
0.8.2 (they said they were going t
The following metrics are reported via JMX.
1. Samza exposes a per-partition metric called "$topicname-$partition_name-
messages-behind-high-watermark". This measures the number of messages
behind the high watermark of your consumer. Ideally, at steady state, you
would expect this metric to be zer
