Hi,

We are writing our own consumer implementation and need to store offsets
manually in ZooKeeper. Currently we are storing them in a top-level znode
with a structure similar to what is found in /consumers. For example, we
have

/[consumer-name]/offsets/partition/[partition-id]

In looking into some monitoring tools like Burrow and KafkaOffsetMonitor,
they seem to only be geared toward the high-level consumer. So my question
is - is the recommended practice for writing one's own consumer to still
store offsets within the /consumers znode? Ie, store my offsets in
/consumers/[consumer-name]/offsets/partition/[partition-id] despite not
actually using the high-level consumer?

My concerns for doing so are whether Kafka makes assumptions about data
within the /consumers znode such that it would cause issues to store data
in there.

-- 

*Buck Ryan*
Software Engineer

1339 Chestnut Street, Ste 1500
Philadelphia, PA 19107

Website <http://rjmetrics.com>   |   Email <fr...@rjmetrics.com>   |
Twitter <http://twitter.com/_buckryan>

Reply via email to