Hi there, I'm trying to track down why __consumer_offsets is never balancing and what to do about it.
We're using Shopify's sarama <https://gowalker.org/github.com/Shopify/sarama> client, which explicitly states that it doesn't do rebalancing for internal topics ... ever. It seems like sarama is one of the better-regarded golang clients for kafka, despite this fairly glaring and confusing oversight. (is this true? does anyone have a better recommendation?) There is a sarama-cluster <https://github.com/bsm/sarama-cluster> extension with a CLI. It looks like if you compile the CLI and connect to the cluster, it seems it will kick off a rebalance and remove downed brokers. This ... seems incredibly awkward to me, but at least it's *a* solution. But it only works if you're using consumer groups. Which we are not. We can rewrite the kafka consumer to use consumer groups, though after reading docs for a day I still don't understand why we would do that other than to make sarama-cluster work, or what the impact would be other than letting us use this thing that lets us rebalance. Last question. We are using kafka-0.9, and we seem to keep running into shit that is not very well documented and doesn't seem to be used by very many people. I'm thinking of downgrading to 0.8 because none of the 0.9 features are very compelling to us, and we would really not rather be blazing any unnecessary trails. Does anyone have an informed opinion here? Thanks for answering all the noob questions.
