Re: Kafka Advisory Topic

2021-02-03 Thread Christopher Shannon
After discussing this a bit more and looking into what it would take to implement this feature Knowles and I are going to hold off until we see what the removal of Zookeeper will look like. We are still interested in this feature but we figure that things will change significantly enough that it's

Re: Kafka Advisory Topic

2021-01-26 Thread Knowles Atchison Jr
Gwen, Yes, Chris and I work together; we can put together a KIP. If someone would please grant me wiki permissions, username is katchison. Thank you. On Mon, Jan 25, 2021 at 2:58 PM Gwen Shapira wrote: > Agree that this sounds like a good idea. > > Would be good to have a more formal proposal

Re: Kafka Advisory Topic

2021-01-25 Thread Gwen Shapira
Agree that this sounds like a good idea. Would be good to have a more formal proposal (i.e. a KIP) with the details. I can think of about 100 different questions (will there be "levels" like in logs, what type of events are in or out of scope, rate limiting, data formats, etc). I am also curious o

Re: Kafka Advisory Topic

2021-01-21 Thread Christopher Shannon
Hi, I am on the ActiveMQ PMC and I think this is a very good idea to have a way to do advisories/notifications/events (whatever you want to call it). In ActiveMQ classic you have advisories and in Artemis you have notifications. Having management messages that can be subscribed to in real time is

Re: Kafka Advisory Topic

2021-01-20 Thread Boyang Chen
Hey Knowles, in Kafka people normally use admin clients to get those metadata. I'm not sure why you mentioned specifically that having a topic to manage these information is useful, but a good news is that in KIP-500

Kafka Advisory Topic

2021-01-20 Thread Knowles Atchison Jr
Good afternoon all, In our Kafka clusters we have a need to know when certain activities are performed, mainly topics being created, but brokers coming up/down is also useful. This would be akin to what ActiveMQ does via advisory messages ( https://activemq.apache.org/advisory-message). Since the