Sorry Ivan and Garmes, I misunderstood the suggestion earlier. I think this will be a great idea for a KIP.
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals You were referring to metadata for the actual topic and not its contents. Sorry about that confusion. On Mon, Jun 14, 2021 at 1:05 PM Ivan Yurchenko <ivan0yurche...@gmail.com> wrote: > Hi, > > Having metadata for topics seems pretty useful. Currently, one has to use > external storage for this (e.g. a database) and the question of keeping > topic and metadata in sync exists: A topic is deleted, how to delete its > metadata? How to deal with delete-then-recreate scenarios (well, we have > topic IDs now)? Making Kafka self-sufficient in this aspect sounds good. > > Ivan > > > On Mon, 14 Jun 2021 at 19:55, Israel Ekpo <israele...@gmail.com> wrote: > > > Garmes, > > > > I had similar questions in the past but @Matthias J. Sax > > <mj...@apache.org> pointed > > me to this > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-244%3A+Add+Record+Header+support+to+Kafka+Streams+Processor+API > > > > With the headers, you can filter based on the header content and not just > > the contents of the topic. > > > > > > > > > > On Fri, Jun 11, 2021 at 9:04 AM Garmes Amine > <aminegar...@yahoo.fr.invalid > > > > > wrote: > > > > > Hello Apache Kafka community, > > > > > > In the driection to make Kafka more cloud friendly, do you think it > make > > > sense to have key-velue metadata in the topic definition, > > > this will allow to search, filter, select ... topics not only by name > but > > > also by metadata > > > > > > Ex: > > > topic name: test > > > topic metadata: > > > - namespace: space1 (will allow to have multi tenant Kafka cluster) > > > - project: test > > > - phase: dev > > > - type: json > > > - area: logging > > > > > > > > > Best regards, > > > Garmes > > >
