Adrian, Trying to follow up the discussion here. Is my understanding correct that if we have topic hierarchies then we can do without namespaces. To me namespace is an abstraction, it can be implemented with topic hierarchies as well, would you agree? If so I guess topic hierarchies is the way to go.
One more thing I would like to put forward as an advantage of having topic hierarchy is that we can support acls/ permissions inheritance in topic hierarchies. This will avoid bootstrapping acls for each new topic. On Wed, Apr 29, 2015 at 7:42 AM, Adrian Preston <prest...@uk.ibm.com> wrote: > Thanks for your response. > > I agree. I think it would be useful to get consensus on how > namespaces and topic-hierarchies relate to one another. To seed the > discussion - here's my viewpoint, which I hope others will challenge: > > I see namespaces as something the broker provides to ensure that > two tenants can never interact with one another - even if, > for example, they both choose to use a topic called 'TOPIC1'. I > imagine that this would be achieved by having the broker silently > add a per tenant prefix to the topic name in each request, and > strip it off in the response. So, for example, if 'TENANT1' sends > a message to 'TOPIC1', this would be re-written so that the send is > to 'TENANT1-TOPIC1'. > > If topic-hierarchies were available, then I think the prefix, > added/removed to implement namespaces, would be the first level of > qualification in the hierarchy. So, for example, if 'TENANT1' sends > a message to 'TOPIC1', this would be re-written so that the send is to > 'TENANT1/TOPIC1'. > > Extrapolating from what's currently in KIP-21 (dynamic configuration) > I guess that topic-hierarchies might result in the possibility for > even finer grain topic configuration - e.g. a ZNode structure of: > '/config/topics/<topic_level1>/<topic_level2>/...'. This would work > best with an implementation of namespaces that was based on-top of > topic-hierarchies, allowing configuration to be applied at the scope > of: all tenants, one tenant, or one tenant's topics. > > So in summary: I think that namespaces can be usefully implemented > independently of topic-hierarchies, and when topic-hierarchies are > implemented would be easily integrated. > > Regards > - Adrian > > -----Gwen Shapira <gshap...@cloudera.com> wrote: ----- > To: "dev@kafka.apache.org" <dev@kafka.apache.org> > From: Gwen Shapira <gshap...@cloudera.com> > Date: 04/28/2015 06:54PM > Subject: Re: Adding multi-tenancy capabilities to Kafka > > I think recent discussion showed some need for topic namespaces - for > example, Jun's use case for reserving topic names for specific users > discussed under authorization. > > I think some discussion should happen on namespaces vs more full-fledged > topic-hierarchy. > I like the simplicity of namespaces, but there may be other requirements > (such as inheriting configuration). > > Gwen > > On Tue, Apr 28, 2015 at 10:42 AM, Adrian Preston <prest...@uk.ibm.com> > wrote: > > > Hi all, > > > > I've been looking at how a Kafka cluster could be used be deployed so > that > > it can be used by multiple tenants. Specifically: a scheme where the > > clients belonging to each tenant receive the impression they are > operating > > against their own cluster. The ongoing security and quota work looks > like > > it might provide a lot of the isolation requirements, but each tenant > would > > currently share the same namespace for topics and consumer groups. So > the > > illusion of "it is my own cluster" is broken as soon as two tenants try > > independent to use the same topic name. > > > > I wondered if other people on this list are interested in being able to > > support multiple tenants in this way? And / or if the ability to avoid > > clashes in the topic namespace would be useful? I am considering > > submitting a KIP in this area - but first wanted to get a feeling for > > whether these kinds of capabilities are of interest to others. > > > > Thanks in advance, > > - Adrian > > > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > > > > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > -- Regards, Ashish
