Thanks a lot Soenke, Your explanation make a lot of sense.
On Mon, Feb 26, 2018 at 10:05 PM Sönke Liebau <soenke.lie...@opencore.com.invalid> wrote: > Hi Reema, hi Naresh, > > I'll try and answer both your questions together by expanding on the > topic a bit. Also, rereading my message I realize, that I phrased that > somewhat ambiguously, since a few of the terms in there are > overloaded. > > First of, if you are using the java consumer or producer (which you > most probably are) then there is no need for these to have access to > the Zookeeper nodes. Only the old scala clients needed to talk to > Zookeeper. This allows you to firewall your Zookeeper cluster so that > only Kafka brokers can connect to them. > > Moving on to the topic of listing topics things become a bit more > complex because both things are possible. If you run the shell command > "kafka-topics --list" that will connect to Zookeeper and retrieve a > list of topics. And this is black and white, you either see all topics > when you can access Zookeeper or none if you can't. > There is also the Java Admin Client that can list topics and this > talks to a Kafka broker to retrieve the topics. For this case, ACLs > apply and you will only see the topics you are allowed to access. The > main drawback of this method is, that there is no command line tool > for this yet, it is "just" a java api. > > When I said "access the Kafka nodes" I meant being able to connect to > the Kafka brokers port on those machines, that would be enough to use > the java admin client as described above. > > Hope this helps. > > Best regards, > Sönke > > > On Mon, Feb 26, 2018 at 5:25 PM, naresh Goud <nareshgoud.du...@gmail.com> > wrote: > > It should require zookeeper connection always, because intern kafka > brokers > > will interact with zookeeper for all meta data about topics. > > But its interesting, how would you give departments to access to kafka > nodes > > > > @Sönke, > > > > Could you please shade some light on giving departements access to kafka > > nodes.? Is it like departments able to ssh to kafka nodes and run > describe > > command? so it will show topics metadata only topics in that node? > > > > Apologies, if my question is very basic. > > > > Thank you, > > Naresh > > > > > > > > Thanks, > > Naresh > > www.linkedin.com/in/naresh-dulam > > http://hadoopandspark.blogspot.com/ > > > > > > On Mon, Feb 26, 2018 at 5:30 PM, Reema Chugani <reemachug...@outlook.com > > > > wrote: > > > >> Hi Sönke, > >> > >> Thanks for the info, it is helpful! > >> > >> I can modify so that the departments can only access the Kafka nodes > >> themselves. However how would the consumers connect to the topics then? > >> Don't the consumer clients require to connect via Zookeeper? > >> > >> Thanks, > >> Reema > >> > >> On Fri, Feb 23, 2018 at 10:50 PM, Sönke Liebau < > soenke.lie...@opencore.com > >> .invalid<mailto:soenke.lie...@opencore.com.invalid>> wrote: > >> Hi Reema, > >> > >> if your departments have access to Zookeeper then there probably is not > >> much you can do about them accessing data on other departments topics. I > >> assume that you have enabled Zookeeper ACLs, but even with those in > place, > >> the topic metadata is world readable, so listing topics can be done by > >> anyone who has access to Zookeeper. > >> > >> If your departments can only access the Kafka nodes themselves then the > >> DESCRIBE action on Topics is I believe what you are looking for, > without an > >> ACL in place to grant this, the topic should not be listed in Metadata > >> responses. > >> > >> I hope that helps, if you need more information let us know! > >> > >> Best regards, > >> Sönke > >> > >> Am 24.02.2018 06:32 schrieb "Reema Chugani" <reemachug...@outlook.com< > >> mailto:reemachug...@outlook.com>>: > >> > >> Hi, > >> > >> I am using Kafka 0.10.2. > >> > >> I have multiple topics & consumers set up with ACLS such that consumer > can > >> only read from a particular topic. I am wondering how I can prevent a > >> consumer from accessing metadata in zookeeper about other topics? i.e, > >> prevent consumers from listing or getting info about topics in the > cluster. > >> (Not let marketing dept see the topic names of finance topics.) > >> > >> Thanks, > >> Reema > >> > >> > >> > > > > -- > Sönke Liebau > Partner > Tel. +49 179 7940878 > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany > -- Thanks, Naresh www.linkedin.com/in/naresh-dulam http://hadoopandspark.blogspot.com/
