Thanks a lot Soenke,
Your explanation make a lot of sense.


On Mon, Feb 26, 2018 at 10:05 PM Sönke Liebau
<soenke.lie...@opencore.com.invalid> wrote:

> Hi Reema, hi Naresh,
>
> I'll try and answer both your questions together by expanding on the
> topic a bit. Also, rereading my message I realize, that I phrased that
> somewhat ambiguously, since a few of the terms in there are
> overloaded.
>
> First of, if you are using the java consumer or producer (which you
> most probably are) then there is no need for these to have access to
> the Zookeeper nodes. Only the old scala clients needed to talk to
> Zookeeper. This allows you to firewall your Zookeeper cluster so that
> only Kafka brokers can connect to them.
>
> Moving on to the topic of listing topics things become a bit more
> complex because both things are possible. If you run the shell command
> "kafka-topics --list" that will connect to Zookeeper and retrieve a
> list of topics. And this is black and white, you either see all topics
> when you can access Zookeeper or none if you can't.
> There is also the Java Admin Client that can list topics and this
> talks to a Kafka broker to retrieve the topics. For this case, ACLs
> apply and you will only see the topics you are allowed to access. The
> main drawback of this method is, that there is no command line tool
> for this yet, it is "just" a java api.
>
> When I said "access the Kafka nodes" I meant being able to connect to
> the Kafka brokers port on those machines, that would be enough to use
> the java admin client as described above.
>
> Hope this helps.
>
> Best regards,
> Sönke
>
>
> On Mon, Feb 26, 2018 at 5:25 PM, naresh Goud <nareshgoud.du...@gmail.com>
> wrote:
> > It should require zookeeper connection always, because intern kafka
> brokers
> > will interact with zookeeper for all meta data about topics.
> > But its interesting, how would you give departments to access to kafka
> nodes
> >
> > @Sönke,
> >
> > Could you please shade some light on giving departements access to kafka
> > nodes.? Is it like  departments able to ssh to kafka nodes and run
> describe
> > command? so it will show topics metadata only topics in that node?
> >
> > Apologies, if my question is very basic.
> >
> > Thank you,
> > Naresh
> >
> >
> >
> > Thanks,
> > Naresh
> > www.linkedin.com/in/naresh-dulam
> > http://hadoopandspark.blogspot.com/
> >
> >
> > On Mon, Feb 26, 2018 at 5:30 PM, Reema Chugani <reemachug...@outlook.com
> >
> > wrote:
> >
> >> Hi Sönke,
> >>
> >> Thanks for the info, it is helpful!
> >>
> >> I can modify so that the departments can only access the Kafka nodes
> >> themselves. However how would the consumers connect to the topics then?
> >> Don't the consumer clients require to connect via Zookeeper?
> >>
> >> Thanks,
> >> Reema
> >>
> >> On Fri, Feb 23, 2018 at 10:50 PM, Sönke Liebau <
> soenke.lie...@opencore.com
> >> .invalid<mailto:soenke.lie...@opencore.com.invalid>> wrote:
> >> Hi Reema,
> >>
> >> if your departments have access to Zookeeper then there probably is not
> >> much you can do about them accessing data on other departments topics. I
> >> assume that you have enabled Zookeeper ACLs, but even with those in
> place,
> >> the topic metadata is world readable, so listing topics can be done by
> >> anyone who has access to Zookeeper.
> >>
> >> If your departments can only access the Kafka nodes themselves then the
> >> DESCRIBE action on Topics is I believe what you are looking for,
> without an
> >> ACL in place to grant this, the topic should not be listed in Metadata
> >> responses.
> >>
> >> I hope that helps, if you need more information let us know!
> >>
> >> Best regards,
> >> Sönke
> >>
> >> Am 24.02.2018 06:32 schrieb "Reema Chugani" <reemachug...@outlook.com<
> >> mailto:reemachug...@outlook.com>>:
> >>
> >> Hi,
> >>
> >> I am using Kafka 0.10.2.
> >>
> >> I have multiple topics & consumers set up with ACLS such that consumer
> can
> >> only read from a particular topic. I am wondering how I can prevent a
> >> consumer from accessing metadata in zookeeper about other topics? i.e,
> >> prevent consumers from listing or getting info about topics in the
> cluster.
> >> (Not let marketing dept see the topic names of finance topics.)
> >>
> >> Thanks,
> >> Reema
> >>
> >>
> >>
>
>
>
> --
> Sönke Liebau
> Partner
> Tel. +49 179 7940878
> OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
>
-- 
Thanks,
Naresh
www.linkedin.com/in/naresh-dulam
http://hadoopandspark.blogspot.com/

Reply via email to