Bhavesh I am from the Apache Ranger team, so let me answer the questions pertaining to Ranger…
> 1) Is there any performance impact with Brokers/Producer/Consumer while using > Apache Ranger ? As such Ranger overhead is negligible. Specifically for Kafka, we did some more optimization for the unique usage pattern of Kafka consumers and publishers. Considering Kafka itself is optimized for super throughput, expected some overhead penalty. We don’t have the exact numbers yet. Would you be willing to help us some run some real-world scenarios to determine the overhead? If so, we can discuss it in the Ranger mailing list. Thanks. > 2) Is Audit log really useful out-of-box ? or let me know what sort of > reports you run on audit logs (e.g pumping Apache Ranger audit log into any > other system for reporting purpose). If you are referring to Ranger audit logs, then it is configurable. OOTB, it has options to go to HDFS, SOLR and/or DB. We have extensions to send it to Kafka and any Log4J appender. Audit logs has its own purpose, from compliance requirements to operation monitoring usage to anomaly detection. Ranger portal has its own UI for querying. Since the audits are stored in normalized format, you can write your own reports on top of it. Bosco On 11/3/15, 9:55 PM, "Bhavesh Mistry" <mistry.p.bhav...@gmail.com> wrote: >+ Kafka Dev team to see if Kafka Dev team know or recommend any Auth >engine for Producers/Consumers. > >Thanks, > >Bhavesh > >Please pardon me, I accidentally send previous blank email. > >On Tue, Nov 3, 2015 at 9:52 PM, Bhavesh Mistry ><mistry.p.bhav...@gmail.com> wrote: >> On Sun, Nov 1, 2015 at 11:15 PM, Bhavesh Mistry >> <mistry.p.bhav...@gmail.com> wrote: >>> HI All, >>> >>> Have any one used Apache Ranger as Authorization Engine for Kafka Topic >>> creation, consumption (read) and write operation on a topic. I am looking >>> at having audit log and regulating consumption/ write to particular topic >>> (for example, having production environment access does not mean that anyone >>> can run console consumer etc on particular topic. Basically, regulate who >>> can read/write to a topic as first use case). >>> >>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide#ApacheRanger0.5-UserGuide-KAFKA >>> >>> If you have used Apache Ranger in production, I have following question: >>> 1) Is there any performance impact with Brokers/Producer/Consumer while >>> using Apache Ranger ? >>> 2) Is Audit log really useful out-of-box ? or let me know what sort of >>> reports you run on audit logs (e.g pumping Apache Ranger audit log into any >>> other system for reporting purpose). >>> >>> Please share your experience using Kafka with any other Authorization engine >>> if you are not using Apache Ranger (This is based on >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface). >>> >>> Thanks and looking forward to hear back from Apache Kafka Community members. >>> >>> Thanks, >>> >>> Bhavesh
