Hello, Viktor. Do you want to implement the exact approach as it described in the current KIP? Or you have another proposal on how it has to be implemented?
I abandoned this KIP due to lack of interest from community. Guess we can collaborate during implementation. > 7 сент. 2020 г., в 13:13, Viktor Somogyi-Vass <viktorsomo...@gmail.com> > написал(а): > > Hi folks, > > It's been a few days since I last pinged and nobody replied so I assume > that this KIP is abandoned and I can take this over (but please let me know > if it's not). I will keep the current version of the KIP and move it to a > sub-page if it's ever needed. > > Thanks, > Viktor > > On Fri, Aug 28, 2020 at 11:35 AM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> wrote: > >> Hi folks, >> >> I have a use-case and a non-trivial implementation with Apache Atlas for >> this KIP and since this kip seems to be dormant for a while now, I'd take >> it over and drive it to completion if you don't mind. >> The current state of the PoC can be found on my fork at >> https://github.com/viktorsomogyi/kafka/tree/atlas-audit-impl. >> >> Viktor >> >> On Wed, Jan 29, 2020 at 9:32 AM Игорь Мартемьянов <ledost...@gmail.com> >> wrote: >> >>> Hello, Nikolai. >>> >>> >>> >>>> Can you, please, make it more specific? >>> >>>> Why does a business want to have this information? >>> >>> It is very demanded for security department to know who/when/where create >>> or edit ACL settings. The same situation about topics. >>> >>> >>> >>>> What are the use-cases for it? >>> >>> This KIP are able to help catching some unauthorized changes of cluster >>> resources or save history these changes. >>> >>> >>> >>>> Who will be analyzing these events and how? >>> >>> Events could be analyzed by some security department or developers who >>> builds applications that can change cluster resources. >>> >>> >>> >>>> Why it’s not convenient to implement it with third-party tools? >>> >>> It is imposible to my mind getting detailed information about these events >>> through third-party tools, because we don`t have the ability to catch >>> these >>> events outside Kafka core. >>> >>> >>> >>>> 1. `audit` name sounds too general for me. How about `onEvent`? >>> >>> Thanks, good point. I`ll change the method name. >>> >>> >>> >>>> 2. Should we introduce a special marker interface for audit events? >>> >>>> `AuditableRequest`, for example? >>> >>> No, we shouldn`t. We are going to create these events if we process >>> special >>> request in KafkaApis class (ApiKeys.CREATE_TOPICS, ApiKeys.DELETE_TOPICS, >>> ApiKeys.CREATE_ACLS, ApiKeys.DELETE_ACLS, ApiKeys.DESCRIBE_ACLS, >>> ApiKeys.ALTER_CONFIGS) >>> >>> >>> >>>>> public interface AuditEvent<T extends AbstractRequest> { >>> >>>>> String guid(); >>> >>> >>> >>>> Where this `guid` comes from? >>> >>> Guid generated automatically when the audit event is created. >>> >>> >>> >>>> Will it be the same on each node that receives an auditable event? >>> >>> Event is creating while request that changes cluster resource is processed >>> and will destroyed after the prosessing will finished on each node. >>> >>> >>> >>>> Do we have `guid` for any extensions of `AbstractRequest`? >>> >>> No, we don`t. >>> >>> >>> >>>> If this field is `guid` why do we format this as a String on the API >>> level? >>> >>> Thanks for the point. >>> >>> I have changed event interface recently. >>> >>> >>> >>>> Can you, please, add a full list of proposed new configuration >>> properties >>> and >>> >>>> examples for each to clarify your intentions? >>> >>> Yes, I have added some examples of new configuration properties. >>> >>> ср, 29 янв. 2020 г., 10:23 Владимир Беруненко <vvberune...@gmail.com>: >>> >>>> Hi Nikolai! >>>> >>>>> Can you, please, make it more specific? >>>> Why does a business want to have this information? >>>>> What are the use-cases for it? >>>>> Who will be analyzing these events and how? >>>>> Why it’s not convenient to implement it with third-party tools? >>>> >>>> This is required by the guys from information security to detect >>> potential >>>> threats of violation the rules for providing access to the data layer. >>>> Analysis in the audit system is usually based on identifying >>>> uncharacteristic integrations. The key feature of the audit, is that >>>> cluster administrators do not have access to modification of audit >>> events. >>>> Third-party tools are great for data analysis, but its not good idea to >>> use >>>> it for audit events collection. >>>> >>>>> It’s not clear for me where and when AuditEvents will be sent? >>>> Who will be the receiver of events? >>>> >>>> In my opinion, sending an audit event should be initiated when the >>> broker >>>> receives a request that matches the audit parameters. Each organization >>> has >>>> its own receiver system, so a common interface is required that the >>>> organization’s development team can implement to integrate with their >>> audit >>>> system. >>>> >>>> Best wishes, Vladimir >>>> >>>> Hello, Igor. >>>>> >>>>> Thanks for the KIP. >>>>> I have a couple of comments for it: >>>>> >>>>>> Motivation >>>>>> It is highly demanded in most businesses to have the ability of >>>>> obtaining audit information in case someone changes cluster >>> configuration >>>>> (like creation/deletion/modify/description of any topic or ACLs). >>>>> >>>>> Can you, please, make it more specific? >>>>> Why does a business want to have this information? >>>>> What are the use-cases for it? >>>>> Who will be analyzing these events and how? >>>>> Why it’s not convenient to implement it with third-party tools? >>>>> >>>>> It’s not clear for me where and when AuditEvents will be sent? >>>>> Who will be the receiver of events? >>>>> >>>>>> void audit<T extends AbstractRequest>(AuditEvent<T> event); >>>>> >>>>> 1. `audit` name sounds too general for me. How about `onEvent`? >>>>> 2. Should we introduce a special marker interface for audit events? >>>>> `AuditableRequest`, for example? >>>>> >>>>>> public interface AuditEvent<T extends AbstractRequest> { >>>>>> String guid(); >>>>> >>>>> Where this `guid` comes from? >>>>> Will it be the same on each node that receives an auditable event? >>>>> Do we have `guid` for any extensions of `AbstractRequest`? >>>>> If this field is `guid` why do we format this as a String on the API >>>> level? >>>>> >>>>>> One or more of AuditExtension's implementations can be configured >>> via >>>>> the configuration audit.extension.classes >>>>> >>>>> Can you, please, add a full list of proposed new configuration >>> properties >>>>> and examples for each to clarify your intentions? >>>>> >>>>> >>>>>> 24 янв. 2020 г., в 18:12, Alexander Dunayevsky < >>>> a.a.dunayev...@gmail.com> >>>>> написал(а): >>>>>> >>>>>> Hello Igor, >>>>>> >>>>>> Thanks for your KIP 🙌🏽 >>>>>> It would be great to adopt this functionality and getting the best >>> of >>>>>> tracking cluster activity. >>>>>> >>>>>> +1 vote from me >>>>>> >>>>>> Cheers, >>>>>> Alex Dunayevsky >>>>>> >>>>>> >>>>>> On Fri, 24 Jan 2020, 15:35 Игорь Мартемьянов, <ledost...@gmail.com> >>>>> wrote: >>>>>> >>>>>>> Motivation: >>>>>>> >>>>>>> >>>>>>> *It is highly demanded in most businesses to have the ability of >>>>> obtaining >>>>>>> audit information in case someone changes cluster configuration >>> (like >>>>>>> creation/deletion/modify/description of any topic or ACLs).We may >>> add >>>>> this >>>>>>> ability. Since audit requirements are so broad, it's impractical to >>>>> support >>>>>>> all of them.Hence we have to provide ability for users to plug >>>> resources >>>>>>> helping to achieve required capabilities.* >>>>>>> >>>>>>> >>>>>>> >>>>> >>>> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-567%3A+Kafka+Cluster+Audit >>>>>>> >>>>>>> >>>>>>> пт, 24 янв. 2020 г., 17:29 Игорь Мартемьянов <ledost...@gmail.com >>>> : >>>>>>> >>>>>>>> Hello there. >>>>>>>> Please review this KIP. >>>>>>>> Thanks. >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>> >>
