Hi Edoardo, reading the KIP-170 at this point :
3) API Authorization Given the potential damage that can be caused if this API is used by mistake, it is important that we limit its usage to only authorized users. For this matter, we can take advantage of the existing authorization framework implemented in KIP-11<https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface>. deleteDataBefore() will have the same authorization setting as deleteTopic(). Its operation type is be DELETE and its resource type is TOPIC. it seems to me that the KIP doesn't suggest a policy as you are saying but using the authorizer mechanism with operation = DELETE and resource = TOPIC. Is my understanding right ? Paolo Patierno Senior Software Engineer (IoT) @ Red Hat Microsoft MVP on Azure & IoT Microsoft Azure Advisor Twitter : @ppatierno<http://twitter.com/ppatierno> Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> Blog : DevExperience<http://paolopatierno.wordpress.com/> ________________________________ From: Edoardo Comar <eco...@uk.ibm.com> Sent: Tuesday, September 26, 2017 1:59 PM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API Hi Paolo, our the motivation for suggesting a delete policy as described in KIP-170 was to avoid deletion of topics that have a specific usage (essential to other services). As you say, it would be entirely replaceable by an Authorizer implementation that performs the equivalent check. thanks Edo -------------------------------------------------- Edoardo Comar IBM Message Hub IBM UK Ltd, Hursley Park, SO21 2JN From: Paolo Patierno <ppatie...@live.com> To: "dev@kafka.apache.org" <dev@kafka.apache.org> Date: 26/09/2017 14:10 Subject: Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API Hi Tom, the KIP-170 doesn't propose to use a TopicDeletePolicy as policy classes are meant to be. It's referring to the authorization interface (KIP-11< https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D11-2B-2D-2BAuthorization-2BInterface&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=NQw-765HFBIbCYIZLpYIJrvCaZ4-Ml0L1e4z4wkx8fg&e= >) with operation = DELETE and resource = TOPIC. You know that when a request comes in there is the "authorizer" first which is based on KIP-11 and then the operation is called and then you could have policy. I.e. for create topics, first the "authorizer" is called in the KafkaApis (operation = CREATE, resource = TOPIC) and then the CreateTopicPolicy is applied in the AdminManager. In any case what KIP-170 is proposing is not good. Maybe we could have operation = DELETE and a new resource = MESSAGE/RECORD. What could be useful use cases for having a RecordsDeletePolicy ? Records can't be deleted for a topic name ? Starting from a specific offset ? Paolo Patierno Senior Software Engineer (IoT) @ Red Hat Microsoft MVP on Azure & IoT Microsoft Azure Advisor Twitter : @ppatierno< https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_ppatierno&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=XDNIqoF_Lv9mDGsBvFaRn8bazYxBs3a4lWKrekvRxrw&e= > Linkedin : paolopatierno< https://urldefense.proofpoint.com/v2/url?u=http-3A__it.linkedin.com_in_paolopatierno&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=6dww9uXw2L0IZ3JKqvoxImEA07L5d7ORCtWi3_wxwcw&e= > Blog : DevExperience< https://urldefense.proofpoint.com/v2/url?u=http-3A__paolopatierno.wordpress.com_&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=RcYQ4n1Fcl2vHgy2Aj_J9wUO3ZUaFgl8vCZ4slCD5ro&e= > ________________________________ From: Tom Bentley <t.j.bent...@gmail.com> Sent: Tuesday, September 19, 2017 8:55 AM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API Hi Paolo, Thanks for the KIP. What errors can be anticipated for the API method (See https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_KAFKA-2D5445-29-3F&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=EaM1W6ZCxDBgKxKpVthbGgK2riVb9EESHu3EfeLMvOQ&e= It's not mentioned in KIP-107, but maybe now is a good time to consider whether there should be some kind of DeleteRecordsPolicy, like there is for creating topics? Note: KIP-170 proposes a TopicDeletePolicy, but I don't think you could use that as currently proposed, because it would be unable to distinguish the deletion of an entire topic from the deletion of some records within a topic. Thanks again, Tom On 18 September 2017 at 21:06, Dong Lin <lindon...@gmail.com> wrote: > +1 (non-binding) > > On Mon, Sep 18, 2017 at 1:04 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > +1 > > > > On Mon, Sep 18, 2017 at 9:19 AM, Paolo Patierno <ppatie...@live.com> > > wrote: > > > > > Hi devs, > > > > > > > > > I'd like to start a discussion around adding the delete records > > operation, > > > already available at protocol level and in the "legacy" Admin Client in > > > Scala, to the "new" Admin Client API in Java. > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=_ojBO1P6pVw7qcLfB7T9wAgFWqhfd-JswSnJ8hRtoEs&e= > > > 204+%3A+adding+records+deletion+operation+to+the+new+Admin+Client+API > > > > > > > > > Thanks, > > > > > > > > > Paolo Patierno > > > Senior Software Engineer (IoT) @ Red Hat > > > Microsoft MVP on Azure & IoT > > > Microsoft Azure Advisor > > > > > > Twitter : @ppatierno< https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_ppatierno&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=XDNIqoF_Lv9mDGsBvFaRn8bazYxBs3a4lWKrekvRxrw&e= > > > > Linkedin : paolopatierno< https://urldefense.proofpoint.com/v2/url?u=http-3A__it.linkedin.com_in_paolopatierno&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=6dww9uXw2L0IZ3JKqvoxImEA07L5d7ORCtWi3_wxwcw&e= > > > > Blog : DevExperience< https://urldefense.proofpoint.com/v2/url?u=http-3A__paolopatierno.wordpress.com_&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=7f9IdD9OiWfZojnX4FN8OUimAl_0Q72yxvE8Nz_QdpU&s=RcYQ4n1Fcl2vHgy2Aj_J9wUO3ZUaFgl8vCZ4slCD5ro&e= > > > > > > > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
