Hello Colin, There is a POC PR for this KIP, and it contains most changes we are proposing now.
Best, Yaodong On Tue, Feb 26, 2019 at 6:51 PM Yaodong Yang <yangyaodon...@gmail.com> wrote: > Hello Colin, > > CIL, > > Thanks! > Yaodong > > > On Tue, Feb 26, 2019 at 9:59 AM Colin McCabe <cmcc...@apache.org> wrote: > >> Hi Yaodong, >> >> I don't understand how the proposed API would be used. It talks about >> adding a ConfigResource type for clients and users, but doesn't explain >> what can be done with these. >> > > Sorry for the confusion. I just updated the KIP, and hopefully it will > make it easier for you and other people. Looking forward to your feedback! > > >> In the compatibility section (?) it says "We only add a new way to >> configure the quotas" which suggests that quotas are involved somehow What >> relationship does this have with KIP-257? >> > > Let me give you more context, feel free to correct me if I'm wrong 😁 > > 1. Originally we hit an issue that we can not config client quota through > AdminClient. The only way available for us is directly talk to ZK and > manage quota directly. > > 2. As our client service may not in the same DC as ZooKeeper, there could > be some cross DC communication which is less desirable. > > 3. We deicide to add the quota configuration feature in the AdminClient, > which will perfectly solve this issue for us. > > 4. In addition, we realized that this change can also serve as a way to > config other users or clients configuration in Zookeeper. For instance, if > we have a new client configuration introduced in the future and they need > to be in the Zookeeper as well, we can mange it through the same API. > Therefore, this KIP is renamed to manage users/clients configurations. > Quota management is one use case for this configuration management. > > 5. KIP-257 is also compatible with the current KIP. For instance, if user > want to update a quota for a metric, the client side need to parse it, and > eventually pass in a user or client config to AdminClient. AdminClient will > make sure such configuration changes are applied in the Zookeeper. > > >> best, >> Colin >> >> >> On Fri, Feb 22, 2019, at 15:11, Yaodong Yang wrote: >> > Hi Colin, >> > >> > CIL, >> > >> > Thanks! >> > Yaodong >> > >> > >> > On Fri, Feb 22, 2019 at 10:56 AM Colin McCabe <cmcc...@apache.org> >> wrote: >> > >> > > Hi Yaodong, >> > > >> > > KIP-422 says that it would be good if "applications [could] leverage >> the >> > > unified KafkaAdminClient to manage their user/client configurations, >> > > instead of the direct dependency on Zookeeper." But the KIP doesn't >> talk >> > > about any changes to KafkaAdminClient. Instead, the only changes >> proposed >> > > are to AdminZKClient. But that is an internal class-- we don't need >> a KIP >> > > to change it, and it's not a public API that users can use. >> > > >> > >> > Sorry for the confusion in the KIP. Actually there is no change to >> > AdminZKClient needed for this KIP, we just leverage them to configure >> the >> > properties in the ZK. You can find the details from this PR >> > https://github.com/apache/kafka/pull/6189 >> > >> > As you can see from the PR, we need the client side and server process >> > changes, so I feel like we still need the KIP for this change. >> > >> > >> > > I realize that the naming might be a bit confusing, but >> > > kafka.zk.AdminZKClient and kafka.admin.AdminClient are internal >> classes. >> > > As the JavaDoc says, kafka.admin.AdminClient is deprecated as well. >> The >> > > public class that we would be adding new methods to is >> > > org.apache.kafka.clients.admin.AdminClient. >> > > >> > >> > I agree. Thanks for pointing this out! >> > >> > >> > > best, >> > > Colin >> > > >> > > On Tue, Feb 19, 2019, at 15:21, Yaodong Yang wrote: >> > > > Hello Jun, Viktor, Snoke and Stan, >> > > > >> > > > Thanks for taking time to look at this KIP-422! For some reason, >> this >> > > email >> > > > was put in my spam folder. Sorry about that. >> > > > >> > > > Jun is right, the main motivation for this KIP-422 is to allow >> users to >> > > > config user/clientId quota through AdminClient. In addition, this >> KIP-422 >> > > > also allows users to set or update any config related to a user or >> > > clientId >> > > > entity if needed in the future. >> > > > >> > > > For the KIP-257, I agree with Jun that we should add support for >> it. I >> > > will >> > > > look at the current implementation and update the KIP-422 with new >> > > change. >> > > > >> > > > I will ping this thread once I updated the KIP. >> > > > >> > > > Thanks again! >> > > > Yaodong >> > > > >> > > > On Fri, Feb 15, 2019 at 1:28 AM Viktor Somogyi-Vass < >> > > viktorsomo...@gmail.com> >> > > > wrote: >> > > > >> > > > > Hi Guys, >> > > > > >> > > > > I wanted to reject that KIP, split it up and revamp it as in the >> > > meantime >> > > > > there were some overlapping works I just didn't get to it due to >> other >> > > > > higher priority work. >> > > > > One of the splitted KIPs would have been the quota part of that >> and >> > > I'd be >> > > > > happy if that lived in this KIP if Yaodong thinks it's worth to >> > > > > incorporate. I'd be also happy to rebase that wire protocol and >> > > contribute >> > > > > it to this KIP. >> > > > > >> > > > > Viktor >> > > > > >> > > > > On Wed, Feb 13, 2019 at 7:14 PM Jun Rao <j...@confluent.io> wrote: >> > > > > >> > > > > > Hi, Yaodong, >> > > > > > >> > > > > > Thanks for the KIP. As Stan mentioned earlier, it seems that >> this is >> > > > > > mostly covered by KIP-248, which was originally proposed by >> Victor. >> > > > > > >> > > > > > Hi, Victor, >> > > > > > >> > > > > > Do you still plan to work on KIP-248? It seems that you already >> got >> > > > > pretty >> > > > > > far on that. If not, would you mind letting Yaodong take over >> this? >> > > > > > >> > > > > > For both KIP-248 and KIP-422, one thing that I found missing is >> the >> > > > > > support for customized quota ( >> > > > > > >> > > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-257+-+Configurable+Quota+Management >> > > > > ). >> > > > > > With KIP-257, it's possible for one to construct a customized >> quota >> > > > > defined >> > > > > > through a map of metric tags. It would be useful to support >> that in >> > > the >> > > > > > AdminClient API and the wire protocol. >> > > > > > >> > > > > > Hi, Sonke, >> > > > > > >> > > > > > I think the proposal is to support the user/clientId level quota >> > > through >> > > > > > an AdminClient api. The user can be obtained from any existing >> > > > > > authentication mechanisms. >> > > > > > >> > > > > > Thanks, >> > > > > > >> > > > > > Jun >> > > > > > >> > > > > > On Thu, Feb 7, 2019 at 5:59 AM Sönke Liebau >> > > > > > <soenke.lie...@opencore.com.invalid> wrote: >> > > > > > >> > > > > >> Hi Yaodong, >> > > > > >> >> > > > > >> thanks for the KIP! >> > > > > >> >> > > > > >> If I understand your intentions correctly then this KIP would >> only >> > > > > >> address a fairly specific use case, namely SASL-PLAIN with >> users >> > > > > >> defined in Zookeeper. For all other authentication mechanisms >> like >> > > > > >> SSL, SASL-GSSAPI or SASL-PLAIN with users defined in jaas >> files I >> > > > > >> don't see how the AdminClient could directly create new users. >> > > > > >> Is this correct, or am I missing something? >> > > > > >> >> > > > > >> Best regards, >> > > > > >> Sönke >> > > > > >> >> > > > > >> On Thu, Feb 7, 2019 at 2:47 PM Stanislav Kozlovski >> > > > > >> <stanis...@confluent.io> wrote: >> > > > > >> > >> > > > > >> > This KIP seems to duplicate some of the functionality >> proposed in >> > > > > >> KIP-248 >> > > > > >> > < >> > > > > >> >> > > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-248+-+Create+New+ConfigCommand+That+Uses+The+New+AdminClient >> > > > > >> >. >> > > > > >> > KIP-248 has been stuck in a vote thread since July 2018. >> > > > > >> > >> > > > > >> > Viktor, do you plan on working on the KIP? >> > > > > >> > >> > > > > >> > On Thu, Feb 7, 2019 at 1:27 PM Stanislav Kozlovski < >> > > > > >> stanis...@confluent.io> >> > > > > >> > wrote: >> > > > > >> > >> > > > > >> > > Hey there Yaodong, thanks for the KIP! >> > > > > >> > > >> > > > > >> > > I'm not too familiar with the user/client configurations we >> > > > > currently >> > > > > >> > > allow, is there a KIP describing the initial feature? If >> there >> > > is, >> > > > > it >> > > > > >> would >> > > > > >> > > be useful to include in KIP-422. >> > > > > >> > > >> > > > > >> > > I also didn't see any authorization in the PR, have we >> thought >> > > about >> > > > > >> > > needing to authorize the alter/describe requests per the >> > > > > user/client? >> > > > > >> > > >> > > > > >> > > Thanks, >> > > > > >> > > Stanislav >> > > > > >> > > >> > > > > >> > > On Fri, Jan 25, 2019 at 5:47 PM Yaodong Yang < >> > > > > yangyaodon...@gmail.com >> > > > > >> > >> > > > > >> > > wrote: >> > > > > >> > > >> > > > > >> > >> Hi folks, >> > > > > >> > >> >> > > > > >> > >> I've published KIP-422 which is about adding support for >> > > > > user/client >> > > > > >> > >> configurations in the Kafka Admin Client. >> > > > > >> > >> >> > > > > >> > >> Basically the story here is to allow KafkaAdminClient to >> > > configure >> > > > > >> the >> > > > > >> > >> user >> > > > > >> > >> or client configurations for users, instead of requiring >> users >> > > to >> > > > > >> directly >> > > > > >> > >> talk to ZK. >> > > > > >> > >> >> > > > > >> > >> The link for this KIP is >> > > > > >> > >> following: >> > > > > >> > >> >> > > > > >> >> > > > > >> > > >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97555704 >> > > > > >> > >> >> > > > > >> > >> I'd be happy to receive some feedback about the KIP I >> > > published. >> > > > > >> > >> >> > > > > >> > >> -- >> > > > > >> > >> Best, >> > > > > >> > >> Yaodong Yang >> > > > > >> > >> >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > -- >> > > > > >> > > Best, >> > > > > >> > > Stanislav >> > > > > >> > > >> > > > > >> > >> > > > > >> > >> > > > > >> > -- >> > > > > >> > Best, >> > > > > >> > Stanislav >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> -- >> > > > > >> Sönke Liebau >> > > > > >> Partner >> > > > > >> Tel. +49 179 7940878 >> > > > > >> OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - >> > > Germany >> > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> >
