Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-12 Thread Tom Bentley
Hi Ismael, Thanks for your comments. Regarding the loophole issue, keep in mind that the alter topics > authorization would still be required, so I don't think it's an issue > It could be an issue for people trying to provide a Kafka-as-a-Service offering, couldn't it? I mean if the providers ar

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-12 Thread Ismael Juma
Hi Tom, OK, I suggest not calling any policy then. We can do a separate KIP for overhauling topic policies so that they work with all operations for 1.1.0. Regarding the loophole issue, keep in mind that the alter topics authorization would still be required, so I don't think it's an issue. Users

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-12 Thread Tom Bentley
2. About using the create topics policy, I'm not sure. Aside from the > naming issue, there's also the problem that the policy doesn't know if a > creation or update is taking place. This matters because one may not want > to allow the number of partitions to be changed after creation as it > affec

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-11 Thread Ismael Juma
>> >> > > Then if an API for decreasing were ever implemented it could be >> >> > > removePartitions() with a RemovedPartitions class if necessary. >> >> > > >> >> > > Cheers, >> >> > > >> >> > > Tom >> >> > > >

Re: Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Kamal Chandraprakash
t...@gmail.com] > Sent: Friday, September 08, 2017 11:28 AM > To: dev@kafka.apache.org > Subject: EXT: Re: [DISCUSS] KIP-195: AdminClient.increasePartitions > > I've gone with createPartitions() and NewPartitions. > > To answer my own question about the replication factor

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Ismael Juma
8 September 2017 at 12:13, Paolo Patierno > >> wrote: > >> > > > >> > > > My 2 cents about naming ... > >> > > > > >> > > > > >> > > > PartitionCount or NumPartitions sound better to me providing an > >> > > "absolute" > >> > > > value

RE: Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Selvamalar Arunkumar
, September 08, 2017 11:28 AM To: dev@kafka.apache.org Subject: EXT: Re: [DISCUSS] KIP-195: AdminClient.increasePartitions I've gone with createPartitions() and NewPartitions. To answer my own question about the replication factor, the current code (specifically AdminUtils.addPartitions()) just use

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Tom Bentley
t; > > > PartitionCount or NumPartitions sound better to me providing an >> > > "absolute" >> > > > value (as today the kafka-topics script work) for an idempotent >> > operation >> > > > while the NumPartitionsIncrease name s

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Tom Bentley
t; > > > > > > > Paolo Patierno > > > > Senior Software Engineer (IoT) @ Red Hat > > > > Microsoft MVP on Windows Embedded & IoT > > > > Microsoft Azure Advisor > > > > > > > > Twitter : @ppatierno<http://twitter.com/ppati

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Ted Yu
at > > > Microsoft MVP on Windows Embedded & IoT > > > Microsoft Azure Advisor > > > > > > Twitter : @ppatierno<http://twitter.com/ppatierno> > > > Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> > > > Blog : DevExperience<http://paolopatierno.wordpre

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Ismael Juma
aolopatierno<http://it.linkedin.com/in/paolopatierno> > > Blog : DevExperience<http://paolopatierno.wordpress.com/> > > > > > > > > From: Tom Bentley > > Sent: Friday, September 8, 2017 9:39 AM > > To: dev@kafka.apache

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Tom Bentley
opatierno<http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience<http://paolopatierno.wordpress.com/> > > > > From: Tom Bentley > Sent: Friday, September 8, 2017 9:39 AM > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-195: AdminClien

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Paolo Patierno
t; From: Tom Bentley Sent: Friday, September 8, 2017 9:39 AM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-195: AdminClient.increasePartitions Hi Ismael, Thanks for the comments. My bad for not noticing the custom assignment requirement. The current >

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Tom Bentley
Hi Ismael, Thanks for the comments. My bad for not noticing the custom assignment requirement. The current > proposal has the following example (I updated it a little so that 2 > partitions are added): > > increasePartitionCount(4, asList(asList(1, 2), asList(2, 3)) > > Why not simply provide the

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Ismael Juma
My bad for not noticing the custom assignment requirement. The current proposal has the following example (I updated it a little so that 2 partitions are added): increasePartitionCount(4, asList(asList(1, 2), asList(2, 3)) Why not simply provide the assignment? For example, if you want to add 2 p

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-08 Thread Tom Bentley
Hi Ted and Ismael, Thanks for the comments. Ted, I've fixed the KIP for your comments. Ismael, see responses inline: On 8 September 2017 at 02:00, Ismael Juma wrote: > Thanks Tom. Thanks for the KIP. A few comments: > > 1. Does the `PartitionCount` class still make sense given that the method

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-07 Thread Ismael Juma
Thanks Tom. Thanks for the KIP. A few comments: 1. Does the `PartitionCount` class still make sense given that the method can only increase the number of partitions now? 2,. In `NewTopic`, we have `numPartitions`. Should we keep using that variant and call the method `increaseNumPartitions`? 3. Si

Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-07 Thread Ted Yu
Tom: Looks good overall. bq. for the topic from the AlterPartitionCountsResult Please align the name of Result with current proposal. Please also fill in JIRA number when you have it. On Thu, Sep 7, 2017 at 9:38 AM, Tom Bentley wrote: > As suggested by Ismael, I've factored the increasePartit

[DISCUSS] KIP-195: AdminClient.increasePartitions

2017-09-07 Thread Tom Bentley
As suggested by Ismael, I've factored the increasePartitionCounts() API out of KIP-179 out into a separate KIP which hopefully can progress more quickly. https://cwiki.apache.org/confluence/display/KAFKA/KIP-195%3A+AdminClient.increasePartitions If you've looked at KIP-179 in the last few days th