Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2018-12-07 Thread Tom Bentley
t; > > interface AlterTopicRequest extends AbstractRequestMetadata { > > > > > > // updated topic configs, or null if not changed > > > > > > map updatedConfigs(); > > > > > > // proposed replica assignment in this requ

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2018-12-13 Thread Tom Bentley
with the proposal? Or do you have use cases where the policy maker is more interested in what the request is changing? Kind regards, Tom On Fri, 7 Dec 2018 at 08:41, Tom Bentley wrote: > Hi Anna and Mickael, > > Sorry for remaining silent on this for so long. I should have time to lo

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2019-01-09 Thread Tom Bentley
On Thu, 13 Dec 2018 at 18:11, Tom Bentley wrote: > Hi Anna, > > Firstly, let me apologise again about having missed your previous emails > about this. > > Thank you for the feedback. You raise some valid points about ambiguity. > The problem with pulling the metad

Re: [ANNOUNCE] New Committer: Vahid Hashemian

2019-01-16 Thread Tom Bentley
Congratulations Vahid! On Wed, Jan 16, 2019 at 10:13 AM Edoardo Comar wrote: > Bravo Vahid!!! > > Edo > > > > From: Mickael Maison > To: Users > Cc: dev > Date: 16/01/2019 08:48 > Subject:Re: [ANNOUNCE] New Committer: Vahid Hashemian > > > > Congratulations Vahid! Well don

Re: [DISCUSS] Kafka 2.2.0 in February 2018

2019-01-18 Thread Tom Bentley
Hi Matthias, Hopefully we can get the work for KIP 183 merged too. Thanks, Tom On Fri, 18 Jan 2019, 19:33 Matthias J. Sax Just a quick update on the release. > > > We have 22 KIP atm: > > 81, 207, 258, 289, 313, 328, 331, 339, 341, 351, 359, 361, 367, 368, > 371, 376, 377, 380, 386, 393, 394,

Zookeeper dependency

2018-03-14 Thread Tom Bentley
I saw a comment in https://issues.apache.org/jira/browse/KAFKA-6598 about the possibility of removing Kafka's dependency on Zookeeper. While I'm sure the promised KIP will have a lot of the interesting technical details, I was wondering if anyone was able to shed any light on this in terms of what

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

2018-03-19 Thread Tom Bentley
anisms. > > best, > Colin > > > On Fri, Dec 22, 2017, at 06:48, Tom Bentley wrote: > > Hi Steven, > > > > I must admit that I didn't really considered that option. I can see how > > attractive it is from your perspective. In practice it would come w

[DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-25 Thread Tom Bentley
Hi, I'd like to start a discussion for KIP-201. The basic point is that new AdminClient APIs for modifying topics should have a configurable policy to allow the administrator to veto the modifications. Just adding a ModifyTopicPolicy would make for awkwardness by having separate policies for creat

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-25 Thread Tom Bentley
Hi Ted, Thanks for the feedback! bq. topic.action.policy.class.name > > Since the policy would cover more than one action, how about using actions > for the second word ? > Good point, done. > For TopicState interface, the abstract modifier for its methods are not > needed. > Fixed. bq. KIP-

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-25 Thread Tom Bentley
Hi Mickael, Thanks for the reply. Thanks for the KIP. Is this meant to superseed KIP-170 ? > If so, one of our key requirements was to be able to access the > topics/partitions list from the policy, so an administrator could > enforce a partition limit for example. > It's not meant to replace KI

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-25 Thread Tom Bentley
Hi Ismael, On 25 September 2017 at 17:51, Ismael Juma wrote: > We don't have this policy today for what it's worth. > Thanks for the clarification. On re-reading I realise I misinterpreted Guozhang Wang's suggestion when 1.0.0 was first mooted: > Just to clarify, my proposal is that moving fo

Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API

2017-09-26 Thread Tom Bentley
Hi Edoardo and Paolo, On 26 September 2017 at 14:10, Paolo Patierno wrote: > What could be useful use cases for having a RecordsDeletePolicy ? Records > can't be deleted for a topic name ? Starting from a specific offset ? > I can imagine some users wanting to prohibit using this API completel

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-26 Thread Tom Bentley
the delete topic response > (to tell why it's failed). > > I'm happy if you incorporate the enhancements to create/alter that allow a > > check against the cluster metadata > and leave out - to anther KIP, or maybe I'll rewrite 170 the changes to > delete. > > thanks > Edo > >

Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API

2017-09-26 Thread Tom Bentley
re Engineer (IoT) @ Red Hat > Microsoft MVP on Azure & IoT > Microsoft Azure Advisor > > Twitter : @ppatierno<http://twitter.com/ppatierno> > Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience<http://paolopatierno.wordpress.com/&g

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-26 Thread Tom Bentley
Hi Edoardo, what about a single method in ClusterState > > interface ClusterState { > public Map topicsState(); > > } > > which could return a read-only snapshot of the cluster metadata ? > Sure that would work too. A concern with that is that we end up allocating a potentially r

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-27 Thread Tom Bentley
icy.validate method > would just lazily tap into MetadataCache. No need for big upfront > allocations. > > ciao, > Edo > -- > > Edoardo Comar > > IBM Message Hub > > IBM UK Ltd, Hursley Park, SO21 2JN > >

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-09-27 Thread Tom Bentley
Hi Ismael, Thanks for looking at the KIP and explaining the thinking behind the original API. Looking at the updated KIP, I notice that we actually have a > TopicDeletionPolicy with a separate config. That seems to be a halfway > house. Not sure about that. > I can certainly see that point of vi

Re: [VOTE] KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule

2017-09-27 Thread Tom Bentley
+1 (nonbinding) On 27 September 2017 at 16:10, Manikumar wrote: > Hi All, > > I'd like to start the vote on KIP-203. Details are here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 203%3A+Add+toLowerCase+support+to+sasl.kerberos.principal.to.local+rule > > Thanks, >

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-09-28 Thread Tom Bentley
I'm starting to think about KIP-179 again. In order to have more manageably-scoped KIPs and PRs I think it might be worth factoring-out the throttling part into a separate KIP. Wdyt? Keeping the throttling discussion in this thread for the moment... The throttling behaviour is currently spread ac

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-10-02 Thread Tom Bentley
e throttle API to be useful for a broad range of use cases, rather than being too narrowly focussed on what's needed by the existing CLI tools. Thanks, Tom On 28 September 2017 at 17:22, Tom Bentley wrote: > I'm starting to think about KIP-179 again. In order to have more

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-10-02 Thread Tom Bentley
we > > > decided to introduce policies incrementally). > > > > > > The main reason why I think the original approach doesn't work well is > > > that there is no direct mapping between an operation and the policy. > That > > > is, we initially thought

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-10-04 Thread Tom Bentley
> number of partitions). > > thanks > Edo > -- > > Edoardo Comar > > IBM Message Hub > > IBM UK Ltd, Hursley Park, SO21 2JN > > > > From: Tom Bentley > To: dev@kafka.apache.org > Date: 02/10/2017 15:15 > Subject:Re: [

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-10-04 Thread Tom Bentley
ors to clients is much less useful > 2. Testing policies is much easier with `validateOnly` > > Ismael > > On Wed, Oct 4, 2017 at 9:20 AM, Tom Bentley wrote: > > > Thanks Edoardo, > > > > I've added that motivation to the KIP. > > >

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-10-05 Thread Tom Bentley
* they were explicitly requested by the client. */ boolean generatedReplicaAssignments(); } Thoughts? On 4 October 2017 at 11:06, Tom Bentley wrote: > Good point. Then I guess I can do those items too. I would also need to do > the same changes for DeleteRecordsRequest and

Re: [DISCUSS] KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change

2017-10-05 Thread Tom Bentley
Hi Colin, Is it really true that "the period when the offset is unavailable should be brief"? I'm thinking about a producer with acks=1, so the old leader returns the ProduceResponse immediately and then is replaced before it can sent a FetchResponse to any followers. The new leader is then waitin

Re: [DISCUSS] KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change

2017-10-06 Thread Tom Bentley
Thanks Colin, it makes sense now, it was the HWM party I was missing. Cheers, Tom On 6 Oct 2017 6:44 pm, "Colin McCabe" wrote: On Thu, Oct 5, 2017, at 12:06, Tom Bentley wrote: > Hi Colin, > > Is it really true that "the period when the offset is unavailable should >

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2017-10-09 Thread Tom Bentley
anks, Tom On 5 October 2017 at 12:41, Tom Bentley wrote: > I'd like to raise a somewhat subtle point about how the proposed API > should behave. > > The current CreateTopicPolicy gets passed either the request partition > count and replication factor, or the requested assig

[VOTE] KIP-201: Rationalising policy interfaces

2017-10-11 Thread Tom Bentley
I would like to start a vote on KIP-201, which proposes to replace the existing policy interfaces with a single new policy interface that also extends policy support to cover new and existing APIs in the AdminClient. https://cwiki.apache.org/confluence/display/KAFKA/KIP-201%3A+Rationalising+Policy

Re: [DISCUSS] KIP-59 : Proposal for a kafka broker command

2017-10-16 Thread Tom Bentley
Hi Jayesh, Thanks, for the KIP. I few questions/points: 1. Could you elaborate on the motivation a little? Currently it seems to boil down to "Kafka doesn't have this, yet", but that's not, in itself, a reason to add it. What can't be done without this change? 2. The second bullet in the "Broker

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-17 Thread Tom Bentley
Hi Clebert, The motivation section is written as more of a summary and doesn't really give any motivation for this change. Can you explain why it would be beneficial for Kafka to have this change? For example, if you have use cases where the current way of instantiating a producer, consumer or adm

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-10-20 Thread Tom Bentley
9+-+Change+ReassignPartitionsCommand+to+use+AdminClient Thanks, Tom On 2 October 2017 at 13:15, Tom Bentley wrote: > One question I have is about whether/how to scope throttling to a > reassignment. Currently throttles are only loosely associated with > reassignment: You can start a reass

Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API

2017-10-23 Thread Tom Bentley
48 AM, Paolo Patierno < > > > > > ppatie...@live.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > maybe we want to start w

Re: Please subscribe me in your updates

2017-10-23 Thread Tom Bentley
As detailed at https://kafka.apache.org/contact, to subscribe, send an email to dev-subscr...@kafka.apache.org < dev-subscr...@kafka.apache.org>. On 22 October 2017 at 15:15, Veeramani S wrote: > >

Re: [VOTE] KIP-201: Rationalising policy interfaces

2017-10-25 Thread Tom Bentley
s to the protocol) it now > supersedes KIP-170 > > +1 non-binding > -- > > Edoardo Comar > > IBM Message Hub > > IBM UK Ltd, Hursley Park, SO21 2JN > > > > From: Tom Bentley > To: dev@kafka.apache.org &g

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-10-25 Thread Tom Bentley
If there are no further comments, I will start a vote on this next week. Thanks, Tom On 20 October 2017 at 08:33, Tom Bentley wrote: > Hi, > > I've made a fairly major update to KIP-179 to propose APIs for setting > throttled rates and throttled replicas with the abilit

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-11-01 Thread Tom Bentley
sCommand+to+use+AdminClient Thanks, Tom On 25 October 2017 at 10:33, Tom Bentley wrote: > If there are no further comments, I will start a vote on this next week. > > Thanks, > > Tom > > On 20 October 2017 at 08:33, Tom Bentley wrote: > >> Hi, >> >> I

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-11-02 Thread Tom Bentley
Hi Steven, I notice you've renamed the template's "Rejected Alternatives" section to "Other Alternatives", suggesting they're not rejected yet (or, if you have rejected them, I think you should give your reasons). Personally, I'd like to understand the arguments against simply replacing KafkaFutu

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

2017-11-02 Thread Tom Bentley
these APIs, Kafka can be used for two broad classes of > >> application: > >> > > > >> > > ** Building real-time streaming data pipelines that reliably get > data > >> > between > >> > > systems or applications. > >> >

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

2017-11-06 Thread Tom Bentley
Hi Jorge, Thanks for the KIP. A few initial comments: 1. The AdminClient doesn't have any API like `listConsumerGroups()` currently, so in general how does a client know the group ids it is interested in? 2. Could you fill in the API of DescribeConsumerGroupResult, just so everyone knows exactly

[VOTE] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-11-07 Thread Tom Bentley
Hi, I would like to start a vote on KIP-179 which would add an AdminClient API for partition reassignment and interbroker replication throttling. https://cwiki.apache.org/confluence/display/KAFKA/KIP-179+-+Change+ReassignPartitionsCommand+to+use+AdminClient Thanks, Tom

Re: [ANNOUNCE] New committer: Onur Karaman

2017-11-07 Thread Tom Bentley
Well done Onur. On 7 November 2017 at 06:52, Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Congratulations Onur!! > On Tue, 7 Nov 2017 at 06:30, Jaikiran Pai > wrote: > > > Congratulations Onur! > > > > -Jaikiran > > > > > > On 06/11/17 10:54 PM, Jun Rao wrote: > > > Hi, ever

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

2017-11-07 Thread Tom Bentley
t; > A quick question: I think we do not yet have the `list consumer > groups` > > > > func as in the old AdminClient. Without this `describe group` given > the > > > > group id would not be very useful. Could you include this as well in > > your > > >

Re: [VOTE] KIP-201: Rationalising policy interfaces

2017-11-07 Thread Tom Bentley
] > > Stephane Maarek | Developer > > +61 416 575 980 > steph...@simplemachines.com.au > simplemachines.com.au > Level 2, 145 William Street, Sydney NSW 2010 > > On 25 October 2017 at 19:45, Tom Bentley wrote: > > > It's been two weeks since I started the

Re: [DISCUSS] Kafka 2.0.0 in June 2018

2017-11-09 Thread Tom Bentley
Hi Stephane, I think the version number rules are based on semantic versioning, so Kafka can't remove even deprecated APIs in a minor release (it is a breaking change, after all). Therefore until Kafka 2.0 we will have to carry the weight of the deprecated APIs, and Java 7. Cheers, Tom On 9 N

0.11 git tags

2017-11-10 Thread Tom Bentley
I just noticed that although every other release is tagged in git there are no tags for 0.11.0.0 and 0.11.0.1. Did something go wrong? Please can they be tagged retrospectively? Cheers, Tom

Re: 0.11 git tags

2017-11-10 Thread Tom Bentley
Not when I run `git tag` locally though. On 10 November 2017 at 12:44, Ismael Juma wrote: > They seem to be there: > > https://github.com/apache/kafka/tree/0.11.0.0 > https://github.com/apache/kafka/tree/0.11.0.1 > > Ismael > > On Fri, Nov 10, 2017 at 12:41 PM, Tom B

Re: 0.11 git tags

2017-11-10 Thread Tom Bentley
it fetch --tags ? > > > > On Fri, Nov 10, 2017 at 12:54 PM, Tom Bentley > > wrote: > > > Not when I run `git tag` locally though. > > > > > > On 10 November 2017 at 12:44, Ismael Juma wrote: > > > > > >> They seem to be there: >

Re: [VOTE] KIP-201: Rationalising policy interfaces

2017-11-22 Thread Tom Bentley
want the policy > to be > > > implemented using Scala ? (like the Authorizer) > > > It's usually not best practice to mix in scala / java code. > > > > > > Thanks! > > > Stephane > > > > > > Kind regards

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

2017-11-22 Thread Tom Bentley
Hi Rajini, Just to clarify, are you proposing this as a way to configure interbroker throttling/quotas? I don't think you are, just wanted to check (since KIP-179 proposes a different mechanism for setting them which supports their automatic removal). Cheers, Tom On 22 November 2017 at 18:28, R

Re: [VOTE] KIP-201: Rationalising policy interfaces

2017-12-01 Thread Tom Bentley
I have a PR for this (https://github.com/apache/kafka/pull/4281) in case anyone wants to look at the implementation in detail, but right now this KIP still lacks any committer votes. Cheers, Tom On 22 November 2017 at 17:32, Tom Bentley wrote: > Hi everyone, > > I just wanted to hig

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-12-01 Thread Tom Bentley
> > > > able to decide without this discussion. > > > > > > > > I see two major factors to decide: > > > > - how soon will KIP-118 (drop support of java 7) be implemented? > > > > - for which reasons do we drop backwards compatibility for p

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-12-04 Thread Tom Bentley
i.apache.org/confluence/display/KAFKA/KIP- > 218%3A+Make+KafkaFuture.Function+java+8+lambda+compatible > > Thanks for your patience, > > >Steven > > > Op vr 1 dec. 2017 om 11:55 schreef Tom Bentley : > > > Hi Steven, > > > > I'm particularly interes

Re: [DISCUSS]KIP-235 DNS alias and secured connections

2017-12-05 Thread Tom Bentley
Hi Jonathan, It might be worth mentioning in the KIP that this is necessary only for *Kerberos* on SASL, and not other SASL mechanisms. Reading the JIRA it makes sensem, but I was confused up until that point. Cheers, Tom On 5 December 2017 at 17:53, Skrzypek, Jonathan wrote: > Hi, > > I woul

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-12-05 Thread Tom Bentley
eassignments to be added while some are already running. API methods would then be provided to discover all the currently running reassignments, determine if a reassignment is still running etc. Cheers, Tom On 1 November 2017 at 10:20, Tom Bentley wrote: > This thread has been very quiet for

Re: [VOTE] KIP-179: Change ReassignPartitionsCommand to use AdminClient

2017-12-05 Thread Tom Bentley
I am withdrawing this KIP. See the [DISCUSS] thread for the reasons why. On 7 November 2017 at 08:15, Tom Bentley wrote: > Hi, > > I would like to start a vote on KIP-179 which would add an AdminClient API > for partition reassignment and interbroker replication throttlin

[DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-06 Thread Tom Bentley
Hi, This is still very new, but I wanted some quick feedback on a preliminary KIP which could, I think, help with providing an AdminClient API for partition reassignment. https://cwiki.apache.org/confluence/display/KAFKA/KIP-236%3A+Interruptible+Partition+Reassignment I wasn't sure whether to st

Re: [VOTE] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-12-08 Thread Tom Bentley
+1 On 8 December 2017 at 18:34, Ted Yu wrote: > +1 > > On Fri, Dec 8, 2017 at 3:49 AM, Steven Aerts > wrote: > > > Hello everybody, > > > > > > I think KIP-218 is crystallized enough to start voting. > > > > KIP documentation: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 2

Re: [VOTE] KIP-86: Configurable SASL callback handlers

2017-12-11 Thread Tom Bentley
+1 (non-binding) On 5 May 2017 at 11:57, Mickael Maison wrote: > Thanks for the KIP Rajini, this will significantly simplify providing > custom credential providers > +1 (non binding) > > On Wed, May 3, 2017 at 8:25 AM, Rajini Sivaram > wrote: > > Can we have some more reviews or votes for this

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-12-12 Thread Tom Bentley
rge. > > > > > > Thanks! > > > Xavier > > > > > > > > > On Mon, Dec 4, 2017 at 4:03 AM Steven Aerts > > > wrote: > > > > > > > Tom, > > > > > > > > Thanks for the review. > > > > updated the motiv

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2017-12-12 Thread Tom Bentley
+1 (nonbinding) On 12 December 2017 at 15:34, Ted Yu wrote: > +1 > > On Tue, Dec 12, 2017 at 5:44 AM, Rajini Sivaram > wrote: > > > Since there are no more outstanding comments, I would like to start vote > > for KIP-226: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 226+-+

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-13 Thread Tom Bentley
onfigs. For example, every time we change the > > assignment for a set of partitions, we could further write a sequential > > node /admin/reassignment_changes/[change_x]. That way, the controller > > only needs to watch the change path. Once a change is triggered, the > > controller ca

Re: [VOTE] Allowing write access to GitHub repositories (aka GitBox)

2017-12-14 Thread Tom Bentley
+1 On 12 December 2017 at 20:38, Sriram Subramanian wrote: > +1 > > On Tue, Dec 12, 2017 at 8:22 AM, Manikumar > wrote: > > > +1 > > > > On Tue, Dec 12, 2017 at 9:49 PM, Rajini Sivaram > > > wrote: > > > > > +1 > > > > > > Thanks, Ismael! > > > > > > On Tue, Dec 12, 2017 at 4:18 PM, Damian Guy

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-14 Thread Tom Bentley
dmin/reassignment/$topic/$partition > > Note the lack of 's' for reassignment. It would be good to make zookeeper > paths consistent. > > Thanks > > On Wed, Dec 13, 2017 at 9:49 AM, Tom Bentley > wrote: > > > Hi Jun and Ted, > > > > Jun, you'r

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-15 Thread Tom Bentley
> Should we seek to improve this algorithm in this KIP, or leave that as a later optimisation? I've updated the KIP with a proposed algorithm. On 14 December 2017 at 09:57, Tom Bentley wrote: > Thanks Ted, now fixed. > > On 13 December 2017 at 18:38, Ted Yu wrote: > &

[DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

2017-12-15 Thread Tom Bentley
Hi, KIP-236 lays the foundations for AdminClient APIs to do with partition reassignment. I'd now like to start discussing KIP-240, which adds APIs to the AdminClient to list and describe the current reassignments. https://cwiki.apache.org/confluence/display/KAFKA/KIP-240%3A+AdminClient.listReassi

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-15 Thread Tom Bentley
Just wanted to mention that I've started KIP-240, which builds on top of this one to provide an AdminClient API for listing and describing reassignments. On 15 December 2017 at 14:34, Tom Bentley wrote: > > Should we seek to improve this algorithm in this KIP, or leave that a

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-18 Thread Tom Bentley
> 14. Implementation wise, currently, we register a watcher of the isr path > of each partition being reassigned. This has the potential issue of > registering many listeners. An improvement could be just piggybacking on > the existing IsrChangeNotificationHandler, which only watches a single

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

2017-12-18 Thread Tom Bentley
y poll for changes. > > No clue how difficult it would be to implement, maybe you can piggyback on > some version number in the repartition messages or on zookeeper. > > This is just an idea, not a must have feature for me. We can always poll > over > the proposed api. > &

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

2017-12-18 Thread Tom Bentley
s(); > the return value should be a Collection. > > > On Fri, Dec 15, 2017 at 10:16 AM, Tom Bentley > wrote: > > > Hi, > > > > KIP-236 lays the foundations for AdminClient APIs to do with partition > > reassignment. I'd now like to start discussing

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

2017-12-18 Thread Tom Bentley
bloat. AdminClient.listPartitions() already provides for passing a null collection of partitions to discover all the reassignments, so overall there is no loss of functionality. I'm happy to add it back if people really think it's necessary. On 18 December 2017 at 10:49, Tom Bentley wrote: > Hi

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

2017-12-19 Thread Tom Bentley
+1 On 18 December 2017 at 23:28, Vahid S Hashemian wrote: > +1 > > Thanks for the KIP. > > --Vahid > > > > From: Ted Yu > To: dev@kafka.apache.org > Date: 12/18/2017 02:45 PM > Subject:Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig > constructors public > > > > +1 >

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-19 Thread Tom Bentley
; refer to the current assignment? Could you also describe > where the length of the original assignment is stored in ZK? > > 13. Hmm, I am not sure that the cancellation needs to be done for the whole > batch. The reason that I brought this up is for consistency. The KIP allows >

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

2017-12-22 Thread Tom Bentley
idea. > But I assume that in practice, this will imply too much change to the > code base to be > viable. > > > Regards, > > >Steven > > > 2017-12-18 11:49 GMT+01:00 Tom Bentley : > > Hi Steven, > > > > I think it would be useful to be able

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-22 Thread Tom Bentley
the reassignment of partition my-topic/42 = [4,5,6] 3. Update /admin/reassign_partitions to add your-topic/12=[7,8,9] 4. New controller resumes > Thanks, > > Jun > > On Tue, Dec 19, 2017 at 2:43 AM, Tom Bentley > wrote: > > > Hi Jun, > > > > 10. Another concern of

Re: [VOTE] KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient

2019-01-28 Thread Tom Bentley
y can look at https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=73632065&selectedPageVersions=20&selectedPageVersions=18 and https://github.com/apache/kafka/commit/269b65279c746bc54c611141a5a6509f9b310f11 Kind regards, Tom On Fri, 8 Sep 2017 at 16:30, Tom B

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2019-04-09 Thread Tom Bentley
has > been around a while and reading through the threads, it looks like there > has been a lot of interest in it. > > Thank you, > > Rajini > > > On Wed, Jan 9, 2019 at 11:25 AM Tom Bentley wrote: > > > Hi Anna and Mickael, > > > > Anna, did you hav

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2019-04-12 Thread Tom Bentley
to support broker config updates, it may be > good to start a new vote thread since the other one is quite old and > perhaps the KIP has changed since then. > > > On Wed, Apr 10, 2019 at 3:58 AM Tom Bentley wrote: > > > Hi Rajini, > > > > I'd be happy to

Re: [DISCUSS] KIP-482: The Kafka Protocol should Support Optional Fields

2019-07-02 Thread Tom Bentley
Hi Colin, In the example given, of a FooResponse, both the optional fields have the tag 0x0001. The text says "This number must be unique within the context it appears in.". My first thought was the example tags, must be wrong. But I think I misunderstood what you meant by "the context" – I assume

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-05 Thread Tom Bentley
Hi Colin, Thanks for the KIP. Currently ZooKeeper provides a convenient notification mechanism for knowing that broker and topic configuration has changed. While KIP-500 does suggest that incremental metadata update is expected to come to clients eventually, that would seem to imply that for some

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

2019-08-09 Thread Tom Bentley
+1 (non-binding). Thanks! On Fri, Aug 9, 2019 at 12:37 PM Satish Duggana wrote: > +1 (non-binding) Thanks for the KIP, so useful. > > On Fri, Aug 9, 2019 at 4:42 PM Mickael Maison > wrote: > > > +1 (non binding) > > Thanks for the KIP! > > > > On Fri, Aug 9, 2019 at 9:36 AM Andrew Schofield > >

[DISCUSS] KIP-506: Allow setting SCRAM password via Admin interface

2019-08-12 Thread Tom Bentley
Hi All, I've written KIP-506 proposing an RPC and Admin interface for setting SCRAM user passwords. I think there could be an interesting discussion over the relative merits of hashing on the broker or client. In any case I'd be grateful for any comments you may have: https://cwiki.apache.org/con

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2019-08-12 Thread Tom Bentley
do I thought I'd check whether anyone has any more comments. So please let me know any feedback for this KIP. Many thanks, Tom On Fri, Apr 12, 2019 at 10:46 AM Tom Bentley wrote: > Hi Rajini, > > I've made a number of changes to the KIP. > > 1. I've added RequestedTop

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

2019-08-13 Thread Tom Bentley
to > have Principals for PLAINTEXT sessions or use the default ANONYMOUS > Principal. > > Thanks > > > On Mon, Aug 12, 2019 at 10:52 AM Tom Bentley wrote: > > > > Hi folks, > > > > As far as I can see the motivation for KIP-201 is still valid, and as far >

Re: Test fail reason on doc fix

2017-06-12 Thread Tom Bentley
Hi Paolo, Usually you can just follow the links added by asfbot on the PR, but these are currently giving 404 (and not just for your failures, so maybe an ASF infrastructure problem?) Cheers, Tom On 12 June 2017 at 09:20, Paolo Patierno wrote: > Hi all, > > > I opened this JIRA with a related

JIRA contributor list

2017-06-14 Thread Tom Bentley
Please could I be added to the JIRA contributor list so that I can assign issues to myself? Thanks, Tom

Re: [jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified

2017-06-15 Thread Tom Bentley
It seems to me that in the most common case the partition isn't going to be magically created and in that case it's more helpful to quit with an error than to sit there waiting for the partition to be created.

Improving tools: --help

2017-06-16 Thread Tom Bentley
Hi, I noticed that the command line tools could use a little love. For instance, I was surprised that most of them don't support `--help`, and generally there are a few inconsistencies. KIP-14 is dormant and AFAICS no one is working on https://issues.apache.org/jira/browse/KAFKA-2111 either. So i

Re: Improving tools: --help

2017-06-16 Thread Tom Bentley
lose to 20 commands I have made changes to. I will try to push > something out by end of day today. So I think I will cover 1) from your > list below and not 2) and 3). > > Thanks Tom. > > Regards, > Mariam. > > > > [image: Inactive hide details for Tom Bentley ---06/16/

Re: Improving tools: --help

2017-06-16 Thread Tom Bentley
> I just checked kafka-topics.sh and the only required argument there is > --zookeeper option. Not sure if you were thinking of some other command. > > I meant that for kafka-topics.sh --create requires --topic (amongst others), but --list does not. (For example, using these methods we can say som

Re: JIRA contributor list

2017-06-20 Thread Tom Bentley
According to the website [1] I need to ask to be able to assign JIRAs to myself, but I'm still unable to do this. Could someone set this up for me please? Thanks, Tom [1]: https://kafka.apache.org/contributing On 14 June 2017 at 13:43, Tom Bentley wrote: > Please could I be added to

Re: Contributor

2017-06-21 Thread Tom Bentley
Please can I also be added? My username is tombentley. Thanks Tom On 21 June 2017 at 12:03, Damian Guy wrote: > Hi Andras, > > You should have access now. > > Thanks, > Damian > > On Wed, 21 Jun 2017 at 10:45 Andras Beni wrote: > > > Hi All, > > > > I'd like to contribute to Apache Kafka. > >

Re: Contributor

2017-06-21 Thread Tom Bentley
Thanks! On 21 Jun 2017 4:20 pm, "Damian Guy" wrote: > Done - thanks > > On Wed, 21 Jun 2017 at 12:19 Tom Bentley wrote: > > > Please can I also be added? My username is tombentley. > > > > Thanks > > > > Tom > > > >

Re: Mirroring documentation improvement

2017-06-23 Thread Tom Bentley
Hi Evgeniy, The wiki page you link to there is about contributing improvements for kafka.apache.org. But since you say "mirroring doc in Confluence" I assume you want to edit https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330, if so you will need one of the committers to giv

Re: Mirroring documentation improvement

2017-06-23 Thread Tom Bentley
is evis. > > Best regards, > Evgeny > > С уважением, > Евгений Веретенников > > 2017-06-23 16:43 GMT+03:00 Tom Bentley : > > > Hi Evgeniy, > > > > The wiki page you link to there is about contributing improvements for > > kafka.apache.org. But s

Re: Mirroring documentation improvement

2017-06-23 Thread Tom Bentley
Hi Damian, my username is tombentley Thanks Tom On 23 June 2017 at 14:57, Damian Guy wrote: > Evgeniy, you should now have access. > > Tom & Paolo what are your wiki usernames? > > Thanks, > Damian > > On Fri, 23 Jun 2017 at 14:52 Tom Bentley wrote: > > &

Open PRs

2017-06-26 Thread Tom Bentley
I realise that 0.11.0.0 is imminent and so the committers are rightly going to be rather focussed on that, but I opened some PRs nearly a week ago and they don't seem to have been looked at. Even a comment on the PR to the effect of "We'll look at this right after 0.11.0.0" would at least reassure

Re: From Scala to Java based tools : joptsimple vs argparse4j

2017-07-10 Thread Tom Bentley
Hi Paolo, I don't have an opinion about which you should use, but I certainly agree that two option parsing dependencies appears to be 1 too many. Is there a reason why you prefer joptsimple? Cheers, Tom On 10 July 2017 at 08:38, Paolo Patierno wrote: > Hi devs, > > > working on re-writing t

Comments on JIRAs

2017-07-13 Thread Tom Bentley
The project recently switched from all JIRA events being sent to the dev mailling list, to just issue creations. This seems like a good thing because the dev mailling list was very noisy before, and if you want to see all the JIRA comments etc you can subscribe to the JIRA list. If you don't subscr

[DISCUSS] KIP-178: Change ReassignPartitionsCommand to use AdminClient

2017-07-19 Thread Tom Bentley
Hi, I've written KIP-178 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-178+-+Change+ReassignPartitionsCommand+to+use+AdminClient) for changing the ReassignPartitionsCommand (a.k.a. kafka-reassign-partitions.sh)to remove the dependency on ZooKeeper and instead provide the same functionali

Re: [DISCUSS] KIP-178: Change ReassignPartitionsCommand to use AdminClient

2017-07-19 Thread Tom Bentley
Hi Ismael, Answers in-line: 1. Have you considered how progress would be reported? Partition > reassignment can take a long time and it would be good to have a mechanism > for progress reporting. > The ReassignPartitionsCommand doesn't currently have a mechanism to track progress. All you can do

  1   2   3   4   5   6   >