[VOTE] KIP-632: Add DirectoryConfigProvider

2020-07-07 Thread Tom Bentley
Hi, I'd like to start a vote on KIP-632, which is about making the config provider mechanism more ergonomic on Kubernetes: https://cwiki.apache.org/confluence/display/KAFKA/KIP-632%3A+Add+DirectoryConfigProvider Please take a look if you have time. Many thanks, Tom

Re: [DISCUSS] KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()

2020-07-09 Thread Tom Bentley
I opened a PR for this issue earlier, but > > > your KIP was submitted first. So I closed my one > > > < > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866169 > > > > > > . > > > > > > I have a question: for consistency with

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-07-09 Thread Tom Bentley
Hi Colin, Thanks for the detailed KIP. >> "As described in KIP-500, the controller will store its data in the internal __kafka_metadata topic." The KIP doesn't really explain the extent to which this is a normal topic. I know you say in the Networking section that the only time a client should c

Re: [VOTE] KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()

2020-07-10 Thread Tom Bentley
gt; > > > > > > > > > I tried to think of a better way to do this, but I think you're > right > > > > that > > > > > we probably just need different methods. > > > > > > > > > > +1 (binding). > > > > > >

Re: [VOTE] KIP-554: Add Broker-side SCRAM Config API

2020-07-13 Thread Tom Bentley
+1 (non-binding), thanks for the KIP. On Fri, Jul 10, 2020 at 7:02 PM Boyang Chen wrote: > Hey Colin, thanks for the KIP. One question I have about AlterScramUsers > RPC is whether we could consolidate the deletion list and alteration list, > since in response we only have a single list of resul

Re: [DISCUSS] KIP-363

2020-07-13 Thread Tom Bentley
still using manual serialization, which can't be easily converted to > > using this. So we will probably have to upgrade them to using automatic > > generation, or accept a little inconsistency for a while until they are > > upgraded. > > > > best, > > Colin

Re: [DISCUSS] KIP-595: A Raft Protocol for the Metadata Quorum

2020-07-14 Thread Tom Bentley
Hi Jason, Ghouzang and Boyang, First, thanks for the KIP. I've not a few minor suggestions and nits, but on the whole it was pretty clear and understandable. 1. § Motivation "We have intentionally avoided any assumption about the representation of the log and its semantics." I found this a

Re: [ANNOUNCE] New Kafka PMC Member: Mickael Maison

2020-07-31 Thread Tom Bentley
Congratulations Mickael! On Fri, Jul 31, 2020 at 4:23 PM Jun Rao wrote: > Hi, Everyone, > > Mickael Maison has been a Kafka committer since Nov. 5, 2019. He has > remained active in the community since becoming a committer. It's my > pleasure to announce that Mickael is now a member of Kafka PMC

Re: [VOTE] KIP-632: Add DirectoryConfigProvider

2020-08-06 Thread Tom Bentley
n Wed, Jul 8, 2020 at 11:31 AM Manikumar > wrote: > > > > +1 (bindig) > > > > Thanks for the KIP. > > > > On Tue, Jul 7, 2020 at 10:30 PM David Jacot wrote: > > > > > +1 (non-binding). Thanks for the KIP! > > > > > > On Tue,

Re: New Website Layout

2020-08-06 Thread Tom Bentley
Hi Ben, The documentation for the configs (broker, producer etc) used to function as links as well as anchors, which made the url fragments more discoverable, because you could click on the link and then copy+paste the browser URL: batch.size What seems to have happened with the new layo

Re: [VOTE] KIP-632: Add DirectoryConfigProvider

2020-08-06 Thread Tom Bentley
: > > > +1 (non binding) Looks like a good addition! > > > > On 06/08/2020, Tom Bentley wrote: > > > This pretty minor change has 2 binding and 1 non-binding votes. It > would > > be > > > great if more people could take a look and either vo

Re: [ANNOUNCE] New Kafka PMC Member: John Roesler

2020-08-11 Thread Tom Bentley
Congratulations John! On Tue, Aug 11, 2020 at 7:19 AM Bruno Cadonna wrote: > Wow, that is awesome! Congrats, John! > > Bruno > > On 10.08.20 22:11, Jun Rao wrote: > > Hi, Everyone, > > > > John Roesler has been a Kafka committer since Nov. 5, 2019. He has > remained > > active in the community s

Re: New Website Layout

2020-08-11 Thread Tom Bentley
Thanks. > > On Thu, 6 Aug 2020 at 18:59, Ben Weintraub wrote: > > > Plus one to Tom's request - the ability to easily generate links to > > specific config options is extremely valuable. > > > > On Thu, Aug 6, 2020 at 10:09 AM Tom Bentley wrote: > >

Re: [DISCUSS] KIP-664: Provide tooling to detect and abort hanging transactions

2020-09-09 Thread Tom Bentley
Hi Jason, The KIP looks good to me, but I had one question. AFAIU the LastTimestamp column in the output of --describe-producers and --find-hanging is there so the users of the tool know the txnLastUpdateTimestamp of the TransactionMetadata and from that and the (max) timeout can infer something a

Re: [DISCUSS] KIP-664: Provide tooling to detect and abort hanging transactions

2020-09-10 Thread Tom Bentley
> Jason > > On Wed, Sep 9, 2020 at 1:21 AM Tom Bentley wrote: > > > Hi Jason, > > > > The KIP looks good to me, but I had one question. AFAIU the LastTimestamp > > column in the output of --describe-producers and --find-hanging is there > so

Re: [DISCUSS] KIP-516: Topic Identifiers

2020-09-11 Thread Tom Bentley
Hi Justine, This looks like a very welcome improvement. Thanks! Maybe I missed it, but the KIP doesn't seem to mention changing DeleteTopicsRequest to identify the topic using an id. Maybe that's out of scope, but DeleteTopicsRequest is not listed among the Future Work APIs either. Kind regards,

Re: [DISCUSS] KIP-665 Kafka Connect Hash SMT

2020-09-21 Thread Tom Bentley
Hi Brandon and Mickael, Is it necessary to fix the supported digest? We could just support whatever the JVM's MessageDigest supports? Kind regards, Tom On Fri, Sep 18, 2020 at 6:00 PM Brandon Brown wrote: > Thanks Michael! So proposed hash functions would be MD5, SHA1, SHA256. > > I can expan

Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor

2020-09-21 Thread Tom Bentley
Hi Mickael, A few thoughts about the ReplicaAssignor contract: 1. What happens if a ReplicaAssignor impl returns a Map where some assignments don't meet the given replication factor? 2. Fixing the signature of assignReplicasToBrokers() as you have would make it hard to pass extra information in t

Re: [DISCUSS] KIP-516: Topic Identifiers

2020-09-23 Thread Tom Bentley
t; >> the > >> > > > > > >>> > > DeleteTopicsRequest. I think that this would be out of > >> > scope > >> > > > for > >> > > > > > >>> now, but > >> > > > &g

Re: [DISCUSS] KIP-653: Upgrade log4j to log4j2

2020-09-23 Thread Tom Bentley
Hi Dongjin, I'd like to see this feature, but if I understand correctly, the KIP in its current form breaks a couple of Kafka APIs. For Kafka Connect it says "From log4j2, the name of the root logger becomes empty string from 'root'. It impacts Kafka connect's dynamic logging control feature. (And

Re: [DISCUSS] KIP-673: Emit JSONs with new auto-generated schema

2020-09-25 Thread Tom Bentley
Hi Anastasia, Thanks for the KIP, I can certainly see the benefit of this. I have a few questions: 1. I think it would be helpful to readers to explicitly illustrate the RequestConvertToJson and XYZJsonDataConverter classes (e.g. with method signatures for one or two methods), because currently i

Re: [DISCUSS] KIP-673: Emit JSONs with new auto-generated schema

2020-09-28 Thread Tom Bentley
ia Jackson databind. > > Thanks again, > Anastasia > > On Fri, Sep 25, 2020 at 1:15 AM Tom Bentley wrote: > > > Hi Anastasia, > > > > Thanks for the KIP, I can certainly see the benefit of this. I have a few > > questions: > > > > 1. I think it wo

Re: [DISCUSS] KIP-653: Upgrade log4j to log4j2

2020-09-28 Thread Tom Bentley
t. > > In short, it seems like we can handle the API incompatibility introduced by > the root logger name change by adding a facade. > > How do you think? > > Thanks, > Dongjin > > On Wed, Sep 23, 2020 at 7:36 PM Tom Bentley wrote: > > > Hi Dongjin, >

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-08-11 Thread Tom Bentley
Hi Andrew, Thanks for picking this KIP up. I know you've started a vote, so these are unhelpfully late... sorry about that, but hopefully they're still useful. 1. "The Kafka Java client provides an API for the application to read the generated client instance id to assist in mapping and identific

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-21 Thread Tom Bentley
Congratulations! On Fri, 22 Sept 2023 at 09:11, Sophie Blee-Goldman wrote: > Congrats Lucas! >

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-21 Thread Tom Bentley
Congratulations! On Fri, 22 Sept 2023 at 09:11, Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Congratulations, Yash! > > On Thu 21. Sep 2023 at 21.57, Randall Hauch wrote: > > > Congratulations, Yash! > > > > On Thu, Sep 21, 2023 at 12:31 PM Satish Duggana < > satish.dugg...@

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

2023-09-23 Thread Tom Bentley
Congratulations! On Sun, 24 Sept 2023 at 12:32, Satish Duggana wrote: > Congratulations Justine!! > > On Sat, 23 Sept 2023 at 15:46, Bill Bejeck wrote: > > > > Congrats Justine! > > > > -Bill > > > > On Sat, Sep 23, 2023 at 6:23 PM Greg Harris > > > wrote: > > > > > Congratulations Justine! >

Re: [DISCUSS] KIP-986: Cross-Cluster Replication

2023-10-02 Thread Tom Bentley
Hi Greg, Thanks for this KIP! It is obviously very ambitious, but it's great to have a conversation about it. I'll start with some general points: Do you have a plan in mind for how to proceed with elaborating this KIP? While I like how you're involving the community in elaborating the KIP, I th

Re: [DISCUSS] KIP-987: Connect Static Assignments

2023-10-09 Thread Tom Bentley
Hi Greg, Many thanks for the KIP. Here are a few initial questions 1. Incomplete sentence: "But Connect is not situated to be able to manage resources directly, as workers are given a fixed " 2. You explain how sessioned is now a subset of static, but what happens in a cluster where some workers

Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana

2023-10-28 Thread Tom Bentley
Congratulations! On Sun, 29 Oct 2023 at 5:41 PM, Guozhang Wang wrote: > Congratulations Satish! > > On Sat, Oct 28, 2023 at 12:59 AM Luke Chen wrote: > > > > Congrats Satish! > > > > Luke > > > > On Sat, Oct 28, 2023 at 11:16 AM ziming deng > > wrote: > > > > > Congratulations Satish! > > > >

Re: [DISCUSS] KIP-877: Mechanism for plugins and connectors to register metrics

2024-07-31 Thread Tom Bentley
gt; > >> method should be used. That said, I'm not attached to the name > > so if > > > > > > >> more people prefer "monitor()", or can come up with a better > > name, I'm > > > > > > >> happy to make the ch

Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics

2024-08-01 Thread Tom Bentley
gt; > > > > Hi, > > > > > > > > Following the feedback in the DISCUSS thread, I made significant > > > > changes to the proposal. So I'd like to restart a vote for KIP-877: > > > > > > > > > > > https://cwiki.apache.o

Re: [VOTE] 3.5.2 RC1

2023-12-07 Thread Tom Bentley
Hi, I have validated signatures, checked the Java docs, built from source and run tests. I had a few unit test failures, but I note that others saw them pass and the CI was green too, so I think this is a problem with my system rather than the release. +1 (binding). Thanks! On Wed, 6 Dec 2023 a

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-27 Thread Tom Bentley
Congratulations! On Thu, 28 Dec 2023 at 06:17, Philip Nee wrote: > congrats divij! > > On Wed, Dec 27, 2023 at 8:55 AM Justine Olshan > > wrote: > > > Congratulations Divij! > > > > On Wed, Dec 27, 2023 at 4:20 AM Gaurav Narula wrote: > > > > > Congratulations Divij! > > > > > > Regards, > > >

Re: [DISCUSS] KIP-877: Mechanism for plugins and connectors to register metrics

2024-01-24 Thread Tom Bentley
Hi Mickael, Thanks for the KIP! I can tell a lot of thought went into this. I have a few comments, but they're all pretty trivial and aimed at making the correct use of this API clearer to implementors. 1. Configurable and Reconfigurable both use a verb in the imperative mood for their method nam

Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics

2024-01-24 Thread Tom Bentley
Hi Mickael, You'll have seen that I left some comments on the discussion thread, but they're minor enough that I'm happy to vote +1 here. Thanks, Tom On Thu, 11 Jan 2024 at 06:14, Mickael Maison wrote: > Bumping this thread since I've not seen any feedback. > > Thanks, > Mickael > > On Tue, D

Re: [VOTE] KIP-390: Support Compression Level (rebooted)

2024-02-08 Thread Tom Bentley
n buffer option into the 'Future Works' > > > > section, as Ismael proposed. > > > > > > > > Best, > > > > Dongjin > > > > > > > > > > > > > > > > On Fri, Jun 11, 2021 at 3:03 AM Konstantine Karantasis >

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-28 Thread Tom Bentley
Congratulations! On Wed, 27 Mar 2024 at 21:10, Matthias J. Sax wrote: > Congrats! > > On 3/26/24 9:39 PM, Christo Lolov wrote: > > Thank you everyone! > > > > It wouldn't have been possible without quite a lot of reviews and > extremely > > helpful inputs from you and the rest of the community!

Re: [DISCUSS] KIP-932: Queues for Kafka

2024-04-03 Thread Tom Bentley
Hi Andrew (and Omnia), Thanks for the KIP. I hope to provide some feedback on this KIP soon, but I had a thought on the specific subject of group configs and MM2. If brokers validate for known groups configs then doesn't this induce an ordering requirement on upgrading clusters: Wouldn't you have

Re: [ANNOUNCE] New committer: Igor Soarez

2024-04-24 Thread Tom Bentley
Congratulations Igor! On Thu, 25 Apr 2024 at 6:27 AM, Chia-Ping Tsai wrote: > Congratulations, Igor! you are one of the best Kafka developers!!! > > Mickael Maison 於 2024年4月25日 週四 上午2:16寫道: > > > Congratulations Igor! > > > > On Wed, Apr 24, 2024 at 8:06 PM Colin McCabe wrote: > > > > > > Hi a

[DISCUSS] KIP-585: Conditional SMT

2020-03-24 Thread Tom Bentley
Hi, I've opened KIP-585 which is intended to provide a mechanism to conditionally apply SMTs in Kafka Connect. I'd be grateful for any feedback: https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Conditional+SMT Many thanks, Tom

Re: [DISCUSS] KIP-585: Conditional SMT

2020-04-01 Thread Tom Bentley
Does anyone have any comments, feedback or thoughts about this? Thanks, Tom On Tue, Mar 24, 2020 at 11:56 AM Tom Bentley wrote: > Hi, > > I've opened KIP-585 which is intended to provide a mechanism to > conditionally apply SMTs in Kafka Connect. I'd be grateful for a

Re: [DISCUSS] KIP-585: Conditional SMT

2020-04-01 Thread Tom Bentley
y-header > transforms.conditionalExtract.condition.hasMyHeader: field-value:my-value > > > Also, while writing that, I noticed that you have a version with and > without "name" for your transformation in the KIP: > > transforms.conditionalExtract.condition.hasMyHeader: ha

Re: [DISCUSS] KIP-585: Conditional SMT

2020-04-02 Thread Tom Bentley
to help with > readability; we probably wouldn't need that with the approach of naming > conditions via separate properties, but things may get a little nasty with > literal conditions included there, especially with the possibility of > nesting. Finally, the shift in syntax he

Re: [DISCUSS] KIP-585: Conditional SMT

2020-04-06 Thread Tom Bentley
etc. I think it's less ambiguous, in particular when it > comes to ordering of the different conditions and determining their > precedence. > > Would love to see this feature in one or another way in Connect. > > Best, > > --Gunnar > > > > Am Do., 2. Apr. 20

Re: [DISCUSS] KIP-585: Conditional SMT

2020-04-06 Thread Tom Bentley
ght not to validate() seems like a recipe for confusion. Also problematic is that there's no use for the Config returned from Transformations.validate(). So I'm not convinced that the superficial similarity really gains anything. Kind regards, Tom On Mon, Apr 6, 2020 at 2:29 PM Tom Ben

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-06 Thread Tom Bentley
Hi Boyang, Thanks for the KIP! When a broker proxies a request to the controller how does the authenticated principal get propagated? I think a couple of things might complicate this: 1. A PrincipalBuilder might be in use, 2. A Principal does not have to be serializable. Kind regards, Tom On

Re: [DISCUSS] KIP-585: Conditional SMT

2020-04-08 Thread Tom Bentley
Since no one objected I've updated the KIP with the revised way to configure this transformation. On Mon, Apr 6, 2020 at 2:52 PM Tom Bentley wrote: > To come back about a point Chris made: > > 1. Instead of the new "ConfigDef config(Map props)" method, >> what wo

Re: [DISCUSS] KIP-578: Add configuration to limit number of partitions

2020-04-20 Thread Tom Bentley
Hi Gokul, Leaving aside the question of how Kafka scales, I think the proposed solution, limiting the number of partitions in a cluster or per-broker, is a policy which ought to be addressable via the pluggable policies (e.g. create.topic.policy.class.name). Unfortunately although there's a policy

Re: [DISCUSS] KIP-578: Add configuration to limit number of partitions

2020-04-21 Thread Tom Bentley
tom implementation. I am afraid that would add > complexity that I am not sure we want to undertake. > > Do you see sense in what I am saying? > > Thanks. > > On Mon, Apr 20, 2020 at 3:59 PM Tom Bentley wrote: > > > Hi Gokul, > > > > Leaving aside the

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-21 Thread Tom Bentley
Hi Boyang, The answer to my original question about the request principal was that the forwarding broker would authorize the request and the controller would trust the request since it was from another broker. AFAIU you added the principal purely for logging purposes. In the "EnvelopeRequest Handl

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-22 Thread Tom Bentley
sending one > > unnecessary forwarding request if it couldn't pass the authorization in > the > > first place. > > > > In the meantime, could you give more context on the custom Kafka > principal > > you are referring to? How does that get encode

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-23 Thread Tom Bentley
Hi folks, Colin wrote: > The final broker knows it can trust the principal name in the envelope > (since EnvelopeRequest requires CLUSTERACTION on CLUSTER). So it can use > that principal name for authorization (given who you are, what can you > do?) The forwarded principal name will also be us

Re: [DISCUSS] KIP-585: Conditional SMT

2020-04-28 Thread Tom Bentley
> > > SMT equivalent of an "if" statement, and it's totally > understandable if > > > that's too much to ask. The one concern I have left is that if we > expand > > > the SMT in the future, there become compatibility concerns since

Re: [DISCUSS] KIP-589 Add API to Update Replica State in Controller

2020-04-29 Thread Tom Bentley
Hi David, Thanks for the KIP! In the rejecting the alternative of having an RPC for log dir failures you say: It was also rejected to prevent "leaking" the notion of a log dir to the > public API. > I'm not quite sure I follow that argument, since we already have RPCs for changing replica log d

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-29 Thread Tom Bentley
ncipals. > > > > > > > > We'll probably want to at least add a note on that to the docs - > unless > > > it > > > > is there already, I've only looked for about 30 seconds.. > > > > > > > > Best regards, > > > > Sön

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

2020-04-30 Thread Tom Bentley
Hi David, Thanks for the KIP. If I understand the proposed throttling algorithm, an initial request would be allowed (possibly making K negative) and only subsequent requests (before K became positive) would receive the QUOTA_VIOLATED. That would mean it was still possible to block the controller

Re: [DISCUSS] KIP-585: Conditional SMT

2020-05-01 Thread Tom Bentley
redicates.TopicNameMatch > transforms.t2.?regex: my-prefix-.* > transforms.t2.type: org.apache.kafka.connect.transforms.ExtractField$Key > transforms.t2.field: c1 > > The "?!" means the transform guarded by the predicate runs if the Predicate > returns false. > > I think a

Re: [DISCUSS] KIP-317 - Add end-to-end data encryption functionality to Apache Kafka

2020-05-01 Thread Tom Bentley
Hi Sönke, I never looked at the original version, but what you describe in the new version makes sense to me. Here are a few things which sprang to mind while I was reading: 1. It wasn't immediately obvious why this can't be achieved using custom serializers and deserializers. 2. It would be use

Re: [DISCUSS] KIP-589 Add API to Update Replica State in Controller

2020-05-02 Thread Tom Bentley
cs" section, I > mention that if no topics are present in the RPC request, then all topics > on the broker are implied. And if a topic is given with no partitions, all > partitions for that topic (on the broker) are implied. Does this make > sense? > > Thanks again! I'll u

Re: [DISCUSS] KIP-585: Conditional SMT

2020-05-04 Thread Tom Bentley
type:ExtractField$Key > transforms.t2.field: c1 > > predicates: has-my-prefix > predicates.has-my-prefix.type: TopicNameMatch > predicates.has-my-prefix.negate: true > predicates.has-my-prefix.pattern: my-prefix-.* > > Excited to see how this is evolving and I think we'

Re: [DISCUSS] KIP-585: Conditional SMT

2020-05-05 Thread Tom Bentley
quot;?predicate" to name the predicate alias to use > transforms: t2 > transforms.t2.?predicate: has-my-prefix > transforms.t2.type:ExtractField$Key > transforms.t2.field: c1 > > B) "?" to name the predicate alias to use > transforms: t2 > transforms.t2.?:

Re: [DISCUSS] KIP-589 Add API to Update Replica State in Controller

2020-05-05 Thread Tom Bentley
have. > > Sorry if I'm repeating stuff you already figured out, but I just wanted to > be more clear about this (I found it confusing too until David explained it > to me originally...) > > best, > Colin > > > On Sat, May 2, 2020, at 10:30, Tom Bentley wrote: > &

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-05 Thread Tom Bentley
Hi Colin, SCRAM is better than SASL PLAIN because it doesn't send the password over the wire in the clear. Presumably this property is important for some users who have chosen to use SCRAM. This proposal does send the password in the clear when setting the password. That doesn't mean it can't be u

Re: [DISCUSS] KIP-317 - Add end-to-end data encryption functionality to Apache Kafka

2020-05-07 Thread Tom Bentley
Hi Sönke, Replies inline 1. The functionality in this first phase could indeed be achieved with > custom serializers, that would then need to wrap the actual serializer that > is to be used. However, looking forward I intend to add functionality that > allows configuration to be configured broker

Re: [DISCUSS] KIP-317 - Add end-to-end data encryption functionality to Apache Kafka

2020-05-07 Thread Tom Bentley
Hi Rayanne, You raise some good points there. Similarly, if the whole record is encrypted, it becomes impossible to do > joins, group bys etc, which just need the record key and maybe don't have > access to the encryption key. Maybe only record _values_ should be > encrypted, and maybe Kafka Stre

Re: [DISCUSS] KIP-317 - Add end-to-end data encryption functionality to Apache Kafka

2020-05-07 Thread Tom Bentley
addition to the encryption/decryption keys, so I think it should still work. Unless I've overlooked something else. Cheers, Tom On Thu, May 7, 2020 at 6:04 PM Tom Bentley wrote: > Hi Rayanne, > > You raise some good points there. > > Similarly, if the whole record is e

Re: [DISCUSS] KIP-585: Conditional SMT

2020-05-11 Thread Tom Bentley
> >transforms.t1.!test.pattern: my-prefix-.* > >transforms.t1.then: t1 > >transforms.t1.then.t1.type: ExtractField$Key > >transforms.t1.then.t1.field: c1 > > >I would love to see conditional application of transforms in SMT in >

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

2020-05-12 Thread Tom Bentley
Hi David, Thanks for the reply. >> If I understand the proposed throttling algorithm, an initial request > would > >> be allowed (possibly making K negative) and only subsequent requests > >> (before K became positive) would receive the QUOTA_VIOLATED. That would > >> mean it was still possible t

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-12 Thread Tom Bentley
Hi Colin, It's not clear whether users of the Java API would need to supply the salt and salted password directly, or whether the constructor of ScramCredential would take the password and perform the hashing itself. I also wonder a little about consistency with the other APIs which have separate

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

2020-05-13 Thread Tom Bentley
Hi David, Thanks for the explanation and confirmation that evolving the APIs is not off the table in the longer term. Kind regards, Tom

[VOTE] KIP 585: Filter and conditional SMTs

2020-05-13 Thread Tom Bentley
Hi, I'd like to start a vote on KIP-585: Filter and conditional SMTs https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs Those involved in the discussion seem to be positively disposed to the idea, but in the absence of any committer participation it's been d

Re: [DISCUSS] KIP-585: Conditional SMT

2020-05-14 Thread Tom Bentley
ently examples are neither java properties nor json. Should we > comply with one of the two formats to avoid confusing users that will read > the KIP before trying this feature? > > Thanks again for driving this proposal. > > Konstantine > > On Mon, May 11, 2020 at 7:57 AM

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-15 Thread Tom Bentley
Hi Colin, The AdminClient should do the hashing, right? I don't see any advantage to > doing it externally. I'm happy so long as the AdminClient interface doesn't require users to do the hashing themselves. I do think we should support setting the salt explicitly, but really only > for testing

Re: [VOTE] KIP 585: Filter and conditional SMTs

2020-05-19 Thread Tom Bentley
sis > > : > > > > > > +1 (binding) > > > > > > Thanks Tom. > > > > > > Konstantine > > > > > > On Fri, May 15, 2020 at 5:03 AM Andrew Schofield > > > > > wrote: > > > > > > > +1 (non-bin

Re: [VOTE] KIP 585: Filter and conditional SMTs

2020-05-20 Thread Tom Bentley
e a useful addition. > > +1(binding) > > -Bill > > On Tue, May 19, 2020 at 1:14 PM Tom Bentley wrote: > > > It would be nice to get this into Kafka 2.6. There are 2 binding and 3 > > non-binding votes so far. If you've not looked at it already now would &

Re: [DISCUSS] Apache Kafka 2.6.0 release

2020-05-20 Thread Tom Bentley
Hi Randall, Can we add KIP-585? (I'm not quite sure of the protocol here, but thought it better to ask than to just add it myself). Thanks, Tom On Tue, May 5, 2020 at 6:54 PM Randall Hauch wrote: > Greetings! > > I'd like to volunteer to be release manager for the next time-based feature > re

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

2020-06-03 Thread Tom Bentley
> > > > >> > > 10. Regarding exposing rate or usage as quota. Your argument > is > > > that > > > > >> > usage > > > > >> > > is not very accurate anyway and is harder to implement. So, > > l

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

2020-06-04 Thread Tom Bentley
>> the > >> > > > processing of an operation anyway. (3) If you prefer rate > strongly, > >> I > >> > > don't > >> > > > have strong objections. However, I do feel that the new quota name > >> > should > >>

Re: [VOTE] KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

2020-06-04 Thread Tom Bentley
+1 (non binding). Thanks! On Wed, Jun 3, 2020 at 3:51 PM Rajini Sivaram wrote: > +1 (binding) > > Thanks for the KIP, David! > > Regards, > > Rajini > > > On Sun, May 31, 2020 at 3:29 AM Gwen Shapira wrote: > > > +1 (binding) > > > > Looks great. Thank you for the in-depth design and discussio

[DISCUSS] KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()

2020-06-08 Thread Tom Bentley
Hi all, I've opened a small KIP seeking to deprecate and replace a couple of methods of DescribeLogDirsResult which refer to internal classes in their return type. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158862109 Please take a look if you have the time. Kind regards,

Re: PR for CVE-2022-34917

2022-09-26 Thread Tom Bentley
Hi Swathi, In this case the PR reviews happened on a private repo because the CVE wasn't public at that time. On the 3.3 branches you can look at/cherry-pick commits 015d7aede6cbd350d56d75006930dd2bf89a4a5a and b2b928338c7226b41a73786df27a2127eaa32ab2. Kind regards, Tom On Mon, 26 Sept 2022 at

Re: [ANNOUNCE] New committer: Deng Ziming

2022-10-10 Thread Tom Bentley
Congratulations! On Mon, 10 Oct 2022 at 18:24, John Roesler wrote: > Congratulations, Ziming! > > On Mon, Oct 10, 2022, at 12:05, Justine Olshan wrote: > > Congratulations Ziming! I'll always remember the help you provided with > > topic IDs. > > Very well deserved! > > > > On Mon, Oct 10, 2022

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

2022-11-01 Thread Tom Bentley
Hi Igor, Thanks for the KIP, I've finally managed to take an initial look. 0. You mention the command line tools (which one?) in the public interfaces section, but don't spell out how it changes -- what options are added. Reading the proposed changes it suggests that there are no changes to the s

Re: [ANNOUNCE] New Kafka PMC Member: Bruno Cadonna

2022-11-02 Thread Tom Bentley
Congratulations! On Wed, 2 Nov 2022 at 06:40, David Jacot wrote: > Congrats, Bruno! Well deserved. > > Le mer. 2 nov. 2022 à 06:12, Randall Hauch a écrit : > > > Congratulations, Bruno! > > > > On Tue, Nov 1, 2022 at 11:20 PM Sagar wrote: > > > > > Congrats Bruno! > > > > > > Sagar. > > > > >

Re: Request the account for Kafka Jira

2022-11-15 Thread Tom Bentley
Hi Simon, Usernames have to be all lower case ascii letters and digits, so I used simonzhang. You should have a password reset email where you can complete the account creation. Thanks for your interest in Apache Kafka. On Tue, 15 Nov 2022 at 15:25, zsy112371 wrote: > Hi team, > I would like t

Re: [DISCUSS] KIP-888: Batch describe ACLs and describe client quotas

2022-12-02 Thread Tom Bentley
Hi Mickael, Thanks for the KIP. I can understand the motivation, but to be honest I have a doubt about this. Taking the ACLs first, by allowing multiple filters with the current proposal isn't there the chance that the same ACL will match multiple filters, and be returned multiple times in the re

Re: [ANNOUNCE] New committer: Ron Dagostino

2022-12-15 Thread Tom Bentley
Congratulations! On Thu, 15 Dec 2022 at 07:40, Satish Duggana wrote: > Congratulations, Ron!! > > On Thu, 15 Dec 2022 at 07:48, ziming deng > wrote: > > > Congratulations, Ron! > > Well deserved! > > > > -- > > Ziming > > > > > On Dec 15, 2022, at 09:16, Luke Chen wrote: > > > > > > Congratula

Re: [ANNOUNCE] New Kafka PMC Member: Luke Chen

2022-12-16 Thread Tom Bentley
Congratulations! On Fri, 16 Dec 2022 at 20:40, Viktor Somogyi-Vass wrote: > Congrats Luke! :) > > On Fri, Dec 16, 2022, 21:26 Randall Hauch wrote: > > > Congratulations, Luke! > > > > On Fri, Dec 16, 2022 at 2:08 PM Josep Prat > > wrote: > > > > > Congrats Luke! > > > > > > On Fri, Dec 16, 202

Re: [ANNOUNCE] New committer: Josep Prat

2022-12-21 Thread Tom Bentley
Congratulations! On Wed, 21 Dec 2022 at 03:05, John Roesler wrote: > Congratulations, Josep! > -John > > On Tue, Dec 20, 2022, at 20:02, Luke Chen wrote: > > Congratulations, Josep! > > > > Luke > > > > On Wed, Dec 21, 2022 at 6:26 AM Viktor Somogyi-Vass > > wrote: > > > >> Congrats Josep! > >>

Re: [ANNOUNCE] New committer: Satish Duggana

2022-12-24 Thread Tom Bentley
Congratulations! On Sat, 24 Dec 2022 at 05:05, Luke Chen wrote: > Congratulations, Satish! > > On Sat, Dec 24, 2022 at 4:12 AM Federico Valeri > wrote: > > > Hi Satish, congrats! > > > > On Fri, Dec 23, 2022, 8:46 PM Viktor Somogyi-Vass > > wrote: > > > > > Congrats Satish! > > > > > > On Fri,

Re: [ANNOUNCE] New committer: Justine Olshan

2023-01-03 Thread Tom Bentley
Congratulations! On Mon, 2 Jan 2023 at 18:25, Rajini Sivaram wrote: > Congratulations, Justine, well deserved! > > Regards, > > Rajini > > On Mon, Jan 2, 2023 at 5:29 PM Ismael Juma wrote: > > > Congratulations Justine! > > > > Ismael > > > > On Thu, Dec 29, 2022, 12:58 PM David Jacot wrote: >

Re: [ANNOUNCE] New committer: Edoardo Comar

2023-01-09 Thread Tom Bentley
Congratulations! On Sun, 8 Jan 2023 at 01:14, Satish Duggana wrote: > Congratulations, Edorado! > > On Sun, 8 Jan 2023 at 00:15, Viktor Somogyi-Vass > wrote: > > > > Congrats Edoardo! > > > > On Sat, Jan 7, 2023, 18:15 Bill Bejeck wrote: > > > > > Congratulations, Edoardo! > > > > > > -Bill >

Re: [VOTE] 3.3.2 RC1

2023-01-09 Thread Tom Bentley
+1 binding. I have checked signatures, given the javadocs a quick look, followed the quickstart for KRaft and run the tests. Thanks Chris for running the release. Tom On Mon, 9 Jan 2023 at 15:29, Chris Egerton wrote: > Hi all, > > Thanks to everyone who has tested and voted for the release

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

2023-01-10 Thread Tom Bentley
Hi Igor, 20. The description of the changes to meta.properties says "If there any meta.properties file is missing directory.id a new UUID is generated, and assigned to that log directory by updating the file", and the upgrade/migration section says "As the upgraded brokers come up, the existing me

Re: [ANNOUNCE] New committer: Stanislav Kozlovski

2023-01-17 Thread Tom Bentley
Congratulations! On Tue, 17 Jan 2023 at 16:52, Bill Bejeck wrote: > Congratulations Stan! > > -Bill > > On Tue, Jan 17, 2023 at 11:37 AM Bruno Cadonna wrote: > > > Congrats Stan! > > > > Well deserved! > > > > Best, > > Bruno > > > > On 17.01.23 16:50, Jun Rao wrote: > > > Hi, Everyone, > > > >

Re: [ANNOUNCE] New committer: Walker Carlson

2023-01-17 Thread Tom Bentley
Congratulations! On Wed, 18 Jan 2023 at 01:26, John Roesler wrote: > Congratulations, Walker! > -John > > On Tue, Jan 17, 2023, at 18:50, Guozhang Wang wrote: > > Congrats, Walker! > > > > On Tue, Jan 17, 2023 at 2:20 PM Chris Egerton > > wrote: > > > >> Congrats, Walker! > >> > >> On Tue, Jan

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

2023-01-23 Thread Tom Bentley
Hi Igor, Thanks for your replies on points 21-25. Those all LGTM. See below for a further thought on the meta.properties upgrade. Kind regards, Tom On Fri, 13 Jan 2023 at 18:09, Igor Soarez wrote: > Hi Tom, > > Thank you for having another look. > > 20. Upon a downgrade to a Kafka version th

What happened to KIP-158?

2019-11-25 Thread Tom Bentley
Hi, KIP-158 proposed a way to allow source connectors a way to configure their topics prior to creation. This was discussed, but the vote didn't get enough commiter votes at the time (though I think a couple of the +1s came from people who are now committers). Meanwhile in KIP-382 (MirrorMaker 2.

Re: Broker Interceptors

2019-12-06 Thread Tom Bentley
Hi, Couldn't this be done without exposing broker internals at the slightly higher level of AbstractRequest and AbstractResponse? Those classes are public. If the observer interface used Java default methods then adding a new request type would not break existing implementations. I'm thinking some

<    1   2   3   4   5   6   >