Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Just to update - still working on it. Get to work only on and off on it :( On Thu, Aug 8, 2019 at 4:05 PM Maulin Vasavada wrote: > Hi Harsha > > Let me try to write samples and will let you know. > > Thanks > Maulin > > On Thu, Aug 8, 2019 at 4:00 PM Harsha Ch wrote: > >> Hi Maulin, >>

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

Chris, I don't understand how the rebalance protocol can be used to give out session tokens in a secure way. It seems that any attacker could just join the group and sign requests with the provided token. Am I missing something? Ryanne On Wed, Aug 14, 2019, 2:31 PM Chris Egerton wrote: > The KI

Apply to be contributor of kafka

Hello, I am a graduate student studying in United States, major in computer science. I am much interested on being a contributor of kafka. Before going to school again, I actually had a couple years working experience on Hadoop ecosystem. I have built clusters to satisfy the real world requirem

Jenkins build is back to normal : kafka-trunk-jdk11 #754

See

Request permission

Hi, I'd like to create a new page for a KIP in the Apache Kafka Confluence site. The implementation of this KIP is already done but changes some things in the API that should be approved. You can check it out here . My ID in Confluence is "mmolimar". Re

Build failed in Jenkins: kafka-trunk-jdk8 #3853

See Changes: [cmccabe] KAFKA-8345: KIP-455: Admin API changes (Part 2) (#7120) [jason] KAFKA-7335; Store clusterId locally to ensure broker joins the right -- [...truncat

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

+1 (binding) Thanks Michael for the suggestion of simplifying offset retrieval/alteration operations. --Vahid On Wed, Aug 14, 2019 at 4:42 PM Bill Bejeck wrote: > Thanks for the KIP Mickael, looks very useful. > +1 (binding) > > -Bill > > On Wed, Aug 14, 2019 at 6:14 PM Harsha Chintalapani >

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

Thanks for the KIP Mickael, looks very useful. +1 (binding) -Bill On Wed, Aug 14, 2019 at 6:14 PM Harsha Chintalapani wrote: > Thanks for the KIP Mickael. LGTM +1 (binding). > -Harsha > > > On Wed, Aug 14, 2019 at 1:10 PM, Colin McCabe wrote: > > > Thanks, Mickael. +1 (binding) > > > > best, >

[jira] [Created] (KAFKA-8806) Kafka.poll spends significant amount of time in KafkaConsumer.updateAssignmentMetadataIfNeeded

Xavier Léauté created KAFKA-8806: Summary: Kafka.poll spends significant amount of time in KafkaConsumer.updateAssignmentMetadataIfNeeded Key: KAFKA-8806 URL: https://issues.apache.org/jira/browse/KAFKA-8806

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

Thanks for the KIP Mickael. LGTM +1 (binding). -Harsha On Wed, Aug 14, 2019 at 1:10 PM, Colin McCabe wrote: > Thanks, Mickael. +1 (binding) > > best, > Colin > > On Wed, Aug 14, 2019, at 12:07, Gabor Somogyi wrote: > > +1 (non-binding) > I've read it through in depth and as Jungtaek said Spark

Build failed in Jenkins: kafka-2.3-jdk8 #87

See Changes: [bill] MINOR: Correct typo in test name `TimetampedSegmentsTest` (#7210) -- [...truncated 2.92 MB...] kafka.zk.KafkaZkClientTest > testCreateTokenChangeNotificati

Build failed in Jenkins: kafka-trunk-jdk8 #3852

See Changes: [bbejeck] MINOR: Correct typo in test name `TimetampedSegmentsTest` (#7210) -- [...truncated 6.51 MB...] org.apache.kafka.connect.connector.policy.PrincipalC

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

Thanks, Mickael. +1 (binding) best, Colin On Wed, Aug 14, 2019, at 12:07, Gabor Somogyi wrote: > +1 (non-binding) > I've read it through in depth and as Jungtaek said Spark can make good use > of it. > > On Wed, 14 Aug 2019, 17:06 Jungtaek Lim, wrote: > > > +1 (non-binding) > > > > I found i

Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface

Hi Rajini, I think it would be good to rename KafkaRequestContext to something like AuthorizableRequestContext, and put it in the org.apache.kafka.server.authorizer namespace. If we put it in the org.apache.kafka.common namespace, then it's not really clear that it's part of the Authorizer AP

[jira] [Created] (KAFKA-8805) Bump producer epoch following recoverable errors

Bob Barrett created KAFKA-8805: -- Summary: Bump producer epoch following recoverable errors Key: KAFKA-8805 URL: https://issues.apache.org/jira/browse/KAFKA-8805 Project: Kafka Issue Type: Improv

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

The KIP page can be found at https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints, by the way. Apologies for neglecting to include it in my initial email! On Wed, Aug 14, 2019 at 12:29 PM Chris Egerton wrote: > Hi all, > > I'd like to start discus

[DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

Hi all, I'd like to start discussion on a KIP to secure the internal "POST /connectors//tasks" endpoint for the Connect framework. The proposed changes address a vulnerability in the framework in its current state that allows malicious users to write arbitrary task configurations for connectors; i

[jira] [Created] (KAFKA-8804) Internal Connect REST endpoints are insecure

Chris Egerton created KAFKA-8804: Summary: Internal Connect REST endpoints are insecure Key: KAFKA-8804 URL: https://issues.apache.org/jira/browse/KAFKA-8804 Project: Kafka Issue Type: Bug

Build failed in Jenkins: kafka-trunk-jdk11 #753

See Changes: [bbejeck] MINOR: Correct typo in test name `TimetampedSegmentsTest` (#7210) -- [...truncated 2.59 MB...] org.apache.kafka.connect.runtime.standalone.Standalo

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

+1 (non-binding) I've read it through in depth and as Jungtaek said Spark can make good use of it. On Wed, 14 Aug 2019, 17:06 Jungtaek Lim, wrote: > +1 (non-binding) > > I found it very useful for Spark's case. (Discussion on KIP-505 described > it.) > > Thanks for driving the effort! > > 2019년

Re: [VOTE] KIP-503: deleted topics metric

+1 (non-binding) Thanks for the KIP! Simple yet very useful. Best, David On Wed, Aug 14, 2019 at 9:24 AM Robert Barrett wrote: > +1 (non-binding) > > This will be good to have, thanks David! > > Bob > > On Wed, Aug 14, 2019 at 6:08 AM Mickael Maison > wrote: > > > +1 non binding > > Thank you

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

My main concern is to require the overloaded `initTransactions` to be called repeatedly while the original `initTransactions` still called once throughout the life time, which is a bit confusing. Looking into the current POC PR, we actually only need the latest generation id when fetching offsets,

[jira] [Created] (KAFKA-8803) Stream will not start due to TimeoutException: Timeout expired after 60000milliseconds while awaiting InitProducerId

Raman Gupta created KAFKA-8803: -- Summary: Stream will not start due to TimeoutException: Timeout expired after 6milliseconds while awaiting InitProducerId Key: KAFKA-8803 URL: https://issues.apache.org/jira/brows

[jira] [Created] (KAFKA-8802) ConcurrentSkipListMap shows performance regression in cache and in-memory store

Sophie Blee-Goldman created KAFKA-8802: -- Summary: ConcurrentSkipListMap shows performance regression in cache and in-memory store Key: KAFKA-8802 URL: https://issues.apache.org/jira/browse/KAFKA-8802

[jira] [Reopened] (KAFKA-7912) In-memory key-value store does not support concurrent access

[ https://issues.apache.org/jira/browse/KAFKA-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman reopened KAFKA-7912: The fix for this caused a performance regression and is being rolled back. We should l

Re: [VOTE] KIP-497: Add inter-broker API to alter ISR

+1 (non-binding), Thanks Jason for nice improvements on ISR propagation protocol! On Wed, Aug 14, 2019 at 8:29 PM David Arthur wrote: > > +1 binding, this looks great! > > -David > > On Tue, Aug 13, 2019 at 4:55 PM Guozhang Wang wrote: > > > +1 (binding). This is a great KIP, thanks Jason! > > >

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

Yeah, my reasoning is that the group metadata is only relevant to the subscription API. So it makes sense to only expose it to the rebalance listener. One option we could consider is bring back the `initTransactions` overload. Then usage looks something like this: consumer.subscribe(topics, new R

Re: [VOTE] KIP-503: deleted topics metric

+1 (non-binding) This will be good to have, thanks David! Bob On Wed, Aug 14, 2019 at 6:08 AM Mickael Maison wrote: > +1 non binding > Thank you! > > On Tue, Aug 13, 2019 at 9:07 PM Stanislav Kozlovski > wrote: > > > > +1 (non-binding) > > > > Thanks for the simple but very useful KIP! > > Be

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

Thanks for the review Konstantine! I think the terminology suggestion definitely makes things clearer - I will update the documentation based on your suggestion (e.g. s/Consumer/Sink Converter/g and s/Producer/Source Converter/g). Cheers, Almog On Wed, Aug 14, 2019 at 8:13 AM Konstantine Karanta

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

Thanks Almog for preparing this KIP! I think it will improve usability and troubleshooting with JSON data a lot. The finalized plan seems quite concrete now. I also liked that some implementation specific implications (such as setting the ObjectMapper to deserialize floating point as BigDecimal) a

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

+1 (non-binding) I found it very useful for Spark's case. (Discussion on KIP-505 described it.) Thanks for driving the effort! 2019년 8월 14일 (수) 오후 8:49, Mickael Maison 님이 작성: > Hi Guozhang, > > Thanks for taking a look. > > 1. Right, I updated the titles of the code blocks > > 2. Yes that's a g

Re: [VOTE] KIP-497: Add inter-broker API to alter ISR

+1 binding, this looks great! -David On Tue, Aug 13, 2019 at 4:55 PM Guozhang Wang wrote: > +1 (binding). This is a great KIP, thanks Jason! > > Regarding the naming of the zkVersion, I'm actually fine to name it more > generally and leave a note that at the moment its value is defined as the >

Build failed in Jenkins: kafka-trunk-jdk8 #3851

See Changes: [ismael] MINOR: Avoid unnecessary leaderFor calls when ProducerBatch queue empty [github] KAFKA-8765: Remove interface annotations in Streams API (#7174) [gwen] KAFKA-8792; Default ZK configuration

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks for the KIP Arjun. FYI, I left a few comments on the discussion thread, but mentioning here too since I noticed that the vote started a few hours ago. Konstantine On Wed, Aug 14, 2019 at 12:13 AM Cyrus Vafadari wrote: > I am excited to see this implemented +1 nonbinding > > On Tue, Aug

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

And one thing I forgot is also related to Chris's comment above. I agree that an example on how a user is expected to set the log level (for instance to DEBUG) would be nice, even if it's showing only one out of the many possible ways to achieve that. - Konstantine On Wed, Aug 14, 2019 at 4:38 PM

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks Arjun for tackling the need to support this very useful feature. One thing I noticed while reading the KIP is that I would have loved to see more info regarding how this proposal depends on the underlying logging APIs and implementations. For instance, my understanding is that slf4j can not

Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface

Hi Rajini, Thanks for the KIP! I really like that authorize() will be able to take a batch of requests, this will speed up many implementations! On Tue, Aug 13, 2019 at 5:57 PM Rajini Sivaram wrote: > > Thanks David! I have fixed the typo. > > Also made a couple of changes to make the context in

Re: [VOTE] KIP-503: deleted topics metric

+1 non binding Thank you! On Tue, Aug 13, 2019 at 9:07 PM Stanislav Kozlovski wrote: > > +1 (non-binding) > > Thanks for the simple but very useful KIP! > Best, > Stanislav > > On Tue, Aug 13, 2019 at 8:32 PM Harsha Chintalapani wrote: > > > +1 (binding) > > > > Thanks, > > Harsha > > > > > > On

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

Sorry, reading my email the second time I probably wasn't clear. So basically the concept is that there is a user who can add other users as resources (such as userB and userC) prior to creating the "userA can create delegation token for userB and userC" association with CreateTokens. To limit who

Build failed in Jenkins: kafka-trunk-jdk8 #3850

See Changes: [bbejeck] KAFKA-8736: Track size in InMemoryKeyValueStore (#7177) [wangguoz] MINOR: remove unnecessary #remove overrides (#7178) [github] KAFKA-8791: RocksDBTimestampedStore should open in regular

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

Hi Guozhang, Thanks for taking a look. 1. Right, I updated the titles of the code blocks 2. Yes that's a good idea. I've updated the KIP Thank you On Wed, Aug 14, 2019 at 11:05 AM Mickael Maison wrote: > > Hi Colin, > > Thanks for raising these 2 valid points. I've updated the KIP accordingly

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

Hi, Why do we need new ACL operation "CreateUsers"? I think, "CreateTokens" Operation is sufficient to create "UserA can create tokens for UserB, UserC" association. Thanks, On Tue, Aug 13, 2019 at 3:37 PM Viktor Somogyi-Vass wrote: > Hi Manikumar, > > Yea, I just brought up superuser for t

Build failed in Jenkins: kafka-trunk-jdk11 #752

See Changes: [wangguoz] MINOR: remove unnecessary #remove overrides (#7178) [github] KAFKA-8791: RocksDBTimestampedStore should open in regular mode by [ismael] MINOR: Avoid unnecessary leaderFor calls when Pro

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

Hi Colin, Thanks for raising these 2 valid points. I've updated the KIP accordingly. On Tue, Aug 13, 2019 at 9:50 PM Guozhang Wang wrote: > > Hi Mickael, > > Thanks for the KIP! > > Just some minor comments. > > 1. Java class names are stale, e.g. "CommitOffsetsOptions.java" should be > "AlterOf

[jira] [Created] (KAFKA-8801) electLeaderForPartitions redundancy for some success elect partitions

shilin Lu created KAFKA-8801: Summary: electLeaderForPartitions redundancy for some success elect partitions Key: KAFKA-8801 URL: https://issues.apache.org/jira/browse/KAFKA-8801 Project: Kafka