Re: Request for wiki permission to create KIP

Hi Kamal, I just gave you wiki permissions; Thanks, On Sat, Dec 1, 2018 at 11:29 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi, > > I would like to propose a KIP for KAFKA-7499 > . Please grant me > permission to create K

Request for wiki permission to create KIP

Hi, I would like to propose a KIP for KAFKA-7499 . Please grant me permission to create KIP. ID: ckamal

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

Thanks Jason for the reply! Since the overall motivation and design is pretty clear, I will go ahead to start implementation and we could discuss the underlying details in the PR. Best, Boyang From: Matthias J. Sax Sent: Saturday, December 1, 2018 3:12 AM To: de

[jira] [Resolved] (KAFKA-4219) Permit setting of event time in stream processor

[ https://issues.apache.org/jira/browse/KAFKA-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4219. -- Resolution: Fixed Fix Version/s: 2.0.0 > Permit setting of event time in stream processo

Re: [VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

Hi All, Thanks for the participation and votes. This vote has passed and the KIP is accepted! +1 (binding) votes: - Manikumar - Gwen - Jason +1 (non-binding) votes: - Mickael - Bill Regards, Kevin On Fri, Nov 30, 2018 at 1:45 PM Jason Gustafson wrote: > +1 Thanks for the KIP! > > -Jason > >

Build failed in Jenkins: kafka-trunk-jdk11 #130

See Changes: [github] MINOR: improve state directory test (#5961) -- [...truncated 2.24 MB...] org.apache.kafka.connect.json.JsonConverterTest > nullSchemaAndMapToJson PA

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hi Boyang, For Streams, I think we do not need an extra config for the instance id, instead, we can re-use the way we construct the embedded consumer's client id as: [streams client-id] + "-StreamThread-" + [thread-id] + "-consumer" So as long as user's specify the unique streams client-id, the

Build failed in Jenkins: kafka-trunk-jdk8 #3233

See Changes: [github] MINOR: improve state directory test (#5961) -- [...truncated 2.61 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldPassIfKeyAndValue

Re: [VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

+1 Thanks for the KIP! -Jason On Wed, Nov 28, 2018 at 5:26 PM Kevin Lu wrote: > @Manikumar I have updated the Compatibility section to note that this > option is not supported with the deprecated "--zookeeper" option. > > We still need 1 more binding vote for this to be accepted. > > Thanks~ >

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hi Adam, I'd agree with the minimal approach. As far as I'm concerned, it's fine to just implement inner join for starters. Thanks for considering the API feedback! Regarding #6, yes, please scrutinize it! It *seems* right to me, but it also seems suspiciously too simple. I haven't done a proof

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

Thanks Sönke. > it just feels to me like an awful lot of Connect functionality would need to be reimplemented or at least wrapped Connect currently has two drivers, ConnectDistributed and ConnectStandalone. Both set up a Herder, which manages Workers. I've implemented a third driver which sets up

[jira] [Resolved] (KAFKA-7660) Stream Metrics - Memory Analysis

[ https://issues.apache.org/jira/browse/KAFKA-7660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7660. -- Resolution: Fixed Assignee: John Roesler Fix Version/s: 2.2.0 > Stream Metrics

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

SGTM. On 11/30/18 10:17 AM, Jason Gustafson wrote: > Using the session expiration logic we already have seems like the simplest > option (this is probably a one or two line change). The rejoin should be > quick anyway, so I don't think it's worth optimizing for unjoined new > members. Just my two

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

Using the session expiration logic we already have seems like the simplest option (this is probably a one or two line change). The rejoin should be quick anyway, so I don't think it's worth optimizing for unjoined new members. Just my two cents. This is more of an implementation detail, so need not

[DISCUSS] KIP-398: Support reading trust store from classpath

I wrote a KIP for my minimal suggested change to support reading a truststore from the classpath as well as from a file. The KIP is available here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-398%3A+Support+reading+trust+store+from+classpath

Build failed in Jenkins: kafka-trunk-jdk8 #3232

See Changes: [manikumar.reddy] KAFKA-7617: Add authorization primitives to security page -- [...truncated 2.61 MB...] org.apache.kafka.connect.runtime.ConnectMetricsTest

kafka??????????

??kafka ??100kafka??broker.

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

I like Guozhang's suggestion to not have to wait for session timeout in case we know that we want to downsize the consumer group and redistribute the partitions among the remaining consumers. IIUC, with the above suggestions, the admin api "removeMemberFromGroup(groupId, list[instanceId])" or "remo

Heads up: spotBugs enabled for Java 11 builds

Hi, As per https://issues.apache.org/jira/browse/KAFKA-7389, spotBugs is now enabled for Java 11 builds and the Java 11 PR build will fail for old PRs that don't include the commit from KAFKA-7389. The fix is to rebase/merge. Ismael

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hi John Thanks for the feedback. It is extremely valuable. Responses as follows: 1. I could definitely override the join function. The API you proposed is fine by me: KTable join(KTable other, ValueMapper foreignKeyExtractor, ValueJoiner joiner,

[DISCUSS] KIP-397: Add methods to override fetch offsets based on timestamp

Hello! I added a KIP to to add a method that would determine the offsets for a list of topic partitions based on offset and update the consumer assignment to seek to those offsets. https://cwiki.apache.org/confluence/display/KAFKA/KIP-397%3A+Add+methods+to+override+fetch+offsets+based+on+timestam

[jira] [Resolved] (KAFKA-7676) Compile Kafka failed with Gradle 5.0

[ https://issues.apache.org/jira/browse/KAFKA-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-7676. Resolution: Cannot Reproduce > Compile Kafka failed with Gradle 5.0 > -

Build failed in Jenkins: kafka-trunk-jdk8 #3231

See Changes: [manikumar.reddy] KAFKA-7259; Remove deprecated ZKUtils usage from ZkSecurityMigrator [me] MINOR: Fix handling of dummy record in EndToEndLatency tool [me] MINOR: Add logging to Connect SMTs [me]

[jira] [Created] (KAFKA-7691) Encypt-then-MAC Delegation token metadata

Attila Sasvari created KAFKA-7691: - Summary: Encypt-then-MAC Delegation token metadata Key: KAFKA-7691 URL: https://issues.apache.org/jira/browse/KAFKA-7691 Project: Kafka Issue Type: Improve

Re: [DISCUSS] KIP-396: Add Commit/List Offsets Operations to AdminClient

Thanks for the KIP Mickael! two initial observations `commitOffsets` methods could be renamed to `commitConsumerGroupOffsets` to better match the existing `listConsumerGroupOffsets` `ListOffsetsOptions` offers you a per-request isolation level option and timestamp, but timestamp is a per-parti

[DISCUSS] KIP-396: Add Commit/List Offsets Operations to AdminClient

Hi, I've written a KIP to add commit/list offsets operation to the AdminClient: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484 Please take a look and let me know if you have any feedback. Thanks Mickael

Build failed in Jenkins: kafka-2.0-jdk8 #194

See Changes: [me] MINOR: Add logging to Connect SMTs -- [...truncated 886.96 KB...] kafka.controller.PartitionStateMachineTest > testInvalidNonexistentPartitionToOfflinePart

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

Hi Ryanne, thanks. I missed the remote to remote replication scenario in my train of thought, you are right. That being said I have to admit that I am not yet fully on board with the concept, sorry. But I might just be misunderstanding what your intention is. Let me try and explain what I think i

[jira] [Resolved] (KAFKA-7617) Document security primitives

[ https://issues.apache.org/jira/browse/KAFKA-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-7617. -- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 5906 [https://github.com/a

Re: [DISCUSS] KIP-394: Require member.id for initial join group request

Thanks Matthias for the question. I'm thinking of having a separate hash set called `registeredMemberIds` which will be cleared out every time a group finishes one round of rebalance. Since storing one id is pretty trivial, using purgatory to track the id removal is a bit wasteful in my opinion.

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Hey Stanislav, What do you think about the use case I mentioned in my previous reply about > a more resilient self-service Kafka? I believe the benefit there is bigger. I see this config as analogous to the open file limit. Probably this limit was intended to be prescriptive at some point about