Jenkins build is back to normal : kafka-2.1-jdk8 #230

See

Re: [DISCUSS] KIP-479: Add Materialized to Join

Hi Bill, For the record, the current proposal looks good to me also. Thanks, -John On Tue, Sep 17, 2019 at 5:06 PM Matthias J. Sax wrote: > > > Just to clarify I'll update `as(StoreSupplier, StoreSupplier)` to > > `with(..., ...)` and change the class name from `StreamJoined` to > > `StreamJoin

Build failed in Jenkins: kafka-trunk-jdk8 #3907

See Changes: [wangguoz] MINOR: Default to 5 partitions of the __consumer_offsets topic in [wangguoz] KAFKA-8839 : Improve streams debug logging (#7258) -- [...truncated 8

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Kevin, Thanks for adding useful metrics with the KIP. On Wed, 18 Sep, 2019, 1:49 AM Kevin Lu, wrote: > Hi Manikumar, > > Thanks for the support. > > Since we have added a couple additional metrics, I have renamed the KIP > title to reflect the content better: KIP-517: Add consumer metrics to

[jira] [Resolved] (KAFKA-8919) Flaky Test kafka.api.AuthorizerIntegrationTest.testSimpleConsumeWithOffsetLookupAndNoGroupAccess

[ https://issues.apache.org/jira/browse/KAFKA-8919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-8919. -- Resolution: Fixed Realize it is fixed as part of polling (1L) in https://github.com/apache/kaf

Build failed in Jenkins: kafka-2.2-jdk8 #166

See Changes: [matthias] KAFKA-8913: Document topic based configs & ISR settings for Streams apps -- Started by an SCM change [EnvInject] - Loading node environment variables

Build failed in Jenkins: kafka-2.0-jdk8 #294

See Changes: [matthias] KAFKA-8913: Document topic based configs & ISR settings for Streams apps -- Started by an SCM change [EnvInject] - Loading node environment variables

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

Thanks for picking up this KIP. My personal take about repartition topics is, that it seems to be ok to apply the handler for those, too (in addition to output topics). It seems to be more flexible and also simplifies the code. In the end, the topic name is passed via `ProducerRecord` into the han

Re: [DISCUSS] KIP-479: Add Materialized to Join

> Just to clarify I'll update `as(StoreSupplier, StoreSupplier)` to > `with(..., ...)` and change the class name from `StreamJoined` to > `StreamJoin` Thanks Bill. SGTM. -Matthias On 9/17/19 4:52 PM, aishwarya kumar wrote: > Hi Bill, > > Thanks for clarifying, and the KIP looks great!! > >

Re: [DISCUSS] KIP-479: Add Materialized to Join

Hi Bill, Thanks for clarifying, and the KIP looks great!! Best regards, Aishwarya On Tue, Sep 17, 2019, 6:16 PM Bill Bejeck wrote: > Hi Aishwarya, > > On Tue, Sep 17, 2019 at 1:41 PM aishwarya kumar > wrote: > > > Will this be applicable to Kstream-Ktable joins as well? Or do users > always >

Jenkins build is back to normal : kafka-trunk-jdk11 #816

See

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks so much everyone for your time and comments. Closing out this vote with three binding votes (Jason, Gwen and Randall) and 3 non-binding (Chris, Cyrus and Konstantine). Much appreciated! On Tue, Sep 17, 2019 at 10:41 AM Randall Hauch wrote: > +1 (binding). > > Thanks for adding this, Ar

[jira] [Created] (KAFKA-8919) Flaky Test kafka.api.AuthorizerIntegrationTest.testSimpleConsumeWithOffsetLookupAndNoGroupAccess

Guozhang Wang created KAFKA-8919: Summary: Flaky Test kafka.api.AuthorizerIntegrationTest.testSimpleConsumeWithOffsetLookupAndNoGroupAccess Key: KAFKA-8919 URL: https://issues.apache.org/jira/browse/KAFKA-8919

Re: [DISCUSS] KIP-479: Add Materialized to Join

Hi Aishwarya, On Tue, Sep 17, 2019 at 1:41 PM aishwarya kumar wrote: > Will this be applicable to Kstream-Ktable joins as well? Or do users always > materialize these joins explicitly? > No, this change applies to KStream-KStream joins only. With KStream-KTable joins KafkaStreams has materiali

Issues with partition assignments after Kafka Connect 2.3 Upgrade

Hello all, After an upgrade to Kafka Connect 2.3.0, we've started seeing an abundance of Exceptions pertaining to Kafka Connect Workers and their partition assignments. The exception is: java.lang.IllegalStateException: No current assignment for partition [redacted] at org.apache.kafka.clients.co

Re: [DISCUSS] KIP-479: Add Materialized to Join

Just to clarify I'll update `as(StoreSupplier, StoreSupplier)` to `with(..., ...)` and change the class name from `StreamJoined` to `StreamJoin` -Bill On Tue, Sep 17, 2019 at 2:09 PM Bill Bejeck wrote: > Thanks for the comments, Matthias. > > I like both suggestions regarding the names and I'll

[jira] [Created] (KAFKA-8918) Flaky Test org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskCancellation

Guozhang Wang created KAFKA-8918: Summary: Flaky Test org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskCancellation Key: KAFKA-8918 URL: https://issues.apache.org/jira/browse/KAFKA-8918 Pr

Re: [DISCUSS] KIP-479: Add Materialized to Join

Thanks for the comments, Matthias. I like both suggestions regarding the names and I'll update the KIP accordingly. On Tue, Sep 17, 2019 at 11:22 AM Matthias J. Sax wrote: > Thanks Bill! > > Using a new configuration object was suggested by John in the original > DISCUSS thread already. We reje

Re: [DISCUSS] KIP-479: Add Materialized to Join

Will this be applicable to Kstream-Ktable joins as well? Or do users always materialize these joins explicitly? I'm not sure if its even necessary (or makes sense), just trying to understand why the change is applicable to Kstream joins only? Best, Aishwarya On Tue, Sep 17, 2019 at 4:05 PM Bill B

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Manikumar, Thanks for the support. Since we have added a couple additional metrics, I have renamed the KIP title to reflect the content better: KIP-517: Add consumer metrics to observe user poll behavior

Re: [DISCUSS] KIP-479: Add Materialized to Join

Guozhang, Thanks for the comments. Yes, you are correct in your assessment regarding names, *if* the users provide their own StoreSuppliers When constructing as StoreSupplier, the name can't be null, and additionally, there is no way to update the name. Further downstream, the underlying StateSt

[DISCUSS] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

Dev team, Requesting discussion for improvement to the producer when dealing with a large number of topics. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-526%3A+Reduce+Producer+Metadata+Lookups+for+Large+Number+of+Topics JIRA: https://issues.apache.org/jira/browse/KAFKA-8904 Though

Re: [VOTE] KIP-481: SerDe Improvements for Connect Decimal type in JSON

+1 (binding). Thanks Almog! Guozhang On Tue, Sep 17, 2019 at 11:56 AM Bill Bejeck wrote: > +1 (binding) > > On Tue, Sep 17, 2019 at 11:32 AM Matthias J. Sax > wrote: > > > +1 (binding) > > > > -Matthias > > > > On 9/17/19 12:20 AM, Arjun Satish wrote: > > > Thanks for the KIP, Almog. > > > >

About Testing Stream Applications Documentation

Hi, I am not sure if it is the right email address to write about this topic, please correct me if I am wrong. As I read documentation of Testing Stream Application, I believe I noticed wrong naming of method called shouldNotUpdateStoreForLargerValue. As I understand it, either the name should ch

Re: [VOTE] KIP-481: SerDe Improvements for Connect Decimal type in JSON

Thanks everyone for the comments and discussion! Closing the voting out for this KIP and marking accepted: * 3 Binding (Randall, Matthias, Bill) * 2 Non-Binding (Arjun, Konstantine) Cheers, Almog On Tue, Sep 17, 2019 at 11:56 AM Bill Bejeck wrote: > +1 (binding) > > On Tue, Sep 17, 2019 at 11:

Re: [VOTE] KIP-481: SerDe Improvements for Connect Decimal type in JSON

+1 (binding) On Tue, Sep 17, 2019 at 11:32 AM Matthias J. Sax wrote: > +1 (binding) > > -Matthias > > On 9/17/19 12:20 AM, Arjun Satish wrote: > > Thanks for the KIP, Almog. > > > > +1 (non-binding) > > > > On Mon, Sep 16, 2019 at 4:45 PM Randall Hauch wrote: > > > >> Thanks for the nice improv

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

Aishwarya, thanks for the KIP. Overall, I think it makes sense to allow converting a KStream into a KTable. From the KIP: > materializing these KTables should only be allowed if the overloaded function > with Materialized is used (and if optimization is turned on it may still be > only logical

Re: [VOTE] KIP-481: SerDe Improvements for Connect Decimal type in JSON

+1 (binding) -Matthias On 9/17/19 12:20 AM, Arjun Satish wrote: > Thanks for the KIP, Almog. > > +1 (non-binding) > > On Mon, Sep 16, 2019 at 4:45 PM Randall Hauch wrote: > >> Thanks for the nice improvement, Almog! >> >> +1 (binding) >> >> Randall >> >> On Thu, Aug 15, 2019 at 11:59 AM Konst

Re: [DISCUSS] KIP-479: Add Materialized to Join

Thanks Bill! Using a new configuration object was suggested by John in the original DISCUSS thread already. We rejected it because we wanted to avoid a new configuration class. However, given your analysis, it seems it's actually the right choice to introduce a new configuration class. Hence, ove

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Kevin, Thanks for the KIP. LGTM. This will be useful. Thanks, On Mon, Sep 16, 2019 at 10:17 PM Harsha Chintalapani wrote: > Thanks. +1 LGTM. > > > On Mon, Sep 16, 2019 at 9:19 AM, Kevin Lu wrote: > > > Hi Harsha, > > > > Thanks for the feedback. I have added *last-poll-seconds-ago* to the

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

+1 (binding). Thanks for adding this, Arjun! On Thu, Sep 12, 2019 at 11:58 AM Gwen Shapira wrote: > +1 (binding) - Thank you. Love the new REST API for logs. > > On Tue, Aug 20, 2019 at 3:05 PM Konstantine Karantasis > wrote: > > > > Thanks for the updates on the KIP Arjun! > > > > +1 by me (n

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

Hi Randall, I've added your suggested paragraph to the KIP; it definitely clarifies the intended behavior more than the content that it replaced. Thanks! As far as I can tell, the two items remaining open for discussion are the names for the new configs (which should be made less REST request-spe

Re: [DISCUSS] KIP-479: Add Materialized to Join

Hello Bill, Thanks for the updated KIP. I made a pass on the StreamJoined section. Just a quick question from user's perspective: if a user wants to provide a customized store-supplier, she is forced to also provide a name via the store-supplier. So there's no way to say "I want to provide my own

[jira] [Created] (KAFKA-8917) When performing a Left/Right-Join, pick the headers of the same side

Pascal Büttiker created KAFKA-8917: -- Summary: When performing a Left/Right-Join, pick the headers of the same side Key: KAFKA-8917 URL: https://issues.apache.org/jira/browse/KAFKA-8917 Project: Kafka

Re: [VOTE] KIP-496: Administrative API to delete consumer offsets

Hi all, We haven't included the changes in the command line tool to support the new API. Therefore, I would like to amend the current KIP to cover the changes in the `kafka-consumer-groups` command line tool. The change is rather small and it does not need to add any new arguments to the command l

Re: [DISCUSS] KIP-479: Add Materialized to Join

Bumping this discussion as we need to re-vote before the KIP deadline. On Fri, Sep 13, 2019 at 10:29 AM Bill Bejeck wrote: > Hi All, > > While working on the implementation of KIP-479, some issues came to light > that the KIP as written won't work. I have updated the KIP with a solution > I bel

Re: Requesting Permission

Hi, Brian, Thanks for your interest. Just gave you the wiki permission. Jun On Mon, Sep 16, 2019 at 7:10 PM Brian Byrne wrote: > Hello, > > I'm requesting permission to the Kafka Wiki, specifically to create a KIP. > Wiki ID is 'bbyrne'. > > Thanks, > Brian >

Re: [DISCUSS] KIP-491: Preferred Leader Deprioritized List (Temporary Blacklist)

Hey Harsha, > If we want to go with making this an option and providing a tool which abstracts moving the broker to end preferred leader list , it needs to do it for all the partitions that broker is leader for. As said in the above comment a broker i.e leader for 1000 partitions we have to this f

[jira] [Created] (KAFKA-8916) Unreliable kafka-reassign-partitions.sh affecting performance

VinayKumar created KAFKA-8916: - Summary: Unreliable kafka-reassign-partitions.sh affecting performance Key: KAFKA-8916 URL: https://issues.apache.org/jira/browse/KAFKA-8916 Project: Kafka Issue

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

Looks great. Thanks again, Arjun! On Tue, Sep 17, 2019 at 1:17 AM Arjun Satish wrote: > Answers inline > > On Mon, Sep 16, 2019 at 5:06 PM Randall Hauch wrote: > > > Thanks for the updates, Arjun. If possible, it'd be great to have the KIP > > clarify a few things: > > > > 1) IIUC, the loggers

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

Hi All, Keeping this thread alive!! The aim is to add two methods Kstream.toTable() & Kstream.toTable(Materialized), so users can choose to convert their event stream into a changelog stream at any stage. wiki link : https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable

Build failed in Jenkins: kafka-trunk-jdk8 #3906

See Changes: [wangguoz] KAFKA-8859: Expose built-in streams metrics version in -- [...truncated 6.07 MB...] org.apache.kafka.trogdor.common.WorkerUtilsTest > testCreateT

Build failed in Jenkins: kafka-trunk-jdk11 #815

See Changes: [wangguoz] KAFKA-8859: Expose built-in streams metrics version in -- [...truncated 6.08 MB...] org.apache.kafka.connect.transforms.TimestampConverterTest > t

RE: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

Good point about the two callers of SslFactory. We can move the SslEngine validation to a separate class and call it in both places. That SslEngine validation class would not be part of the public API and therefore we don't need to fuss about its API. -Original Message- From: Maulin Va

Re: [DISCUSS] KIP-525 - Return topic metadata and configs in CreateTopics response

Hi all, Since this is minor KIP, I will start vote tomorrow if there are no concerns. Thank you, Rajini On Fri, Sep 13, 2019 at 10:17 PM Rajini Sivaram wrote: > Hi all, > > I would like to start discussion on KIP-525 to return topic configs in > CreateTopics response: > >- > > https:/

[jira] [Created] (KAFKA-8915) 无法修改partition

lingyi.zhong created KAFKA-8915: --- Summary: 无法修改partition Key: KAFKA-8915 URL: https://issues.apache.org/jira/browse/KAFKA-8915 Project: Kafka Issue Type: Bug Reporter: lingyi.zhong

Re: [DISCUSS] KIP-511: Collect and Expose Client's Name and Version in the Brokers

Hey Jose, Yes, we have considered it. Check "Put clientName and clientVersion in the RequestHeader" in the rejected alternatives. Best, David On Tue, Sep 17, 2019 at 12:57 AM Jose Armando Garcia Sancio < jsan...@confluent.io> wrote: > On Mon, Sep 9, 2019 at 4:00 PM Colin McCabe wrote: > > > >

Re: [VOTE] KIP-481: SerDe Improvements for Connect Decimal type in JSON

Thanks for the KIP, Almog. +1 (non-binding) On Mon, Sep 16, 2019 at 4:45 PM Randall Hauch wrote: > Thanks for the nice improvement, Almog! > > +1 (binding) > > Randall > > On Thu, Aug 15, 2019 at 11:59 AM Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Thanks Almog! > > Nicely