Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

Hi John, Thanks for the feedback. I will open the Vote thread now. Best, Dongjin On Fri, Sep 11, 2020 at 2:00 AM John Roesler wrote: > Hi Dongjin, > > Sorry for the delay. I'm glad you're still pushing this > forward. It would be nice to get this in to the 2.7 release. > > I just took another

[VOTE] KIP-508: Make Suppression State Queriable

Hi devs, I hope to start the vote for KIP-508: Make Suppression State Queriable - it makes suppression state queriable by introducing KTable#suppress(Suppressed, Materialized) to the public API. Thanks, Dongjin -- *Dongjin Lee* *A hitchhiker in the mathematical world.* *github:

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

Hi All, Here is the voting thread: https://lists.apache.org/thread.html/r5653bf2dafbb27b247bf20dbe6f070c151b3823d96c9c9ca94183e20%40%3Cdev.kafka.apache.org%3E Thanks, Dongjin On Fri, Sep 11, 2020 at 4:23 PM Dongjin Lee wrote: > Hi John, > > Thanks for the feedback. I will open the Vote thread

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #57

See Changes: [github] KAFKA-5636: Add Sliding Windows documentation (#9264) -- [...truncated 3.28 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [DISCUSS] KIP-516: Topic Identifiers

Hi Justine, This looks like a very welcome improvement. Thanks! Maybe I missed it, but the KIP doesn't seem to mention changing DeleteTopicsRequest to identify the topic using an id. Maybe that's out of scope, but DeleteTopicsRequest is not listed among the Future Work APIs either. Kind regards,

Re: [DISCUSS] KIP idea: Support of multipart messages

See my answers below, On Thu, Sep 10, 2020 at 5:03 PM Gwen Shapira wrote: > There is another option of doing the splitting on the server and hiding > this from the clients. My personal (and highly controversial) take is that > Kafka clients could use less complexity rather than more. They are >

Re: [DISCUSS] KIP idea: Support of multipart messages

Hello, I don't have an answer for this. Ideally, the bigger the better. For our use case: working with Web data, hundreds of megabytes should be enough. But there could be other use cases, such as transferring of binary or media files. There is another dimension to consider: amount of large messa

Re: [DISCUSS] KIP-654 Aborting Transaction with non-flushed data should throw a non-fatal Exception

All, As the vote has passed, I have raised a PR here: https://github.com/apache/kafka/pull/9280 Please help review. Thanks, Gokul On 04-09-2020 00:48, Gokul Srinivas wrote: Appreciate the help! On 04-09-2020 00:46, Sophie Blee-Goldman wrote: Yep, you can go ahead and call for a vote on the

Jenkins build is back to normal : Kafka » kafka-trunk-jdk11 #57

See

[jira] [Created] (KAFKA-10476) Error when calling (kafka-configs.sh --add-config): org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient..... Caused by: javax.crypto.BadPadding

AbdulRahman Mahmoud created KAFKA-10476: --- Summary: Error when calling (kafka-configs.sh --add-config): org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient. Caused by: javax.crypto.BadPaddingException: Given fin

Re: [DISCUSS] KIP-516: Topic Identifiers

Hello Justine, Thanks for the KIP! I happen to have been confronted recently with the need to keep track of a large number of topics as compactly as possible. I was going to come up with some way to dictionary encode the topic names as integers, but this seems much better! Apologies if this h

Re: [DISCUSS] KIP-516: Topic Identifiers

Hi. Thanks for the KIP. Have you thought about whether it makes sense to support authorizing a principal for a topic ID rather than a topic name to achieve tighter security? Or is the topic ID fundamentally an internal detail similar to epochs used in a bunch of other places in Kafka? Thanks.

Kip creation permission

Can you give me permission to create a Kip? username: wcarlson thanks, Walker

Re: Kip creation permission

Done. On 9/11/20 9:02 AM, Walker Carlson wrote: > Can you give me permission to create a Kip? username: wcarlson > > thanks, > Walker > signature.asc Description: OpenPGP digital signature

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

Hi Unmesh, I think you're right that we should use a duration here rather than a time. As you said, the clock on the controller will probably not match the one on the broker. I have updated the KIP. > > It's important to keep in mind that messages may be delayed in the > > network, or arrive

Re: [VOTE] KIP-664: Provide tooling to detect and abort hanging transactions

+1 (non-binding) Thanks Jason! On Tue, Sep 8, 2020 at 5:28 PM Guozhang Wang wrote: > +1. Thanks! > > Guozhang > > On Tue, Sep 8, 2020 at 3:04 PM Ron Dagostino wrote: > > > +1 (non-binding) -- Thanks, Jason! > > > > Ron > > > > On Tue, Sep 8, 2020 at 2:04 PM Jason Gustafson > wrote: > > > > >

Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor

Thanks Mickael, I think adding the new Exception resolves my concerns. On Thu, Sep 3, 2020 at 9:47 AM Mickael Maison wrote: > Thanks Robert and Ryanne for the feedback. > > ReplicaAssignor implementations can throw an exception to indicate an > assignment can't be computed. This is already what

Re: [DISCUSS] KIP-516: Topic Identifiers

Hello all, Thanks for continuing the discussion! I have a few responses to your points. Tom: You are correct in that this KIP has not mentioned the DeleteTopicsRequest. I think that this would be out of scope for now, but may be something worth adding in the future. John: We did consider sequence

Re: [DISCUSS] KIP-516: Topic Identifiers

Thanks, Justine! Your response seems compelling to me. -John On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote: > Hello all, > Thanks for continuing the discussion! I have a few responses to your points. > > Tom: You are correct in that this KIP has not mentioned the > DeleteTopicsRequest

[DISCUSS] KIP-671: Shutdown Streams Application when appropriate exception is thrown

Hello all, I have created KIP-671 to give the option to shutdown a streams application in response to an error. https://cwiki.apache.org/confluence/display/KAFKA/KIP-671%3A+Shutdown+Streams+Application+when+appropriate+exception+is+thrown This is because of the Jira ticket

Re: Contributor Access to JIRA

Hi, I haven't got any response from you all. Please give contributor access to Kafka JIRA. On Thu, Sep 3, 2020 at 7:19 PM Kp k wrote: > Hi, > > Can you please provide me Contributor access to Kafka JIRA, as I am > interested in contributing. > > Thanks, > Kalpitha >

Re: Contributor Access to JIRA

There was a reply on 9/3. > Please create an account (self-service) and share you account info here, > so we can add you. Did you miss it? (There was multiple emails with the same subject and GMail might have collapsed both threads into one. -Matthias On 9/11/20 10:35 AM, Kp k wrote: > Hi, >

Re: [DISCUSS] KIP-671: Shutdown Streams Application when appropriate exception is thrown

Thanks for the KIP. It seem that the new exception would need to be thrown by user code? However, in the motivation you mention the scenario of a missing source topic that a user cannot detect, but KafkaStreams runtime would be responsible to handle. How do both things go together? -Matthias O

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

Thanks for updating the KIP. I think there is still one open question. `suppress()` can be used on non-windowed KTable for rate control, as well as on a windowed-KTable (also for rate control, but actually mainly) for only emitting the final result of a windowed aggregation. For the non-windowed c

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

I think separating the different contexts make sense. In fact, we could even go one step further and remove the record context from the processor context completely and we add a third parameter to `process(key, value, recordContext)`. This would make it clear that the context is for the input reco

Re: How do I place a JIRA in status "Patch Available"? (For KAFKA-10473)

Hi James, Sorry, I overlooked your reply until now. I've granted you access. Thanks, -John On Wed, 2020-09-09 at 21:44 -0700, James Cheng wrote: > Thanks John. My wiki user ID is wushujames > > -James > > Sent from my iPhone > > > On Sep 9, 2020, at 7:03 PM, John Roesler wrote: > > > > Hi

[jira] [Created] (KAFKA-10477) Sink Connector fails with DataException when trying to convert Kafka record with empty key to Connect Record

Shaik Zakir Hussain created KAFKA-10477: --- Summary: Sink Connector fails with DataException when trying to convert Kafka record with empty key to Connect Record Key: KAFKA-10477 URL: https://issues.apache.org

Re: KIP-669: Preserve Source Partition in Kafka Streams from context

With regard to KIP-478, there is the idea to introduce a `RecordContext` class. Thus, we could just change the `StreamPartitioner` to take this new class as parameter to `partition()`? This might actually kill two birds with one stone, because I could imagine use cases in which users want to parti

Re: KIP-658 Deprecate all setters of Headers

Well, I am not totally sure if immutability is actually desired for `Headers`? Consider the case when there is an application that just wants to add for example a tracing header field. It would just call `add()` on the consumer record's headers and pass the modified `Headers` object to the `Produc

[jira] [Resolved] (KAFKA-7970) Missing topic causes service shutdown without exception

[ https://issues.apache.org/jira/browse/KAFKA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-7970. Fix Version/s: 2.7.0 Assignee: Bruno Cadonna Resolution: Fixed > Mi

Re: [DISCUSS] KIP-634: Complementary support for headers in Kafka Streams DSL

Jorge, thanks a lot for this KIP. Being able to modify headers is a very valuable feature. However, before we actually expose them in the DSL, I am wondering if we should improve how headers can be modified in the PAPI? Currently, it is possible but very clumsy to work with headers in the Process

Re: [DISCUSS] KIP-671: Shutdown Streams Application when appropriate exception is thrown

Hey Walker, The proposal makes sense to me, but while reading up on those old tickets I started wondering if we should give users two options: one that would shut down the entire application, as described in the current KIP, and another that would only shut down an individual instance. I think th

Re: How do I place a JIRA in status "Patch Available"? (For KAFKA-10473)

Thanks John! I can access and edit the wiki now. And I improved the instructions for how to change the status of a JIRA to "Patch Available". https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=59689925&selectedPageVersions=36&selectedPageVersions=37

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #58

See Changes: [github] Adding reverse iterator usage for sliding windows processing (extending KIP-450) (#9239) -- [...truncated 3.29 MB...] org.apache.kafka

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #58

See Changes: [github] Adding reverse iterator usage for sliding windows processing (extending KIP-450) (#9239) -- [...truncated 6.57 MB...] org.apache.kafka

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #56

See Changes: [github] Adding reverse iterator usage for sliding windows processing (extending KIP-450) (#9239) -- [...truncated 6.52 MB...] org.apache.kafka.s

Re: How do I place a JIRA in status "Patch Available"? (For KAFKA-10473)

Thanks, James! That change looks good to me. -John On Fri, 2020-09-11 at 15:54 -0700, James Cheng wrote: > Thanks John! I can access and edit the wiki now. > > And I improved the instructions for how to change the status of a JIRA to > "Patch Available". > https://cwiki.apache.org/confluence/

[jira] [Created] (KAFKA-10478) advertised.listeners should allow duplicated ports

Andre Araujo created KAFKA-10478: Summary: advertised.listeners should allow duplicated ports Key: KAFKA-10478 URL: https://issues.apache.org/jira/browse/KAFKA-10478 Project: Kafka Issue Type