dev@kafka.apache.org

[ https://issues.apache.org/jira/browse/KAFKA-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464184#comment-15464184 ] zhang shuai commented on KAFKA-4119: Nobody get this problem ? > Get topic offset wit

Re: [DISCUSS] KIP-78: Cluster Id

Hi Dong, Please find my answers inline. Hopefully they address your concerns this time ! Sumit On Sun, Sep 4, 2016 at 4:54 PM, Dong Lin wrote: > Hey Ismael, > > Thanks for the explanation Ismael. Please see my comment inline. > > On Sun, Sep 4, 2016 at 8:58 AM, Ismael Juma wrote: > > > Hi Do

Build failed in Jenkins: kafka-trunk-jdk8 #860

See Changes: [wangguoz] KAFKA-4105: Queryable state tests -- [...truncated 3450 lines...] kafka.integration.SaslPlaintextTopicMetadataTest > testAutoCreateTopic STARTED kafka.integration.SaslPl

[jira] [Commented] (KAFKA-4120) byte[] keys in RocksDB state stores do not work as expected

[ https://issues.apache.org/jira/browse/KAFKA-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464171#comment-15464171 ] Greg Fodor commented on KAFKA-4120: --- We were able to work around it by just creating a p

[jira] [Commented] (KAFKA-4120) byte[] keys in RocksDB state stores do not work as expected

[ https://issues.apache.org/jira/browse/KAFKA-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464109#comment-15464109 ] Guozhang Wang commented on KAFKA-4120: -- Hi [~gfodor], thanks for reporting this issue

[jira] [Commented] (KAFKA-4105) Queryable state tests for concurrency and rebalancing

[ https://issues.apache.org/jira/browse/KAFKA-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464083#comment-15464083 ] ASF GitHub Bot commented on KAFKA-4105: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1806: KAFKA-4105: Queryable state tests

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1806 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-4105) Queryable state tests for concurrency and rebalancing

[ https://issues.apache.org/jira/browse/KAFKA-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4105. -- Resolution: Fixed Issue resolved by pull request 1806 [https://github.com/apache/kafka/pull/1806

[jira] [Commented] (KAFKA-3769) KStream job spending 60% of time writing metrics

[ https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15463861#comment-15463861 ] Greg Fodor commented on KAFKA-3769: --- I've done some additional profiling and I have foun

Re: [DISCUSS] KIP-78: Cluster Id

Hey Ismael, Thanks for the explanation Ismael. Please see my comment inline. On Sun, Sep 4, 2016 at 8:58 AM, Ismael Juma wrote: > Hi Dong, > > Sorry for the delay, I was offline for 24 hours. Thankfully Sumit was > around to explain the reasoning for the current approach. From reading the > thr

Re: Queryable state client read guarantees

Hi Eno, thanks for the response and sorry for not getting back earlier. I think it makes sense and the example is great! To make it possible to experiment with the guarantees/semantics, I've created a tool available at: https://github.com/mkhq/kafka-qs-verify. Basically it can be used to trace rea

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

> Processor code should always work; independently if caching is enabled or not. If we want to get this, I guess we need a quite different design (see (1)). The point is, that we want to dedup the output, and not state updates. It just happens that our starting point was KTable, for which state

Re: Plans to improve SSL performance in Kafka, for 0.10.x?

We've been using SSL for produce traffic (mirror makers only right now, but that's a very large percentage of traffic for us), and we're in the process of turning it on for inter broker traffic as well. Our experience is that this does not present a significant amount of overhead to the brokers. Sp

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Sure, you can use a non-cached state. However, if you write code like below for a non-cached state, and learn about caching later on, and think, caching is a cool feature, I want to use it, you would simply want to enable caching (without breaking your code). Processor code should always work inde

Plans to improve SSL performance in Kafka, for 0.10.x?

We are using 0.10.0.1 of Kafka and (Java) client libraries. We recently decided to start using SSL for Kafka communication between broker and clients. Right now, we have a pretty basic setup with just 1 broker with SSL keystore setup and the Java client(s) communicate using the Producer/Consume

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Thanks for clarifying On 4 September 2016 at 17:36, Matthias J. Sax wrote: > Sorry for not being precise. What I meant be "completely" is for a > single processor. Assume I want to have the following pattern: > > process(...) { > if (someCondition) { > state.put(...) > context.

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi Matthias, Thanks for the good questions. There is still the option of not using cached state. If one uses cached state it will dedup for stores and forwarding further. But you can always disable caching and do what you say. Eno > On 4 Sep 2016, at 17:36, Matthias J. Sax wrote: > > Sorry

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Sorry for not being precise. What I meant be "completely" is for a single processor. Assume I want to have the following pattern: process(...) { if (someCondition) { state.put(...) context.forward(...); } else { context.forward(...); } Ie, for some record I do update

[jira] [Commented] (KAFKA-3900) High CPU util on broker

[ https://issues.apache.org/jira/browse/KAFKA-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15463149#comment-15463149 ] Michael Saffitz commented on KAFKA-3900: We're seeing a similar issue-- we have a

Re: [DISCUSS] KIP-78: Cluster Id

Hi Dong, Sorry for the delay, I was offline for 24 hours. Thankfully Sumit was around to explain the reasoning for the current approach. From reading the thread, it seems like it may help to reiterate a few important goals for the auditing use case where there is a requirement to associate a messa

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi Matthias, Thanks for bringing the conversation across to the thread. I think a main limitation would be, that you cannot mix the 4 patterns > within a single application anymore (iff you use a "caches state"). If > you have processor with a "cached state" this disables direct usage of > contex

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

I think a main limitation would be, that you cannot mix the 4 patterns within a single application anymore (iff you use a "caches state"). If you have processor with a "cached state" this disables direct usage of context.forward() completely -- if I understand the design correctly. Thus, if a "cach

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

We had a recent discussion about KIP-63, and I just c&p from the JIRA discussion: Damian: > During the code walk-through, Matthias raised a very good point about the use > of context().forward being coupled to whether or not caching is enabled. Now > that i've had the chance to think about it I

WARN log message flooding broker logs for a pretty typical SSL setup

We just started enabling SSL for our Kafka brokers and (Java) clients and among some of the issues we are running into, one of them is the flooding of the server/broker Kafka logs where we are seeing these messages: [2016-09-02 08:07:13,773] WARN SSL peer is not authenticated, returning ANONY

Re: [DISCUSS] KIP-78: Cluster Id

Hi Flavio, Thanks for reviewing the KIP. Comments inline. On Sat, Sep 3, 2016 at 4:48 PM, Flavio Junqueira wrote: > Thanks for the KIP, Ismael, looks great. I have just a couple of comments > and questions: > > - I assume the znode containing the cluster id is persistent. Is there > ever the ne