[GitHub] kafka pull request #1837: MINOR: Update the README.md to include a note abou...

GitHub user jaikiran opened a pull request: https://github.com/apache/kafka/pull/1837 MINOR: Update the README.md to include a note about GRADLE_USER_HOME Trying to build the source and publish it to internal Maven repo, I ran into an issue that I explain in the mailing list discuss

Re: Building Kafka from source and publishing to Maven

A bit more debugging of the build file suggested that the ~/.gradle/gradle.properties file wasn't being used at all. Turns out my setup had the GRADLE_USER_HOME pointing to some other directory (I don't remember why I had set it up that way) and it was looking for the gradle.properties in that

[jira] [Updated] (KAFKA-3664) When subscription set changes on new consumer, the partitions may be removed without offset being committed.

[ https://issues.apache.org/jira/browse/KAFKA-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-3664: --- Resolution: Fixed Status: Resolved (was: Patch Available) > When subscription set cha

[jira] [Commented] (KAFKA-3664) When subscription set changes on new consumer, the partitions may be removed without offset being committed.

[ https://issues.apache.org/jira/browse/KAFKA-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475944#comment-15475944 ] Vahid Hashemian commented on KAFKA-3664: Fix was provided under [KIP-70|https://c

[jira] [Commented] (KAFKA-3664) When subscription set changes on new consumer, the partitions may be removed without offset being committed.

[ https://issues.apache.org/jira/browse/KAFKA-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475941#comment-15475941 ] ASF GitHub Bot commented on KAFKA-3664: --- Github user vahidhashemian closed the pull

[GitHub] kafka pull request #1363: KAFKA-3664: Commit offset of unsubscribed partitio...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/1363 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Build failed in Jenkins: kafka-trunk-jdk7 #1528

See Changes: [jason] KAFKA-4033; Revise partition assignment semantics on consumer -- [...truncated 12725 lines...] org.apache.kafka.streams.processor.internals.StreamThreadTest > testPartition

Build failed in Jenkins: kafka-trunk-jdk8 #871

See Changes: [jason] KAFKA-4033; Revise partition assignment semantics on consumer -- [...truncated 3474 lines...] kafka.coordinator.GroupMetadataManagerTest > testStoreNonEmptyGroup STARTED ka

Building Kafka from source and publishing to Maven

I am trying to build Kafka 0.10 branch from source to get a SSL fix and use it for our internal tests. In order to use it internally in our application, we'll need these built artifacts to be published to our internal Maven repo (backed by Nexus). I build instructions in README.md says this:

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Hi Jun, > 1. latestOffsets() returns the next offset. So there won't be a timestamp > associated with it. Would we use something like -1 for timestamp? The returned value would high watermark, so there might be an associated timestamp. But if it is log end offset, it seems that -1 is a reasonable

[jira] [Updated] (KAFKA-4033) KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change

[ https://issues.apache.org/jira/browse/KAFKA-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4033: --- Resolution: Fixed Fix Version/s: 0.10.1.0 Status: Resolved (was: Patch Avail

[jira] [Commented] (KAFKA-4033) KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change

[ https://issues.apache.org/jira/browse/KAFKA-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475716#comment-15475716 ] ASF GitHub Bot commented on KAFKA-4033: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1726: KAFKA-4033: KIP-70: Revise Partition Assignment Se...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1726 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

Hi Jun, 1. yes, it is my own personal opinion that people use queued.max.requests as an indirect way to bound memory consumption. once a more direct memory bound mechanism exists (and works) i dont think queued.max.requests woul dbe required. having said that I was not planning on making any chang

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Hi, Jiangjie, Thanks for the updated KIP. A couple of minor comments. 1. latestOffsets() returns the next offset. So there won't be a timestamp associated with it. Would we use something like -1 for timestamp? 2. Jason mentioned that if no message has timestamp >= the provided timestamp, we retu

Re: [VOTE] KIP-78 Cluster Id (second attempt)

Thanks for the writeup. +1. Jun On Tue, Sep 6, 2016 at 7:46 PM, Ismael Juma wrote: > Hi all, > > I would like to (re)initiate[1] the voting process for KIP-78 Cluster Id: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-78%3A+Cluster+Id > > As explained in the KIP and discussion thread

Build failed in Jenkins: kafka-trunk-jdk7 #1527

See Changes: [wangguoz] MINOR: fix transient QueryableStateIntegration test failure -- [...truncated 6861 lines...] kafka.coordinator.GroupCoordinatorResponseTest > testValidLeaveGroup PASSED

Re: [VOTE] KIP-63: Unify store and downstream caching in streams

+1 (binding) On Thu, Sep 8, 2016 at 2:34 PM Gwen Shapira wrote: > +1 (binding) > > Looks great and very detailed. Nice job, Eno :) > > On Thu, Aug 25, 2016 at 3:57 AM, Eno Thereska > wrote: > > > Hi folks, > > > > We'd like to start the vote for KIP-63. At this point the Wiki addresses > > all

[jira] [Created] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

Jason Gustafson created KAFKA-4145: -- Summary: Avoid redundant integration testing in ProducerSendTests Key: KAFKA-4145 URL: https://issues.apache.org/jira/browse/KAFKA-4145 Project: Kafka Is

Re: [VOTE] KIP-63: Unify store and downstream caching in streams

+1 (binding) Looks great and very detailed. Nice job, Eno :) On Thu, Aug 25, 2016 at 3:57 AM, Eno Thereska wrote: > Hi folks, > > We'd like to start the vote for KIP-63. At this point the Wiki addresses > all previous questions and we believe the PoC is feature-complete. > > Thanks > Eno > -

[GitHub] kafka pull request #1836: KAFKA-3703: Graceful close for consumers and produ...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/1836 KAFKA-3703: Graceful close for consumers and producer with acks=0 Process requests received from channels before they were closed. For consumers, wait for coordinator requests to complete befo

[jira] [Commented] (KAFKA-3703) Selector.close() doesn't complete outgoing writes

[ https://issues.apache.org/jira/browse/KAFKA-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474933#comment-15474933 ] ASF GitHub Bot commented on KAFKA-3703: --- GitHub user rajinisivaram opened a pull req

Build failed in Jenkins: kafka-trunk-jdk7 #1526

See Changes: [jason] KAFKA-4139; Reset findCoordinatorFuture when brokers are unavailable -- [...truncated 12297 lines...] org.apache.kafka.streams.processor.internals.PartitionGroupTest > test

Build failed in Jenkins: kafka-trunk-jdk8 #870

See Changes: [wangguoz] MINOR: fix transient QueryableStateIntegration test failure -- [...truncated 4896 lines...] kafka.utils.ReplicationUtilsTest > testUpdateLeaderAndIsr PASSED kafka.utils.

Build failed in Jenkins: kafka-trunk-jdk8 #869

See Changes: [jason] KAFKA-4139; Reset findCoordinatorFuture when brokers are unavailable -- [...truncated 12315 lines...] org.apache.kafka.streams.kstream.internals.KTableKTableLeftJoinTest > t

Re: [VOTE] KIP-63: Unify store and downstream caching in streams

Guozhang, I think the info is there. The next sentence says "The optimization is applicable to aggregations. It is not applicable to other operators like joins." Also the bit about the per-thread cache is one paragraph lower: "Specifically, for a streams instance with T threads and cache size C,

[GitHub] kafka pull request #1833: MINOR: fix transient QueryableStateIntegration tes...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1833 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE] KIP-63: Unify store and downstream caching in streams

Just read the KIP wiki again: https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in+streams Just one minor comments but otherwise I'm +1: In "proposed changes" section: "The cache has two functions. First, it continues to serve as a read buffer for dat

Re: [VOTE] KIP-63: Unify store and downstream caching in streams

There have been a couple of changes to KIP-63 since the voting started, after more feedback, most notably the fact that this KIP applies to the DSL only, and not to the Processor API. At this point I'd like to restart the voting process. Thanks Eno > On 31 Aug 2016, at 17:16, Jim Jagielski w

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi, Users have many options for buffering in the Processor API and it doesn't seem right we should prescribe a particular one. Also, there is value in continuing to keep the Processor API simple. As such, we'll remove the ".enableCaching" for a store used in the processor API from the KIP and

[jira] [Updated] (KAFKA-4139) Kafka consumer stuck in ensureCoordinatorReady after broker failure

[ https://issues.apache.org/jira/browse/KAFKA-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4139: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Kafka consumer stuck in e

[jira] [Assigned] (KAFKA-4136) KafkaBasedLog should include offsets it is trying to/successfully read to in log messages

[ https://issues.apache.org/jira/browse/KAFKA-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liquan Pei reassigned KAFKA-4136: - Assignee: Liquan Pei (was: Ewen Cheslack-Postava) > KafkaBasedLog should include offsets it is t

[GitHub] kafka pull request #1831: KAFKA-4139: Reset findCoordinatorFuture when broke...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1831 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4139) Kafka consumer stuck in ensureCoordinatorReady after broker failure

[ https://issues.apache.org/jira/browse/KAFKA-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474369#comment-15474369 ] ASF GitHub Bot commented on KAFKA-4139: --- Github user asfgit closed the pull request

[jira] [Created] (KAFKA-4144) Allow per stream/table timestamp extractor

Elias Levy created KAFKA-4144: - Summary: Allow per stream/table timestamp extractor Key: KAFKA-4144 URL: https://issues.apache.org/jira/browse/KAFKA-4144 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-3590) KafkaConsumer fails with "Messages are rejected since there are fewer in-sync replicas than required." when polling

[ https://issues.apache.org/jira/browse/KAFKA-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473820#comment-15473820 ] Dustin Cote commented on KAFKA-3590: [~hachikuji] is going to take this one over for n

[jira] [Updated] (KAFKA-3590) KafkaConsumer fails with "Messages are rejected since there are fewer in-sync replicas than required." when polling

[ https://issues.apache.org/jira/browse/KAFKA-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Cote updated KAFKA-3590: --- Status: Open (was: Patch Available) > KafkaConsumer fails with "Messages are rejected since there are

[jira] [Updated] (KAFKA-3590) KafkaConsumer fails with "Messages are rejected since there are fewer in-sync replicas than required." when polling

[ https://issues.apache.org/jira/browse/KAFKA-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Cote updated KAFKA-3590: --- Assignee: Jason Gustafson (was: Dustin Cote) > KafkaConsumer fails with "Messages are rejected since

[jira] [Commented] (KAFKA-3590) KafkaConsumer fails with "Messages are rejected since there are fewer in-sync replicas than required." when polling

[ https://issues.apache.org/jira/browse/KAFKA-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473815#comment-15473815 ] ASF GitHub Bot commented on KAFKA-3590: --- Github user cotedm closed the pull request

[GitHub] kafka pull request #1681: KAFKA-3590: KafkaConsumer fails with "Messages are...

Github user cotedm closed the pull request at: https://github.com/apache/kafka/pull/1681 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Assigned] (KAFKA-4133) Provide a configuration to control consumer max in-flight fetches

[ https://issues.apache.org/jira/browse/KAFKA-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-4133: - Assignee: Mickael Maison > Provide a configuration to control consumer max in-flight fetc

[jira] [Commented] (KAFKA-4130) [docs] Link to Varnish architect notes is broken

[ https://issues.apache.org/jira/browse/KAFKA-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473469#comment-15473469 ] ASF GitHub Bot commented on KAFKA-4130: --- GitHub user oscerd opened a pull request:

[GitHub] kafka pull request #1835: [DOCS] KAFKA-4130: Link to Varnish architect notes...

GitHub user oscerd opened a pull request: https://github.com/apache/kafka/pull/1835 [DOCS] KAFKA-4130: Link to Varnish architect notes is broken Hi all, The PR is related to https://issues.apache.org/jira/browse/KAFKA-4130 Thanks, Andrea You can merge

[jira] [Assigned] (KAFKA-4130) [docs] Link to Varnish architect notes is broken

[ https://issues.apache.org/jira/browse/KAFKA-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Cosentino reassigned KAFKA-4130: --- Assignee: Andrea Cosentino > [docs] Link to Varnish architect notes is broken > -

Multiple active Kafka producers/consumers with users authentication

Hello, I'm using Kafka 0.10+ with an SASL authentication on the client: // kafka_client_jaas.conf KafkaClient { org.apache.kafka.common.security.plain.PlainLoginModule required username="guillaume" password="secret"; }; Everything is working as expected and I want to use the same aut

Re: Queryable state client read guarantees

Hi Mikael, A fix for KAFKA-4123 (the issue you found with receiving null values) has now been committed to trunk. I've tried it with your github repo and it appears to be working. You will have to make a small change to your code as we now throw I