Re: [DISCUSS] Kafka 0.10.1.0 Release Plan

+1 Plan looks good to me. Thanks a lot for putting this together! On Fri, Sep 9, 2016 at 3:45 PM, Jason Gustafson wrote: > Hi All, > > I've volunteered to be release manager for the upcoming 0.10.1 release and > would like to propose the following timeline: > > Feature Freeze (Sep. 19): The 0.

[jira] [Updated] (KAFKA-4131) Multiple Regex KStream-Consumers cause Null pointer exception in addRawRecords in RecordQueue class

[ https://issues.apache.org/jira/browse/KAFKA-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-4131: --- Status: Patch Available (was: In Progress) Working on getting an integration test for this bug fix.

[jira] [Commented] (KAFKA-4131) Multiple Regex KStream-Consumers cause Null pointer exception in addRawRecords in RecordQueue class

[ https://issues.apache.org/jira/browse/KAFKA-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478891#comment-15478891 ] ASF GitHub Bot commented on KAFKA-4131: --- GitHub user bbejeck opened a pull request:

[GitHub] kafka pull request #1843: KAFKA-4131: moved updates for topics matching rege...

GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/1843 KAFKA-4131: moved updates for topics matching regex subscription meth… …od, allows for followers to get subscribed topics You can merge this pull request into a Git repository by running: $

Build failed in Jenkins: kafka-trunk-jdk8 #873

See Changes: [ismael] KAFKA-3807; Fix transient test failure caused by race on future -- [...truncated 4771 lines...] kafka.admin.DeleteConsumerGroupTest > testGroupWideDeleteInZK STARTED kafka.

[VOTE] KIP-79 - ListOffsetRequest v1 and search by timestamp methods in new consumer.

Hi all, I'd like to start the voting for KIP-79 In short we propose to : 1. add a ListOffsetRequest/ListOffsetResponse v1, and 2. add earliestOffsts(), latestOffsets() and offsetForTime() methods in the new consumer. The KIP wiki is the following: https://cwiki.apache.org/confluence/pages/viewpa

[jira] [Created] (KAFKA-4148) KIP-79 add ListOffsetRequest v1 and search by timestamp interface to consumer.

Jiangjie Qin created KAFKA-4148: --- Summary: KIP-79 add ListOffsetRequest v1 and search by timestamp interface to consumer. Key: KAFKA-4148 URL: https://issues.apache.org/jira/browse/KAFKA-4148 Project: K

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Hey Jun, I am thinking about the interface. It seems that returning a timestamp for earliestOffsets() and latestOffsets() is not that useful because the timestamp of the first message is not necessarily the earliest timestamp, and the timestamp of the last offset might be -1. Also exposing the tim

[jira] [Updated] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-4145: --- Status: Patch Available (was: In Progress) > Avoid redundant integration testing in ProducerS

[jira] [Commented] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478602#comment-15478602 ] ASF GitHub Bot commented on KAFKA-4145: --- GitHub user vahidhashemian opened a pull re

[GitHub] kafka pull request #1842: KAFKA-4145: Avoid redundant integration testing in...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/1842 KAFKA-4145: Avoid redundant integration testing in ProducerSendTests You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka

[jira] [Resolved] (KAFKA-4146) Kafka Stream ignores Serde exceptions leading to silently broken apps

[ https://issues.apache.org/jira/browse/KAFKA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elias Levy resolved KAFKA-4146. --- Resolution: Invalid > Kafka Stream ignores Serde exceptions leading to silently broken apps >

Build failed in Jenkins: kafka-trunk-jdk7 #1530

See Changes: [ismael] KAFKA-3807; Fix transient test failure caused by race on future -- [...truncated 12377 lines...] org.apache.kafka.streams.kstream.internals.KTableFilterTest > testSkipNull

[jira] [Commented] (KAFKA-4133) Provide a configuration to control consumer max in-flight fetches

[ https://issues.apache.org/jira/browse/KAFKA-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478521#comment-15478521 ] Jason Gustafson commented on KAFKA-4133: [~mimaison] You should have permission no

[DISCUSS] Kafka 0.10.1.0 Release Plan

Hi All, I've volunteered to be release manager for the upcoming 0.10.1 release and would like to propose the following timeline: Feature Freeze (Sep. 19): The 0.10.1 release branch will be created. Code Freeze (Oct. 3): The first RC will go out. Final Release (~Oct. 17): Assuming no blocking issu

[jira] [Commented] (KAFKA-4147) Transient failure in ConsumerCoordinatorTest.testAutoCommitDynamicAssignment

[ https://issues.apache.org/jira/browse/KAFKA-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478406#comment-15478406 ] ASF GitHub Bot commented on KAFKA-4147: --- GitHub user hachikuji opened a pull request

[GitHub] kafka pull request #1841: KAFKA-4147: Fix transient failure in ConsumerCoord...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/1841 KAFKA-4147: Fix transient failure in ConsumerCoordinatorTest.testAutoCommitDynamicAssignment You can merge this pull request into a Git repository by running: $ git pull https://github.com/

[jira] [Commented] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478402#comment-15478402 ] Ashish K Singh commented on KAFKA-4145: --- +1, testFlush has been one of the most flak

[jira] [Created] (KAFKA-4147) Transient failure in ConsumerCoordinatorTest.testAutoCommitDynamicAssignment

Jason Gustafson created KAFKA-4147: -- Summary: Transient failure in ConsumerCoordinatorTest.testAutoCommitDynamicAssignment Key: KAFKA-4147 URL: https://issues.apache.org/jira/browse/KAFKA-4147 Projec

Re: [ANNOUNCE] New committer: Jason Gustafson

Congrats Jason. On Fri, Sep 9, 2016 at 5:30 AM, Michael Noll wrote: > My compliments, Jason -- well deserved! :-) > > -Michael > > > > On Wed, Sep 7, 2016 at 6:49 PM, Grant Henke wrote: > > > Congratulations and thank you for all of your contributions to Apache > > Kafka Jason! > > > > On Wed,

[jira] [Resolved] (KAFKA-3807) OffsetValidationTest - transient failure on test_broker_rolling_bounce

[ https://issues.apache.org/jira/browse/KAFKA-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3807. Resolution: Fixed Reviewer: Ismael Juma Fix Version/s: 0.10.1.0 > OffsetValidationTe

[GitHub] kafka pull request #1821: KAFKA-3807: Fix transient test failure caused by r...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1821 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3807) OffsetValidationTest - transient failure on test_broker_rolling_bounce

[ https://issues.apache.org/jira/browse/KAFKA-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478255#comment-15478255 ] ASF GitHub Bot commented on KAFKA-3807: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478241#comment-15478241 ] Ismael Juma commented on KAFKA-4145: I'd probably remove `testFlush`. The rest sounds

[jira] [Commented] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478226#comment-15478226 ] Vahid Hashemian commented on KAFKA-4145: How about reducing the above list to: #

[jira] [Commented] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478195#comment-15478195 ] Ismael Juma commented on KAFKA-4145: I am not sure about the `testClose` ones. It seem

[jira] [Work started] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4145 started by Vahid Hashemian. -- > Avoid redundant integration testing in ProducerSendTests >

[jira] [Assigned] (KAFKA-4145) Avoid redundant integration testing in ProducerSendTests

[ https://issues.apache.org/jira/browse/KAFKA-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-4145: -- Assignee: Vahid Hashemian > Avoid redundant integration testing in ProducerSendTests >

[jira] [Updated] (KAFKA-4135) Inconsistent javadoc for KafkaConsumer.poll behavior when there is no subscription

[ https://issues.apache.org/jira/browse/KAFKA-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-4135: --- Status: Patch Available (was: Open) > Inconsistent javadoc for KafkaConsumer.poll behavior wh

[GitHub] kafka pull request #1840: MINOR: catch InvalidStateStoreException in Queryab...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/1840 MINOR: catch InvalidStateStoreException in QueryableStateIntegrationTest A couple of the tests may transiently fail in QueryableStateIntegrationTest as they are not catching InvalidStateStoreException.

[jira] [Commented] (KAFKA-4135) Inconsistent javadoc for KafkaConsumer.poll behavior when there is no subscription

[ https://issues.apache.org/jira/browse/KAFKA-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477973#comment-15477973 ] ASF GitHub Bot commented on KAFKA-4135: --- GitHub user vahidhashemian opened a pull re

[GitHub] kafka pull request #1839: KAFKA-4135: Consumer polls when not subscribed to ...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/1839 KAFKA-4135: Consumer polls when not subscribed to any topic or assigned any partition should raise an exception When the consumer is not subscribed to any topic or, in the case of manual ass

Re: Subscribe me

Hi Kiran, You can follow the instructions here to subscribe: http://kafka.apache.org/contact.html Guozhang On Fri, Sep 9, 2016 at 6:57 AM, Kiran Potladurthi < kiran.potladur...@gmail.com> wrote: > Please subscribe me for this mailing list. > > Regards > Kiran > -- -- Guozhang

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Completely agree that we should have a consistent representation of something missing/unknown in the code. My understanding of the convention is that -1 means "not available"/"unknown". For example, when producer request failed, the offset we used in the callback is -1. And Message.NoTimestamp is

Build failed in Jenkins: kafka-trunk-jdk7 #1529

See Changes: [ismael] HOTFIX: Temporarily ignoring this test until fixed -- [...truncated 6229 lines...] kafka.log.BrokerCompressionTest > testBrokerSideCompression[10] PASSED kafka.log.Broker

[jira] [Created] (KAFKA-4146) Kafka Stream ignores Serde exceptions leading to silently broken apps

Elias Levy created KAFKA-4146: - Summary: Kafka Stream ignores Serde exceptions leading to silently broken apps Key: KAFKA-4146 URL: https://issues.apache.org/jira/browse/KAFKA-4146 Project: Kafka

Re: Agenda item for next KIP meeting

Hi, I'd like to discuss KIP-72. could you please add me? On Fri, Sep 9, 2016 at 7:48 AM, Ismael Juma wrote: > Hi Vahid, > > Sounds good. Jun is unavailable on the day, so I'll send the invite. Any > other KIPs that people would like to discuss on the day (Tuesday at 11am > PT)? The following KI

Re: Queryable state client read guarantees

Hi Mikael, During rebalance both instances should throw IllegalStateStoreException until the rebalance has completed. Once the rebalance has completed if the key is not found on the local store, then you would get a null value. You can always find the Kafka Streams instance that will have that key

Build failed in Jenkins: kafka-trunk-jdk8 #872

See Changes: [ismael] HOTFIX: Temporarily ignoring this test until fixed -- [...truncated 6794 lines...] kafka.api.ProducerBounceTest > testBrokerFailure STARTED kafka.api.ProducerBounceTest >

Re: Queryable state client read guarantees

Hi Damian, thanks for fixing this so quickly, I re-ran the test and it works fine. The next test I tried was to read from two service instances implementing the same string count topology. First, the client is started sending two read requests, one per instance, every second. Next, I start the fi

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Our inconsistent usage of magic values like `-1` and `null` to represent missing values makes it hard to reason about code. We had a recent major regression in MirrorMaker as a result of this[1], for example. `null` by itself is hard enough (often called the billion dollar mistake), so can we decid

Re: Agenda item for next KIP meeting

Hi Vahid, Sounds good. Jun is unavailable on the day, so I'll send the invite. Any other KIPs that people would like to discuss on the day (Tuesday at 11am PT)? The following KIPs are under discussion and have either been submitted or updated recently: KIP-68: Add a consumed log retention before

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Jiangjie, Returning TimestampOffset(-1, -1) sounds reasonable to me. Thanks, Jun On Thu, Sep 8, 2016 at 8:46 PM, Becket Qin wrote: > Hi Jun, > > > 1. latestOffsets() returns the next offset. So there won't be a timestamp > > associated with it. Would we use something like -1 for timestamp? >

[GitHub] kafka pull request #1838: HOTFIX: Temporarily ignoring this test until fixed

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1838 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Subscribe me

Please subscribe me for this mailing list. Regards Kiran

Re: [ANNOUNCE] New committer: Jason Gustafson

My compliments, Jason -- well deserved! :-) -Michael On Wed, Sep 7, 2016 at 6:49 PM, Grant Henke wrote: > Congratulations and thank you for all of your contributions to Apache > Kafka Jason! > > On Wed, Sep 7, 2016 at 10:12 AM, Mayuresh Gharat < > gharatmayures...@gmail.com > > wrote: > > > c

[GitHub] kafka pull request #1838: HOTFIX: Temporarily ignoring this test until fixed

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/1838 HOTFIX: Temporarily ignoring this test until fixed You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka hotfix-ignore-smoke-tes

[jira] [Commented] (KAFKA-4133) Provide a configuration to control consumer max in-flight fetches

[ https://issues.apache.org/jira/browse/KAFKA-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15476427#comment-15476427 ] Mickael Maison commented on KAFKA-4133: --- I don't seem to have permissions to edit pa

Transaction Support in Kafka

Hi, I have use case where in I want to Publish the message only after my Primary Transaction getting committed into SQL Store. My Primary store is update may involve multiple tables under single Transactions. At the same time, if Publish in Kafka fails for some reason, I should be able to get ro