[GitHub] kafka pull request #1688: MINOR: Remove unnecessary synchronized block in or...
GitHub user PierreCoquentin opened a pull request: https://github.com/apache/kafka/pull/1688 MINOR: Remove unnecessary synchronized block in org.apache.kafka.streams.processor.internals.StreamTask The StreamTask is owned by a specific thread, so it doesn't seem necessary to synchronized the processing of the records as discussed with @guozhangwang on the dev mailing list You can merge this pull request into a Git repository by running: $ git pull https://github.com/PierreCoquentin/kafka trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1688 commit 9f2296e79582bcc2811c8ebd5c8c52e95fb46031 Author: PierreCoquentin Date: 2016-07-28T15:36:18Z Remove unnecessary synchronized block The StreamTask is owned by a specific thread, so it doesn't seem necessary to synchronized the processing of the records. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #1682: HOTFIX: non-unique state.dirs in integration tests...
Github user dguy closed the pull request at: https://github.com/apache/kafka/pull/1682 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #1679: MINOR: replace reference to HoppingWindows in stre...
Github user dguy closed the pull request at: https://github.com/apache/kafka/pull/1679 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #1689: HOTFIX: Fix punctuation timestamp in PunctuationQu...
GitHub user antonnazaruk opened a pull request: https://github.com/apache/kafka/pull/1689 HOTFIX: Fix punctuation timestamp in PunctuationQueue.java Actually, there are two problems: 1) Processor's "punctuate" method is not calling if there are NO new messages in "source" topic 2) When message comes after let's say 20 seconds delay (punctuation timeout was set to 2 seconds) - method punctuate is calling 10 times in a loop for every "missed" call within delayed period with the SAME timestamp Actually suggested change only fixes second item (every missed call will be using it's timestamp). Can anyone, please, comment if first item's statement is intended behavior or a bug? You can merge this pull request into a Git repository by running: $ git pull https://github.com/antonnazaruk/kafka patch-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1689.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1689 commit 900cc7f8d6ff893fcc30ec8ce5bb01cb6903c189 Author: antonnazaruk Date: 2016-07-30T21:15:46Z Fix punctuation timestamp in PunctuationQueue.java Actually, there are two problems: 1) Processor's "punctuate" method is not calling if there are new messages in "source" topic 2) When message comes after let's say 20 seconds delay (punctuation timeout was set to 2 seconds) - method punctuate is calling 10 times in a loop for every "missed" call within delayed period with the SAME timestamp (what doesn't have sense to me), Actually suggested change only fixes second item (every missed call will be using it's timestamp). Can anyone, please, comment if first item's statement is intended behavior or a bug? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (KAFKA-4008) Module "tools" should ne be dependent on "core"
[
https://issues.apache.org/jira/browse/KAFKA-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismael Juma updated KAFKA-4008:
---
Priority: Blocker (was: Critical)
> Module "tools" should ne be dependent on "core"
> ---
>
> Key: KAFKA-4008
> URL: https://issues.apache.org/jira/browse/KAFKA-4008
> Project: Kafka
> Issue Type: Bug
> Components: core, tools
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
>Priority: Blocker
> Fix For: 0.10.0.1
>
>
> The newly introduced "Stream Application Reset Tool" added the dependency to
> {{core}} into module {{tools}}. We want to get rid of this dependency.
> Solution: move {{StreamsResetter}} into module {{core}}
> Remark: actually, {{StreamsResetter}} should be in module {{streams}}
> however, this change is blocked by KIP-4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-4008) Module "tools" should ne be dependent on "core"
[
https://issues.apache.org/jira/browse/KAFKA-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400920#comment-15400920
]
Ismael Juma edited comment on KAFKA-4008 at 7/31/16 2:36 AM:
-
Note that `connect-runtime` has a dependency on `tools` and due to this change
it ends up depending on a particular Scala version due to the `core` dependency.
was (Author: ijuma):
Note that `connect-runtime` has a dependency on `tools` and this change it ends
up depending on a particular Scala version due to the `core` dependency.
> Module "tools" should ne be dependent on "core"
> ---
>
> Key: KAFKA-4008
> URL: https://issues.apache.org/jira/browse/KAFKA-4008
> Project: Kafka
> Issue Type: Bug
> Components: core, tools
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
>Priority: Blocker
> Fix For: 0.10.0.1
>
>
> The newly introduced "Stream Application Reset Tool" added the dependency to
> {{core}} into module {{tools}}. We want to get rid of this dependency.
> Solution: move {{StreamsResetter}} into module {{core}}
> Remark: actually, {{StreamsResetter}} should be in module {{streams}}
> however, this change is blocked by KIP-4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-4008) Module "tools" should ne be dependent on "core"
[
https://issues.apache.org/jira/browse/KAFKA-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400920#comment-15400920
]
Ismael Juma commented on KAFKA-4008:
Note that `connect-runtime` has a dependency on `tools` and this change it ends
up depending on a particular Scala version due to the `core` dependency.
> Module "tools" should ne be dependent on "core"
> ---
>
> Key: KAFKA-4008
> URL: https://issues.apache.org/jira/browse/KAFKA-4008
> Project: Kafka
> Issue Type: Bug
> Components: core, tools
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
>Priority: Critical
> Fix For: 0.10.0.1
>
>
> The newly introduced "Stream Application Reset Tool" added the dependency to
> {{core}} into module {{tools}}. We want to get rid of this dependency.
> Solution: move {{StreamsResetter}} into module {{core}}
> Remark: actually, {{StreamsResetter}} should be in module {{streams}}
> however, this change is blocked by KIP-4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [VOTE] 0.10.0.1 RC0
Hi Harsha, I agree that we should fix KAFKA-3950 and include it in the 0.10.0 branch. I worked with Manikumar on the PR to improve the tests and to fix a bug for the case where the subscription pattern matches internal topics and `exclude.internal.topics` is true (the default). We should be able to merge it soon. Having said that, KAFKA-3950 is a bug that has existed since the introduction of the Authorizer in 0.9.0.0 (i.e. it's not a regression) and we have received no reports of the issue until a few weeks ago. Given that, it would not warrant another 0.10.0.1 release candidate by itself, in my opinion. I am mentioning this to make sure we are all on the same page with regards to expectations. The bar for the respin of a bug fix release candidate is pretty high (regressions mostly). There are always more bugs to be fixed and they can be included in the subsequent bug fix release. By the way, the "Fix version" field in JIRA is a good way to communicate intent to the release manager (it was not set for KAFKA-3950). I wrote the above for future releases as it turns out that we do have a regression in the first release candidate for 0.10.0.1 (reported by Matthias). As such, we will need another RC and we should be able to include KAFKA-3950. :) Ismael On Fri, Jul 29, 2016 at 3:57 PM, Harsha Ch wrote: > Hi Ismael, > I would like to this JIRA included in the minor release > https://issues.apache.org/jira/browse/KAFKA-3950. > Thanks, > Harsha > > On Fri, Jul 29, 2016 at 7:46 AM Ismael Juma wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the first candidate for the release of Apache Kafka 0.10.0.1. > This > > is a bug fix release and it includes fixes and improvements from 50 JIRAs > > (including a few critical bugs). See the release notes for more details: > > > > http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/RELEASE_NOTES.html > > > > *** Please download, test and vote by Monday, 1 August, 8am PT *** > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > http://kafka.apache.org/KEYS > > > > * Release artifacts to be voted upon (source and binary): > > http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/ > > > > * Maven artifacts to be voted upon: > > https://repository.apache.org/content/groups/staging > > > > * Javadoc: > > http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/javadoc/ > > > > * Tag to be voted upon (off 0.10.0 branch) is the 0.10.0.1-rc0 tag: > > > > > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=0c2322c2cf7ab7909cfd8b834d1d2fffc34db109 > > > > * Documentation: > > http://kafka.apache.org/0100/documentation.html > > > > * Protocol: > > http://kafka.apache.org/0100/protocol.html > > > > * Successful Jenkins builds for the 0.10.0 branch: > > Unit/integration tests: > > https://builds.apache.org/job/kafka-0.10.0-jdk7/170/ > > System tests: > > https://jenkins.confluent.io/job/system-test-kafka-0.10.0/130/ > > > > Thanks, > > Ismael > > >
Re: [kafka-clients] [VOTE] 0.10.0.1 RC0
Hi Dana, Thanks for testing releases so promptly. Very much appreciated! It's funny, Ewen had suggested something similar with regards to the release notes a couple of days ago. We now have a Python script for generating the release notes: https://github.com/apache/kafka/blob/trunk/release_notes.py It should be straightforward to change it to do the grouping. Contributions encouraged. :) Ismael On Fri, Jul 29, 2016 at 5:02 PM, Dana Powers wrote: > +1 > > tested against kafka-python integration test suite = pass. > > Aside: as the scope of kafka gets bigger, it may be useful to organize > release notes into functional groups like core, brokers, clients, > kafka-streams, etc. I've found this useful when organizing > kafka-python release notes. > > -Dana > > On Fri, Jul 29, 2016 at 7:46 AM, Ismael Juma wrote: > > Hello Kafka users, developers and client-developers, > > > > This is the first candidate for the release of Apache Kafka 0.10.0.1. > This > > is a bug fix release and it includes fixes and improvements from 50 JIRAs > > (including a few critical bugs). See the release notes for more details: > > > > http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/RELEASE_NOTES.html > > > > *** Please download, test and vote by Monday, 1 August, 8am PT *** > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > http://kafka.apache.org/KEYS > > > > * Release artifacts to be voted upon (source and binary): > > http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/ > > > > * Maven artifacts to be voted upon: > > https://repository.apache.org/content/groups/staging > > > > * Javadoc: > > http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/javadoc/ > > > > * Tag to be voted upon (off 0.10.0 branch) is the 0.10.0.1-rc0 tag: > > > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=0c2322c2cf7ab7909cfd8b834d1d2fffc34db109 > > > > * Documentation: > > http://kafka.apache.org/0100/documentation.html > > > > * Protocol: > > http://kafka.apache.org/0100/protocol.html > > > > * Successful Jenkins builds for the 0.10.0 branch: > > Unit/integration tests: > https://builds.apache.org/job/kafka-0.10.0-jdk7/170/ > > System tests: > https://jenkins.confluent.io/job/system-test-kafka-0.10.0/130/ > > > > Thanks, > > Ismael > > > > -- > > You received this message because you are subscribed to the Google Groups > > "kafka-clients" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to kafka-clients+unsubscr...@googlegroups.com. > > To post to this group, send email to kafka-clie...@googlegroups.com. > > Visit this group at https://groups.google.com/group/kafka-clients. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/kafka-clients/CAD5tkZYz8fbLAodpqKg5eRiCsm4ze9QK3ufTz3Q4U%3DGs0CRb1A%40mail.gmail.com > . > > For more options, visit https://groups.google.com/d/optout. >
Re: [VOTE] 0.10.0.1 RC0
Thanks Matthias. We do indeed need another RC. The fact that `tools` acquired a dependency on `core` means that `connect-runtime` also depends on `core` (and all of its dependencies like `Scala`). Ismael On Sat, Jul 30, 2016 at 4:55 AM, Matthias J. Sax wrote: > Hi, > > I think we need a new RC because of > > https://issues.apache.org/jira/browse/KAFKA-4008 > > -Matthias > > > On 07/29/2016 04:59 PM, Harsha Chintalapani wrote: > > Hi Ismael, > > I would like this JIRA to be included in the minor release > > https://issues.apache.org/jira/browse/KAFKA-3950 > > Thanks, > > Harsha > > On Fri, Jul 29, 2016 at 7:46 AM Ismael Juma wrote: > > > >> Hello Kafka users, developers and client-developers, > >> > >> This is the first candidate for the release of Apache Kafka 0.10.0.1. > This > >> is a bug fix release and it includes fixes and improvements from 50 > JIRAs > >> (including a few critical bugs). See the release notes for more details: > >> > >> http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/RELEASE_NOTES.html > >> > >> *** Please download, test and vote by Monday, 1 August, 8am PT *** > >> > >> Kafka's KEYS file containing PGP keys we use to sign the release: > >> http://kafka.apache.org/KEYS > >> > >> * Release artifacts to be voted upon (source and binary): > >> http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/ > >> > >> * Maven artifacts to be voted upon: > >> https://repository.apache.org/content/groups/staging > >> > >> * Javadoc: > >> http://home.apache.org/~ijuma/kafka-0.10.0.1-rc0/javadoc/ > >> > >> * Tag to be voted upon (off 0.10.0 branch) is the 0.10.0.1-rc0 tag: > >> > >> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=0c2322c2cf7ab7909cfd8b834d1d2fffc34db109 > >> > >> * Documentation: > >> http://kafka.apache.org/0100/documentation.html > >> > >> * Protocol: > >> http://kafka.apache.org/0100/protocol.html > >> > >> * Successful Jenkins builds for the 0.10.0 branch: > >> Unit/integration tests: > >> https://builds.apache.org/job/kafka-0.10.0-jdk7/170/ > >> System tests: > >> https://jenkins.confluent.io/job/system-test-kafka-0.10.0/130/ > >> > >> Thanks, > >> Ismael > >> > > > >
[jira] [Updated] (KAFKA-3950) kafka mirror maker tool is not respecting whitelist option
[
https://issues.apache.org/jira/browse/KAFKA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar Reddy updated KAFKA-3950:
---
Affects Version/s: 0.10.0.0
> kafka mirror maker tool is not respecting whitelist option
> --
>
> Key: KAFKA-3950
> URL: https://issues.apache.org/jira/browse/KAFKA-3950
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.10.0.0
>Reporter: Raghav Kumar Gautam
>Assignee: Manikumar Reddy
>Priority: Critical
> Fix For: 0.10.0.1
>
>
> A mirror maker launched like this:
> {code}
> /usr/bin/kinit -k -t /home/kfktest/hadoopqa/keytabs/kfktest.headless.keytab
> kfkt...@example.com
> JAVA_HOME=/usr/jdk64/jdk1.8.0_77 JMX_PORT=9112
> /usr/kafka/bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/config/mirror_consumer_12.properties
> --producer.config
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/config/mirror_producer_12.properties
> --new.consumer --whitelist="test.*" >>
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/logs/mirror_maker-12/mirror_maker_12.log
> 2>&1 & echo pid:$! >
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/logs/mirror_maker-12/entity_12_pid
> {code}
> Lead to TopicAuthorizationException:
> {code}
> WARN Error while fetching metadata with correlation id 44 :
> {__consumer_offsets=TOPIC_AUTHORIZATION_FAILED}
> (org.apache.kafka.clients.NetworkClient)
> [2016-06-20 13:24:49,983] FATAL [mirrormaker-thread-0] Mirror maker thread
> failure due to (kafka.tools.MirrorMaker$MirrorMakerThread)
> org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to
> access topics: [__consumer_offsets]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-3950) kafka mirror maker tool is not respecting whitelist option
[
https://issues.apache.org/jira/browse/KAFKA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar Reddy updated KAFKA-3950:
---
Reviewer: Ismael Juma
Fix Version/s: 0.10.0.1
> kafka mirror maker tool is not respecting whitelist option
> --
>
> Key: KAFKA-3950
> URL: https://issues.apache.org/jira/browse/KAFKA-3950
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.10.0.0
>Reporter: Raghav Kumar Gautam
>Assignee: Manikumar Reddy
>Priority: Critical
> Fix For: 0.10.0.1
>
>
> A mirror maker launched like this:
> {code}
> /usr/bin/kinit -k -t /home/kfktest/hadoopqa/keytabs/kfktest.headless.keytab
> kfkt...@example.com
> JAVA_HOME=/usr/jdk64/jdk1.8.0_77 JMX_PORT=9112
> /usr/kafka/bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/config/mirror_consumer_12.properties
> --producer.config
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/config/mirror_producer_12.properties
> --new.consumer --whitelist="test.*" >>
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/logs/mirror_maker-12/mirror_maker_12.log
> 2>&1 & echo pid:$! >
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/logs/mirror_maker-12/entity_12_pid
> {code}
> Lead to TopicAuthorizationException:
> {code}
> WARN Error while fetching metadata with correlation id 44 :
> {__consumer_offsets=TOPIC_AUTHORIZATION_FAILED}
> (org.apache.kafka.clients.NetworkClient)
> [2016-06-20 13:24:49,983] FATAL [mirrormaker-thread-0] Mirror maker thread
> failure due to (kafka.tools.MirrorMaker$MirrorMakerThread)
> org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to
> access topics: [__consumer_offsets]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-3950) kafka mirror maker tool is not respecting whitelist option
[
https://issues.apache.org/jira/browse/KAFKA-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar Reddy updated KAFKA-3950:
---
Affects Version/s: (was: 0.10.0.0)
0.9.0.0
> kafka mirror maker tool is not respecting whitelist option
> --
>
> Key: KAFKA-3950
> URL: https://issues.apache.org/jira/browse/KAFKA-3950
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Raghav Kumar Gautam
>Assignee: Manikumar Reddy
>Priority: Critical
> Fix For: 0.10.0.1
>
>
> A mirror maker launched like this:
> {code}
> /usr/bin/kinit -k -t /home/kfktest/hadoopqa/keytabs/kfktest.headless.keytab
> kfkt...@example.com
> JAVA_HOME=/usr/jdk64/jdk1.8.0_77 JMX_PORT=9112
> /usr/kafka/bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/config/mirror_consumer_12.properties
> --producer.config
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/config/mirror_producer_12.properties
> --new.consumer --whitelist="test.*" >>
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/logs/mirror_maker-12/mirror_maker_12.log
> 2>&1 & echo pid:$! >
> /usr/kafka/system_test/mirror_maker_testsuite/testcase_15001/logs/mirror_maker-12/entity_12_pid
> {code}
> Lead to TopicAuthorizationException:
> {code}
> WARN Error while fetching metadata with correlation id 44 :
> {__consumer_offsets=TOPIC_AUTHORIZATION_FAILED}
> (org.apache.kafka.clients.NetworkClient)
> [2016-06-20 13:24:49,983] FATAL [mirrormaker-thread-0] Mirror maker thread
> failure due to (kafka.tools.MirrorMaker$MirrorMakerThread)
> org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to
> access topics: [__consumer_offsets]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
