[GitHub] kafka pull request: HOTFIX: Fix checkstyle failure in KStreams by ...

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1000 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request: HOTFIX: Fix checkstyle failure in KStreams by ...

2016-03-02 Thread ewencp
GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/1000 HOTFIX: Fix checkstyle failure in KStreams by providing fully qualified class names. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafk

[jira] [Commented] (KAFKA-2967) Move Kafka documentation to ReStructuredText

2016-03-02 Thread Vamsi Subhash Achanta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177342#comment-15177342 ] Vamsi Subhash Achanta commented on KAFKA-2967: -- Emacs org mode works very wel

[jira] [Updated] (KAFKA-3323) Negative offsets in Log Segment Index files due to Integer overflow when compaction is enabled

2016-03-02 Thread Michael Schiff (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Schiff updated KAFKA-3323: -- Attachment: log_dump.txt index_dump.txt Index and Log dump from the 0 segment of

[jira] [Created] (KAFKA-3323) Negative offsets in Log Segment Index files due to Integer overflow when compaction is enabled

2016-03-02 Thread Michael Schiff (JIRA)
Michael Schiff created KAFKA-3323: - Summary: Negative offsets in Log Segment Index files due to Integer overflow when compaction is enabled Key: KAFKA-3323 URL: https://issues.apache.org/jira/browse/KAFKA-3323

[jira] [Resolved] (KAFKA-2375) Implement elasticsearch Copycat sink connector

2016-03-02 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2375. -- Resolution: Won't Fix Reviewer: Ewen Cheslack-Postava Fix Version

Build failed in Jenkins: kafka-trunk-jdk8 #411

2016-03-02 Thread Apache Jenkins Server
See Changes: [cshapi] KAFKA-2784: swallow exceptions when mirror maker exits. [me] KAFKA-2944: Replaced the NPE with a nicer error and clean exit and added -- [...truncated 6576 lines...] org.ap

Build failed in Jenkins: kafka-trunk-jdk7 #1084

2016-03-02 Thread Apache Jenkins Server
See Changes: [cshapi] KAFKA-2784: swallow exceptions when mirror maker exits. [me] KAFKA-2944: Replaced the NPE with a nicer error and clean exit and added -- [...truncated 1488 lines...] kafk

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Becket Qin
Hi Jason, I was thinking that every time when we connect to a broker, we first send the version check request. (The version check request itself should be very simple and never changes across all server releases.) This does add an additional round trip, but given reconnect is rare, it is probably

Build failed in Jenkins: kafka-trunk-jdk8 #410

2016-03-02 Thread Apache Jenkins Server
See Changes: [cshapi] KAFKA-3290: fix transient test failures in WorkerSourceTaskTest [cshapi] MINOR: fix typo -- [...truncated 4744 lines...] org.apache.kafka.clients.consumer.internals.Consume

Build failed in Jenkins: kafka-trunk-jdk7 #1083

2016-03-02 Thread Apache Jenkins Server
See Changes: [cshapi] KAFKA-3290: fix transient test failures in WorkerSourceTaskTest [cshapi] MINOR: fix typo -- [...truncated 2916 lines...] kafka.api.PlaintextProducerSendTest > testAutoCre

[jira] [Commented] (KAFKA-2944) NullPointerException in KafkaConfigStorage when config storage starts right before shutdown request

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177056#comment-15177056 ] ASF GitHub Bot commented on KAFKA-2944: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-2944) NullPointerException in KafkaConfigStorage when config storage starts right before shutdown request

2016-03-02 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2944: - Assignee: Gwen Shapira (was: Ewen Cheslack-Postava) > NullPointerException in Kaf

[GitHub] kafka pull request: KAFKA-2944: Replaced the NPE with a nicer erro...

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/993 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Created] (KAFKA-3322) recurring errors

2016-03-02 Thread jackie (JIRA)
jackie created KAFKA-3322: - Summary: recurring errors Key: KAFKA-3322 URL: https://issues.apache.org/jira/browse/KAFKA-3322 Project: Kafka Issue Type: Bug Components: core Affects Versi

[jira] [Commented] (KAFKA-2784) Mirror maker should swallow exceptions during shutdown.

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177007#comment-15177007 ] ASF GitHub Bot commented on KAFKA-2784: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-2784) Mirror maker should swallow exceptions during shutdown.

2016-03-02 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2784: Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 47

[GitHub] kafka pull request: KAFKA-2784: swallow exceptions when mirror mak...

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/478 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Dana Powers
In kafka-python we've been doing something like: if version >= (0, 9): Do cool new stuff elif version >= (0, 8, 2): Do some older stuff else: raise UnsupportedVersionError This will break if / when the new 0.9 apis are completely removed from some future release, but should handle inte

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Jason Gustafson
Hey Becket, I was thinking about that too. I guess there are two scenarios: upgrades and downgrades. For upgrades, the client may continue using the old API version until the next metadata refresh, which seems ok. For downgrades, we still need some way to signal the client that an API version is n

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Jason Gustafson
I'm assuming that the broker would continue to support older versions of the requests, so the check would probably be more like this: if (version > 0.9) // send 0.9 requests else if (version > 0.8) // send 0.8 requests Does that not work? This version can also can be used by clients to tell wheth

Build failed in Jenkins: kafka-trunk-jdk7 #1082

2016-03-02 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: Improve JavaDoc for some public classes. -- [...truncated 1481 lines...] kafka.api.PlaintextConsumerTest > testInterceptorsWithWrongKeyValue PASSED kafka.api.Pla

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Jay Kreps
But won't it be the case that what clients end up doing would be something like if(version != 0.8.1) throw new UnsupportedVersionException() which then means the client is broken as soon as we release a new server version even though the protocol didn't change. I'm actually not sure how yo

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Jason Gustafson
Hey Jay, Yeah, I wasn't suggesting that we eliminate request API versions. They're definitely needed on the broker to support compatibility. I was just saying that if a client wants to support multiple broker versions (e.g. 0.8 and 0.9), then it makes more sense to me to make the kafka release ver

[GitHub] kafka pull request: MINOR: fix typo

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/996 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Resolved] (KAFKA-3290) WorkerSourceTask testCommit transient failure

2016-03-02 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-3290. - Resolution: Fixed Fix Version/s: 0.10.0.0 Issue resolved by pull request 998 [https://githu

[GitHub] kafka pull request: MINOR: Fixed error in test by moving commit va...

2016-03-02 Thread gwenshap
Github user gwenshap closed the pull request at: https://github.com/apache/kafka/pull/994 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[jira] [Commented] (KAFKA-3290) WorkerSourceTask testCommit transient failure

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176909#comment-15176909 ] ASF GitHub Bot commented on KAFKA-3290: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3290: fix transient test failures in Wor...

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/998 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Becket Qin
I think using Kafka release version makes sense. More particularly, we can use the ApiVersion and this will cover all the interval version as well. In KAFKA-3025, we added the ApiVersion to message format version mapping, We can add the ApiKey to version mapping to ApiVersion as well. We can move A

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Jay Kreps
Hey guys, I think we want the clients to depend not on the broker version but rather on the protocol version. If you build against protocol version x it is valid with any Kafka version that supports x even as Kafka releases progress. We discussed originally whether to version the protocol as a wh

Build failed in Jenkins: kafka-trunk-jdk8 #409

2016-03-02 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: Improve JavaDoc for some public classes. -- [...truncated 1478 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest > testCleanShutdownFile PASSE

[jira] [Commented] (KAFKA-3236) Honor Producer Configuration "block.on.buffer.full"

2016-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176875#comment-15176875 ] Jiangjie Qin commented on KAFKA-3236: - [~tgraves] Got it. It is true that to achieve a

Jenkins build is back to normal : kafka-trunk-jdk7 #1081

2016-03-02 Thread Apache Jenkins Server
See

[GitHub] kafka pull request: MINOR: Improve JavaDoc for some public classes...

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/999 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] kafka pull request: MINOR: Improve JavaDoc for some public classes...

2016-03-02 Thread guozhangwang
GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/999 MINOR: Improve JavaDoc for some public classes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka KJavaDoc Alternatively y

Build failed in Jenkins: kafka-trunk-jdk8 #408

2016-03-02 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-3311; Prepare internal source topics before calling partition -- [...truncated 5658 lines...] org.apache.kafka.connect.runtime.WorkerTaskTest > stopBeforeStarting P

[jira] [Commented] (KAFKA-3236) Honor Producer Configuration "block.on.buffer.full"

2016-03-02 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176728#comment-15176728 ] Thomas Graves commented on KAFKA-3236: -- Yeah it would be possible to do it outside in

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2016-03-02 Thread Jason Gustafson
I think Dana's suggestion to include the Kafka release version makes a lot of sense. I'm actually wondering why you would need the individual API versions if you have that? It sounds like keeping track of all the api version information would add a lot of complexity to clients since they'll have to

[jira] [Commented] (KAFKA-1037) switching to gzip compression no error message for missing snappy jar on classpath

2016-03-02 Thread Andrew Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176683#comment-15176683 ] Andrew Stein commented on KAFKA-1037: - This is a severe issue AFAIK. It took three of

[jira] [Commented] (KAFKA-3236) Honor Producer Configuration "block.on.buffer.full"

2016-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176626#comment-15176626 ] Jiangjie Qin commented on KAFKA-3236: - [~tgraves] I remember there were some discussio

[GitHub] kafka pull request: KAFKA-3290: fix transient test failures in Wor...

2016-03-02 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/998 KAFKA-3290: fix transient test failures in WorkerSourceTaskTest You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-3290 Alt

[jira] [Commented] (KAFKA-3290) WorkerSourceTask testCommit transient failure

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176591#comment-15176591 ] ASF GitHub Bot commented on KAFKA-3290: --- GitHub user hachikuji opened a pull request

[jira] [Commented] (KAFKA-3236) Honor Producer Configuration "block.on.buffer.full"

2016-03-02 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176573#comment-15176573 ] Thomas Graves commented on KAFKA-3236: -- [~becket_qin] So if your concern that this c

[jira] [Commented] (KAFKA-3229) Root statestore is not registered with ProcessorStateManager, inner state store is registered instead

2016-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176570#comment-15176570 ] Guozhang Wang commented on KAFKA-3229: -- [~tom_dearman] Sorry my previous message was

[jira] [Commented] (KAFKA-3261) Consolidate class kafka.cluster.BrokerEndPoint and kafka.cluster.EndPoint

2016-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176557#comment-15176557 ] Guozhang Wang commented on KAFKA-3261: -- [~ijuma][~chenzhu] Since both of these classe

[jira] [Resolved] (KAFKA-3311) Move co-partition checking to PartitionAssignor and auto-create internal topics

2016-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3311. -- Resolution: Fixed Fix Version/s: 0.10.0.0 Issue resolved by pull request 990 [https://git

[jira] [Commented] (KAFKA-3311) Move co-partition checking to PartitionAssignor and auto-create internal topics

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176537#comment-15176537 ] ASF GitHub Bot commented on KAFKA-3311: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3311: Prepare internal source topics bef...

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/990 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Created] (KAFKA-3321) KafkaConfigStorage should never encounter commit without config data

2016-03-02 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-3321: --- Summary: KafkaConfigStorage should never encounter commit without config data Key: KAFKA-3321 URL: https://issues.apache.org/jira/browse/KAFKA-3321 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk7 #1080

2016-03-02 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-3196; Added checksum and size to RecordMetadata and ConsumerRecord -- [...truncated 1354 lines...] kafka.log.LogSegmentTest > testNextOffsetCalculation PASSED kafk

[jira] [Updated] (KAFKA-3188) Add system test for KIP-31 and KIP-32 - Compatibility Test

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3188: --- Priority: Blocker (was: Major) > Add system test for KIP-31 and KIP-32 - Compatibility Test > ---

[jira] [Updated] (KAFKA-3197) Producer can send message out of order even when in flight request is set to 1.

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3197: --- Priority: Blocker (was: Major) > Producer can send message out of order even when in flight request i

[jira] [Updated] (KAFKA-1215) Rack-Aware replica assignment option

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1215: --- Priority: Blocker (was: Major) > Rack-Aware replica assignment option > -

[jira] [Updated] (KAFKA-3202) Add system test for KIP-31 and KIP-32 - Change message format version on the fly

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3202: --- Priority: Blocker (was: Major) > Add system test for KIP-31 and KIP-32 - Change message format versio

[jira] [Updated] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3201: --- Priority: Blocker (was: Major) > Add system test for KIP-31 and KIP-32 - Upgrade Test > -

[jira] [Updated] (KAFKA-3303) Pass partial record metadata to Interceptor onAcknowledgement in case of errors

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3303: --- Priority: Blocker (was: Major) > Pass partial record metadata to Interceptor onAcknowledgement in cas

[jira] [Updated] (KAFKA-3316) Add Connect REST API to list available connector classes

2016-03-02 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3316: - Priority: Blocker (was: Major) > Add Connect REST API to list available connector

[jira] [Updated] (KAFKA-2370) Add pause/unpause connector support

2016-03-02 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2370: - Priority: Blocker (was: Major) > Add pause/unpause connector support > --

[jira] [Updated] (KAFKA-3315) Add Connect API to expose connector configuration info

2016-03-02 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3315: - Priority: Blocker (was: Major) > Add Connect API to expose connector configuratio

Build failed in Jenkins: kafka-trunk-jdk8 #407

2016-03-02 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-3196; Added checksum and size to RecordMetadata and ConsumerRecord -- [...truncated 2890 lines...] kafka.api.SaslSslEndToEndAuthorizationTest > testNoConsumeAcl PASS

[jira] [Updated] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-03-02 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-3310: Priority: Blocker (was: Major) > fetch requests can trigger repeated NPE when quota is enabled > --

[jira] [Created] (KAFKA-3320) Add successful acks verification to ProduceConsumeValidateTest

2016-03-02 Thread Anna Povzner (JIRA)
Anna Povzner created KAFKA-3320: --- Summary: Add successful acks verification to ProduceConsumeValidateTest Key: KAFKA-3320 URL: https://issues.apache.org/jira/browse/KAFKA-3320 Project: Kafka I

Re: Upgrading to Kafka 0.9.x

2016-03-02 Thread Cody Koeninger
Jay, thanks for the response. Regarding the new consumer API for 0.9, I've been reading through the code for it and thinking about how it fits in to the existing Spark integration. So far I've seen some interesting challenges, and if you (or anyone else on the dev list) have time to provide some h

[jira] [Commented] (KAFKA-3314) Add CDDL license to LICENSE and NOTICE file

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176146#comment-15176146 ] ASF GitHub Bot commented on KAFKA-3314: --- GitHub user junrao opened a pull request:

[jira] [Updated] (KAFKA-3314) Add CDDL license to LICENSE and NOTICE file

2016-03-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3314: --- Status: Patch Available (was: Open) > Add CDDL license to LICENSE and NOTICE file > -

[GitHub] kafka pull request: KAFKA-3314: Add CDDL license to LICENSE and NO...

2016-03-02 Thread junrao
GitHub user junrao opened a pull request: https://github.com/apache/kafka/pull/997 KAFKA-3314: Add CDDL license to LICENSE and NOTICE file You can merge this pull request into a Git repository by running: $ git pull https://github.com/junrao/kafka kafka-3314 Alternatively you

Re: Possible bug in Streams → KStreamImpl.java

2016-03-02 Thread Avi Flax
On Mon, Feb 29, 2016 at 6:34 PM, Guozhang Wang wrote: > Thanks for pointing it out! My pleasure! > And yes it is indeed a bug in the code. Could > you file a HOTFIX PR fixing this and also modify the existing unit test to > cover this case? Sorry, I missed your email until just now. I’m glad to

[jira] [Created] (KAFKA-3319) Improve description of group min and max session timeouts

2016-03-02 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-3319: -- Summary: Improve description of group min and max session timeouts Key: KAFKA-3319 URL: https://issues.apache.org/jira/browse/KAFKA-3319 Project: Kafka I

[jira] [Created] (KAFKA-3318) Improve consumer rebalance error messaging

2016-03-02 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-3318: -- Summary: Improve consumer rebalance error messaging Key: KAFKA-3318 URL: https://issues.apache.org/jira/browse/KAFKA-3318 Project: Kafka Issue Type: Impr

[jira] [Updated] (KAFKA-3196) KIP-42 (part 2): add record size and CRC to RecordMetadata and ConsumerRecords

2016-03-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3196: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 951 [https:/

Re: [DISCUSS] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

2016-03-02 Thread Ashish Singh
Here is link to the KIP, https://cwiki.apache.org/confluence/display/KAFKA/KIP-50+-+Enhance+Authorizer+interface+to+be+aware+of+supported+Principal+Types On Wed, Mar 2, 2016 at 9:46 AM, Ashish Singh wrote: > Hi Guys, > > I would like to initiate a discuss thread on KIP-50. Kafka authorizer is >

[DISCUSS] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

2016-03-02 Thread Ashish Singh
Hi Guys, I would like to initiate a discuss thread on KIP-50. Kafka authorizer is agnostic of principal types it supports, so are the acls CRUD methods in kafka.security.auth.Authorizer. The intent behind is to keep Kafka authorization pluggable, which is really great. However, this leads to Acls

[GitHub] kafka pull request: KAFKA-3196: Added checksum and size to RecordM...

2016-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/951 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Commented] (KAFKA-3196) KIP-42 (part 2): add record size and CRC to RecordMetadata and ConsumerRecords

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176067#comment-15176067 ] ASF GitHub Bot commented on KAFKA-3196: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-3290) WorkerSourceTask testCommit transient failure

2016-03-02 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176061#comment-15176061 ] Jason Gustafson commented on KAFKA-3290: [~ewencp] I haven't been able to reproduc

[jira] [Updated] (KAFKA-3317) too many open files between kafka borkers or may between borker and clients

2016-03-02 Thread david (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] david updated KAFKA-3317: - Description: there is much open files in my java app client which write msg to kafka broker, i used {quote}

[jira] [Commented] (KAFKA-2934) Offset storage file configuration in Connect standalone mode is not included in StandaloneConfig

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175765#comment-15175765 ] ASF GitHub Bot commented on KAFKA-2934: --- GitHub user ZoneMayor reopened a pull reque

[jira] [Commented] (KAFKA-2934) Offset storage file configuration in Connect standalone mode is not included in StandaloneConfig

2016-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175764#comment-15175764 ] ASF GitHub Bot commented on KAFKA-2934: --- Github user ZoneMayor closed the pull reque

[GitHub] kafka pull request: KAFKA-2934: Offset storage file configuration ...

2016-03-02 Thread ZoneMayor
Github user ZoneMayor closed the pull request at: https://github.com/apache/kafka/pull/734 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] kafka pull request: KAFKA-2934: Offset storage file configuration ...

2016-03-02 Thread ZoneMayor
GitHub user ZoneMayor reopened a pull request: https://github.com/apache/kafka/pull/734 KAFKA-2934: Offset storage file configuration in Connect standalone mode is not included in StandaloneConfig Added offsetBackingStore config to StandaloneConfig and DistributedConfig; Added c

[jira] [Created] (KAFKA-3317) too many open files between kafka borkers or may between borker and clients

2016-03-02 Thread david (JIRA)
david created KAFKA-3317: Summary: too many open files between kafka borkers or may between borker and clients Key: KAFKA-3317 URL: https://issues.apache.org/jira/browse/KAFKA-3317 Project: Kafka I

intellij make errors...

2016-03-02 Thread vasanth loka
I am attempting to compile kafka 0.9 and run a few tests in intellij. I am able to import the project into IntelliJ. I have scala 2.1.6 and Jdk 1.8. When I attempt a make, i get the few errors and all indicating: can't expand macros compiled by previous versions of Scala for example: E

[GitHub] kafka pull request: MINOR: remove unnecessary imports

2016-03-02 Thread xuwei-k
Github user xuwei-k closed the pull request at: https://github.com/apache/kafka/pull/65 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Updated] (KAFKA-3240) Replication issues

2016-03-02 Thread Jan Omar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Omar updated KAFKA-3240: Summary: Replication issues (was: Replication issues on FreeBSD) > Replication issues > --

[jira] [Commented] (KAFKA-3240) Replication issues on FreeBSD

2016-03-02 Thread Jan Omar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175423#comment-15175423 ] Jan Omar commented on KAFKA-3240: - To sum it up: The issue does NOT come up when we use 2

Build failed in Jenkins: kafka-trunk-jdk8 #406

2016-03-02 Thread Apache Jenkins Server
See Changes: [me] KAFKA-3257: disable bootstrap-test-env.sh --colour option -- [...truncated 3563 lines...] org.apache.kafka.clients.producer.RecordSendTest > testBlocking PASSED org.apache.kaf

Jenkins build is back to normal : kafka-trunk-jdk7 #1079

2016-03-02 Thread Apache Jenkins Server
See

[jira] [Updated] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3310: --- Affects Version/s: 0.9.0.0 > fetch requests can trigger repeated NPE when quota is enabled > -

[jira] [Updated] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3310: --- Fix Version/s: 0.10.0.0 > fetch requests can trigger repeated NPE when quota is enabled >

[jira] [Updated] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3310: --- Assignee: Aditya Auradkar > fetch requests can trigger repeated NPE when quota is enabled > --

[jira] [Updated] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3310: --- Reviewer: Jun Rao Status: Patch Available (was: Open) > fetch requests can trigger repeated NPE

[GitHub] kafka pull request: MINOR: fix typo

2016-03-02 Thread xuwei-k
GitHub user xuwei-k opened a pull request: https://github.com/apache/kafka/pull/996 MINOR: fix typo You can merge this pull request into a Git repository by running: $ git pull https://github.com/xuwei-k/kafka patch-1 Alternatively you can review and apply these changes as th

[jira] [Updated] (KAFKA-2951) Additional authorization test cases

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2951: --- Fix Version/s: (was: 0.10.0.0) > Additional authorization test cases > ---

[jira] [Updated] (KAFKA-2834) kafka-merge-pr.py should run unit tests before pushing it to trunk

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2834: --- Fix Version/s: (was: 0.10.0.0) > kafka-merge-pr.py should run unit tests before pushing it to trun

[jira] [Commented] (KAFKA-3173) Error while moving some partitions to OnlinePartition state

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175344#comment-15175344 ] Ismael Juma commented on KAFKA-3173: [~fpj] Are you planning to work on this? > Error

[jira] [Closed] (KAFKA-2547) Make DynamicConfigManager to use the ZkNodeChangeNotificationListener introduced as part of KAFKA-2211

2016-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma closed KAFKA-2547. -- This was merged a while ago. > Make DynamicConfigManager to use the ZkNodeChangeNotificationListener > int

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-03-02 Thread Rajini Sivaram
Jun, Thanks, I have added a note to the KIP. I will add a comment in the implementation and also add a unit test to ensure that conflicts are avoided when version number is modified. On Tue, Mar 1, 2016 at 5:43 PM, Jun Rao wrote: > Rajini, > > Thanks for the explanation. For 1, this implies tha