Build failed in Jenkins: kafka-trunk-jdk8 #1327

2017-03-06 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-4828: ProcessorTopologyTestDriver does not work when using through -- [...truncated 848.06 KB...] org.apache.kafka.streams.integration.KStre

[jira] [Created] (KAFKA-4846) Use KafkaProducer without allocating a new ProducerRecord for each message sent

2017-03-06 Thread JIRA
Mikael Ståldal created KAFKA-4846: - Summary: Use KafkaProducer without allocating a new ProducerRecord for each message sent Key: KAFKA-4846 URL: https://issues.apache.org/jira/browse/KAFKA-4846 Proje

[jira] [Commented] (KAFKA-4846) Use KafkaProducer without allocating a new ProducerRecord for each message sent

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897013#comment-15897013 ] Ismael Juma commented on KAFKA-4846: Thanks for the report. There are a number of pla

[jira] [Updated] (KAFKA-4845) KafkaConsumer.seekToEnd cannot take effect when integrating with spark streaming

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4845: --- Description: When integrating with spark streaming, kafka consumer cannot get the latest offsets exce

[jira] [Commented] (KAFKA-4845) KafkaConsumer.seekToEnd cannot take effect when integrating with spark streaming

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897022#comment-15897022 ] Ismael Juma commented on KAFKA-4845: [~vahid], you've worked on this code, would you b

[jira] [Updated] (KAFKA-4834) Kafka cannot delete topic with ReplicaStateMachine went wrong

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4834: --- Labels: reliability (was: ) > Kafka cannot delete topic with ReplicaStateMachine went wrong > ---

Re: Allow to replace Zookeeper with different Coordination Service (etcd/consul)

2017-03-06 Thread Alexander Binzberger
I would also be interested in this. (etcd) Am 02.03.2017 um 12:24 schrieb Molnár Bálint: Hi, I was wonderring to refactor Kafka core code to be able to use different Coordination Service than Zookeeper. I know I will need to create a KIP for that. I think the first part of this task to refactor

[GitHub] kafka pull request #2635: MINOR: additional refactoring around the use of Er...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2635 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Created] (KAFKA-4847) 1Y30J

2017-03-06 Thread Vamsi Jakkula (JIRA)
Vamsi Jakkula created KAFKA-4847: Summary: 1Y30J Key: KAFKA-4847 URL: https://issues.apache.org/jira/browse/KAFKA-4847 Project: Kafka Issue Type: Bug Reporter: Vamsi Jakkula Cre

[GitHub] kafka pull request #1712: KAFKA-3989: initial support for adding a JMH bench...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1712 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3989) Add JMH module for Benchmarks

2017-03-06 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897165#comment-15897165 ] ASF GitHub Bot commented on KAFKA-3989: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-4739) KafkaConsumer poll going into an infinite loop

2017-03-06 Thread Jin Tianfan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897164#comment-15897164 ] Jin Tianfan commented on KAFKA-4739: unfortunately,I met the same problem.Is this prob

Build failed in Jenkins: kafka-trunk-jdk8 #1328

2017-03-06 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: additional refactoring around the use of Errors -- [...truncated 157.71 KB...] kafka.integration.SaslPlaintextTopicMetadataTest > testIsrAft

[jira] [Comment Edited] (KAFKA-4739) KafkaConsumer poll going into an infinite loop

2017-03-06 Thread Jin Tianfan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897164#comment-15897164 ] Jin Tianfan edited comment on KAFKA-4739 at 3/6/17 12:28 PM: -

Build failed in Jenkins: kafka-trunk-jdk8 #1329

2017-03-06 Thread Apache Jenkins Server
See Changes: [ismael] KAFKA-3989; Initial support for adding a JMH benchmarking module -- [...truncated 102.18 KB...] kafka.api.PlaintextConsumerTest > testEarliestOrLate

[GitHub] kafka pull request #1997: KAFKA-4266: ReassignPartitionsClusterTest - Ensure...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1997 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-3989) Add JMH module for Benchmarks

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3989: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Add JMH module for Benchmarks > -

[jira] [Commented] (KAFKA-4266) Replication Quota Tests: Ensure ZK updated before tests start

2017-03-06 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897270#comment-15897270 ] ASF GitHub Bot commented on KAFKA-4266: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4266) Replication Quota Tests: Ensure ZK updated before tests start

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4266. Resolution: Fixed > Replication Quota Tests: Ensure ZK updated before tests start >

[jira] [Updated] (KAFKA-4266) Replication Quota Tests: Ensure ZK updated before tests start

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4266: --- Fix Version/s: 0.11.0.0 > Replication Quota Tests: Ensure ZK updated before tests start >

[jira] [Assigned] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-3514: --- Assignee: Eno Thereska > Stream timestamp computation needs some further thoughts > -

[jira] [Commented] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897290#comment-15897290 ] Eno Thereska commented on KAFKA-3514: - [~mihbor] 2) above is slightly different from t

[jira] [Work started] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3514 started by Eno Thereska. --- > Stream timestamp computation needs some further thoughts > --

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-03-06 Thread Ismael Juma
Thanks Colin. It seems like you replied to me accidentally instead of the list, so leaving your reply below for the benefit of others. Regarding the disadvantage of having to hunt through the request class, don't people have to do that anyway with the Options classes? Aside from that, it would be

[jira] [Commented] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Michal Borowiecki (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897371#comment-15897371 ] Michal Borowiecki commented on KAFKA-3514: -- Hi [~enothereska], I have to disagree

Re: [DISCUSS] KIP-81: Max in-flight fetches

2017-03-06 Thread Mickael Maison
Following the feedback from the Voting thread I've updated the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-81%3A+Bound+Fetch+memory+usage+in+the+consumer Instead of trying to identify Coordinator's messages by their size, now the proposal is to mark the Node/Channel used to communic

[jira] [Commented] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897381#comment-15897381 ] Eno Thereska commented on KAFKA-3514: - [~mihbor] I understand now. I had thought the e

[jira] [Commented] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Michal Borowiecki (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897413#comment-15897413 ] Michal Borowiecki commented on KAFKA-3514: -- Thank you for responding. Just now I

Build failed in Jenkins: kafka-trunk-jdk8 #1330

2017-03-06 Thread Apache Jenkins Server
See Changes: [ismael] KAFKA-4266; ReassignPartitionsClusterTest: ensure ZK publication is -- [...truncated 100.80 KB...] kafka.api.PlaintextConsumerTest > testEarliestOrL

[jira] [Created] (KAFKA-4848) Stream thread getting into deadlock state while trying to get rocksdb lock in retryWithBackoff

2017-03-06 Thread Sachin Mittal (JIRA)
Sachin Mittal created KAFKA-4848: Summary: Stream thread getting into deadlock state while trying to get rocksdb lock in retryWithBackoff Key: KAFKA-4848 URL: https://issues.apache.org/jira/browse/KAFKA-4848

[jira] [Commented] (KAFKA-4848) Stream thread getting into deadlock state while trying to get rocksdb lock in retryWithBackoff

2017-03-06 Thread Sachin Mittal (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897462#comment-15897462 ] Sachin Mittal commented on KAFKA-4848: -- The deadlock issue is like this. If a thread

[jira] [Commented] (KAFKA-4848) Stream thread getting into deadlock state while trying to get rocksdb lock in retryWithBackoff

2017-03-06 Thread Sachin Mittal (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897463#comment-15897463 ] Sachin Mittal commented on KAFKA-4848: -- Please refer https://github.com/apache/kafka/

[jira] [Created] (KAFKA-4849) Bug in KafkaStreams documentation

2017-03-06 Thread Seweryn Habdank-Wojewodzki (JIRA)
Seweryn Habdank-Wojewodzki created KAFKA-4849: - Summary: Bug in KafkaStreams documentation Key: KAFKA-4849 URL: https://issues.apache.org/jira/browse/KAFKA-4849 Project: Kafka Iss

[jira] [Created] (KAFKA-4850) RocksDb cannot use Bloom Filters

2017-03-06 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-4850: --- Summary: RocksDb cannot use Bloom Filters Key: KAFKA-4850 URL: https://issues.apache.org/jira/browse/KAFKA-4850 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-4846) Use KafkaProducer without allocating a new ProducerRecord for each message sent

2017-03-06 Thread Matt Sicker (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897564#comment-15897564 ] Matt Sicker commented on KAFKA-4846: I know I'd love this feature personally for distr

Re: Allow to replace Zookeeper with different Coordination Service (etcd/consul)

2017-03-06 Thread Ofir Manor
I suggest you check KIP-30 and the extensive discussion about it in the mailing list from around December 2015 called "[DISCUSS] KIP-30 Allow for brokers to have plug-able consensus and meta data storage sub systems" If I remember correctly, it ran into some objections, as the existing commiters t

[jira] [Created] (KAFKA-4851) SessionStore.fetch(key) is a performance bottleneck

2017-03-06 Thread Damian Guy (JIRA)
Damian Guy created KAFKA-4851: - Summary: SessionStore.fetch(key) is a performance bottleneck Key: KAFKA-4851 URL: https://issues.apache.org/jira/browse/KAFKA-4851 Project: Kafka Issue Type: Bug

java.lang.IllegalStateException: Correlation id for response () does not match request ()

2017-03-06 Thread Mickael Maison
Hi, In one of our clusters, some of our clients occasionally see this exception: java.lang.IllegalStateException: Correlation id for response (4564) does not match request (4562) at org.apache.kafka.clients.NetworkClient.correlate(NetworkClient.java:486) at org.apache.kafka.clients.NetworkClient.p

[GitHub] kafka pull request #2644: hotfix: broken link

2017-03-06 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2644 hotfix: broken link You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfixBrokerLink Alternatively you can review and apply these cha

[GitHub] kafka pull request #2645: KAFKA-4851: only search available segments during ...

2017-03-06 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2645 KAFKA-4851: only search available segments during Segments.segments(from, to) restrict the locating of segments in {{Segments#segments(..)}} to only the segments that are currently available, i.e., ra

[jira] [Commented] (KAFKA-4851) SessionStore.fetch(key) is a performance bottleneck

2017-03-06 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897714#comment-15897714 ] ASF GitHub Bot commented on KAFKA-4851: --- GitHub user dguy opened a pull request:

[jira] [Commented] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897729#comment-15897729 ] Matthias J. Sax commented on KAFKA-3514: [~mihbor] I am not sure about this. To me

[GitHub] kafka pull request #2644: Hotfix: broken link

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2644 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: java.lang.IllegalStateException: Correlation id for response () does not match request ()

2017-03-06 Thread Ismael Juma
Hi Mickael, This looks to be the same as KAFKA-4669. In theory, this should never happen and it's unclear when/how it can happen. Not sure if someone has investigated it in more detail. Ismael On Mon, Mar 6, 2017 at 5:15 PM, Mickael Maison wrote: > Hi, > > In one of our clusters, some of our c

[jira] [Commented] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

2017-03-06 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897747#comment-15897747 ] Edoardo Comar commented on KAFKA-4669: -- It's easy to discard and recreate the consume

[jira] [Commented] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

2017-03-06 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897746#comment-15897746 ] Edoardo Comar commented on KAFKA-4669: -- We've seen this same exception using 10.0.1 c

[jira] [Commented] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Michal Borowiecki (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897771#comment-15897771 ] Michal Borowiecki commented on KAFKA-3514: -- Oh, I wouldn't mind that at all. Just

[jira] [Commented] (KAFKA-4845) KafkaConsumer.seekToEnd cannot take effect when integrating with spark streaming

2017-03-06 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897780#comment-15897780 ] Vahid Hashemian commented on KAFKA-4845: [~ijuma], sure, I'll take a look. > Kafk

[jira] [Assigned] (KAFKA-4845) KafkaConsumer.seekToEnd cannot take effect when integrating with spark streaming

2017-03-06 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-4845: -- Assignee: Vahid Hashemian > KafkaConsumer.seekToEnd cannot take effect when integrating

[jira] [Created] (KAFKA-4852) ByteBufferSerializer not compatible with offsets

2017-03-06 Thread Werner Daehn (JIRA)
Werner Daehn created KAFKA-4852: --- Summary: ByteBufferSerializer not compatible with offsets Key: KAFKA-4852 URL: https://issues.apache.org/jira/browse/KAFKA-4852 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-4853) ByteBufferSerializer does a memcopy

2017-03-06 Thread Werner Daehn (JIRA)
Werner Daehn created KAFKA-4853: --- Summary: ByteBufferSerializer does a memcopy Key: KAFKA-4853 URL: https://issues.apache.org/jira/browse/KAFKA-4853 Project: Kafka Issue Type: Bug Com

Re: [DISCUSS] KIP-82 - Add Record Headers

2017-03-06 Thread Colin McCabe
As others have mentioned, it seems clear that we want to preserve the ordering of message headers, so that we can implement things like lineage tracing. (For example, each stage could add a "lineage:" header.) I also think that we want the ability to add and remove headers as needed. It would be

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

2017-03-06 Thread Guozhang Wang
1) Sounds good. 2) Yeah what I meant is to emphasize that TimestampExtractor to be stateless in the docs somewhere. Guozhang On Sun, Mar 5, 2017 at 4:27 PM, Matthias J. Sax wrote: > Guozhang, > > about renaming the config parameters. I like this idea, but want to > point out, that this chang

[jira] [Commented] (KAFKA-3874) Transient test failure: org.apache.kafka.streams.integration.JoinIntegrationTest.shouldCountClicksPerRegion

2017-03-06 Thread Armin Braun (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897882#comment-15897882 ] Armin Braun commented on KAFKA-3874: I think this one is still pretty flaky. I see thi

[jira] [Commented] (KAFKA-3514) Stream timestamp computation needs some further thoughts

2017-03-06 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897891#comment-15897891 ] Guozhang Wang commented on KAFKA-3514: -- There are some discussions on pros and cons o

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-03-06 Thread Colin McCabe
On Mon, Mar 6, 2017, at 05:50, Ismael Juma wrote: > Thanks Colin. It seems like you replied to me accidentally instead of the > list, so leaving your reply below for the benefit of others. Thanks, Ismael. I actually realized my mistake right after I sent to you, and re-posted it to the mailing li

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-03-06 Thread Ismael Juma
Thanks Colin. I am familiar with the protocol semantics, but we need to document the API for users who don't know the protocol. I still think it would be valuable to have some examples of how the API would be used for common use cases. For example, say someone creates a topic and then produces to

Build failed in Jenkins: kafka-trunk-jdk8 #1331

2017-03-06 Thread Apache Jenkins Server
See Changes: [wangguoz] HOTFIX: fix broken link for wordcount demo example -- [...truncated 919.74 KB...] org.apache.kafka.connect.runtime.WorkerSinkTaskThreadedTest > tes

Build failed in Jenkins: kafka-0.10.2-jdk7 #97

2017-03-06 Thread Apache Jenkins Server
See Changes: [wangguoz] HOTFIX: fix broken link for wordcount demo example -- [...truncated 321.65 KB...] kafka.server.KafkaConfigTest > testUncleanLeaderElectionDefault S

[jira] [Created] (KAFKA-4854) Producer RecordBatch executes callbacks with `null` provided for metadata if an exception is encountered

2017-03-06 Thread Robert Quinlivan (JIRA)
Robert Quinlivan created KAFKA-4854: --- Summary: Producer RecordBatch executes callbacks with `null` provided for metadata if an exception is encountered Key: KAFKA-4854 URL: https://issues.apache.org/jira/browse/

[GitHub] kafka pull request #2646: MINOR: Rename RecordBatch to ProducerBatch to allo...

2017-03-06 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2646 MINOR: Rename RecordBatch to ProducerBatch to allow reuse in KIP-98 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka rename-rec

[GitHub] kafka pull request #2639: KAFKA-3155: Avoid Long Overflow in org.apache.kafk...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2639 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-3155) Transient Failure in kafka.api.PlaintextProducerSendTest.testFlush

2017-03-06 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3155. Resolution: Fixed Issue resolved by pull request 2639 [https://github.com/apache/kafka/pull/2639] >

[jira] [Commented] (KAFKA-3155) Transient Failure in kafka.api.PlaintextProducerSendTest.testFlush

2017-03-06 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898129#comment-15898129 ] ASF GitHub Bot commented on KAFKA-3155: --- Github user asfgit closed the pull request

[jira] [Created] (KAFKA-4855) Struct SchemaBuilder should not allow duplicate fields.

2017-03-06 Thread Jeremy Custenborder (JIRA)
Jeremy Custenborder created KAFKA-4855: -- Summary: Struct SchemaBuilder should not allow duplicate fields. Key: KAFKA-4855 URL: https://issues.apache.org/jira/browse/KAFKA-4855 Project: Kafka

[jira] [Commented] (KAFKA-4845) KafkaConsumer.seekToEnd cannot take effect when integrating with spark streaming

2017-03-06 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898258#comment-15898258 ] Vahid Hashemian commented on KAFKA-4845: [~DanC], The issue you raised sounds very

[jira] [Comment Edited] (KAFKA-4845) KafkaConsumer.seekToEnd cannot take effect when integrating with spark streaming

2017-03-06 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898258#comment-15898258 ] Vahid Hashemian edited comment on KAFKA-4845 at 3/6/17 10:20 PM: ---

[GitHub] kafka pull request #2647: MINOR: Add varint serde utilities for new message ...

2017-03-06 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2647 MINOR: Add varint serde utilities for new message format You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka add-varint-serdes Al

[jira] [Updated] (KAFKA-4856) Calling KafkaProducer.close() from multiple threads may cause spurious error

2017-03-06 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté updated KAFKA-4856: - Description: Calling KafkaProducer.close() from multiple threads simultaneously may cause the fol

[jira] [Created] (KAFKA-4856) Calling KafkaProducer.close() from multiple threads may cause spurious error

2017-03-06 Thread JIRA
Xavier Léauté created KAFKA-4856: Summary: Calling KafkaProducer.close() from multiple threads may cause spurious error Key: KAFKA-4856 URL: https://issues.apache.org/jira/browse/KAFKA-4856 Project: K

[jira] [Updated] (KAFKA-4856) Calling KafkaProducer.close() from multiple threads may cause spurious error

2017-03-06 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté updated KAFKA-4856: - Description: Calling KafkaProducer.close() from multiple threads simultaneously may cause the fol

[jira] [Updated] (KAFKA-4856) Calling KafkaProducer.close() from multiple threads may cause spurious error

2017-03-06 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté updated KAFKA-4856: - Description: Calling {{KafkaProducer.close()}} from multiple threads simultaneously may cause the

[jira] [Commented] (KAFKA-4825) Likely Data Loss in ReassignPartitionsTest System Test

2017-03-06 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898312#comment-15898312 ] Jun Rao commented on KAFKA-4825: [~benstopford], thanks for reporting this. Took a look a

[jira] [Comment Edited] (KAFKA-4825) Likely Data Loss in ReassignPartitionsTest System Test

2017-03-06 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898312#comment-15898312 ] Jun Rao edited comment on KAFKA-4825 at 3/6/17 10:53 PM: - [~bensto

[jira] [Updated] (KAFKA-4794) Add access to OffsetStorageReader from SourceConnector

2017-03-06 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4794: --- Labels: needs-kip (was: ) > Add access to OffsetStorageReader from SourceConnector >

[jira] [Commented] (KAFKA-4794) Add access to OffsetStorageReader from SourceConnector

2017-03-06 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898347#comment-15898347 ] Jason Gustafson commented on KAFKA-4794: [~fhussonnois] Thanks for creating the ti

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

2017-03-06 Thread Jeyhun Karimov
Sorry I was late. I will update javadocs in related methods to emphasize that TimestampExtractor is stateless. Cheers, Jeyhun On Mon, Mar 6, 2017 at 8:17 PM Guozhang Wang wrote: > 1) Sounds good. > > 2) Yeah what I meant is to emphasize that TimestampExtractor to be > stateless in the docs some

[jira] [Commented] (KAFKA-4849) Bug in KafkaStreams documentation

2017-03-06 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898381#comment-15898381 ] Matthias J. Sax commented on KAFKA-4849: With regard to http://docs.confluent.io/3

[jira] [Assigned] (KAFKA-4849) Bug in KafkaStreams documentation

2017-03-06 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-4849: -- Assignee: Matthias J. Sax > Bug in KafkaStreams documentation > ---

[jira] [Created] (KAFKA-4857) Improve Client handling

2017-03-06 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-4857: -- Summary: Improve Client handling Key: KAFKA-4857 URL: https://issues.apache.org/jira/browse/KAFKA-4857 Project: Kafka Issue Type: Bug Component

[jira] [Commented] (KAFKA-2729) Cached zkVersion not equal to that in zookeeper, broker not recovering.

2017-03-06 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898415#comment-15898415 ] Jun Rao commented on KAFKA-2729: [~mjuarez], did you see ZK session expiration in the serv

Jenkins build is back to normal : kafka-trunk-jdk8 #1332

2017-03-06 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka_0.9.0_jdk7 #139

2017-03-06 Thread Apache Jenkins Server
See Changes: [ismael] KAFKA-4251: fix test driver not launching in Vagrant 1.8.6 -- [...truncated 31.60 KB...] /tmp/sbt_361d0c98/xsbt/ConsoleInterface.scala:12: error: erro

Re: [DISCUSS] KIP-126 - Allow KafkaProducer to batch based on uncompressed size

2017-03-06 Thread Dong Lin
Hey Becket, I am wondering if we should first vote for the KIP before reviewing the patch. I have two comments below: - Should we specify the new sensors as part of interface change in the KIP? - The KIP proposes to increase estimated compression ratio by 0.05 for each underestimation and decreme

[jira] [Comment Edited] (KAFKA-4849) Bug in KafkaStreams documentation

2017-03-06 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898381#comment-15898381 ] Matthias J. Sax edited comment on KAFKA-4849 at 3/7/17 1:10 AM:

[GitHub] kafka pull request #2648: KAFKA-4849: Bug in KafkaStreams documentation

2017-03-06 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2648 KAFKA-4849: Bug in KafkaStreams documentation - removed ZK config from Streams cods - removed deprecated ZK config parameter from configs table You can merge this pull request into a Git reposit

[jira] [Commented] (KAFKA-4849) Bug in KafkaStreams documentation

2017-03-06 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898528#comment-15898528 ] ASF GitHub Bot commented on KAFKA-4849: --- GitHub user mjsax opened a pull request:

[jira] [Updated] (KAFKA-4849) Bug in KafkaStreams documentation

2017-03-06 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4849: --- Status: Patch Available (was: In Progress) > Bug in KafkaStreams documentation >

[jira] [Work started] (KAFKA-4849) Bug in KafkaStreams documentation

2017-03-06 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4849 started by Matthias J. Sax. -- > Bug in KafkaStreams documentation > - > >

[GitHub] kafka pull request #2646: MINOR: Rename RecordBatch to ProducerBatch to allo...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2646 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4739) KafkaConsumer poll going into an infinite loop

2017-03-06 Thread Jin Tianfan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898623#comment-15898623 ] Jin Tianfan commented on KAFKA-4739: is this problem sloved? > KafkaConsumer poll goi

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

2017-03-06 Thread Matthias J. Sax
Hi, I want to give a first respond: 1. Producer per task: First, we did some performance tests, indicating that the performance penalty is small. Please have a look here: https://docs.google.com/spreadsheets/d/18aGOB13-ibwHJl5VE27HnHBlNkZAIeLF9yQ99FDxkuM/edit?usp=sharing For the test, we ran

[jira] [Commented] (KAFKA-2273) KIP-54: Add rebalance with a minimal number of reassignments to server-defined strategy list

2017-03-06 Thread Jeff Widman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898678#comment-15898678 ] Jeff Widman commented on KAFKA-2273: This KIP was accepted, so I added it to the 0.11

[jira] [Updated] (KAFKA-2273) KIP-54: Add rebalance with a minimal number of reassignments to server-defined strategy list

2017-03-06 Thread Jeff Widman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-2273: --- Fix Version/s: 0.11.0.0 > KIP-54: Add rebalance with a minimal number of reassignments to > server-de

[VOTE] KIP-128: Add ByteArrayConverter for Kafka Connect

2017-03-06 Thread Ewen Cheslack-Postava
Hi all, I'd like to kick off voting for KIP-128: Add ByteArrayConverter for Kafka Connect: https://cwiki.apache.org/confluence/display/KAFKA/KIP-128%3A+Add+ByteArrayConverter+for+Kafka+Connect There was a small amount of discussion, see the original thread here: https://lists.apache.org/thread.h

[jira] [Commented] (KAFKA-4566) Can't Symlink to Kafka bins

2017-03-06 Thread Tyler Hale (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898696#comment-15898696 ] Tyler Hale commented on KAFKA-4566: --- I know this is a bit more than intended than the or

Build failed in Jenkins: kafka-trunk-jdk8 #1333

2017-03-06 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: Rename RecordBatch to ProducerBatch to free the name for KIP-98 -- [...truncated 321.09 KB...] kafka.security.auth.SimpleAclAuthorizerTest >

[jira] [Commented] (KAFKA-4566) Can't Symlink to Kafka bins

2017-03-06 Thread Marc Bush (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898711#comment-15898711 ] Marc Bush commented on KAFKA-4566: -- on REL, at least, I found the issue to be with the ca

Becoming a contributor

2017-03-06 Thread Tyler Hale
Kafka devs, I'm looking to start contributing to kafka and have been reading over the getting started and contributing guides. How do I get added to the contribution list so that I can assign myself JIRAs? Thanks, Tyler Sent from my iPhone

Re: [DISCUSS] KIP-128: Add ByteArrayConverter for Kafka Connect

2017-03-06 Thread Guozhang Wang
Sounds good. Thanks. On Wed, Mar 1, 2017 at 8:21 PM, Gwen Shapira wrote: > Yeah, you are right, it is best not to make converters actively break the > data structures :) > > On Wed, Mar 1, 2017 at 4:55 PM, Ewen Cheslack-Postava > wrote: > > > Guozhang, > > > > I'm fine w/ adjusting if people wa

  1   2   >