[GitHub] kafka pull request: KAFKA-3675; Add lz4 to parametrized `test_upgr...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1343 KAFKA-3675; Add lz4 to parametrized `test_upgrade` system test You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka kafka-3675-lz4-test-upg

[jira] [Commented] (KAFKA-3675) Add lz4 to parametrized `test_upgrade` system test

[ https://issues.apache.org/jira/browse/KAFKA-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275506#comment-15275506 ] ASF GitHub Bot commented on KAFKA-3675: --- GitHub user ijuma opened a pull request:

[jira] [Updated] (KAFKA-3675) Add lz4 to parametrized `test_upgrade` system test

[ https://issues.apache.org/jira/browse/KAFKA-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3675: --- Status: Patch Available (was: In Progress) > Add lz4 to parametrized `test_upgrade` system test > ---

Re: [VOTE] KIP-45: Standardize KafkaConsumer API to use Collection

Hi Harsha, See inline. On Sun, May 8, 2016 at 3:02 AM, Harsha wrote: > Ismael, >Do we need to add old assign and subscribe that accepts List. Yes, to maintain binary compatibility. If you compile your jar with kafka-clients 0.9.0.1 and then run it with kafka-clients 0.10.0.0 you will

[GitHub] kafka pull request: KAFKA-3670; ControlledShutdownLeaderSelector s...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1338 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-3670) ControlledShutdownLeaderSelector should pick the preferred replica as the new leader, if possible

[ https://issues.apache.org/jira/browse/KAFKA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3670: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 1338 [https:

[jira] [Commented] (KAFKA-3670) ControlledShutdownLeaderSelector should pick the preferred replica as the new leader, if possible

[ https://issues.apache.org/jira/browse/KAFKA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275638#comment-15275638 ] ASF GitHub Bot commented on KAFKA-3670: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-3587) LogCleaner fails due to incorrect offset map computation on a replica

[ https://issues.apache.org/jira/browse/KAFKA-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275650#comment-15275650 ] Guozhang Wang commented on KAFKA-3587: -- If I understand correctly, let's day the prod

Build failed in Jenkins: kafka-trunk-jdk7 #1265

See Changes: [junrao] KAFKA-3670; ControlledShutdownLeaderSelector should pick the preferred -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H1

Build failed in Jenkins: kafka-trunk-jdk8 #601

See Changes: [junrao] KAFKA-3670; ControlledShutdownLeaderSelector should pick the preferred -- [...truncated 5594 lines...] org.apache.kafka.streams.processor.internals.assignment.TaskAssignorT

Build failed in Jenkins: kafka-0.10.0-jdk7 #64

See Changes: [junrao] KAFKA-3670; ControlledShutdownLeaderSelector should pick the preferred -- [...truncated 6410 lines...] org.apache.kafka.connect.storage.KafkaOffsetBackingStoreTest > testSe

reading the consumer offsets topic

I'm having difficulty reading the consumer offsets topic from the command line. I try the following but it doesn't seem to work (along with a few related variants including specifying the zookeeper hosts): ./kafka-console-consumer.sh --broker-list localhost:9092 --topic __consumer_offsets Is the

[jira] [Commented] (KAFKA-3587) LogCleaner fails due to incorrect offset map computation on a replica

[ https://issues.apache.org/jira/browse/KAFKA-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275741#comment-15275741 ] Jun Rao commented on KAFKA-3587: [~guozhang], I don't think (a:4) will ever be duplicated

[GitHub] kafka pull request: KAFKA-3579 - Update reference to the outdated ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1239 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275761#comment-15275761 ] ASF GitHub Bot commented on KAFKA-3579: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3579: --- Resolution: Fixed Fix Version/s: 0.10.0.0 Status: Resolved (was: Patch Available) I

[jira] [Commented] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275764#comment-15275764 ] Ismael Juma commented on KAFKA-3579: Noting here that the command now references both

[jira] [Updated] (KAFKA-3587) LogCleaner fails due to incorrect offset map computation on a replica

[ https://issues.apache.org/jira/browse/KAFKA-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3587: --- Reviewer: Jun Rao > LogCleaner fails due to incorrect offset map computation on a replica > --

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275793#comment-15275793 ] Jun Rao commented on KAFKA-3565: [~becket_qin], thanks for the latest analysis. The differ

[jira] [Created] (KAFKA-3676) Add system tests for connector pause/resume

Jason Gustafson created KAFKA-3676: -- Summary: Add system tests for connector pause/resume Key: KAFKA-3676 URL: https://issues.apache.org/jira/browse/KAFKA-3676 Project: Kafka Issue Type: Tes

Build failed in Jenkins: kafka-trunk-jdk7 #1266

See Changes: [ismael] KAFKA-3579; Reference both old and new consumer properties in -- [...truncated 6412 lines...] org.apache.kafka.connect.storage.KafkaOffsetBackingStoreTest > testReloadOnS

Build failed in Jenkins: kafka-trunk-jdk8 #602

See Changes: [ismael] KAFKA-3579; Reference both old and new consumer properties in -- [...truncated 3310 lines...] kafka.log.LogSegmentTest > testRecoveryWithCorruptMessage PASSED kafka.log.Lo

Build failed in Jenkins: kafka-0.10.0-jdk7 #65

See Changes: [ismael] KAFKA-3579; Reference both old and new consumer properties in -- [...truncated 5554 lines...] org.apache.kafka.streams.StreamsConfigTest > testGetProducerConfigs PASSED or

Re: reading the consumer offsets topic

It looks like you’re just missing the proper message formatter. Of course, that largely depends on your version of the broker. Try: ./kafka-console-consumer.sh --broker-list localhost:9092 --topic __consumer_offsets --formatter kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter If f

[jira] [Updated] (KAFKA-3674) Connector target state changes not propagated to all workers

[ https://issues.apache.org/jira/browse/KAFKA-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3674: --- Fix Version/s: 0.10.0.0 > Connector target state changes not propagated to all workers > -

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275847#comment-15275847 ] Ismael Juma commented on KAFKA-3565: [~becket_qin], would you be able to submit a PR t

[jira] [Commented] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

[ https://issues.apache.org/jira/browse/KAFKA-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275852#comment-15275852 ] ASF GitHub Bot commented on KAFKA-3160: --- GitHub user dpkp opened a pull request:

[GitHub] kafka pull request: Fixup KAFKA-3160: catch decompression errors i...

GitHub user dpkp opened a pull request: https://github.com/apache/kafka/pull/1344 Fixup KAFKA-3160: catch decompression errors in constructor After testing KAFKA-3160 a bit more, I found that the error code was not being set properly in ProduceResponse. This happened because the val

[jira] [Commented] (KAFKA-3665) Default ssl.endpoint.identification.algorithm should be https

[ https://issues.apache.org/jira/browse/KAFKA-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275873#comment-15275873 ] Ismael Juma commented on KAFKA-3665: [~junrao], good questions. With regards to 1, bot

[jira] [Commented] (KAFKA-3585) Shutdown slow when there is only one broker which is controller

[ https://issues.apache.org/jira/browse/KAFKA-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275880#comment-15275880 ] Pengwei commented on KAFKA-3585: do you used the kafka-server-stop.sh to shutdown? > Shut

[jira] [Issue Comment Deleted] (KAFKA-3585) Shutdown slow when there is only one broker which is controller

[ https://issues.apache.org/jira/browse/KAFKA-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengwei updated KAFKA-3585: --- Comment: was deleted (was: do you used the kafka-server-stop.sh to shutdown?) > Shutdown slow when there is o

[jira] [Commented] (KAFKA-3585) Shutdown slow when there is only one broker which is controller

[ https://issues.apache.org/jira/browse/KAFKA-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275879#comment-15275879 ] Pengwei commented on KAFKA-3585: do you used the kafka-server-stop.sh to shutdown? > Shut

[jira] [Commented] (KAFKA-3676) Add system tests for connector pause/resume

[ https://issues.apache.org/jira/browse/KAFKA-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275904#comment-15275904 ] ASF GitHub Bot commented on KAFKA-3676: --- GitHub user hachikuji opened a pull request

[GitHub] kafka pull request: KAFKA-3676: system tests for connector pause/r...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/1345 KAFKA-3676: system tests for connector pause/resume You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-3676 Alternatively

[jira] [Commented] (KAFKA-1981) Make log compaction point configurable

[ https://issues.apache.org/jira/browse/KAFKA-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275920#comment-15275920 ] Eric Wasserman commented on KAFKA-1981: --- That did it. Thanks. I created https://cwi

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275924#comment-15275924 ] Jiangjie Qin commented on KAFKA-3565: - [~junrao] Yes, you are right. I changed the pro

[jira] [Commented] (KAFKA-3665) Default ssl.endpoint.identification.algorithm should be https

[ https://issues.apache.org/jira/browse/KAFKA-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275926#comment-15275926 ] Jun Rao commented on KAFKA-3665: Interesting, the difference is that in https, if a VIP is

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275930#comment-15275930 ] Guozhang Wang commented on KAFKA-3565: -- Just curious what is the {{batch.size}} in yo

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275934#comment-15275934 ] Jiangjie Qin commented on KAFKA-3565: - [~ijuma] We should absolutely mention the 8 byt

[GitHub] kafka pull request: MINOR: Add virtual env to Kafka system test RE...

GitHub user Ishiihara opened a pull request: https://github.com/apache/kafka/pull/1346 MINOR: Add virtual env to Kafka system test README.md You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ishiihara/kafka add-venv Alternatively

[jira] [Created] (KAFKA-3677) Add a producer performance tuning tool

Jiangjie Qin created KAFKA-3677: --- Summary: Add a producer performance tuning tool Key: KAFKA-3677 URL: https://issues.apache.org/jira/browse/KAFKA-3677 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275948#comment-15275948 ] Jiangjie Qin commented on KAFKA-3565: - BTW, I just created KAFKA-3677 to provide a too

[GitHub] kafka pull request: MINOR: Move connect.start() to try catch block

GitHub user Ishiihara opened a pull request: https://github.com/apache/kafka/pull/1347 MINOR: Move connect.start() to try catch block You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ishiihara/kafka connect-standalone Alternative

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275957#comment-15275957 ] Jun Rao commented on KAFKA-3565: [~becket_qin], thanks for confirming this. I guess defaul

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275959#comment-15275959 ] Jun Rao commented on KAFKA-3565: [~becket_qin], also, it seems that you patched ProducerPe

[GitHub] kafka pull request: KAFKA 3671: Move topics to SinkConnectorConfig

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-3671) Topics should not be in common ConnectorConfig definitions

[ https://issues.apache.org/jira/browse/KAFKA-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3671. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Fixed by https://github.com/apa

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275970#comment-15275970 ] Jiangjie Qin commented on KAFKA-3565: - [~junrao] Yes, the value bound was supposed to

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275971#comment-15275971 ] Jiangjie Qin commented on KAFKA-3565: - The batch size was 80K. It is not quite clear t

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275974#comment-15275974 ] Jiangjie Qin commented on KAFKA-3565: - Actually, never mind. The buffer size is not th

[jira] [Commented] (KAFKA-3647) Unable to set a ssl provider

[ https://issues.apache.org/jira/browse/KAFKA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275992#comment-15275992 ] Johan Abbors commented on KAFKA-3647: - Thanks Elvar. Re-generating the keystores with

[jira] [Commented] (KAFKA-3647) Unable to set a ssl provider

[ https://issues.apache.org/jira/browse/KAFKA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275995#comment-15275995 ] Johan Abbors commented on KAFKA-3647: - The -keyalg RSA fix Elvar found should be inclu

[GitHub] kafka pull request: KAFKA-1489: Global threshold on data retention...

GitHub user bendrees opened a pull request: https://github.com/apache/kafka/pull/1348 KAFKA-1489: Global threshold on data retention size Implemented a "log retention policy" based on keeping a certain percentage of disk space free. In dynamic situations where topics are adde

[jira] [Commented] (KAFKA-1489) Global threshold on data retention size

[ https://issues.apache.org/jira/browse/KAFKA-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276004#comment-15276004 ] ASF GitHub Bot commented on KAFKA-1489: --- GitHub user bendrees opened a pull request:

[DISCUSS] KIP-59 - Proposal for a kafka broker command - kafka-brokers.sh

Hi All, This is to start off a discussion on the above KIP at  https://cwiki.apache.org/confluence/display/KAFKA/KIP-59+-+Proposal+for+a+kafka+broker+command+-+kafka-brokers.sh The proposal is to fill the void of a command line tool/utility that can provide information on the cluster and brokers i

[GitHub] kafka pull request: KAFKA-3673: Connect tests don't handle concurr...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-3673) Connect tests dont handle concurrent config changes

[ https://issues.apache.org/jira/browse/KAFKA-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3673. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request

[jira] [Commented] (KAFKA-3673) Connect tests dont handle concurrent config changes

[ https://issues.apache.org/jira/browse/KAFKA-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276019#comment-15276019 ] ASF GitHub Bot commented on KAFKA-3673: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: MINOR: Move connect.start() to try catch block

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1347 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab