[jira] [Updated] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Attachment: KAFKA-1456_2014-05-19_23:24:27.patch > Add LZ4 and LZ4C as a compression codec > --

[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002856#comment-14002856 ] James Oliver commented on KAFKA-1456: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/ --- (Updated May 20, 2014, 6:24 a.m.) Review request for kafka. Summary (updated)

[jira] [Issue Comment Deleted] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1432: Comment: was deleted (was: Updated reviewboard https://reviews.apache.org/r/21663/diff/ against br

[jira] [Updated] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1432: Attachment: (was: KAFKA-1432_2014-05-19_23:13:01.patch) > Make num.producerThreads configurable

Re: Review Request 21663: Patch for KAFKA-1432

> On May 20, 2014, 1:22 a.m., James Oliver wrote: > > Test cases failing: > > > > :clients:test > > ... > > org.apache.kafka.common.record.MemoryRecordsTest > testIterator[3] FAILED > > java.lang.IllegalArgumentException: Unknown compression type: LZ4 > > at > > org.apache.kafka.com

[jira] [Updated] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1432: Attachment: KAFKA-1432_2014-05-19_23:13:01.patch > Make num.producerThreads configurable on new Mir

[jira] [Commented] (KAFKA-1432) Make num.producerThreads configurable on new MirrrorMaker

[ https://issues.apache.org/jira/browse/KAFKA-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002843#comment-14002843 ] James Oliver commented on KAFKA-1432: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 21663: Patch for KAFKA-1432

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/ --- (Updated May 20, 2014, 6:13 a.m.) Review request for kafka. Summary (updated)

Review Request 21676: Patch for KAFKA-1298

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21676/ --- Review request for kafka. Bugs: KAFKA-1298 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1298) Controlled shutdown tool doesn't seem to work out of the box

[ https://issues.apache.org/jira/browse/KAFKA-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1298: -- Attachment: KAFKA-1298.patch > Controlled shutdown tool doesn't seem to work ou

[jira] [Commented] (KAFKA-1298) Controlled shutdown tool doesn't seem to work out of the box

[ https://issues.apache.org/jira/browse/KAFKA-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002703#comment-14002703 ] Sriharsha Chintalapani commented on KAFKA-1298: --- Created reviewboard https:/

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/#review43455 --- Test cases failing: :clients:test ... org.apache.kafka.common.recor

[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002677#comment-14002677 ] James Oliver commented on KAFKA-1456: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Attachment: KAFKA-1456_2014-05-19_18:19:32.patch > Add LZ4 and LZ4C as a compression codec > --

Re: Review Request 21663: Patch for KAFKA-1456

> On May 20, 2014, 1:16 a.m., Joe Stein wrote: > > system_test/producer_perf/bin/run-compression-test-all.sh, line 1 > > > > > > In addition to what you added here can you also please add test cases > > for the compress

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/ --- (Updated May 20, 2014, 1:19 a.m.) Review request for kafka. Bugs: KAFKA-1456

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/#review43453 --- system_test/producer_perf/bin/run-compression-test-all.sh

Re: Review Request 21663: Patch for KAFKA-1456

> On May 19, 2014, 11 p.m., Joe Stein wrote: > > system_test/producer_perf/config/server.properties, line 18 > > > > > > There are dozens of changes to this file. When making a patch start > > with a clean trunk that

[jira] [Updated] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Attachment: KAFKA-1456_2014-05-19_16:39:01.patch > Add LZ4 and LZ4C as a compression codec > --

[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002583#comment-14002583 ] James Oliver commented on KAFKA-1456: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/ --- (Updated May 19, 2014, 11:39 p.m.) Review request for kafka. Bugs: KAFKA-1456

Re: Review Request 21663: Patch for KAFKA-1456

> On May 19, 2014, 11:08 p.m., Joe Stein wrote: > > clients/src/main/java/org/apache/kafka/common/record/CompressionType.java, > > line 45 > > > > > > This is the id that comes back to make the COMPRESSION_CODEC_MASK &

Re: Review Request 21663: Patch for KAFKA-1456

> On May 19, 2014, 10:58 p.m., Joe Stein wrote: > > core/src/main/scala/kafka/message/Message.scala, line 60 > > > > > > Why does the CompressionCodeMask have to change from 0x03 to 0x07 ? I > > see this as a client br

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/#review43441 --- clients/src/main/java/org/apache/kafka/common/record/CompressionTyp

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/#review43439 --- system_test/producer_perf/config/server.properties

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/#review43440 --- Can you please add test cases for this in MessageCompressionTest.sca

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/#review43437 --- core/src/main/scala/kafka/message/Message.scala

[jira] [Updated] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Attachment: KAFKA-1456_2014-05-19_15:01:10.patch > Add LZ4 and LZ4C as a compression codec > --

[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002477#comment-14002477 ] James Oliver commented on KAFKA-1456: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/ --- (Updated May 19, 2014, 10:01 p.m.) Review request for kafka. Bugs: KAFKA-1456

[jira] [Updated] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Attachment: (was: KAFKA-1456.patch) > Add LZ4 and LZ4C as a compression codec > ---

[jira] [Issue Comment Deleted] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Comment: was deleted (was: Created reviewboard against branch origin/trunk) > Add LZ4 and LZ4C as

[jira] [Updated] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Attachment: KAFKA-1456.patch > Add LZ4 and LZ4C as a compression codec > --

[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002424#comment-14002424 ] James Oliver commented on KAFKA-1456: - Created reviewboard https://reviews.apache.org/

Review Request 21663: Patch for KAFKA-1456

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663/ --- Review request for kafka. Bugs: KAFKA-1456 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002418#comment-14002418 ] James Oliver commented on KAFKA-1456: - Created reviewboard against branch origin/trun

[jira] [Updated] (KAFKA-1456) Add LZ4 and LZ4C as a compression codec

[ https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Oliver updated KAFKA-1456: Attachment: KAFKA-1456.patch > Add LZ4 and LZ4C as a compression codec > --

[jira] [Commented] (KAFKA-1447) Controlled shutdown deadlock when trying to send state updates

[ https://issues.apache.org/jira/browse/KAFKA-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002177#comment-14002177 ] Sam Meder commented on KAFKA-1447: -- I don't have those logs anymore, should have grabbed

[jira] Subscription: outstanding kafka patches

Issue Subscription Filter: outstanding kafka patches (89 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-1460 NoReplicaOnlineException: No replica for partition https://issues.apache.org/jira/browse/KAFKA-1460 KAFKA-1450 check in

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/#review43395 --- clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerCon

[jira] [Created] (KAFKA-1461) Replica fetcher threads do not implement any back-off behavior

Sam Meder created KAFKA-1461: Summary: Replica fetcher threads do not implement any back-off behavior Key: KAFKA-1461 URL: https://issues.apache.org/jira/browse/KAFKA-1461 Project: Kafka Issue T

[jira] [Updated] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-1461: - Summary: Replica fetcher thread does not implement any back-off behavior (was: Replica fetcher threads d

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/#review43385 --- clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.ja

Re: Review Request 19731: Patch for KAFKA-1328

> On May 19, 2014, 5:30 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java, > > lines 70-71 > > > > > > Instead of introducing ConsumerRecords, I'd prefer just returning >

[jira] [Resolved] (KAFKA-1437) ConsumerMetadataResponse should always include coordinator information

[ https://issues.apache.org/jira/browse/KAFKA-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-1437. --- Resolution: Fixed Updated the wiki. > ConsumerMetadataResponse should always include coordinator inf

[jira] [Closed] (KAFKA-1437) ConsumerMetadataResponse should always include coordinator information

[ https://issues.apache.org/jira/browse/KAFKA-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy closed KAFKA-1437. - > ConsumerMetadataResponse should always include coordinator information > --

[jira] [Commented] (KAFKA-1448) Filter-plugins for messages

[ https://issues.apache.org/jira/browse/KAFKA-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002028#comment-14002028 ] Jim Hoagland commented on KAFKA-1448: - Having separate topics is the probably the best

Re: Review Request 19731: Patch for KAFKA-1328

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19731/#review43365 --- clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.ja

Re: Make kafka storage engine pluggable and provide a HDFS plugin?

Which version of Kafka are you using? Cluster expansion should be reliable in 0.8.1.1. As for using HDFS, we thought about it when implement replication in Kafka. The short answer is that using HDFS in Kafka is not easy. You can see the discussion in https://issues.apache.org/jira/browse/KAFKA-50

Make kafka storage engine pluggable and provide a HDFS plugin?

Hi there, I recently started to use Kafka for our data analysis pipeline and it works very well. One problem to us so far is expanding our cluster when we need more storage space. Kafka provides some scripts for helping do this but the process wasn't smooth. To make it work perfectly, seems Kafk