[jira] [Commented] (KAFKA-235) Add a 'log.file.age' configuration parameter to force rotation of log files after they've reached a certain age

2012-12-05 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511208#comment-13511208 ] Swapnil Ghike commented on KAFKA-235: - Yes, KAFKA-475 added a feature to 0.7 and 0.8 th

[jira] [Created] (KAFKA-655) IOException due to yammer metrics on Kafka brokers

2012-12-05 Thread Neha Narkhede (JIRA)
Neha Narkhede created KAFKA-655: --- Summary: IOException due to yammer metrics on Kafka brokers Key: KAFKA-655 URL: https://issues.apache.org/jira/browse/KAFKA-655 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511152#comment-13511152 ] Jun Rao commented on KAFKA-642: --- Another thing. 62. FetchResponsePartitionData.initialOffset

[jira] [Updated] (KAFKA-654) Irrecoverable error while trying to roll a segment that already exists

2012-12-05 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-654: Description: I tried setting up a 5 broker 0.8 cluster and sending messages to 100s of topics on it

[jira] [Created] (KAFKA-654) Irrecoverable error while trying to roll a segment that already exists

2012-12-05 Thread Neha Narkhede (JIRA)
Neha Narkhede created KAFKA-654: --- Summary: Irrecoverable error while trying to roll a segment that already exists Key: KAFKA-654 URL: https://issues.apache.org/jira/browse/KAFKA-654 Project: Kafka

[jira] [Commented] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511139#comment-13511139 ] Jun Rao commented on KAFKA-642: --- Thanks for the patch. A couple of minor comments: 60. Topic

[jira] [Commented] (KAFKA-597) Refactor KafkaScheduler

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511129#comment-13511129 ] Jay Kreps commented on KAFKA-597: - Ready for review. > Refactor KafkaSched

[jira] [Resolved] (KAFKA-518) Replace kafka.log.SegmentList with ConcurrentSkipListMap

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-518. - Resolution: Fixed Assignee: Jay Kreps This was done as part of KAFKA-521 > Replace

[jira] [Assigned] (KAFKA-521) Refactor Log subsystem

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-521: --- Assignee: Jay Kreps > Refactor Log subsystem > -- > > Key: KAF

[jira] [Assigned] (KAFKA-201) Support for mirroring from multiple sources

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-201: --- Assignee: Joel Koshy Mirror maker does this, no? > Support for mirroring from multipl

[jira] [Assigned] (KAFKA-235) Add a 'log.file.age' configuration parameter to force rotation of log files after they've reached a certain age

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-235: --- Assignee: Swapnil Ghike Didn't we do this? > Add a 'log.file.age' configuration param

[jira] [Resolved] (KAFKA-561) Rebuild index file for a log segment if there is none

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-561. - Resolution: Fixed Fix Version/s: 0.8.1 Assignee: Jay Kreps This was fixed as part of the l

[jira] [Updated] (KAFKA-653) Allow getTopicMetadata to get metadata for all topics

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-653: Labels: newbie (was: ) > Allow getTopicMetadata to get metadata for all topics > --

[jira] [Commented] (KAFKA-631) Implement log compaction

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1355#comment-1355 ] Jay Kreps commented on KAFKA-631: - Yes, I am actually working on it now (forgot to assign t

[jira] [Resolved] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-642. - Resolution: Fixed > Protocol tweaks for 0.8 > --- > > Key: KAFKA-6

[jira] [Created] (KAFKA-653) Allow getTopicMetadata to get metadata for all topics

2012-12-05 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-653: --- Summary: Allow getTopicMetadata to get metadata for all topics Key: KAFKA-653 URL: https://issues.apache.org/jira/browse/KAFKA-653 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-653) Allow getTopicMetadata to get metadata for all topics

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-653: Issue Type: New Feature (was: Bug) > Allow getTopicMetadata to get metadata for all topics > --

[jira] [Commented] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511073#comment-13511073 ] Jay Kreps commented on KAFKA-642: - Neha: 12. No, that should be fine, right? The alternativ

[jira] [Commented] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511022#comment-13511022 ] Joel Koshy commented on KAFKA-642: -- +1 BTW, is there a separate jira open on fetching met

[jira] [Updated] (KAFKA-597) Refactor KafkaScheduler

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-597: Attachment: KAFKA-597-v3.patch Attached v3 patch, same as before but fully rebased to trunk.

[jira] [Assigned] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-642: --- Assignee: Jay Kreps > Protocol tweaks for 0.8 > --- > > Key: K

[jira] [Commented] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510878#comment-13510878 ] Neha Narkhede commented on KAFKA-642: - +1. Minor observations - 12. In OffsetRequest,

[jira] [Updated] (KAFKA-633) AdminTest.testShutdownBroker fails

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-633: -- Priority: Blocker (was: Major) > AdminTest.testShutdownBroker fails > -- >

[jira] [Updated] (KAFKA-648) Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-648: -- Priority: Blocker (was: Major) > Use uniform convention for naming properties keys for clientId, groupId,

[jira] [Updated] (KAFKA-513) Add state change log to Kafka brokers

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-513: -- Priority: Blocker (was: Major) > Add state change log to Kafka brokers > --

[jira] [Updated] (KAFKA-598) decouple fetch size from max message size

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-598: -- Priority: Blocker (was: Major) > decouple fetch size from max message size > --

[jira] [Updated] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-642: -- Component/s: core Affects Version/s: 0.8 > Protocol tweaks for 0.8 > --- > >

[jira] [Updated] (KAFKA-411) Message Error in high cocurrent environment

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-411: -- Fix Version/s: (was: 0.8) 0.8.1 Not sure if this is still an issue in 0.8. Move it to 0.8

[jira] [Closed] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-545. - > Add a Performance Suite for the Log subsystem > - > >

[jira] [Commented] (KAFKA-652) Create testcases for clean shut-down

2012-12-05 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510791#comment-13510791 ] John Fung commented on KAFKA-652: - This test will be set up as followings: 1. Configure a c

[jira] [Created] (KAFKA-652) Create testcases for clean shut-down

2012-12-05 Thread John Fung (JIRA)
John Fung created KAFKA-652: --- Summary: Create testcases for clean shut-down Key: KAFKA-652 URL: https://issues.apache.org/jira/browse/KAFKA-652 Project: Kafka Issue Type: Task Reporter:

[jira] [Commented] (KAFKA-598) decouple fetch size from max message size

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510696#comment-13510696 ] Jun Rao commented on KAFKA-598: --- Thanks for patch v2. Looks good overhead. A couple of commen

Re: changes in zkClient library

2012-12-05 Thread Bae, Jae Hyeon
Thanks a lot for your answer. I am using zookeeper-3.3.5.jar now. On Wed, Dec 5, 2012 at 10:41 AM, Neha Narkhede wrote: > Jae, > > We modified the zkclient read API to expose the zookeeper Stat object. > This is because we need to do conditional writes in some places in > Kafka. Exposing the Sta

Re: changes in zkClient library

2012-12-05 Thread Neha Narkhede
Jae, We modified the zkclient read API to expose the zookeeper Stat object. This is because we need to do conditional writes in some places in Kafka. Exposing the Stat object allows us to read the previous version of the zookeeper value, and we can use that to write the new value if the expected v

[jira] [Commented] (KAFKA-608) getTopicMetadata does not respect producer config settings

2012-12-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510663#comment-13510663 ] Jun Rao commented on KAFKA-608: --- I guess that you are talking about ClientUtils. Currently, f

changes in zkClient library

2012-12-05 Thread Bae, Jae Hyeon
Could you share what changed in zkclient-20120522.jar? I found that watchers were canceled when the zookeeper connection was interrupted and reconnected with another application. If the new zkclient-20120522 resolved this issue, I need to update this library in my other projects. I really appreci

[jira] [Updated] (KAFKA-642) Protocol tweaks for 0.8

2012-12-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-642: Attachment: KAFKA-642-v6.patch Patch v6. - Rebased to 0.8 head. - Removed changes to ClientUtils since they