[jira] [Updated] (KAFKA-598) decouple fetch size from max message size

2012-12-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-598: - Attachment: KAFKA-598-v2.patch Here is an updated patch. After discussing with Jay, here is a proposal tha

[jira] [Commented] (KAFKA-608) getTopicMetadata does not respect producer config settings

2012-12-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510264#comment-13510264 ] Jay Kreps commented on KAFKA-608: - Guys, previously we had a terrible api to fetch metadata

[jira] [Closed] (KAFKA-450) Test in sync replica changes due to slow or stuck followers

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-450. --- > Test in sync replica changes due to slow or stuck followers > --

[jira] [Resolved] (KAFKA-450) Test in sync replica changes due to slow or stuck followers

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung resolved KAFKA-450. - Resolution: Fixed testcase_0301 ~ testcase_0308 cover the test requirements of this JIRA. So mark this cl

[jira] [Closed] (KAFKA-446) Message replication with failures and varied producer acks

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-446. --- > Message replication with failures and varied producer acks > ---

[jira] [Closed] (KAFKA-447) Message replication with failures and multiple partitions and log segments

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-447. --- > Message replication with failures and multiple partitions and log segments > ---

[jira] [Closed] (KAFKA-448) Message replication with failures and varied replication factor

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-448. --- > Message replication with failures and varied replication factor > --

[jira] [Closed] (KAFKA-445) Message replication with failures

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-445. --- > Message replication with failures > - > > Key: KAFKA-445 >

[jira] [Closed] (KAFKA-442) Simple message replication test with varied producer acks

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-442. --- > Simple message replication test with varied producer acks >

[jira] [Closed] (KAFKA-443) Simple message replication test with multiple partitions

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-443. --- > Simple message replication test with multiple partitions > -

[jira] [Closed] (KAFKA-444) Simple message replication test with multiple partitions and log segments

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-444. --- > Simple message replication test with multiple partitions and log segments >

[jira] [Closed] (KAFKA-441) Simple message replication test

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung closed KAFKA-441. --- > Simple message replication test > --- > > Key: KAFKA-441 >

[jira] [Closed] (KAFKA-408) ProducerPerformance does not work with all producer config options

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-408. - > ProducerPerformance does not work with all producer config options > ---

[jira] [Updated] (KAFKA-408) ProducerPerformance does not work with all producer config options

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-408: -- Resolution: Duplicate Fix Version/s: 0.8 Status: Resolved (was: Patch Available) That's fixed

[jira] [Updated] (KAFKA-496) high level producer send should return a response

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-496: -- Fix Version/s: (was: 0.8) 0.8.1 Moving this to 0.8.1. > high level produ

[jira] [Resolved] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-514. --- Resolution: Duplicate Fixed in kafka-551. > Replication with Leader Failure Test: Log segment

[jira] [Closed] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-514. - > Replication with Leader Failure Test: Log segment files checksum mismatch >

[jira] [Updated] (KAFKA-451) follower replica may need to backoff the fetching if leader is not ready yet

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-451: -- Priority: Major (was: Blocker) Fix Version/s: (was: 0.8) 0.8.1 Move to 0.8.1 si

[jira] [Updated] (KAFKA-566) Add last modified time to the TopicMetadataRequest

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-566: -- Priority: Major (was: Blocker) Fix Version/s: (was: 0.8) 0.8.1 > Add last m

[jira] [Closed] (KAFKA-588) Index truncation doesn't seem to remove the last entry properly

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-588. - > Index truncation doesn't seem to remove the last entry properly > --

[jira] [Closed] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-544. - > Retain key in producer and expose it in the consumer > > >

[jira] [Closed] (KAFKA-532) Multiple controllers can co-exist during soft failures

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-532. - > Multiple controllers can co-exist during soft failures > --

[jira] [Resolved] (KAFKA-630) Auto create topic doesn't reflect the new topic and throws UnknownTopicOrPartitionException

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-630. --- Resolution: Duplicate > Auto create topic doesn't reflect the new topic and throws > UnknownTopicOrPartit

[jira] [Closed] (KAFKA-630) Auto create topic doesn't reflect the new topic and throws UnknownTopicOrPartitionException

2012-12-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-630. - > Auto create topic doesn't reflect the new topic and throws > UnknownTopicOrPartitionException > ---

[jira] [Created] (KAFKA-651) Create testcases on auto create topics

2012-12-04 Thread John Fung (JIRA)
John Fung created KAFKA-651: --- Summary: Create testcases on auto create topics Key: KAFKA-651 URL: https://issues.apache.org/jira/browse/KAFKA-651 Project: Kafka Issue Type: Task Reporte

[jira] [Created] (KAFKA-650) Create a testcase for Migration Tool with hundreds of topics

2012-12-04 Thread John Fung (JIRA)
John Fung created KAFKA-650: --- Summary: Create a testcase for Migration Tool with hundreds of topics Key: KAFKA-650 URL: https://issues.apache.org/jira/browse/KAFKA-650 Project: Kafka Issue Type: T

[jira] [Created] (KAFKA-649) Cleanup log4j logging

2012-12-04 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-649: --- Summary: Cleanup log4j logging Key: KAFKA-649 URL: https://issues.apache.org/jira/browse/KAFKA-649 Project: Kafka Issue Type: Improvement Reporter: Jay Kre

[jira] [Updated] (KAFKA-513) Add state change log to Kafka brokers

2012-12-04 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-513: Priority: Major (was: Blocker) > Add state change log to Kafka brokers > --

[jira] [Updated] (KAFKA-644) System Test should run properly with mixed File System Pathname

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-644: Attachment: (was: kafka-644-v1.patch) > System Test should run properly with mixed File System Pathn

[jira] [Updated] (KAFKA-644) System Test should run properly with mixed File System Pathname

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-644: Attachment: kafka-644-v1.patch Uploaded kafka-644-v1.patch which covers the testcase of auto create topic

[jira] [Updated] (KAFKA-648) Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId

2012-12-04 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-648: Fix Version/s: 0.8.1 > Use uniform convention for naming properties keys for clientId, groupId,

[jira] [Commented] (KAFKA-631) Implement log compaction

2012-12-04 Thread Prashanth Menon (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510029#comment-13510029 ] Prashanth Menon commented on KAFKA-631: --- Hey Jay, is this something you wanted to tak

[jira] [Commented] (KAFKA-648) Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId

2012-12-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510021#comment-13510021 ] Jay Kreps commented on KAFKA-648: - I actually recommend that we do a full config review and

[jira] [Commented] (KAFKA-648) Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId

2012-12-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510016#comment-13510016 ] Jay Kreps commented on KAFKA-648: - A bunch of the config values are non standard. For examp

[jira] [Updated] (KAFKA-644) System Test should run properly with mixed File System Pathname

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-644: Status: Patch Available (was: Open) > System Test should run properly with mixed File System Pathname >

[jira] [Updated] (KAFKA-644) System Test should run properly with mixed File System Pathname

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-644: Attachment: kafka-644-v1.patch > System Test should run properly with mixed File System Pathname > -

[jira] [Commented] (KAFKA-648) Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId

2012-12-04 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509991#comment-13509991 ] Neha Narkhede commented on KAFKA-648: - Not sure I got the part about config.val, where

[jira] [Assigned] (KAFKA-636) Make log segment delete asynchronous

2012-12-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-636: --- Assignee: Jay Kreps > Make log segment delete asynchronous >

[jira] [Updated] (KAFKA-642) Protocol tweaks for 0.8

2012-12-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-642: Attachment: KAFKA-642-v4.patch Updated patch. This patch also makes the message format consistent with the

[jira] [Commented] (KAFKA-648) Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId

2012-12-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509982#comment-13509982 ] Jay Kreps commented on KAFKA-648: - word. > Use uniform convention for nami

[jira] [Commented] (KAFKA-622) Create mbeans per client

2012-12-04 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509976#comment-13509976 ] Swapnil Ghike commented on KAFKA-622: - Created KAFKA-646 to address your comments.

[jira] [Assigned] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

2012-12-04 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike reassigned KAFKA-646: --- Assignee: Swapnil Ghike > Provide aggregate stats at the high level Producer and > Zookee

[jira] [Created] (KAFKA-648) Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId

2012-12-04 Thread Swapnil Ghike (JIRA)
Swapnil Ghike created KAFKA-648: --- Summary: Use uniform convention for naming properties keys for clientId, groupId, consumerId, correlationId Key: KAFKA-648 URL: https://issues.apache.org/jira/browse/KAFKA-648

[jira] [Assigned] (KAFKA-647) Provide a property in System Test for no. of topics and topics string will be generated automatically

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung reassigned KAFKA-647: --- Assignee: John Fung > Provide a property in System Test for no. of topics and topics string will b

[jira] [Created] (KAFKA-647) Provide a property in System Test for no. of topics and topics string will be generated automatically

2012-12-04 Thread John Fung (JIRA)
John Fung created KAFKA-647: --- Summary: Provide a property in System Test for no. of topics and topics string will be generated automatically Key: KAFKA-647 URL: https://issues.apache.org/jira/browse/KAFKA-647

[jira] [Updated] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

2012-12-04 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-646: Labels: bugs (was: ) > Provide aggregate stats at the high level Producer and > ZookeeperConsu

[jira] [Created] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

2012-12-04 Thread Swapnil Ghike (JIRA)
Swapnil Ghike created KAFKA-646: --- Summary: Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level Key: KAFKA-646 URL: https://issues.apache.org/jira/browse/KAFKA-646 Pro

[jira] [Commented] (KAFKA-645) Create a shell script to run System Test with DEBUG details and "tee" console output to a file

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509943#comment-13509943 ] John Fung commented on KAFKA-645: - Attached kafka-645-v1.patch which is a simple patch to p

[jira] [Updated] (KAFKA-645) Create a shell script to run System Test with DEBUG details and "tee" console output to a file

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-645: Status: Patch Available (was: Open) > Create a shell script to run System Test with DEBUG details and "

[jira] [Updated] (KAFKA-645) Create a shell script to run System Test with DEBUG details and "tee" console output to a file

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-645: Attachment: kafka-645-v1.patch > Create a shell script to run System Test with DEBUG details and "tee" c

Re: Protocol documentation draft

2012-12-04 Thread Jun Rao
Thanks for drafting this. Made some minor edits and added some comments. Thanks, Jun On Thu, Nov 29, 2012 at 3:17 PM, Jay Kreps wrote: > I started trying to document the 0.8 protocol from the code and write a > guide to client implementation. This is meant to be a more user-friendly > and up-t

Refactored scheduler

2012-12-04 Thread Jay Kreps
Hey guys, I refactored the scheduler. What I did was the following: 1. Add a Scheduler trait and make everything depend on that, KafkaScheduler implements this 2. Fix broken stuff in KafkaScheduler 3. Add support for one-time tasks (i.e. do this once in the future after a specified delay. 4. Added

[jira] [Updated] (KAFKA-597) Refactor KafkaScheduler

2012-12-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-597: Attachment: KAFKA-597-v2.patch Updated patch, includes misc additional cleanups. > Refactor

[jira] [Created] (KAFKA-645) Create a shell script to run System Test with DEBUG details and "tee" console output to a file

2012-12-04 Thread John Fung (JIRA)
John Fung created KAFKA-645: --- Summary: Create a shell script to run System Test with DEBUG details and "tee" console output to a file Key: KAFKA-645 URL: https://issues.apache.org/jira/browse/KAFKA-645 Proj

[jira] [Updated] (KAFKA-644) System Test should run properly with mixed File System Pathname

2012-12-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-644: Description: Currently, System Test assumes that all the entities (ZK, Broker, Producer, Consumer) are runn

[jira] [Created] (KAFKA-644) System Test should run properly with mixed File System Pathname

2012-12-04 Thread John Fung (JIRA)
John Fung created KAFKA-644: --- Summary: System Test should run properly with mixed File System Pathname Key: KAFKA-644 URL: https://issues.apache.org/jira/browse/KAFKA-644 Project: Kafka Issue Type

Re: maintainers

2012-12-04 Thread Jay Kreps
Yeah that is a good idea. -Jay On Tue, Dec 4, 2012 at 5:32 AM, Joe Stein wrote: > I signed up on some. cool. > > We should maybe match these to JIRA components? > > On Mon, Dec 3, 2012 at 2:42 PM, Jonathan Creasy wrote: > > > I signed up under command line tools since no one had. I'll work o

Re: maintainers

2012-12-04 Thread Joe Stein
I signed up on some. cool. We should maybe match these to JIRA components? On Mon, Dec 3, 2012 at 2:42 PM, Jonathan Creasy wrote: > I signed up under command line tools since no one had. I'll work on getting > up to speed there. > > Looking forward to working with you all! > On Dec 3, 2012 10: