Re: Splitting client code

2012-12-17 Thread ben fleis
As a somewhat related follow up, what are the team's intentions wrt releasing jar files, period? I see source tarballs, but no precompiled, ready-to-run jars in the download section. Is this in the roadmap, easily obtained, etc? For some deployment (like mine, where my client is in another langu

Re: Splitting client code

2012-12-17 Thread David Arthur
On 12/17/12 6:05 AM, ben fleis wrote: As a somewhat related follow up, what are the team's intentions wrt releasing jar files, period? I see source tarballs, but no precompiled, ready-to-run jars in the download section. Is this in the roadmap, easily obtained, etc? For some deployment (like m

[jira] [Commented] (KAFKA-647) Provide a property in System Test for no. of topics and topics string will be generated automatically

2012-12-17 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534116#comment-13534116 ] Neha Narkhede commented on KAFKA-647: - +1, thanks for the patch. > Pro

[jira] [Commented] (KAFKA-657) Add an API to commit offsets

2012-12-17 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534122#comment-13534122 ] Jay Kreps commented on KAFKA-657: - This looks great! Three minor things: 1. Can you change

[jira] [Commented] (KAFKA-657) Add an API to commit offsets

2012-12-17 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534151#comment-13534151 ] David Arthur commented on KAFKA-657: Re 3: Maybe this is a case for the check-and-set f

Offset commit api

2012-12-17 Thread Jay Kreps
Hey Guys, David has made a bunch of progress on the offset commit api implementation. Since this is a public API it would be good to do as much thinking up-front as possible to minimize future iterations. It would be great if folks could do the following: 1. Read the wiki here: https://cwiki.apa

[jira] [Commented] (KAFKA-657) Add an API to commit offsets

2012-12-17 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534162#comment-13534162 ] Jay Kreps commented on KAFKA-657: - I think that actually covers an orthogonal problem right

[jira] Subscription: outstanding kafka patches

2012-12-17 Thread jira
Issue Subscription Filter: outstanding kafka patches (56 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-672 Merge 0.8 changes to trunk https://issues.apache.org/jira/browse/KAFKA-672 KAFKA-664 Kafka server threads die

[jira] [Created] (KAFKA-674) Clean Shutdown Testing - Log segments checksums mismatch

2012-12-17 Thread John Fung (JIRA)
John Fung created KAFKA-674: --- Summary: Clean Shutdown Testing - Log segments checksums mismatch Key: KAFKA-674 URL: https://issues.apache.org/jira/browse/KAFKA-674 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-674) Clean Shutdown Testing - Log segments checksums mismatch

2012-12-17 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534172#comment-13534172 ] John Fung commented on KAFKA-674: - Test description: 1. Start 1 Zookeeper 2. S

[jira] [Updated] (KAFKA-674) Clean Shutdown Testing - Log segments checksums mismatch

2012-12-17 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-674: Attachment: kafka-674-reproduce-issue.patch > Clean Shutdown Testing - Log segments checksums mismatch >

old github kafka

2012-12-17 Thread Jay Kreps
Any objections to my deleting the old github kafka repository. We kept it around for posterity and to preserve the version control history. But I found that it confuses people as to where we are hosted: https://github.com/kafka-dev/kafka We had an active replica of our apache git/svn replica, an

[jira] [Commented] (KAFKA-657) Add an API to commit offsets

2012-12-17 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534178#comment-13534178 ] David Arthur commented on KAFKA-657: Yes, they are somewhat orthogonal. The only real u

[jira] [Commented] (KAFKA-657) Add an API to commit offsets

2012-12-17 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534233#comment-13534233 ] Jay Kreps commented on KAFKA-657: - Makes sense. The API is versioned so we can always add t

Re: old github kafka

2012-12-17 Thread Prashanth Menon
I'm okay deleting the old git repository. Heck, I was confused with which repo to use when I started, too. - Prashanth On Mon, Dec 17, 2012 at 2:04 PM, Jay Kreps wrote: > Any objections to my deleting the old github kafka repository. We kept it > around for posterity and to preserve the versio

[jira] [Updated] (KAFKA-657) Add an API to commit offsets

2012-12-17 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-657: --- Attachment: KAFKA-657v3.patch v3 of the patch 1. info -> debug 2. catching general exceptions and ret

[jira] [Resolved] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

2012-12-17 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-646. - Resolution: Fixed Thanks for v7. Just checked it in. > Provide aggregate stats at

[jira] [Closed] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

2012-12-17 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-646. --- > Provide aggregate stats at the high level Producer and > ZookeeperConsumerConnector level > ---

[jira] [Updated] (KAFKA-647) Provide a property in System Test for no. of topics and topics string will be generated automatically

2012-12-17 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-647: Resolution: Fixed Status: Resolved (was: Patch Available) Checked in v1 >

[jira] [Closed] (KAFKA-647) Provide a property in System Test for no. of topics and topics string will be generated automatically

2012-12-17 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-647. --- > Provide a property in System Test for no. of topics and topics string will be > generated automatic

Re: Offset commit api

2012-12-17 Thread Milind Parikh
Perhaps I don't understand the motivation well enough and perhaps I am misreading the intent. But I thought that the design principle behind kafka is for state (from a consumer standpoint) was to be managed by consumer and not broker. I understand that "These APIs are optional, clients can store

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

2012-12-17 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534388#comment-13534388 ] Neha Narkhede commented on KAFKA-664: - +1 on v4 > Kafka server threads

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

2012-12-17 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534388#comment-13534388 ] Neha Narkhede commented on KAFKA-664: - +1 on v4 > Kafka server threads

[jira] [Updated] (KAFKA-675) Only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Matan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matan updated KAFKA-675: Summary: Only bind to the interface declared in the 'hostname' config property (was: Kafka should only bind to the

[jira] [Created] (KAFKA-675) Kafka should only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Matan (JIRA)
Matan created KAFKA-675: --- Summary: Kafka should only bind to the interface declared in the 'hostname' config property Key: KAFKA-675 URL: https://issues.apache.org/jira/browse/KAFKA-675 Project: Kafka

[jira] [Updated] (KAFKA-675) Only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Matan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matan updated KAFKA-675: Attachment: kafka-675.patch Here is one suggested patch. It works for us in our environments. > On

[jira] [Commented] (KAFKA-675) Only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534439#comment-13534439 ] Jay Kreps commented on KAFKA-675: - Nice. So does the default make it bind to all hosts? Gu

[jira] [Commented] (KAFKA-675) Only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Matan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534442#comment-13534442 ] Matan commented on KAFKA-675: - Nope actually, in this setup, the default would bind to the inte

[jira] [Comment Edited] (KAFKA-675) Only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Matan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534442#comment-13534442 ] Matan edited comment on KAFKA-675 at 12/18/12 12:00 AM: Nope actual

Re: Offset commit api

2012-12-17 Thread David Arthur
Currently, consumers must deal with ZooKeeper directly and implement the consumer re-balancing algorithm (correctly). As this is a rather difficult and error-prone process, the intent of this new API is to provide an easy mechanism for storing offsets for non-Scala clients. At least, that's ho

Re: Offset commit api

2012-12-17 Thread Jay Kreps
There are two questions: 1. Who controls the consumers position in the stream 2. How is that position stored? The theory for (1) is that the consumer should control this so it can chose to move forwards or backwards or wherever it wants and has full control over when position changes take effect.

[jira] [Commented] (KAFKA-675) Only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534462#comment-13534462 ] Jay Kreps commented on KAFKA-675: - I think I probably broke this without realizing it so I

[jira] [Resolved] (KAFKA-664) Kafka server threads die due to OOME during long running test

2012-12-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-664. -- Resolution: Fixed Committed on 0.8 > Kafka server threads die due to OOME during long r

[jira] [Closed] (KAFKA-664) Kafka server threads die due to OOME during long running test

2012-12-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy closed KAFKA-664. > Kafka server threads die due to OOME during long running test > --

[jira] [Commented] (KAFKA-675) Only bind to the interface declared in the 'hostname' config property

2012-12-17 Thread Matan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534485#comment-13534485 ] Matan commented on KAFKA-675: - Sounds like a good plan Jay. getCanonicalHostName is probably m

Re: review board?

2012-12-17 Thread Neha Narkhede
> Will this setup work with git? > Yes. See this - http://ericholscher.com/blog/2011/jan/24/using-reviewboard-git/ > > Is it possible to get patches that are attached in JIRA automatically > entered in review board? If we have to do it manually it might be too > annoying... > Are there any examp