Re: [jira] [Commented] (KAFKA-717) scala 2.10 build support

2013-07-10 Thread Cosmin Lehene
We're also interested in getting this for 0.8. I'm curious if someone maintains an updated branch on Github or willing to do so. Thanks, Cosmin On 7/6/13 6:15 AM, "Matei Zaharia (JIRA)" wrote: > >[ >https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.pl >ugin.system.is

Make documentation part of new features acceptance criteria?

2013-07-10 Thread Cosmin Lehene
I'm not sure if there's already a guideline like this, but I wouldn't it make sense to have it in order to keep documentation in sync with the code? Also, having this type of documentation as part of the codebase to allow proper versioning might be a good idea as well. Cosmin

[jira] [Commented] (KAFKA-966) Allow high level consumer to 'nak' a message and force Kafka to close the KafkaStream without losing that message

2013-07-10 Thread Chris Curtin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704434#comment-13704434 ] Chris Curtin commented on KAFKA-966: Not to be dense, but wouldn't managing the offsets

Re: Make documentation part of new features acceptance criteria?

2013-07-10 Thread Jun Rao
Cosmin, That's a good idea. In the past, for major new features, we tend to create a wiki page to outline the design. The wiki pages can be organized better. Is this what you are looking for? Thanks, Jun On Wed, Jul 10, 2013 at 1:17 AM, Cosmin Lehene wrote: > I'm not sure if there's already

Re: Make documentation part of new features acceptance criteria?

2013-07-10 Thread Jay Kreps
I like the idea of improving our documentation. Help is very much appreciated in this area (but of course the problem is that the people who experience the holes almost by definition can't fill them in). So even just pointing out areas that aren't covered is really helpful. We are in a sort of awk

[jira] [Commented] (KAFKA-717) scala 2.10 build support

2013-07-10 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704719#comment-13704719 ] Jun Rao commented on KAFKA-717: --- We should take this patch in trunk, instead of 0.8.

Re: having problem with 0.8 gzip compression

2013-07-10 Thread Scott Wang
Jun, I did a test this morning and got a very interesting result with you command. I started by wipe all the log files and clean up all zookeeper data files. Once I restarted both server, producer and consumer then execute your command, what I got is a empty log as following: Dumping /Users/sco

[jira] [Created] (KAFKA-968) Typographical Errors in Output

2013-07-10 Thread Rebecca Sealfon (JIRA)
Rebecca Sealfon created KAFKA-968: - Summary: Typographical Errors in Output Key: KAFKA-968 URL: https://issues.apache.org/jira/browse/KAFKA-968 Project: Kafka Issue Type: Bug Compon

[jira] [Updated] (KAFKA-959) DefaultEventHandler can send more produce requests than necesary

2013-07-10 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-959: Assignee: Guozhang Wang > DefaultEventHandler can send more produce requests than necesary > ---

[jira] [Created] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Sriram Subramanian (JIRA)
Sriram Subramanian created KAFKA-969: Summary: Need to prevent failure of rebalance when there are no brokers available when consumer comes up Key: KAFKA-969 URL: https://issues.apache.org/jira/browse/KAFKA-96

[jira] [Updated] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-969: - Attachment: emptybrokeronrebalance-1.patch > Need to prevent failure of rebalance when

[jira] [Updated] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-969: - Status: Patch Available (was: Open) > Need to prevent failure of rebalance when there

formatting if statements

2013-07-10 Thread Jay Kreps
Guys, I am seeing this: if (condition) { // something } else { // something else } This is not the style we are using. Please don't do this or accept code that looks like this. It should be if (condition) { // something } else { //something else } Thanks! -Jay

[jira] [Commented] (KAFKA-965) merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk

2013-07-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705248#comment-13705248 ] Jay Kreps commented on KAFKA-965: - - core/src/main/scala/kafka/admin/AdminUtils.scala: You

[jira] [Commented] (KAFKA-966) Allow high level consumer to 'nak' a message and force Kafka to close the KafkaStream without losing that message

2013-07-10 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705272#comment-13705272 ] Joel Koshy commented on KAFKA-966: -- Yes if you need to implement support for transactions

[jira] [Created] (KAFKA-970) ./sbt +package rebuilds the Hadoop consumer jar N times with the same output file

2013-07-10 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-970: --- Summary: ./sbt +package rebuilds the Hadoop consumer jar N times with the same output file Key: KAFKA-970 URL: https://issues.apache.org/jira/browse/KAFKA-970 Project: Kafka

Re: having problem with 0.8 gzip compression

2013-07-10 Thread Joel Koshy
Weird - I tried your exact code and it worked for me (although I was using 0.8 head and not the beta). Can you re-run with trace logs enabled in your producer and paste that output? Broker logs also if you can? Thanks, Joel On Wed, Jul 10, 2013 at 10:23 AM, Scott Wang wrote: > Jun, > > I did a

[jira] [Commented] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705340#comment-13705340 ] Joel Koshy commented on KAFKA-969: -- This seems reasonable, but I'm not fully convinced abo

[jira] [Commented] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705461#comment-13705461 ] Sriram Subramanian commented on KAFKA-969: -- The other issues you mention are separ

[jira] [Commented] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705484#comment-13705484 ] Guozhang Wang commented on KAFKA-969: - I think KAFKA-956 is orthogonal to this JIRA. It

[jira] [Updated] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-969: -- Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) Thanks for the p

[jira] [Commented] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705490#comment-13705490 ] Joel Koshy commented on KAFKA-969: -- As I already said, I'm +1 on this patch for what it in

[jira] [Commented] (KAFKA-969) Need to prevent failure of rebalance when there are no brokers available when consumer comes up

2013-07-10 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705517#comment-13705517 ] Swapnil Ghike commented on KAFKA-969: - I agree with Joel. Registering a consumers' subs

[jira] [Created] (KAFKA-971) Handle synchronization in updatemetatdata in KafkaApi better

2013-07-10 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-971: - Summary: Handle synchronization in updatemetatdata in KafkaApi better Key: KAFKA-971 URL: https://issues.apache.org/jira/browse/KAFKA-971 Project: Kafka Issue Type: Impro

[jira] [Updated] (KAFKA-965) merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk

2013-07-10 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-965: -- Resolution: Fixed Fix Version/s: 0.8.1 Status: Resolved (was: Patch Available) Thanks for the

[jira] [Assigned] (KAFKA-838) Update design document to match Kafka 0.8 design

2013-07-10 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian reassigned KAFKA-838: Assignee: Sriram Subramanian > Update design document to match Kafka 0.8 design

[jira] [Resolved] (KAFKA-781) Add option to the controlled shutdown tool to timeout after n secs

2013-07-10 Thread Sriram Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian resolved KAFKA-781. -- Resolution: Fixed We have moved this logic into the broker. > Add optio