[jira] [Updated] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Attachment: KAFKA-1018.patch
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791261#comment-13791261
]
Joe Stein commented on KAFKA-1018:
--
Created reviewboard
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Attachment: KAFKA-1018.patch
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791262#comment-13791262
]
Joe Stein commented on KAFKA-1018:
--
Created reviewboard
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
Review Request 14571: Patch for KAFKA-1018
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14571/ --- Review request for kafka. Bugs: KAFKA-1018 https://issues.apache.org/jira/browse/KAFKA-1018 Repository: kafka Description --- KAFKA-1018 tidy up the pom, 0.8.0 release, build the test jars now too Diffs - project/Build.scala 33bb76f4ebb899610e66acd8af1d15c15af1f7cc Diff: https://reviews.apache.org/r/14571/diff/ Testing --- Thanks, Joe Stein
[jira] [Updated] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Attachment: KAFKA-1018.patch
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791264#comment-13791264
]
Joe Stein commented on KAFKA-1018:
--
Created reviewboard
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Attachment: KAFKA-1018.patch
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Attachment: KAFKA-1018.patch
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791267#comment-13791267
]
Joe Stein commented on KAFKA-1018:
--
Created reviewboard https://reviews.apache.org/r/14571/
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791266#comment-13791266
]
Joe Stein commented on KAFKA-1018:
--
Created reviewboard
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791265#comment-13791265
]
Joe Stein commented on KAFKA-1018:
--
Created reviewboard
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Attachment: KAFKA-1018.patch
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Attachment: KAFKA-1018.patch
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791263#comment-13791263
]
Joe Stein commented on KAFKA-1018:
--
Created reviewboard
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
Re: Review Request 14571: Patch for KAFKA-1018
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14571/ --- (Updated Oct. 10, 2013, 7:12 a.m.) Review request for kafka. Bugs: KAFKA-1018 https://issues.apache.org/jira/browse/KAFKA-1018 Repository: kafka Description --- KAFKA-1018 tidy up the pom, 0.8.0 release, build the test jars now too Diffs - project/Build.scala 33bb76f4ebb899610e66acd8af1d15c15af1f7cc Diff: https://reviews.apache.org/r/14571/diff/ Testing (updated) --- deployed a nexus repo in dez cloud, published to it. verified my sbt sanity builds against it. "org.apache.kafka" % "kafka_2.10" % "0.8.0", working as expected without the extra workarounds for previous issues Thanks, Joe Stein
[jira] [Issue Comment Deleted] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Comment: was deleted
(was: Created reviewboard )
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Issue Comment Deleted] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Comment: was deleted
(was: Created reviewboard )
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Issue Comment Deleted] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Comment: was deleted
(was: Created reviewboard )
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Issue Comment Deleted] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Comment: was deleted
(was: Created reviewboard )
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Issue Comment Deleted] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Comment: was deleted
(was: Created reviewboard )
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Issue Comment Deleted] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1018:
-
Comment: was deleted
(was: Created reviewboard )
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1080) why are builds for 2.10 not coming out with the trailing minor version number
[ https://issues.apache.org/jira/browse/KAFKA-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791311#comment-13791311 ] Viktor Taranenko commented on KAFKA-1080: - I think all versions from 2.10.x series are binary compatible between each other. So there is no need in minor version for builds. > why are builds for 2.10 not coming out with the trailing minor version number > - > > Key: KAFKA-1080 > URL: https://issues.apache.org/jira/browse/KAFKA-1080 > Project: Kafka > Issue Type: Task >Reporter: Joe Stein > Fix For: 0.8.1 > > > Joes-MacBook-Air:kafka joestein$ ls -l target/ > total 0 > drwxr-xr-x 4 joestein wheel 136 Sep 26 22:54 resolution-cache > drwxr-xr-x 42 joestein wheel 1428 Oct 10 00:33 scala-2.10 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:37 scala-2.8.0 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:38 scala-2.8.2 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:39 scala-2.9.1 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:40 scala-2.9.2 > drwxr-xr-x 5 joestein wheel 170 Oct 10 00:02 streams -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1080) why are builds for 2.10 not coming out with the trailing minor version number
[ https://issues.apache.org/jira/browse/KAFKA-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791319#comment-13791319 ] Joe Stein commented on KAFKA-1080: -- then there is a way to remove the entire -2.10 completely if it has become obsolete? so our artifact id can just be kafka or core or something? > why are builds for 2.10 not coming out with the trailing minor version number > - > > Key: KAFKA-1080 > URL: https://issues.apache.org/jira/browse/KAFKA-1080 > Project: Kafka > Issue Type: Task >Reporter: Joe Stein > Fix For: 0.8.1 > > > Joes-MacBook-Air:kafka joestein$ ls -l target/ > total 0 > drwxr-xr-x 4 joestein wheel 136 Sep 26 22:54 resolution-cache > drwxr-xr-x 42 joestein wheel 1428 Oct 10 00:33 scala-2.10 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:37 scala-2.8.0 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:38 scala-2.8.2 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:39 scala-2.9.1 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:40 scala-2.9.2 > drwxr-xr-x 5 joestein wheel 170 Oct 10 00:02 streams -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1080) why are builds for 2.10 not coming out with the trailing minor version number
[ https://issues.apache.org/jira/browse/KAFKA-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791324#comment-13791324 ] Viktor Taranenko commented on KAFKA-1080: - I guess it is possible to do that by adjusting sbt build definition. But I'm not sure about the reason making 2.10 kind of default. 2.11 won't be binary compatible with 2.10... Transition between scala versions is not too fast for many libraries. So for those people who use Kafka in scala project probably it's worth keeping builds for couple of latest versions > why are builds for 2.10 not coming out with the trailing minor version number > - > > Key: KAFKA-1080 > URL: https://issues.apache.org/jira/browse/KAFKA-1080 > Project: Kafka > Issue Type: Task >Reporter: Joe Stein > Fix For: 0.8.1 > > > Joes-MacBook-Air:kafka joestein$ ls -l target/ > total 0 > drwxr-xr-x 4 joestein wheel 136 Sep 26 22:54 resolution-cache > drwxr-xr-x 42 joestein wheel 1428 Oct 10 00:33 scala-2.10 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:37 scala-2.8.0 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:38 scala-2.8.2 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:39 scala-2.9.1 > drwxr-xr-x 21 joestein wheel 714 Oct 9 23:40 scala-2.9.2 > drwxr-xr-x 5 joestein wheel 170 Oct 10 00:02 streams -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: Review Request 14571: Patch for KAFKA-1018
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14571/#review26865 --- Ship it! Do we want to address issue #2 in the patch too? - Jun Rao On Oct. 10, 2013, 7:12 a.m., Joe Stein wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14571/ > --- > > (Updated Oct. 10, 2013, 7:12 a.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1018 > https://issues.apache.org/jira/browse/KAFKA-1018 > > > Repository: kafka > > > Description > --- > > KAFKA-1018 tidy up the pom, 0.8.0 release, build the test jars now too > > > Diffs > - > > project/Build.scala 33bb76f4ebb899610e66acd8af1d15c15af1f7cc > > Diff: https://reviews.apache.org/r/14571/diff/ > > > Testing > --- > > deployed a nexus repo in dez cloud, published to it. verified my sbt sanity > builds against it. > > "org.apache.kafka" % "kafka_2.10" % "0.8.0", > > working as expected without the extra workarounds for previous issues > > > Thanks, > > Joe Stein > >
[jira] [Comment Edited] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791199#comment-13791199
]
Joe Stein edited comment on KAFKA-1018 at 10/10/13 3:15 PM:
1) maven central is immutable, resolved in this coming release
2) I think your right, yup. This was fixed on branch after found in maven
central.
3) just going by the docs
4) I have a fix for this I am testing right now
5) yup, agreed. I can create one patch for trunk and one for the 0.8 branch,
leave trunk on SNAPSHOT and then another JIRA for hooking it up to hudson I
gotta look into that
The changes I am testing are in a repo here
http://ec2-54-224-196-57.compute-1.amazonaws.com:8081/nexus/content/repositories/releases
with the pom of
http://ec2-54-224-196-57.compute-1.amazonaws.com:8081/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.0/kafka_2.10-0.8.0.pom
I am running into some issues I am bumping into while testing, working that out
now
was (Author: charmalloc):
1) maven central is immutable, resolved in this coming release
2) I think your right, yup
3) just going by the docs
4) I have a fix for this I am testing right now
5) yup, agreed. I can create one patch for trunk and one for the 0.8 branch,
leave trunk on SNAPSHOT and then another JIRA for hooking it up to hudson I
gotta look into that
The changes I am testing are in a repo here
http://ec2-54-224-196-57.compute-1.amazonaws.com:8081/nexus/content/repositories/releases
with the pom of
http://ec2-54-224-196-57.compute-1.amazonaws.com:8081/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.0/kafka_2.10-0.8.0.pom
I am running into some issues I am bumping into while testing, working that out
now
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
Re: Review Request 14571: Patch for KAFKA-1018
> On Oct. 10, 2013, 2:52 p.m., Jun Rao wrote: > > Do we want to address issue #2 in the patch too? issue #2 was fixed during the beta1 release it just could not make it to maven central because I had already deployed the bonked first publish one reason I setup my own repo was to double check that i could just do "org.apache.kafka" % "kafka_2.10" % "0.8.0", without specifying exclusions, sorry I didn't mention that in the JIRA will ninja edit my comments - Joe --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14571/#review26865 --- On Oct. 10, 2013, 7:12 a.m., Joe Stein wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14571/ > --- > > (Updated Oct. 10, 2013, 7:12 a.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1018 > https://issues.apache.org/jira/browse/KAFKA-1018 > > > Repository: kafka > > > Description > --- > > KAFKA-1018 tidy up the pom, 0.8.0 release, build the test jars now too > > > Diffs > - > > project/Build.scala 33bb76f4ebb899610e66acd8af1d15c15af1f7cc > > Diff: https://reviews.apache.org/r/14571/diff/ > > > Testing > --- > > deployed a nexus repo in dez cloud, published to it. verified my sbt sanity > builds against it. > > "org.apache.kafka" % "kafka_2.10" % "0.8.0", > > working as expected without the extra workarounds for previous issues > > > Thanks, > > Joe Stein > >
[jira] [Resolved] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein resolved KAFKA-1018.
--
Resolution: Fixed
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Resolved] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein resolved KAFKA-981.
-
Resolution: Fixed
See KAFKA-1018
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
>Assignee: Joe Stein
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Resolved] (KAFKA-978) kafka pom file has 2 entries for zookeeper (one with exclusion, one without)
[ https://issues.apache.org/jira/browse/KAFKA-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-978. - Resolution: Fixed Fix Version/s: 0.8 See KAFKA-1018 > kafka pom file has 2 entries for zookeeper (one with exclusion, one without) > > > Key: KAFKA-978 > URL: https://issues.apache.org/jira/browse/KAFKA-978 > Project: Kafka > Issue Type: Bug >Reporter: Jason Rosenberg > Fix For: 0.8 > > > I've noticed, in the 0.8-beta the pom file, there are 2 dependencies listed > for zookeeper, one of which has an exclusion clause, the other does not. I > assume this is not intended: > > org.apache.zookeeper > zookeeper > 3.3.4 > > > log4j > log4j > > > jline > jline > > > > and then > > org.apache.zookeeper > zookeeper > 3.3.4 > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (KAFKA-894) Cannot ./sbt update because https://oss.sonatype.org/content/groups/public/org/scalatest/ doesn't exist anymore
[ https://issues.apache.org/jira/browse/KAFKA-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-894. - Resolution: Implemented > Cannot ./sbt update because > https://oss.sonatype.org/content/groups/public/org/scalatest/ doesn't exist > anymore > --- > > Key: KAFKA-894 > URL: https://issues.apache.org/jira/browse/KAFKA-894 > Project: Kafka > Issue Type: Bug > Components: packaging >Affects Versions: 0.8 >Reporter: François Paradis > > I get these kinds of errors when trying to ./sbt update: > [error] Server access Error: Connection timed out > url=https://oss.sonatype.org/content/groups/public/org/scalatest/com/101tec/zkclient/0.2/zkclient-0.2.pom > It seems like this url doesn't point to anything anymore. > I tried replacing with https://oss.sonatype.org/content/groups/public/ as > https://oss.sonatype.org/content/groups/public/com/101tec/zkclient/0.2/ seems > to exist but I got more errors later on because snappy 1.0.4 wasn't available > there. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (KAFKA-1064) Bad pom.xml definition
[ https://issues.apache.org/jira/browse/KAFKA-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1064. -- Resolution: Fixed See KAFKA-1018 > Bad pom.xml definition > -- > > Key: KAFKA-1064 > URL: https://issues.apache.org/jira/browse/KAFKA-1064 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Ben Manes > > The generated pom.xml includes *two* dependency sections. This results in > Gradle being unable to resolve transitive dependencies as it inspects the > first dependency section, not the second which includes the actual > dependencies. Thus, using Kafka in Gradle requires specifying all the > dependencies by hand or rewriting the pom.xml and deploying to an internal > repository. > http://search.maven.org/remotecontent?filepath=org/apache/kafka/kafka_2.9.1/0.8.0-beta1/kafka_2.9.1-0.8.0-beta1.pom -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (KAFKA-121) pom should include standard maven niceties
[ https://issues.apache.org/jira/browse/KAFKA-121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-121: Fix Version/s: 0.8.1 will keep this open for the 0.8.1 release in case anything is missed > pom should include standard maven niceties > -- > > Key: KAFKA-121 > URL: https://issues.apache.org/jira/browse/KAFKA-121 > Project: Kafka > Issue Type: Improvement > Components: packaging >Reporter: Chris Burroughs >Priority: Minor > Fix For: 0.8.1 > > > * license info, name, description, etc > * webpage link > * parent tags for sub-modules > * groupid pending mailing list discussion > One refernce: > http://vasilrem.com/blog/software-development/from-sbt-to-maven-in-one-move/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791782#comment-13791782 ] Joe Stein commented on KAFKA-865: - Hi, does KAFKA-1018 clear this up or are there other updates/changes to make? > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven and they want to specify the GAV of kafka in > their pom and bang, the client jar and all its dependencies should be added > to the application's classpath. I can't do that right now, because I need to > run ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes together in same jar file, increases the size of the bundle for a > client and also everytime the server's code changes and a release is > performed, the client also needs to update their .JAR file. which is not very > great. We don't want a ton of clients to update their .JAR file, just because > a faster replication strategy for the kafka server cluster changed in a new > release. > Action items are to separate the client and server portions of Kafka, add it > in a pom along with the compile time dependencies and upload it to Maven > Central or if you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (KAFKA-1081) kafka-run-class.sh is broken
Francois Saint-Jacques created KAFKA-1081: - Summary: kafka-run-class.sh is broken Key: KAFKA-1081 URL: https://issues.apache.org/jira/browse/KAFKA-1081 Project: Kafka Issue Type: Bug Affects Versions: 0.8 Reporter: Francois Saint-Jacques Please apply this patch, this is why log4j exists. Rerunning at non-deterministic command twice to catch error message is extremely dangerous. diff --git a/bin/kafka-run-class.sh b/bin/kafka-run-class.sh index eb6ff1b..2f2d8b5 100755 --- a/bin/kafka-run-class.sh +++ b/bin/kafka-run-class.sh @@ -102,19 +102,3 @@ if [ "$1" = "daemon" ] && [ -z "$KAFKA_GC_LOG_OPTS"] ; then fi $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" - -exitval=$? - -if [ $exitval -eq "1" ] ; then - $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" >& exception.txt - exception=`cat exception.txt` - noBuildMessage='Please build the project using sbt. Documentation is available at http://kafka.apache.org/' - pattern="(Could not find or load main class)|(java\.lang\.NoClassDefFoundError)" - match=`echo $exception | grep -E "$pattern"` - if [[ -n "$match" ]]; then - echo $noBuildMessage - fi - rm exception.txt -fi - - -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1081) kafka-run-class.sh is broken
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791816#comment-13791816 ] Joe Stein commented on KAFKA-1081: -- please can you put together reproducible steps for the issue, thanks! > kafka-run-class.sh is broken > > > Key: KAFKA-1081 > URL: https://issues.apache.org/jira/browse/KAFKA-1081 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Francois Saint-Jacques > > Please apply this patch, this is why log4j exists. Rerunning at > non-deterministic command twice to catch error message is extremely dangerous. > diff --git a/bin/kafka-run-class.sh b/bin/kafka-run-class.sh > index eb6ff1b..2f2d8b5 100755 > --- a/bin/kafka-run-class.sh > +++ b/bin/kafka-run-class.sh > @@ -102,19 +102,3 @@ if [ "$1" = "daemon" ] && [ -z "$KAFKA_GC_LOG_OPTS"] ; > then > fi > $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" > - > -exitval=$? > - > -if [ $exitval -eq "1" ] ; then > - $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" >& > exception.txt > - exception=`cat exception.txt` > - noBuildMessage='Please build the project using sbt. Documentation is > available at http://kafka.apache.org/' > - pattern="(Could not find or load main > class)|(java\.lang\.NoClassDefFoundError)" > - match=`echo $exception | grep -E "$pattern"` > - if [[ -n "$match" ]]; then > - echo $noBuildMessage > - fi > - rm exception.txt > -fi > - > - -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1081) kafka-run-class.sh is broken
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791819#comment-13791819 ] Francois Saint-Jacques commented on KAFKA-1081: --- Look for the 3 last lines, not the usage error. $ cd / $ /opt/kafka/bin/kafka-list-topic.sh Missing required argument "[zookeeper]" Option Description -- --- --topic REQUIRED: The topic to be listed. Defaults to all existing topics. (default: ) --unavailable-partitionsif set, only show partitions whose leader is not available --under-replicated-partitions if set, only show under replicated partitions --zookeeper REQUIRED: The connection string for the zookeeper connection in the form host:port. Multiple URLS can be given to allow fail-over. /opt/kafka/bin/kafka-run-class.sh: line 72: exception.txt: Permission denied cat: exception.txt: No such file or directory rm: cannot remove 'exception.txt': No such file or directory > kafka-run-class.sh is broken > > > Key: KAFKA-1081 > URL: https://issues.apache.org/jira/browse/KAFKA-1081 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Francois Saint-Jacques > > Please apply this patch, this is why log4j exists. Rerunning at > non-deterministic command twice to catch error message is extremely dangerous. > diff --git a/bin/kafka-run-class.sh b/bin/kafka-run-class.sh > index eb6ff1b..2f2d8b5 100755 > --- a/bin/kafka-run-class.sh > +++ b/bin/kafka-run-class.sh > @@ -102,19 +102,3 @@ if [ "$1" = "daemon" ] && [ -z "$KAFKA_GC_LOG_OPTS"] ; > then > fi > $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" > - > -exitval=$? > - > -if [ $exitval -eq "1" ] ; then > - $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" >& > exception.txt > - exception=`cat exception.txt` > - noBuildMessage='Please build the project using sbt. Documentation is > available at http://kafka.apache.org/' > - pattern="(Could not find or load main > class)|(java\.lang\.NoClassDefFoundError)" > - match=`echo $exception | grep -E "$pattern"` > - if [[ -n "$match" ]]; then > - echo $noBuildMessage > - fi > - rm exception.txt > -fi > - > - -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1081) kafka-run-class.sh is broken
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791838#comment-13791838 ] Joe Stein commented on KAFKA-1081: -- I don't see removing that function as a solution, we could re work it so that it communicates the error that you need to build the project. i don't have permission issue writing exception.txt but could see another way of fixing the function to not use a file and keep it all in vars if you wanted to rework your patch or chmod a+rw in your folder maybe not sure how you are running things but you should build first (see README) Joes-MacBook-Air:kafka joestein$ bin/kafka-list-topic.sh Missing required argument "[zookeeper]" Option Description -- --- --topic REQUIRED: The topic to be listed. Defaults to all existing topics. (default: ) --unavailable-partitionsif set, only show partitions whose leader is not available --under-replicated-partitions if set, only show under replicated partitions --zookeeper REQUIRED: The connection string for the zookeeper connection in the form host:port. Multiple URLS can be given to allow fail-over. Joes-MacBook-Air:kafka joestein$ > kafka-run-class.sh is broken > > > Key: KAFKA-1081 > URL: https://issues.apache.org/jira/browse/KAFKA-1081 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Francois Saint-Jacques > > Please apply this patch, this is why log4j exists. Rerunning at > non-deterministic command twice to catch error message is extremely dangerous. > diff --git a/bin/kafka-run-class.sh b/bin/kafka-run-class.sh > index eb6ff1b..2f2d8b5 100755 > --- a/bin/kafka-run-class.sh > +++ b/bin/kafka-run-class.sh > @@ -102,19 +102,3 @@ if [ "$1" = "daemon" ] && [ -z "$KAFKA_GC_LOG_OPTS"] ; > then > fi > $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" > - > -exitval=$? > - > -if [ $exitval -eq "1" ] ; then > - $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" >& > exception.txt > - exception=`cat exception.txt` > - noBuildMessage='Please build the project using sbt. Documentation is > available at http://kafka.apache.org/' > - pattern="(Could not find or load main > class)|(java\.lang\.NoClassDefFoundError)" > - match=`echo $exception | grep -E "$pattern"` > - if [[ -n "$match" ]]; then > - echo $noBuildMessage > - fi > - rm exception.txt > -fi > - > - -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] Subscription: outstanding kafka patches
Issue Subscription Filter: outstanding kafka patches (64 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-1049 Encoder implementations are required to provide an undocumented constructor. https://issues.apache.org/jira/browse/KAFKA-1049 KAFKA-1042 Fix segment flush logic https://issues.apache.org/jira/browse/KAFKA-1042 KAFKA-1032 Messages sent to the old leader will be lost on broker GC resulted failure https://issues.apache.org/jira/browse/KAFKA-1032 KAFKA-1020 Remove getAllReplicasOnBroker from KafkaController https://issues.apache.org/jira/browse/KAFKA-1020 KAFKA-1012 Implement an Offset Manager and hook offset requests to it https://issues.apache.org/jira/browse/KAFKA-1012 KAFKA-1011 Decompression and re-compression on MirrorMaker could result in messages being dropped in the pipeline https://issues.apache.org/jira/browse/KAFKA-1011 KAFKA-1008 Unmap before resizing https://issues.apache.org/jira/browse/KAFKA-1008 KAFKA-1005 kafka.perf.ConsumerPerformance not shutting down consumer https://issues.apache.org/jira/browse/KAFKA-1005 KAFKA-1004 Handle topic event for trivial whitelist topic filters https://issues.apache.org/jira/browse/KAFKA-1004 KAFKA-998 Producer should not retry on non-recoverable error codes https://issues.apache.org/jira/browse/KAFKA-998 KAFKA-997 Provide a strict verification mode when reading configuration properties https://issues.apache.org/jira/browse/KAFKA-997 KAFKA-996 Capitalize first letter for log entries https://issues.apache.org/jira/browse/KAFKA-996 KAFKA-984 Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same https://issues.apache.org/jira/browse/KAFKA-984 KAFKA-976 Order-Preserving Mirror Maker Testcase https://issues.apache.org/jira/browse/KAFKA-976 KAFKA-967 Use key range in ProducerPerformance https://issues.apache.org/jira/browse/KAFKA-967 KAFKA-917 Expose zk.session.timeout.ms in console consumer https://issues.apache.org/jira/browse/KAFKA-917 KAFKA-885 sbt package builds two kafka jars https://issues.apache.org/jira/browse/KAFKA-885 KAFKA-881 Kafka broker not respecting log.roll.hours https://issues.apache.org/jira/browse/KAFKA-881 KAFKA-873 Consider replacing zkclient with curator (with zkclient-bridge) https://issues.apache.org/jira/browse/KAFKA-873 KAFKA-868 System Test - add test case for rolling controlled shutdown https://issues.apache.org/jira/browse/KAFKA-868 KAFKA-863 System Test - update 0.7 version of kafka-run-class.sh for Migration Tool test cases https://issues.apache.org/jira/browse/KAFKA-863 KAFKA-859 support basic auth protection of mx4j console https://issues.apache.org/jira/browse/KAFKA-859 KAFKA-855 Ant+Ivy build for Kafka https://issues.apache.org/jira/browse/KAFKA-855 KAFKA-854 Upgrade dependencies for 0.8 https://issues.apache.org/jira/browse/KAFKA-854 KAFKA-815 Improve SimpleConsumerShell to take in a max messages config option https://issues.apache.org/jira/browse/KAFKA-815 KAFKA-745 Remove getShutdownReceive() and other kafka specific code from the RequestChannel https://issues.apache.org/jira/browse/KAFKA-745 KAFKA-735 Add looping and JSON output for ConsumerOffsetChecker https://issues.apache.org/jira/browse/KAFKA-735 KAFKA-717 scala 2.10 build support https://issues.apache.org/jira/browse/KAFKA-717 KAFKA-686 0.8 Kafka broker should give a better error message when running against 0.7 zookeeper https://issues.apache.org/jira/browse/KAFKA-686 KAFKA-674 Clean Shutdown Testing - Log segments checksums mismatch https://issues.apache.org/jira/browse/KAFKA-674 KAFKA-652 Create testcases for clean shut-down https://issues.apache.org/jira/browse/KAFKA-652 KAFKA-649 Cleanup log4j logging https://issues.apache.org/jira/browse/KAFKA-649 KAFKA-645 Create a shell script to run System Test with DEBUG details and "tee" console output to a file https://issues.apache.org/jira/browse/KAFKA-645 KAFKA-598 decouple fetch size from max message size https://issues.apache.org/jira/browse/KAFKA-598 KAFKA-583 SimpleConsumerShell may receive less data inconsistently https://issues.apache.org/jira/browse/KAFKA-583 KAFKA-559 Garbage collect old consumer metadata entries https://issues.apache.org/jira/browse/KAFKA-559 KAFKA-552 No error messages logged for those failing-to-send messages from Producer https://issues.apache.org/jira/browse/KAFKA-552 KAFKA-530 kafka.server.KafkaApis: kafka.common.OffsetOutOfRangeException https://issues.apache.
[jira] [Commented] (KAFKA-1081) kafka-run-class.sh is broken
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791870#comment-13791870 ] Francois Saint-Jacques commented on KAFKA-1081: --- Look, this is an ugly hack. The real problem here is not the directory permission or using a temp file but RERUNNING the first java command. I'm not even trying to build it, I'm trying to correctly package kafka on a production server. Whenever I run any command (bin/*) that doesn't return properly, it borks if kafka-run-class.sh is called within a directory where the user don't have write permission. I understand that these lines are there to help new users who checkout the project and forget to build before running any command, but we're talking of deploying quality code in production. > kafka-run-class.sh is broken > > > Key: KAFKA-1081 > URL: https://issues.apache.org/jira/browse/KAFKA-1081 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Francois Saint-Jacques > > Please apply this patch, this is why log4j exists. Rerunning at > non-deterministic command twice to catch error message is extremely dangerous. > diff --git a/bin/kafka-run-class.sh b/bin/kafka-run-class.sh > index eb6ff1b..2f2d8b5 100755 > --- a/bin/kafka-run-class.sh > +++ b/bin/kafka-run-class.sh > @@ -102,19 +102,3 @@ if [ "$1" = "daemon" ] && [ -z "$KAFKA_GC_LOG_OPTS"] ; > then > fi > $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" > - > -exitval=$? > - > -if [ $exitval -eq "1" ] ; then > - $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" >& > exception.txt > - exception=`cat exception.txt` > - noBuildMessage='Please build the project using sbt. Documentation is > available at http://kafka.apache.org/' > - pattern="(Could not find or load main > class)|(java\.lang\.NoClassDefFoundError)" > - match=`echo $exception | grep -E "$pattern"` > - if [[ -n "$match" ]]; then > - echo $noBuildMessage > - fi > - rm exception.txt > -fi > - > - -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1081) kafka-run-class.sh is broken
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791890#comment-13791890 ] Joe Stein commented on KAFKA-1081: -- We have binary release http://kafka.apache.org/downloads.html since 0.8.0-beta1 and moving forward, you could use that since it is already packaged. I would go so far as to even bootstrap the sbt launcher but that is a lot more changes than just a consistent error when not compiling source before run. > kafka-run-class.sh is broken > > > Key: KAFKA-1081 > URL: https://issues.apache.org/jira/browse/KAFKA-1081 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Francois Saint-Jacques > > Please apply this patch, this is why log4j exists. Rerunning at > non-deterministic command twice to catch error message is extremely dangerous. > diff --git a/bin/kafka-run-class.sh b/bin/kafka-run-class.sh > index eb6ff1b..2f2d8b5 100755 > --- a/bin/kafka-run-class.sh > +++ b/bin/kafka-run-class.sh > @@ -102,19 +102,3 @@ if [ "$1" = "daemon" ] && [ -z "$KAFKA_GC_LOG_OPTS"] ; > then > fi > $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" > - > -exitval=$? > - > -if [ $exitval -eq "1" ] ; then > - $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" >& > exception.txt > - exception=`cat exception.txt` > - noBuildMessage='Please build the project using sbt. Documentation is > available at http://kafka.apache.org/' > - pattern="(Could not find or load main > class)|(java\.lang\.NoClassDefFoundError)" > - match=`echo $exception | grep -E "$pattern"` > - if [[ -n "$match" ]]; then > - echo $noBuildMessage > - fi > - rm exception.txt > -fi > - > - -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1081) kafka-run-class.sh is broken
[ https://issues.apache.org/jira/browse/KAFKA-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791901#comment-13791901 ] Francois Saint-Jacques commented on KAFKA-1081: --- I know you guys have a binary release, this is what I'm deploying. My point is, this specific snippet should never go in production release. > kafka-run-class.sh is broken > > > Key: KAFKA-1081 > URL: https://issues.apache.org/jira/browse/KAFKA-1081 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Francois Saint-Jacques > > Please apply this patch, this is why log4j exists. Rerunning at > non-deterministic command twice to catch error message is extremely dangerous. > diff --git a/bin/kafka-run-class.sh b/bin/kafka-run-class.sh > index eb6ff1b..2f2d8b5 100755 > --- a/bin/kafka-run-class.sh > +++ b/bin/kafka-run-class.sh > @@ -102,19 +102,3 @@ if [ "$1" = "daemon" ] && [ -z "$KAFKA_GC_LOG_OPTS"] ; > then > fi > $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" > - > -exitval=$? > - > -if [ $exitval -eq "1" ] ; then > - $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS > $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@" >& > exception.txt > - exception=`cat exception.txt` > - noBuildMessage='Please build the project using sbt. Documentation is > available at http://kafka.apache.org/' > - pattern="(Could not find or load main > class)|(java\.lang\.NoClassDefFoundError)" > - match=`echo $exception | grep -E "$pattern"` > - if [[ -n "$match" ]]; then > - echo $noBuildMessage > - fi > - rm exception.txt > -fi > - > - -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-417) NullPointerException in ZookeeperConsumerConnector
[
https://issues.apache.org/jira/browse/KAFKA-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791936#comment-13791936
]
Andrew Olson commented on KAFKA-417:
We see this NPE logged numerous times when running our Kafka integration tests.
We're using 0.8.0-beta1.
{code}
2013-10-10 14:13:08,601
[testKafkaNotification_ipfacecobld14.northamerica.net-1381432385832-593508da_watcher_executor]
ERROR kafka.consumer.ZookeeperConsumerConnector -
[testKafkaNotification_ipfacecobld14.northamerica.net-1381432385832-593508da],
error during syncedRebalance
java.lang.NullPointerException
at kafka.utils.ZkUtils$.getChildrenParentMayNotExist(ZkUtils.scala:433)
at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:451)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:372)
at
scala.collection.immutable.Range$ByOne$class.foreach$mVc$sp(Range.scala:282)
at
scala.collection.immutable.Range$$anon$2.foreach$mVc$sp(Range.scala:265)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:369)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(ZookeeperConsumerConnector.scala:326)
{code}
> NullPointerException in ZookeeperConsumerConnector
> --
>
> Key: KAFKA-417
> URL: https://issues.apache.org/jira/browse/KAFKA-417
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.8
>Reporter: Neha Narkhede
>Assignee: Jun Rao
>
> [info] == kafka.integration.LazyInitProducerTest ==
> [info] Test Starting:
> testProduceAndMultiFetch(kafka.integration.LazyInitProducerTest)
> [2012-07-25 11:50:26,394] ERROR group1_consumer2 error during syncedRebalance
> (kafka.consumer.ZookeeperConsumerConnector:99)
> java.lang.NullPointerException
> at
> kafka.utils.ZkUtils$.getChildrenParentMayNotExist(ZkUtils.scala:371)
> at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:389)
> at
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala
> :444)
> at
> scala.collection.immutable.Range$ByOne$class.foreach$mVc$sp(Range.scala:282)
> at
> scala.collection.immutable.Range$$anon$2.foreach$mVc$sp(Range.scala:265)
> at
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:441)
> at
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(ZookeeperConsumerConnector.scala:398)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (KAFKA-1082) zkclient dies after UnknownHostException in zk reconnect
Anatoly Fayngelerin created KAFKA-1082: -- Summary: zkclient dies after UnknownHostException in zk reconnect Key: KAFKA-1082 URL: https://issues.apache.org/jira/browse/KAFKA-1082 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8, 0.7.2 Reporter: Anatoly Fayngelerin Moving this here from the dev list: I've run into the following issue with the Kafka server. The zkclient lib seems to die silently if there is an UnknownHostException(or any IOException) while reconnecting the ZK session. I've filed a bug about this with the zkclient lib(https://github.com/sgroschupf/zkclient/issues/23). The ramifications for Kafka were the silent loss of all ephemeral nodes associated with the affected process. It is fairly easy to reproduce this locally using the following steps: -- Configure a local kafka broker to connect to a local ZK instance using a DNS alias(e.g. add "127.0.0.1 kafka-test-dns" to your /etc/hosts) -- Start the broker, observe that ephemeral nodes have been added to ZK -- Suspend the broker process, preventing it from sending heartbeats to the ZK instance. Observe the loss of ephemeral nodes in ZK. -- Remove the DNS alias(e.g. comment out the /etc/hosts line). -- Upon resuming the broker, the UknownHostException is logged. After this point, the server cannot re-establish its ZK connection. Re-enabling the alias, for example, does not resume normal operation. The broker continues accepting requests, without participating in the ZK protocols. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1082) zkclient dies after UnknownHostException in zk reconnect
[ https://issues.apache.org/jira/browse/KAFKA-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792037#comment-13792037 ] Anatoly Fayngelerin commented on KAFKA-1082: Created reviewboard > zkclient dies after UnknownHostException in zk reconnect > > > Key: KAFKA-1082 > URL: https://issues.apache.org/jira/browse/KAFKA-1082 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8, 0.7.2 >Reporter: Anatoly Fayngelerin > Attachments: KAFKA-1082.patch > > > Moving this here from the dev list: > I've run into the following issue with the Kafka server. The zkclient lib > seems to die silently if there is an UnknownHostException(or any IOException) > while reconnecting the ZK session. I've filed a bug about this with the > zkclient lib(https://github.com/sgroschupf/zkclient/issues/23). The > ramifications for Kafka were the silent loss of all ephemeral nodes > associated with the affected process. > It is fairly easy to reproduce this locally using the following steps: > -- Configure a local kafka broker to connect to a local ZK instance using a > DNS alias(e.g. add "127.0.0.1 kafka-test-dns" to your /etc/hosts) > -- Start the broker, observe that ephemeral nodes have been added to ZK > -- Suspend the broker process, preventing it from sending heartbeats to the > ZK instance. Observe the loss of ephemeral nodes in ZK. > -- Remove the DNS alias(e.g. comment out the /etc/hosts line). > -- Upon resuming the broker, the UknownHostException is logged. After this > point, the server cannot re-establish its ZK connection. Re-enabling the > alias, for example, does not resume normal operation. The broker continues > accepting requests, without participating in the ZK protocols. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (KAFKA-1082) zkclient dies after UnknownHostException in zk reconnect
[ https://issues.apache.org/jira/browse/KAFKA-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anatoly Fayngelerin updated KAFKA-1082: --- Attachment: KAFKA-1082.patch > zkclient dies after UnknownHostException in zk reconnect > > > Key: KAFKA-1082 > URL: https://issues.apache.org/jira/browse/KAFKA-1082 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8, 0.7.2 >Reporter: Anatoly Fayngelerin > Attachments: KAFKA-1082.patch > > > Moving this here from the dev list: > I've run into the following issue with the Kafka server. The zkclient lib > seems to die silently if there is an UnknownHostException(or any IOException) > while reconnecting the ZK session. I've filed a bug about this with the > zkclient lib(https://github.com/sgroschupf/zkclient/issues/23). The > ramifications for Kafka were the silent loss of all ephemeral nodes > associated with the affected process. > It is fairly easy to reproduce this locally using the following steps: > -- Configure a local kafka broker to connect to a local ZK instance using a > DNS alias(e.g. add "127.0.0.1 kafka-test-dns" to your /etc/hosts) > -- Start the broker, observe that ephemeral nodes have been added to ZK > -- Suspend the broker process, preventing it from sending heartbeats to the > ZK instance. Observe the loss of ephemeral nodes in ZK. > -- Remove the DNS alias(e.g. comment out the /etc/hosts line). > -- Upon resuming the broker, the UknownHostException is logged. After this > point, the server cannot re-establish its ZK connection. Re-enabling the > alias, for example, does not resume normal operation. The broker continues > accepting requests, without participating in the ZK protocols. -- This message was sent by Atlassian JIRA (v6.1#6144)
Review Request 14582: Upgrading zkclient to verion 0.4. Adds retry logic in case of a failed ZooKeeper session re-establishment.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14582/ --- Review request for kafka. Repository: kafka Description --- Upgrading zkclient to verion 0.4. Adds retry logic in case of a failed ZooKeeper session re-establishment. Diffs - core/build.sbt b5bcb44f783ae6fe2cfdc091e53e867a56a1e592 core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala 77449f53dc7a8aef14b29790ee6ac62cbafadda7 core/src/main/scala/kafka/consumer/ZookeeperTopicEventWatcher.scala a67c193df9f7cbfc52f75dc1b71dc017de1b5fe2 core/src/main/scala/kafka/controller/KafkaController.scala 88d130f55997b72a8590e4cfe92857a7320e70d5 core/src/main/scala/kafka/server/KafkaHealthcheck.scala 84ea17a068c84592d3d0b007982f7be8fbc231b7 core/src/main/scala/kafka/server/KafkaServer.scala c148fdff61b13c2f06d8d048365daee30a286842 core/src/main/scala/kafka/utils/ZkReconnector.scala PRE-CREATION core/src/main/scala/kafka/utils/ZkUtils.scala d1c4b3d7b3d11c88a1d1474aadbb727704cfb759 core/src/test/scala/unit/kafka/zk/ZkReconnecterTest.scala PRE-CREATION Diff: https://reviews.apache.org/r/14582/diff/ Testing --- Thanks, Anatoly Fayngelerin
[jira] [Commented] (KAFKA-1082) zkclient dies after UnknownHostException in zk reconnect
[ https://issues.apache.org/jira/browse/KAFKA-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792041#comment-13792041 ] Anatoly Fayngelerin commented on KAFKA-1082: Sorry, first time running the reviewboard script: https://reviews.apache.org/r/14582 > zkclient dies after UnknownHostException in zk reconnect > > > Key: KAFKA-1082 > URL: https://issues.apache.org/jira/browse/KAFKA-1082 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8, 0.7.2 >Reporter: Anatoly Fayngelerin > Attachments: KAFKA-1082.patch > > > Moving this here from the dev list: > I've run into the following issue with the Kafka server. The zkclient lib > seems to die silently if there is an UnknownHostException(or any IOException) > while reconnecting the ZK session. I've filed a bug about this with the > zkclient lib(https://github.com/sgroschupf/zkclient/issues/23). The > ramifications for Kafka were the silent loss of all ephemeral nodes > associated with the affected process. > It is fairly easy to reproduce this locally using the following steps: > -- Configure a local kafka broker to connect to a local ZK instance using a > DNS alias(e.g. add "127.0.0.1 kafka-test-dns" to your /etc/hosts) > -- Start the broker, observe that ephemeral nodes have been added to ZK > -- Suspend the broker process, preventing it from sending heartbeats to the > ZK instance. Observe the loss of ephemeral nodes in ZK. > -- Remove the DNS alias(e.g. comment out the /etc/hosts line). > -- Upon resuming the broker, the UknownHostException is logged. After this > point, the server cannot re-establish its ZK connection. Re-enabling the > alias, for example, does not resume normal operation. The broker continues > accepting requests, without participating in the ZK protocols. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (KAFKA-1001) Handle follower transition in batch
[ https://issues.apache.org/jira/browse/KAFKA-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-1001: Assignee: Guozhang Wang > Handle follower transition in batch > --- > > Key: KAFKA-1001 > URL: https://issues.apache.org/jira/browse/KAFKA-1001 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps >Assignee: Guozhang Wang > Fix For: 0.8.1 > > > In KAFKA-615 we made changes to avoid fsync'ing the active segment of the log > due to log roll and maintaining recovery semantics. > One downside of the fix for that issue was that it required checkpointing the > recovery point for the log many times, one for each partition that > transitioned to follower state. > In this ticket I aim to fix that issue by making the following changes: > 1. Add a new API LogManager.truncateTo(m: Map[TopicAndPartition, Long]). This > method will first checkpoint the recovery point, then truncate each of the > given logs to the given offset. This method will have to ensure these two > things happen atomically. > 2. Change ReplicaManager to first stop fetching for all partitions changing > to follower state, then call LogManager.truncateTo then complete the existing > logic. > We think this will, over all, be a good thing. The reason is that the > fetching thread current does something like (a) acquire lock, (b) fetch > partitions, (c) write data to logs, (d) release locks. Since we currently > remove fetchers one at a time this requires acquiring the fetcher lock, and > hence generally blocking for half of the read/write cycle for each partition. > By doing this in bulk we will avoid reacquiring the lock over and over for > each change. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-417) NullPointerException in ZookeeperConsumerConnector
[ https://issues.apache.org/jira/browse/KAFKA-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792071#comment-13792071 ] Neha Narkhede commented on KAFKA-417: - This is a race condition between the shutdown and the rebalance operation. Though, from a cursory look, it seems that we acquire the rebalanceLock in shutdown as well, so this might not be an issue in 0.8 HEAD. [~noslowerdna] Would you mind retrying this using the latest 0.8 HEAD? > NullPointerException in ZookeeperConsumerConnector > -- > > Key: KAFKA-417 > URL: https://issues.apache.org/jira/browse/KAFKA-417 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Jun Rao > > [info] == kafka.integration.LazyInitProducerTest == > [info] Test Starting: > testProduceAndMultiFetch(kafka.integration.LazyInitProducerTest) > [2012-07-25 11:50:26,394] ERROR group1_consumer2 error during syncedRebalance > (kafka.consumer.ZookeeperConsumerConnector:99) > java.lang.NullPointerException > at > kafka.utils.ZkUtils$.getChildrenParentMayNotExist(ZkUtils.scala:371) > at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:389) > at > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala > :444) > at > scala.collection.immutable.Range$ByOne$class.foreach$mVc$sp(Range.scala:282) > at > scala.collection.immutable.Range$$anon$2.foreach$mVc$sp(Range.scala:265) > at > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:441) > at > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(ZookeeperConsumerConnector.scala:398) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792332#comment-13792332
]
Jun Rao commented on KAFKA-1018:
Could you double commit this to trunk? Thanks,
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (KAFKA-1018) tidy up the POM from what feedback has come from the 0.8 beta and publishing to maven
[
https://issues.apache.org/jira/browse/KAFKA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792336#comment-13792336
]
Joe Stein commented on KAFKA-1018:
--
yup, done
> tidy up the POM from what feedback has come from the 0.8 beta and publishing
> to maven
> -
>
> Key: KAFKA-1018
> URL: https://issues.apache.org/jira/browse/KAFKA-1018
> Project: Kafka
> Issue Type: Bug
>Reporter: Joe Stein
> Fix For: 0.8
>
> Attachments: KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch,
> KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch, KAFKA-1018.patch
>
>
> from Chris Riccomini
> 1. Maven central can't resolve it properly (POM is different from Apache
> release). Have to use Apache release repo directly to get things to work.
> 2. Exclusions must be manually applied even though they exist in Kafka's POM
> already. I think Maven can handle this automatically, if the POM is done
> right.
> 3. Weird parent block in Kafka POMs that points to org.apache.
> 4. Would be nice to publish kafka-test jars as well.
> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
> Our hypothesis regarding the first issue is that it was caused by duplicate
> publishing during testing, and it should go away in the future.
> Regarding number 2, I have to explicitly exclude the following when depending
> on Kafka:
> exclude module: 'jms'
> exclude module: 'jmxtools'
> exclude module: 'jmxri'
> I believe these just need to be excluded from the appropriate jars in the
> actual SBT build file, to fix this issue. I see JMS is excluded from ZK, but
> it's probably being pulled in from somewhere else, anyway.
> Regarding number 3, it is indeed listed as something to do on the Apache
> publication page (http://www.apache.org/dev/publishing-maven-artifacts.html).
> I can't find an example of anyone using it, but it doesn't seem to be doing
> any harm.
> Also, regarding your intransitive() call, that is disabling ALL dependencies
> not just the exclusions, I believe. I think that the "proper" way to do that
> would be to do what I've done: exclude("jms", "jmxtools", "jmxri").
> Regardless, fixing number 2, above, should mean that intransitive()/exclude()
> are not required.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
