Problem with published pom?
I tried to pull Kafka in through Ivy this morning and got a strange error:
[ivy:resolve] problem occurred while resolving dependency:
org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with main:
java.lang.IllegalArgumentException: null name not allowed
I have filed https://issues.apache.org/jira/browse/KAFKA-981 to track this.
I'm wondering if underscores are allowed in the artifactId? Wouldn't a
Maven classifier be better for dealing with multiple Scala versions?
Something like:
org.apache.kafka
kafka
0.8-beta
scala-2.9.2
[jira] [Updated] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-982: Attachment: 289.jpeg > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-982) Logo for Kafka
Jay Kreps created KAFKA-982: --- Summary: Logo for Kafka Key: KAFKA-982 URL: https://issues.apache.org/jira/browse/KAFKA-982 Project: Kafka Issue Type: Improvement Reporter: Jay Kreps We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-982: Attachment: 296.png > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 294.jpeg, 296.png > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-982: Attachment: 298.jpeg > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Logo
Hey guys, We need a logo! I got a few designs from a 99 designs contest that I would like to put forward: https://issues.apache.org/jira/browse/KAFKA-982 If anyone else would like to submit a design that would be great. Let's do a vote to choose one. -Jay
[jira] [Assigned] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps reassigned KAFKA-981:
---
Assignee: Joe Stein
Joe any idea what this might be?
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
>Assignee: Joe Stein
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-982: Attachment: 294.jpeg > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 294.jpeg, 296.png > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-981) Unable to pull Kafka binaries with Ivy
David Arthur created KAFKA-981:
--
Summary: Unable to pull Kafka binaries with Ivy
Key: KAFKA-981
URL: https://issues.apache.org/jira/browse/KAFKA-981
Project: Kafka
Issue Type: Bug
Components: packaging
Affects Versions: 0.8
Reporter: David Arthur
I am trying to pull the published Kafka binary with a simple Ivy file.
I get the following exception: [ivy:resolve] problem occurred while resolving
dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-981:
---
Attachment: ivy.xml
Ivy descriptor file I used
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715241#comment-13715241
]
David Arthur commented on KAFKA-981:
For a full example, try cloning this repository and running "ant ivy"
https://github.com/mumrah/trihug-kafka-demo
I encountered this error when setting up this demo
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715237#comment-13715237
]
David Arthur commented on KAFKA-981:
Also should note this is Ivy 2.3.0
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-981:
---
Attachment: ant.log
stdout log from Ant showing full exception
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715390#comment-13715390 ] Sriram Subramanian commented on KAFKA-347: -- v2.4 the reason to expose it is for manual replica assignment. It is more explicit to specify the rep factor and the assignments for those. Rebased without the zkconsumer connector change. > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715392#comment-13715392 ] Jun Rao commented on KAFKA-347: --- Thanks for patch v2. Looks good overall. Some comments. 20. AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(): The info logging should be different, depending on whether the topic is created or updated. Also, fix the indentation in the else clause. 21. AddPartitionsCommand: 21.1 remove unused imports 21.2 Is there any value in having the "replica" option? It seems that it should always be the existing replication factor. 21.3 For the "replica-assignment-list" option, could we make it clear in the description that this is for newly added partitions. 21.4 getManualReplicaAssignment(): We need to make sure the replica factor is the same as the existing one. 22. KafkaController.onNewTopicCreation(): Could you explain why the onNewPartition statement is moved to before the watcher registration? Normally, in order not to miss any watchers, one needs to register the watcher before reading the associated nodes in ZK. 23. PartitionStateMachine.AddPartitionsListener.handleDataChange(): 23.1 In the following statement, we are actually returning all replica assignments. val addedPartitionReplicaAssignment = ZkUtils.getReplicaAssignmentForTopics(zkClient, List(topic)) 23.2 Instead of using controllerContext.partitionLeadershipInfo to filter out existing partitions, it's probably better to use controllerContext.partitionReplicaAssignment, since leaders may not always exist. 23.3 In the error logging, could we add the affected data path? 24. AddPartitionsTest: remove unused imports 25. Did we do any test to make sure that existing consumers can pick up the new partitions? 26. The patch needs to be rebased. > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-347: - Attachment: KAFKA-347-v2-rebased.patch > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715469#comment-13715469 ] Sriram Subramanian commented on KAFKA-347: -- 20. Indentation seems fine to me. 21.2 It is present to make manual assignment more clear. 25 Yes the test was done. I will do another sanity check after the patch is commited. > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-347: - Attachment: KAFKA-347-v3.patch > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch, KAFKA-347-v3.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715473#comment-13715473 ] David Arthur commented on KAFKA-982: +1 for 289, 296 > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715502#comment-13715502 ] Xavier Stevens commented on KAFKA-982: -- +1 for 296, 298 > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Subscription: outstanding kafka patches
Issue Subscription Filter: outstanding kafka patches (68 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-982 Logo for Kafka https://issues.apache.org/jira/browse/KAFKA-982 KAFKA-981 Unable to pull Kafka binaries with Ivy https://issues.apache.org/jira/browse/KAFKA-981 KAFKA-976 Order-Preserving Mirror Maker Testcase https://issues.apache.org/jira/browse/KAFKA-976 KAFKA-974 can't use public release maven repo because of failure of downloaded dependency https://issues.apache.org/jira/browse/KAFKA-974 KAFKA-967 Use key range in ProducerPerformance https://issues.apache.org/jira/browse/KAFKA-967 KAFKA-957 MirrorMaker needs to preserve the key in the source cluster https://issues.apache.org/jira/browse/KAFKA-957 KAFKA-956 High-level consumer fails to check topic metadata response for errors https://issues.apache.org/jira/browse/KAFKA-956 KAFKA-946 Kafka Hadoop Consumer fails when verifying message checksum https://issues.apache.org/jira/browse/KAFKA-946 KAFKA-943 Move all configuration key string to constants https://issues.apache.org/jira/browse/KAFKA-943 KAFKA-932 System Test - set retry.backoff.ms=300 to all test cases https://issues.apache.org/jira/browse/KAFKA-932 KAFKA-925 Add optional partition key override in producer https://issues.apache.org/jira/browse/KAFKA-925 KAFKA-923 Improve controller failover latency https://issues.apache.org/jira/browse/KAFKA-923 KAFKA-922 System Test - set retry.backoff.ms=300 to testcase_0119 https://issues.apache.org/jira/browse/KAFKA-922 KAFKA-917 Expose zk.session.timeout.ms in console consumer https://issues.apache.org/jira/browse/KAFKA-917 KAFKA-915 System Test - Mirror Maker testcase_5001 failed https://issues.apache.org/jira/browse/KAFKA-915 KAFKA-885 sbt package builds two kafka jars https://issues.apache.org/jira/browse/KAFKA-885 KAFKA-883 System Test - update migration tool testsuite after 0.7 ProducerPerformance sends seq MessageID https://issues.apache.org/jira/browse/KAFKA-883 KAFKA-882 Enhance 0.7 ProducerPerformance to send sequential MessageID as in 0.8 https://issues.apache.org/jira/browse/KAFKA-882 KAFKA-881 Kafka broker not respecting log.roll.hours https://issues.apache.org/jira/browse/KAFKA-881 KAFKA-879 In system test, read the new leader from zookeeper instead of broker log on completion of become-leader state transition https://issues.apache.org/jira/browse/KAFKA-879 KAFKA-873 Consider replacing zkclient with curator (with zkclient-bridge) https://issues.apache.org/jira/browse/KAFKA-873 KAFKA-868 System Test - add test case for rolling controlled shutdown https://issues.apache.org/jira/browse/KAFKA-868 KAFKA-863 System Test - update 0.7 version of kafka-run-class.sh for Migration Tool test cases https://issues.apache.org/jira/browse/KAFKA-863 KAFKA-859 support basic auth protection of mx4j console https://issues.apache.org/jira/browse/KAFKA-859 KAFKA-855 Ant+Ivy build for Kafka https://issues.apache.org/jira/browse/KAFKA-855 KAFKA-854 Upgrade dependencies for 0.8 https://issues.apache.org/jira/browse/KAFKA-854 KAFKA-815 Improve SimpleConsumerShell to take in a max messages config option https://issues.apache.org/jira/browse/KAFKA-815 KAFKA-745 Remove getShutdownReceive() and other kafka specific code from the RequestChannel https://issues.apache.org/jira/browse/KAFKA-745 KAFKA-735 Add looping and JSON output for ConsumerOffsetChecker https://issues.apache.org/jira/browse/KAFKA-735 KAFKA-718 kafka-run-class.sh should use reasonable gc settings https://issues.apache.org/jira/browse/KAFKA-718 KAFKA-717 scala 2.10 build support https://issues.apache.org/jira/browse/KAFKA-717 KAFKA-686 0.8 Kafka broker should give a better error message when running against 0.7 zookeeper https://issues.apache.org/jira/browse/KAFKA-686 KAFKA-677 Retention process gives exception if an empty segment is chosen for collection https://issues.apache.org/jira/browse/KAFKA-677 KAFKA-674 Clean Shutdown Testing - Log segments checksums mismatch https://issues.apache.org/jira/browse/KAFKA-674 KAFKA-652 Create testcases for clean shut-down https://issues.apache.org/jira/browse/KAFKA-652 KAFKA-645 Create a shell script to run System Test with DEBUG details and "tee" console output to a file https://issues.apache.org/jira/browse/KAFKA-645 KAFKA-615 Avoid fsync on log segment roll https://issues.apache.org/jira/browse/KAFKA-615 KAFKA-598 decouple fetch size from max message size https://issue
[jira] [Updated] (KAFKA-983) Expose cleanshutdown method in MirrorMaker at the object level
[ https://issues.apache.org/jira/browse/KAFKA-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-983: Status: Patch Available (was: Open) > Expose cleanshutdown method in MirrorMaker at the object level > -- > > Key: KAFKA-983 > URL: https://issues.apache.org/jira/browse/KAFKA-983 > Project: Kafka > Issue Type: Bug >Reporter: Swapnil Ghike >Assignee: Swapnil Ghike > Labels: bugs > Attachments: KAFKA-983.patch > > > Making clean shutdown in MirrorMaker public at the object level will be > useful. Currently if MirrorMaker is run in a container process, the only way > to stop it seems to be triggering the shutdown hook (System.exit(0)) which > may have unwarranted side effects on the other threads running in that > container process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-983) Expose cleanshutdown method in MirrorMaker at the object level
[ https://issues.apache.org/jira/browse/KAFKA-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-983: Attachment: KAFKA-983.patch > Expose cleanshutdown method in MirrorMaker at the object level > -- > > Key: KAFKA-983 > URL: https://issues.apache.org/jira/browse/KAFKA-983 > Project: Kafka > Issue Type: Bug >Reporter: Swapnil Ghike >Assignee: Swapnil Ghike > Labels: bugs > Attachments: KAFKA-983.patch > > > Making clean shutdown in MirrorMaker public at the object level will be > useful. Currently if MirrorMaker is run in a container process, the only way > to stop it seems to be triggering the shutdown hook (System.exit(0)) which > may have unwarranted side effects on the other threads running in that > container process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-983) Expose cleanshutdown method in MirrorMaker at the object level
Swapnil Ghike created KAFKA-983: --- Summary: Expose cleanshutdown method in MirrorMaker at the object level Key: KAFKA-983 URL: https://issues.apache.org/jira/browse/KAFKA-983 Project: Kafka Issue Type: Bug Reporter: Swapnil Ghike Assignee: Swapnil Ghike Making clean shutdown in MirrorMaker public at the object level will be useful. Currently if MirrorMaker is run in a container process, the only way to stop it seems to be triggering the shutdown hook (System.exit(0)) which may have unwarranted side effects on the other threads running in that container process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-983) Expose cleanshutdown method in MirrorMaker at the object level
[ https://issues.apache.org/jira/browse/KAFKA-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715678#comment-13715678 ] Guozhang Wang commented on KAFKA-983: - +1. Looks good to me. > Expose cleanshutdown method in MirrorMaker at the object level > -- > > Key: KAFKA-983 > URL: https://issues.apache.org/jira/browse/KAFKA-983 > Project: Kafka > Issue Type: Bug >Reporter: Swapnil Ghike >Assignee: Swapnil Ghike > Labels: bugs > Attachments: KAFKA-983.patch > > > Making clean shutdown in MirrorMaker public at the object level will be > useful. Currently if MirrorMaker is run in a container process, the only way > to stop it seems to be triggering the shutdown hook (System.exit(0)) which > may have unwarranted side effects on the other threads running in that > container process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same
Guozhang Wang created KAFKA-984: --- Summary: Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same Key: KAFKA-984 URL: https://issues.apache.org/jira/browse/KAFKA-984 Project: Kafka Issue Type: Bug Reporter: Guozhang Wang Assignee: Guozhang Wang Fix For: 0.8 Currently a full rebalance will be triggered on high level consumers even when just a new topic is added to ZK. Better avoid this behavior but only rebalance on this newly added topic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715688#comment-13715688 ] Prashanth Menon commented on KAFKA-982: --- +1 for 296, 298 > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715691#comment-13715691 ] Roger Hoover commented on KAFKA-982: +1 for 298 > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-985) Increasing log retention quickly overflows scala Int
[ https://issues.apache.org/jira/browse/KAFKA-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Fejes updated KAFKA-985: -- Fix Version/s: 0.9 Status: Patch Available (was: Open) > Increasing log retention quickly overflows scala Int > > > Key: KAFKA-985 > URL: https://issues.apache.org/jira/browse/KAFKA-985 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.9 >Reporter: Frank Fejes >Assignee: Jay Kreps >Priority: Minor > Fix For: 0.9 > > > After increasing log.retention.hours from the default of 168 to 744 (31 days) > I noticed that logs were being deleted at every cleanup interval. > scala> val retentionMs: Long = 60 * 60 * 1000 * 31 * 24 > retentionMs: Long = -1616567296 > This only appears to affect trunk. I'll attach the minor patch I'm using > locally. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-985) Increasing log retention quickly overflows scala Int
[ https://issues.apache.org/jira/browse/KAFKA-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Fejes updated KAFKA-985: -- Status: Open (was: Patch Available) > Increasing log retention quickly overflows scala Int > > > Key: KAFKA-985 > URL: https://issues.apache.org/jira/browse/KAFKA-985 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.9 >Reporter: Frank Fejes >Assignee: Jay Kreps >Priority: Minor > Fix For: 0.9 > > > After increasing log.retention.hours from the default of 168 to 744 (31 days) > I noticed that logs were being deleted at every cleanup interval. > scala> val retentionMs: Long = 60 * 60 * 1000 * 31 * 24 > retentionMs: Long = -1616567296 > This only appears to affect trunk. I'll attach the minor patch I'm using > locally. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-985) Increasing log retention quickly overflows scala Int
Frank Fejes created KAFKA-985: - Summary: Increasing log retention quickly overflows scala Int Key: KAFKA-985 URL: https://issues.apache.org/jira/browse/KAFKA-985 Project: Kafka Issue Type: Bug Components: log Affects Versions: 0.9 Reporter: Frank Fejes Assignee: Jay Kreps Priority: Minor After increasing log.retention.hours from the default of 168 to 744 (31 days) I noticed that logs were being deleted at every cleanup interval. scala> val retentionMs: Long = 60 * 60 * 1000 * 31 * 24 retentionMs: Long = -1616567296 This only appears to affect trunk. I'll attach the minor patch I'm using locally. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-985) Increasing log retention quickly overflows scala Int
[ https://issues.apache.org/jira/browse/KAFKA-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Fejes updated KAFKA-985: -- Attachment: KAFKA-985.patch > Increasing log retention quickly overflows scala Int > > > Key: KAFKA-985 > URL: https://issues.apache.org/jira/browse/KAFKA-985 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.9 >Reporter: Frank Fejes >Assignee: Jay Kreps >Priority: Minor > Fix For: 0.9 > > Attachments: KAFKA-985.patch > > > After increasing log.retention.hours from the default of 168 to 744 (31 days) > I noticed that logs were being deleted at every cleanup interval. > scala> val retentionMs: Long = 60 * 60 * 1000 * 31 * 24 > retentionMs: Long = -1616567296 > This only appears to affect trunk. I'll attach the minor patch I'm using > locally. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-986) Topic Consumption Across multiple instances of consumer groups
Subbu Srinivasan created KAFKA-986: -- Summary: Topic Consumption Across multiple instances of consumer groups Key: KAFKA-986 URL: https://issues.apache.org/jira/browse/KAFKA-986 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.8 Environment: Linux Reporter: Subbu Srinivasan Assignee: Neha Narkhede Folks, How can we simulate the notion of queues for consumers from multiple instances? For eg: I have a topic log. >From a single machine ( I tried from different machines also) I started two >consumers on same topic with same group id. Both the consumers get copes of >messages. bin/kafka-console-consumer.sh --zookeeper kafka1:2181 --topic log --group 1 bin/kafka-console-consumer.sh --zookeeper kafka1:2181 --topic log --group 1 >From the design section at http://kafka.apache.org/design.html Each consumer process belongs to a consumer group and each message is delivered to exactly one process within every consumer group. Hence a consumer group allows many processes or machines to logically act as a single consumer. The concept of consumer group is very powerful and can be used to support the semantics of either a queue or topic as found in JMS. To support queue semantics, we can put all consumers in a single consumer group, in which case each message will go to a single consumer. Can someone elaborate on this? Thanks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit
Swapnil Ghike created KAFKA-987: --- Summary: Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit Key: KAFKA-987 URL: https://issues.apache.org/jira/browse/KAFKA-987 Project: Kafka Issue Type: Bug Affects Versions: 0.8 Reporter: Swapnil Ghike Assignee: Swapnil Ghike Fix For: 0.8 We need to fix the Kafka zookeeper consumer to avoid checkpointing offsets that have not changed since the last offset commit. This will help reduce the write load on zookeeper. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same
[
https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715843#comment-13715843
]
Guozhang Wang commented on KAFKA-984:
-
Approach:
1. In ZookeeperConsumerConnector.reinitializeConsumer, add three additional
arguments:
partial : boolean
Seq[String] : addedTopics
Seq[String] : deletedTopics
1. In WildcardStreamsHandler.handleTopicEvent, calling reinitializeConsumer
with partial = true with addedTopics and deletedTopics set.
2. In ZookeeperConsumerConnector.reinitializeConsumer, if partial == true,
branch out the code starting from
// map of {topic -> Set(thread-1, thread-2, ...)}
val consumerThreadIdsPerTopic: Map[String, Set[String]] =
topicCount.getConsumerThreadIdsPerTopic
to
val groupedByTopic = threadQueueStreamPairs.groupBy(_._1._1)
groupedByTopic.foreach(e => {
val topic = e._1
val streams = e._2.map(_._2._2).toList
topicStreamsMap += (topic -> streams)
debug("adding topic %s and %d streams to map.".format(topic,
streams.size))
})
But just update topicThreadIdAndQueues and topicStreamsMap
* Note that we currently do not handle deleted topics, and this issue will not
be fixed in this JIRA.
3. Add another function syncedPartialRebalance in ZKRebalancerListener, and
making the reinitializeConsumer to call this function if partial == true.
syncedPartialRebalance will use the same rebalanceLock as syncedRebalance.
4. ZKRebalancerListener keeps a list of topics it is currently consuming from
in memory.
4. In syncedPartialRebalance, which calls rebalanceForTopic, it first checks if
there are any changes of the topics by reading the ZK and comparing with its in
memory list. If no new topics or deleted topics found, return directly.
4.1. For each deleted topic, simply call closeFetchers, and delete the
ownership/offsets of the topic in ZK.
4.2 For each added topic, read the number of consumers of the group from ZK (we
only do this once for all topics), and read the number of partitions of this
topic, assign the partitions to consumers using the same deterministic
algorithm.
4.3 Try writing to the ZK for all added topics. If succeed, update fetchers
(start new threads) and return true, otherwise return false.
Considerata:
1. If a topic change and consumer/broker change happens at the same time, two
consumers could trigger the corresponding syncedRebalance and
syncedPartialRebalance at different orders. In this case we would prefer to
make syncedPartialRebalance fail fast and make everyone entering the
syncedRebalance phase. So one possible optimization is to check
isWatcherTriggered at the beginning of syncedPartialRebalance, if it is set,
return false directly. Also do not retry in syncedPartialRebalance.
2. Stopping fecthers for certain topics, i.e., only sending a partial
threadIdMap to closeFetchers(cluster: Cluster, messageStreams:
Map[String,List[KafkaStream[_,_]]], relevantTopicThreadIdsMap: Map[String,
Set[String]]) is not used before. Not clear if this works well.
> Avoid a full rebalance in cases when a new topic is discovered but
> container/broker set stay the same
> -
>
> Key: KAFKA-984
> URL: https://issues.apache.org/jira/browse/KAFKA-984
> Project: Kafka
> Issue Type: Bug
>Reporter: Guozhang Wang
>Assignee: Guozhang Wang
> Fix For: 0.8
>
>
> Currently a full rebalance will be triggered on high level consumers even
> when just a new topic is added to ZK. Better avoid this behavior but only
> rebalance on this newly added topic.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-347: - Attachment: KAFKA-347-v4.patch > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch, KAFKA-347-v3.patch, KAFKA-347-v4.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715869#comment-13715869 ] Joe Stein commented on KAFKA-982: - +1 for 298 > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-987: Attachment: kafka-987.patch The only case wherein the code does not have a bug and yet the consumer could end up consuming from an offset < the previously committed offset is when there is an unclean leader election. Hence, this patch will commit new offsets as long as they are different from the previously committed offsets. > Avoid checkpointing offsets in Kafka consumer that have not changed since the > last commit > - > > Key: KAFKA-987 > URL: https://issues.apache.org/jira/browse/KAFKA-987 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Swapnil Ghike >Assignee: Swapnil Ghike > Labels: improvement > Fix For: 0.8 > > Attachments: kafka-987.patch > > > We need to fix the Kafka zookeeper consumer to avoid checkpointing offsets > that have not changed since the last offset commit. This will help reduce the > write load on zookeeper. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same
[ https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715900#comment-13715900 ] Guozhang Wang commented on KAFKA-984: - Following Sriram's comments, I am trying to propose a simpler solution for a special case of "the event handler reports only adding ONE new topic". 1. Add a new function handleAddNewTopicForFilteredStreams, which updates topicThreadIdAndQueues and topicStreamsMap and directly calls syncedPartialRebalance(topic : String) of ZKRebalancerListener. 2. syncedPartialRebalance will use the same rebalanceLock as syncedRebalance. 2.1 For the added topic, first check if its ownership has not be claimed in ZK yet. If it is already claimed (i.e., ZK has the znode for this topic in owners), release the partitions first. 2.2 Read the number of consumers of the group from ZK (we only do this once for all topics), and read the number of partitions of this topic, assign the partitions to consumers using the same deterministic algorithm. 2.2 Try writing to the ZK for all added topics. If succeed, update fetchers (start new threads) and returns, otherwise also returns since it means a syncedRebalance will happen next. Considerata: to make things simple when a topic change and consumer/broker change happens at the same time: 1 At the beginning of syncedPartialRebalance checks if isWatcherTriggered is true, if yes returns directly. 2 During syncedPartialRebalance if the new topic has already been claimed (i.e., there are some znode in owners already), calls the syncedRebalance directly. The correctness of this is based on: 1. If syncedRebalance is triggered after the syncedPartialRebalance, it will see the new topic. 2. If syncedRebalance is triggered before the syncedPartialRebalance and does not see this new topic, a later syncedRebalance can still successfully do the rebalance. > Avoid a full rebalance in cases when a new topic is discovered but > container/broker set stay the same > - > > Key: KAFKA-984 > URL: https://issues.apache.org/jira/browse/KAFKA-984 > Project: Kafka > Issue Type: Bug >Reporter: Guozhang Wang >Assignee: Guozhang Wang > Fix For: 0.8 > > > Currently a full rebalance will be triggered on high level consumers even > when just a new topic is added to ZK. Better avoid this behavior but only > rebalance on this newly added topic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715932#comment-13715932 ] Jay Kreps commented on KAFKA-982: - No love for 294? I particularly liked that one because it has a pen (which works both because kafka is a system for logging or journaling, and also because kafka is a writer) and the pen is made out of two Ks. > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715939#comment-13715939 ] Jakob Homan commented on KAFKA-982: --- +1 298. It can be made recursive. > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715943#comment-13715943 ] Sriram Subramanian commented on KAFKA-347: -- added a script for addpartitions > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch, KAFKA-347-v3.patch, KAFKA-347-v4.patch, > KAFKA-347-v5.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriram Subramanian updated KAFKA-347: - Attachment: KAFKA-347-v5.patch > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch, KAFKA-347-v3.patch, KAFKA-347-v4.patch, > KAFKA-347-v5.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715946#comment-13715946 ] Sriram Subramanian commented on KAFKA-982: -- +1 for 294. 294 is what I like but it does not seem to get any love. Second choice 298 > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715949#comment-13715949 ] Swapnil Ghike commented on KAFKA-982: - +1 for 296. > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715955#comment-13715955
]
David Arthur commented on KAFKA-981:
Looking closer at the pom.xml, I see two sections. One of which
looks like a copy/paste error from ivy.xml, the other looks like normal Maven
pom dependencies. See https://gist.github.com/mumrah/6059092#file-pom-xml-L25
for what I'm talking about.
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
>Assignee: Joe Stein
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715975#comment-13715975
]
Joe Stein commented on KAFKA-981:
-
The first dependency section you see is only from maven central
http://repo1.maven.org/maven2/org/apache/kafka/kafka_2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
its not there in
https://repository.apache.org/content/repositories/releases/org/apache/kafka/kafka_2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
which I believe is due to me posting the release more than once with making
pom changes that did not bump the version in order to get things to work and
through https://issues.apache.org/jira/browse/KAFKA-974
I don't know if that is what is causing the error but looking at the source
here
https://svn.apache.org/repos/asf/ant/ivy/core/trunk/src/java/org/apache/ivy/core/module/id/ModuleId.java
that error is because the name of the module is coming in as null, which is
odd.
Can you try to override the repository to use
https://repository.apache.org/content/repositories/releases instead to see if
it works as expected it has resolved other issues folks have brought up.
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
>Assignee: Joe Stein
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-981) Unable to pull Kafka binaries with Ivy
[
https://issues.apache.org/jira/browse/KAFKA-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715983#comment-13715983
]
David Arthur commented on KAFKA-981:
[~charmalloc], ah yes - that pesky "released versions are immutable" thing.
Changing Ivy to pull from Apache's repo did the trick.
BTW, I debugged and stepped through Ivy during that error and it was failing on
the bad part of the XML (the ivy elements). So it was definitely
caused by the invalid pom.xml.
Now I'm getting failure to resolve "com.sun.jdmk" nonsense, but I can deal with
that.
Should there be a beta2 release to fix this issue in Maven central?
> Unable to pull Kafka binaries with Ivy
> --
>
> Key: KAFKA-981
> URL: https://issues.apache.org/jira/browse/KAFKA-981
> Project: Kafka
> Issue Type: Bug
> Components: packaging
>Affects Versions: 0.8
>Reporter: David Arthur
>Assignee: Joe Stein
> Attachments: ant.log, ivy.xml
>
>
> I am trying to pull the published Kafka binary with a simple Ivy file.
> conf="default->default"/>
> I get the following exception: [ivy:resolve] problem occurred while resolving
> dependency: org.apache.kafka#kafka_2.9.2;0.8.0-beta1 {default=[default]} with
> main: java.lang.IllegalArgumentException: null name not allowed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-347) change number of partitions of a topic online
[ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716043#comment-13716043 ] Jun Rao commented on KAFKA-347: --- Thanks for the patch. Committed to 0.8. Could you also provide a patch to trunk since in trunk, all topic related commands are consolidated to a kafka-topic tool. > change number of partitions of a topic online > - > > Key: KAFKA-347 > URL: https://issues.apache.org/jira/browse/KAFKA-347 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Sriram Subramanian > Labels: features > Fix For: 0.8.1 > > Attachments: kafka-347.patch, kafka-347-v2.patch, > KAFKA-347-v2-rebased.patch, KAFKA-347-v3.patch, KAFKA-347-v4.patch, > KAFKA-347-v5.patch > > > We will need an admin tool to change the number of partitions of a topic > online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-982) Logo for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716050#comment-13716050 ] Jun Rao commented on KAFKA-982: --- +1 for 296. > Logo for Kafka > -- > > Key: KAFKA-982 > URL: https://issues.apache.org/jira/browse/KAFKA-982 > Project: Kafka > Issue Type: Improvement >Reporter: Jay Kreps > Attachments: 289.jpeg, 294.jpeg, 296.png, 298.jpeg > > > We should have a logo for kafka. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
