[jira] [Created] (KAFKA-3638) Using kafka-console-producer with invalid broker list weirds to unexpected behavior
Jarek Jarcec Cecho created KAFKA-3638:
-
Summary: Using kafka-console-producer with invalid broker list
weirds to unexpected behavior
Key: KAFKA-3638
URL: https://issues.apache.org/jira/browse/KAFKA-3638
Project: Kafka
Issue Type: Bug
Reporter: Jarek Jarcec Cecho
I've messed the port when running console producer on my test cluster and
instead of {{ConnectionRefused}} exception or something like that, I didn't get
any immediate error. After I tried to produce some messages I got nothing and
after a minute or so got {{ERROR}} about updating metadata:
{code}
[root@centos6 ~]# kafka-console-producer --broker-list localhost:666 --topic
source
asfasdf
[2016-04-28 14:28:01,950] ERROR Error when sending message to topic source with
key: null, value: 7 bytes with error: Failed to update metadata after 6 ms.
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
{code}
Would it perhaps make sense to throw more accurate exception in this case?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-3638) Using kafka-console-producer with invalid broker list weirds to unexpected behavior
[
https://issues.apache.org/jira/browse/KAFKA-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263187#comment-15263187
]
Jarek Jarcec Cecho commented on KAFKA-3638:
---
Yup KAFKA-3468 definitely seems to be describing the same. Feel free to resolve
this as a duplicate [~granthenke].
> Using kafka-console-producer with invalid broker list weirds to unexpected
> behavior
> ---
>
> Key: KAFKA-3638
> URL: https://issues.apache.org/jira/browse/KAFKA-3638
> Project: Kafka
> Issue Type: Bug
>Reporter: Jarek Jarcec Cecho
>
> I've messed the port when running console producer on my test cluster and
> instead of {{ConnectionRefused}} exception or something like that, I didn't
> get any immediate error. After I tried to produce some messages I got nothing
> and after a minute or so got {{ERROR}} about updating metadata:
> {code}
> [root@centos6 ~]# kafka-console-producer --broker-list localhost:666 --topic
> source
> asfasdf
> [2016-04-28 14:28:01,950] ERROR Error when sending message to topic source
> with key: null, value: 7 bytes with error: Failed to update metadata after
> 6 ms. (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> {code}
> Would it perhaps make sense to throw more accurate exception in this case?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-2316) Drop java 1.6 support
[ https://issues.apache.org/jira/browse/KAFKA-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619146#comment-14619146 ] Jarek Jarcec Cecho commented on KAFKA-2316: --- I've configured the job to use Java "JDK 7 u51". Let me know if you will need further tweaks to that job. > Drop java 1.6 support > - > > Key: KAFKA-2316 > URL: https://issues.apache.org/jira/browse/KAFKA-2316 > Project: Kafka > Issue Type: Bug >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani > Fix For: 0.8.3 > > Attachments: KAFKA-2316.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2316) Drop java 1.6 support
[ https://issues.apache.org/jira/browse/KAFKA-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619271#comment-14619271 ] Jarek Jarcec Cecho commented on KAFKA-2316: --- I've just updated the second job as well [~ijuma]. > Drop java 1.6 support > - > > Key: KAFKA-2316 > URL: https://issues.apache.org/jira/browse/KAFKA-2316 > Project: Kafka > Issue Type: Bug >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani > Fix For: 0.8.3 > > Attachments: KAFKA-2316.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1598) Fix variable typo in Kafka main class
Jarek Jarcec Cecho created KAFKA-1598:
-
Summary: Fix variable typo in Kafka main class
Key: KAFKA-1598
URL: https://issues.apache.org/jira/browse/KAFKA-1598
Project: Kafka
Issue Type: Bug
Reporter: Jarek Jarcec Cecho
Priority: Trivial
I guess that this one is super-nit, but I've noticed that the name
{{kafkaServerStartble}} is miss-spelled in the main {{Kafka.scala}} file.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (KAFKA-1598) Fix variable typo in Kafka main class
[
https://issues.apache.org/jira/browse/KAFKA-1598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated KAFKA-1598:
--
Attachment: KAFKA-1598.patch
Attaching trivial patch that fixes the typo.
> Fix variable typo in Kafka main class
> -
>
> Key: KAFKA-1598
> URL: https://issues.apache.org/jira/browse/KAFKA-1598
> Project: Kafka
> Issue Type: Bug
>Reporter: Jarek Jarcec Cecho
>Priority: Trivial
> Attachments: KAFKA-1598.patch
>
>
> I guess that this one is super-nit, but I've noticed that the name
> {{kafkaServerStartble}} is miss-spelled in the main {{Kafka.scala}} file.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (KAFKA-1598) Fix variable typo in Kafka main class
[
https://issues.apache.org/jira/browse/KAFKA-1598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated KAFKA-1598:
--
Status: Patch Available (was: Open)
> Fix variable typo in Kafka main class
> -
>
> Key: KAFKA-1598
> URL: https://issues.apache.org/jira/browse/KAFKA-1598
> Project: Kafka
> Issue Type: Bug
>Reporter: Jarek Jarcec Cecho
>Priority: Trivial
> Attachments: KAFKA-1598.patch
>
>
> I guess that this one is super-nit, but I've noticed that the name
> {{kafkaServerStartble}} is miss-spelled in the main {{Kafka.scala}} file.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1598) Fix variable typo in Kafka main class
[
https://issues.apache.org/jira/browse/KAFKA-1598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101473#comment-14101473
]
Jarek Jarcec Cecho commented on KAFKA-1598:
---
Thank you for the review [~junrao]!
> Fix variable typo in Kafka main class
> -
>
> Key: KAFKA-1598
> URL: https://issues.apache.org/jira/browse/KAFKA-1598
> Project: Kafka
> Issue Type: Bug
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
>Priority: Trivial
> Fix For: 0.8.2
>
> Attachments: KAFKA-1598.patch
>
>
> I guess that this one is super-nit, but I've noticed that the name
> {{kafkaServerStartble}} is miss-spelled in the main {{Kafka.scala}} file.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1627) Move to Git
[ https://issues.apache.org/jira/browse/KAFKA-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126673#comment-14126673 ] Jarek Jarcec Cecho commented on KAFKA-1627: --- I've already transitioned two projects from SVN to GIT (Sqoop, MRUnit) and hence I'm more then happy to help if this is something that Kafka community wants to do. > Move to Git > --- > > Key: KAFKA-1627 > URL: https://issues.apache.org/jira/browse/KAFKA-1627 > Project: Kafka > Issue Type: Task >Reporter: David Chen > > It appears that Hadoop is moving from SVN to Git: > http://www.linux.com/news/featured-blogs/196-zonker/787127-apache-hadoop-transitions-to-git > Should we do the same? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1627) Move to Git
[ https://issues.apache.org/jira/browse/KAFKA-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126968#comment-14126968 ] Jarek Jarcec Cecho commented on KAFKA-1627: --- Thanks, I've missed that [~joestein]. I was using github mirror from some reason :) > Move to Git > --- > > Key: KAFKA-1627 > URL: https://issues.apache.org/jira/browse/KAFKA-1627 > Project: Kafka > Issue Type: Task >Reporter: David Chen > > It appears that Hadoop is moving from SVN to Git: > http://www.linux.com/news/featured-blogs/196-zonker/787127-apache-hadoop-transitions-to-git > Should we do the same? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1654) Provide a way to override server configuration from command line
Jarek Jarcec Cecho created KAFKA-1654:
-
Summary: Provide a way to override server configuration from
command line
Key: KAFKA-1654
URL: https://issues.apache.org/jira/browse/KAFKA-1654
Project: Kafka
Issue Type: Improvement
Components: core
Reporter: Jarek Jarcec Cecho
Assignee: Jarek Jarcec Cecho
I've been recently playing with Kafka and I found the current way of server
configuration quite inflexible. All the configuration options have to be inside
a properties file and there is no way how they can be overridden for execution.
In order to temporarily change one property I had to copy the config file and
change the property there. Hence, I'm wondering if people would be open to
provide a way how to specify and override the configs from the command line
when starting Kafka?
Something like:
{code}
./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
{code}
or
{code}
./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
{code}
I'm more than happy to take a stab at it, but I would like to see if there is
an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150095#comment-14150095
]
Jarek Jarcec Cecho commented on KAFKA-1654:
---
Thank you for the feedback [~nehanarkhede], yes I'll pick this one up.
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated KAFKA-1654:
--
Fix Version/s: 0.8.3
Affects Version/s: 0.8.1.1
Status: Patch Available (was: Open)
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated KAFKA-1654:
--
Attachment: KAFKA-1654.patch
Finally attaching the patch. My apologies for the delay [~nehanarkhede] - I've
been sidetracked by my day job :(
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1750) handle "topic not exists" scenario
[ https://issues.apache.org/jira/browse/KAFKA-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196584#comment-14196584 ] Jarek Jarcec Cecho commented on KAFKA-1750: --- Thank you for uploading the patch [~Dmitry Pekar]! I think that you should upload the patch to review board and wait on Kafka committers to review and commit before we can resolve the JIRA as "resolved". > handle "topic not exists" scenario > -- > > Key: KAFKA-1750 > URL: https://issues.apache.org/jira/browse/KAFKA-1750 > Project: Kafka > Issue Type: Sub-task > Components: tools >Reporter: Dmitry Pekar >Assignee: Dmitry Pekar > Fix For: 0.8.3 > > Attachments: kafka-1750.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196791#comment-14196791
]
Jarek Jarcec Cecho commented on KAFKA-1654:
---
Thank you for the feedback [~nehanarkhede]! I'll take a look and incorporate
the changes :)
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201096#comment-14201096
]
Jarek Jarcec Cecho commented on KAFKA-1654:
---
Looking at feedback proposal more closely [~nehanarkhede], I wanted to explain
a bit of my thinking. My intention is to add ability to specify command line
arguments while starting Kafka broker similarly as we do have for the command
line tools. In this patch I'm adding one argument only ({{--set}} that will be
renamed to {{--override}} as you've suggested), that happens to have the
ability to be specified multiple times. That explains why I'm expecting that
you have to specify it for every property that is being overriden and also why
I do have the [kafka options] section without mentioning that you have to use
{{--set}} (you might want to add different parameter in the future, like
completely bogus {{--disable-jmx}} or whatever :)). Perhaps I should have also
submitted a second documentation patch to cover the usage. Does the reasoning
resonate with you or would you prefer to simply add ability to override the
config properties without opening the code to introduce additional command line
arguments in the future? I'll definitely address the fact that I'm not properly
failing on error cases, that is a huge problem in my mind - thank you for
uncovering it!
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201096#comment-14201096
]
Jarek Jarcec Cecho edited comment on KAFKA-1654 at 11/6/14 10:42 PM:
-
Looking at feedback proposal more closely [~nehanarkhede], I wanted to explain
a bit of my thinking. My intention is to add ability to specify command line
arguments while starting Kafka broker similarly as we do have for the command
line tools. In this patch I'm adding one argument only ({{\-\-set}} that will
be renamed to {{\-\-override}} as you've suggested), that happens to have the
ability to be specified multiple times. That explains why I'm expecting that
you have to specify it for every property that is being overriden and also why
I do have the [kafka options] section without mentioning that you have to use
{{\-\-set}} (you might want to add different parameter in the future, like
completely bogus {{\-\-disable\-jmx}} or whatever :)). Perhaps I should have
also submitted a second documentation patch to cover the usage. Does the
reasoning resonate with you or would you prefer to simply add ability to
override the config properties without opening the code to introduce additional
command line arguments in the future? I'll definitely address the fact that I'm
not properly failing on error cases, that is a huge problem in my mind - thank
you for uncovering it!
was (Author: jarcec):
Looking at feedback proposal more closely [~nehanarkhede], I wanted to explain
a bit of my thinking. My intention is to add ability to specify command line
arguments while starting Kafka broker similarly as we do have for the command
line tools. In this patch I'm adding one argument only ({{--set}} that will be
renamed to {{--override}} as you've suggested), that happens to have the
ability to be specified multiple times. That explains why I'm expecting that
you have to specify it for every property that is being overriden and also why
I do have the [kafka options] section without mentioning that you have to use
{{--set}} (you might want to add different parameter in the future, like
completely bogus {{--disable-jmx}} or whatever :)). Perhaps I should have also
submitted a second documentation patch to cover the usage. Does the reasoning
resonate with you or would you prefer to simply add ability to override the
config properties without opening the code to introduce additional command line
arguments in the future? I'll definitely address the fact that I'm not properly
failing on error cases, that is a huge problem in my mind - thank you for
uncovering it!
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated KAFKA-1654:
--
Attachment: KAFKA-1654.patch
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch, KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201718#comment-14201718
]
Jarek Jarcec Cecho commented on KAFKA-1654:
---
I've added second version of the patch that:
* Renames the {{\-\-set}} to {{\-\-override}}
* Enforces that only valid parameters are on the command line and there are no
so called "nonOptions"
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch, KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14204442#comment-14204442
]
Jarek Jarcec Cecho commented on KAFKA-1654:
---
Thank you for your additional feedback [~nehanarkhede]! Good pointer to
{{CommandLineUtils}} class -I've updated my patch to use this class and it's
{{printUsageAndDie}} and {{parseKeyValueArgs}} methods. Using the util class
has simplified the main code, but imposed a challenge on test side as those
methods are directly calling {{System.exit()}}. After some googling I've solved
that problem by introducing {{SecurityManager}} that will intercept the
{{System.exit}} call and thrown an unchecked exception that can be verified by
junit.
To answer your specific points:
{quote}
The usage command says "USAGE: java \[options\] %s \[kafka options\]
server.properties", but in order for the kafka options to take effect you also
need to use --set
{quote}
I've updated the message to explicitly state that I'm expecting list of
arguments ({{\[--override key-value\]*}}) rather then using the generic term
{{kafka options}}.
{quote}
If you leave set out, it doesn't error out saying that --set is required and
silently does not end up overriding the property value
{quote}
I've added unit tests to ensure that the only allowed input is to have (list
of) parameter {{--override}} followed by one property that should be overridden.
{quote}
Can you use CommandLineUtils.printUsageAndDie()?
{quote}
Done.
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch, KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated KAFKA-1654:
--
Attachment: KAFKA-1654.patch
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch, KAFKA-1654.patch, KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated KAFKA-1654:
--
Attachment: KAFKA-1654.patch
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch, KAFKA-1654.patch, KAFKA-1654.patch,
> KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222431#comment-14222431
]
Jarek Jarcec Cecho commented on KAFKA-1654:
---
Thanks for the good feedback [~nehanarkhede]!
{quote}
1. The script for starting kafka (bin/kafka-server-start.sh) prints a different
help message compared to the one in kafka.Kafka. Since that is the script used
by everyone to start Kafka, it is worth fixing it there.
{quote}
Good point, I've missed that and fixed in latest patch. It's a bit unfortunate
that we're parsing the arguments in two places, perhaps that is something that
we can improve in subsequent JIRA.
{quote}
2. The properties have to be specified before the server.properties file. I
found it more natural to specify it after the server.properties in the
following manner.
bin/kafka-server-start.sh config/server.properties --override property=value
{quote}
I personally don't have any preference on this matter, so I've incorporated it
into my latest patch :)
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch, KAFKA-1654.patch, KAFKA-1654.patch,
> KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line
[
https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222622#comment-14222622
]
Jarek Jarcec Cecho commented on KAFKA-1654:
---
Thank you [~nehanarkhede] for bearing with me on all those reviews :)
> Provide a way to override server configuration from command line
>
>
> Key: KAFKA-1654
> URL: https://issues.apache.org/jira/browse/KAFKA-1654
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.1.1
>Reporter: Jarek Jarcec Cecho
>Assignee: Jarek Jarcec Cecho
> Fix For: 0.8.3
>
> Attachments: KAFKA-1654.patch, KAFKA-1654.patch, KAFKA-1654.patch,
> KAFKA-1654.patch
>
>
> I've been recently playing with Kafka and I found the current way of server
> configuration quite inflexible. All the configuration options have to be
> inside a properties file and there is no way how they can be overridden for
> execution. In order to temporarily change one property I had to copy the
> config file and change the property there. Hence, I'm wondering if people
> would be open to provide a way how to specify and override the configs from
> the command line when starting Kafka?
> Something like:
> {code}
> ./bin/kafka-server-start.sh -Dmy.cool.property=X kafka.properties
> {code}
> or
> {code}
> ./bin/kafka-server-start.sh --set my.cool.property=X kafka.properties
> {code}
> I'm more than happy to take a stab at it, but I would like to see if there is
> an interest for such capability?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
