Hi Sasaki, Thanks for the update. The KIP looks good to me. I'd suggest moving to a vote.
Thanks, Jason On Mon, May 7, 2018 at 7:08 AM, Sasaki Toru <sasaki...@oss.nttdata.com> wrote: > Hi Manikumar, Colin, > > Thank you for your comment. > > As Colin said, I proposed adding an option to show version information of > "local" tool, > because many software have this option and I think Kafka also needs this > one. > > As you said, the function to show remote Kafka version is useful, > but I think it is better to create new KIP because this function has some > points which should be considered. > > If you have any better ideas, could you please tell us? > > > Many thanks, > Sasaki > > From: Manikumar <manikumar.re...@gmail.com> >> Date: 2018-05-03 4:11 GMT+09:00 >> >> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands >> To: dev <dev@kafka.apache.org> >> >> >> Hi Colin, >> >> Thanks for explanation. It's definitely useful to have --version flag. >> >> kafka-broker-api-versions.sh gives the API versions, not Kafka release >> version. >> Is not easy to figure out release version from API versions. Currently >> release version is available via metric/JMX. >> If required, we can implement this in future. >> >> >> Thanks, >> >> On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cmcc...@apache.org> wrote: >> >> Hi Manikumar, >>> >>> We already have a tool for getting the Kafka broker API versions, >>> "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97. >>> >>> What Saski is proposing here is having a way of getting the version of >>> locally installed Kafka software, which may be different from the server >>> version. Many pieces of software offer a --version flag, and it's always >>> understood to refer to the local version of the software, not a version >>> running somewhere else. The user has no way to get this information now, >>> other than perhaps trying to look at he names of jar files. >>> >>> cheers, >>> Colin >>> >>> On Tue, May 1, 2018, at 08:20, Manikumar wrote: >>> >>>> I assume the intent of the KIP to find out the Kafka broker version. In >>>> this case, maybe we should expose >>>> version using a Kafka request. This will help the remote scripts/tools >>>> >>> to >> >>> query the Kafka version. >>>> scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote >>>> machines and may use >>>> older Kafka versions. In this case, current proposal prints on the older >>>> version. >>>> >>>> On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cmcc...@apache.org> >>>> wrote: >>>> >>>> Thanks, Sasaki. >>>>> >>>>> Colin >>>>> >>>>> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote: >>>>> >>>>>> Hi Colin, Jason, >>>>>> >>>>>> Thank you for your beneficial comment. >>>>>> I have updated my Pull Request to show git commit hash in version >>>>>> information.> In my current Pull Request, we cat get the result such >>>>>> >>>>> below: >>>>> >>>>>> $ bin/kafka-topics.sh --version >>>>>> (snip) >>>>>> 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e) >>>>>> >>>>>> >>>>>> I have also updated to accept double-dash for this option (-- >>>>>> version) only.> >>>>>> >>>>>> Many thanks, >>>>>> Sasaki >>>>>> >>>>>> From: Jason Gustafson <ja...@confluent.io> >>>>>>> Date: 2018-04-25 9:42 GMT+09:00 >>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's >>>>>>> commands> > To: dev <dev@kafka.apache.org> >>>>>>> >>>>>>> >>>>>>> +1 on adding the git commit id to the output. We often encounter >>>>>>> environments which are testing off of trunk or have modifications >>>>>>> >>>>>> on >>> >>>> top of> > an existing release. >>>>>>> >>>>>>> -Jason >>>>>>> >>>>>>> On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmcc...@apache.org >>>>>>> wrote:> > >>>>>>> >>>>>>>> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote: >>>>>>>> >>>>>>>>> Hi Jason, Colin, >>>>>>>>> >>>>>>>>> Thank you for your comment, and I'm sorry for late reply. >>>>>>>>> >>>>>>>>> > we refactored all of the tools so that we could use a common >>>>>>>>> > set of> >>> base options, >>>>>>>>> > it might be a little annoying to have to continue supporting >>>>>>>>> > both> >>> variations. >>>>>>>>> >>>>>>>>> I feel KIP-14 is good idea (I'm sorry, I didn't know about >>>>>>>>> it), and> >>> I think it's no longer necessary both single-dash >>>>>>>>> >>>>>>>> and >>> >>>> double- >>>>> >>>>>> dash are> >>> supported when this KIP will be accepted, as you >>>>>>>>> >>>>>>>> said. >>> >>>> I revise my Pull Request to support single-dash option only. >>>>>>>>> >>>>>>>>> Some my code in kafka-run-class.sh will become unnecessary when >>>>>>>>> KIP-14> >>> is accepted. >>>>>>>>> But I will keep this as temporary, because I seem that it's >>>>>>>>> >>>>>>>> useful> >>> >>>> before KIP-14 accepted. >>>>>>>> >>>>>>>>> >>>>>>>>> > Can you give a little more detail about what would be >>>>>>>>> > displayed when> >>> the version command was used? >>>>>>>>> >>>>>>>>> As Ismael said, the version string is got from >>>>>>>>> AppInfoParser#getVersion.> >>> >>>>>>>>> In my Pull Request, we can get the result such as below:: >>>>>>>>> >>>>>>>>> $ bin/kafka-topics.sh --version >>>>>>>>> (snip) >>>>>>>>> Kafka 1.2.0-SNAPSHOT >>>>>>>>> >>>>>>>> Hi Sasaki, >>>>>>>> >>>>>>>> Thanks for the info. Can you add this to the KIP? >>>>>>>> >>>>>>>> Also, I really think we should include the git hash. >>>>>>>> >>>>>>>> best, >>>>>>>> Colin >>>>>>>> >>>>>>>> >>>>>>>> Many thanks, >>>>>>>>> Sasaki >>>>>>>>> >>>>>>>>> >>>>>>>>> From: Ismael Juma <ism...@juma.me.uk> >>>>>>>>>> Date: 2018-04-24 3:44 GMT+09:00 >>>>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's >>>>>>>>>> commands> >>>> To: dev <dev@kafka.apache.org> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> FYI, the injection via the build process that is mentioned here >>>>>>>>>> >>>>>>>>> already >>>>>>> >>>>>>>> happens. See AppInfoParser. >>>>>>>>>> >>>>>>>>>> Ismael >>>>>>>>>> >>>>>>>>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe >>>>>>>>>> <cmcc...@apache.org>> >> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Sasaki, >>>>>>>>>>> >>>>>>>>>>> Thanks for the KIP. I think a version flag is a good idea. >>>>>>>>>>> >>>>>>>>>>> Can you give a little more detail about what would be >>>>>>>>>>> >>>>>>>>>> displayed >> >>> when> >> the >>>>>>>>>>> version command was used? >>>>>>>>>>> >>>>>>>>>>> We clearly want the version number, but we probably also want >>>>>>>>>>> >>>>>>>>>> to >>> >>>> know> >> if >>>>>>>>>>> this is an official release, or a random SNAPSHOT from a >>>>>>>>>>> >>>>>>>>>> branch. >>> >>>> If> >> this >>>>>>>>>>> is a release candidate, we probably want the RC number as >>>>>>>>>>> >>>>>>>>>> well, >> >>> like> >>>>> "1.1-rc3" We also want a git hash. This can be >>>>>>>>>>> >>>>>>>>>> injected by the> > build >>>>> >>>>>> process. In the case of an official release, where the source >>>>>>>>>>> code> >> is not >>>>>>>>>>> under git, we can pull it from a file. >>>>>>>>>>> >>>>>>>>>>> For example, hadoop's version output looks like this: >>>>>>>>>>> >>>>>>>>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop >>>>>>>>>>> > version> >>>>> > Hadoop 2.8.3 >>>>>>>>>>> > Subversion >>>>>>>>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r> >>>>>>>>>>> >>>>>>>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2 >>>>> >>>>>> > Compiled by jdu on 2017-12-05T03:43Z >>>>>>>>>>> > Compiled with protoc 2.5.0 >>>>>>>>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f1 >>>>>>>>>>> >>>>>>>>>> 9d> >>> >>>> > This command was run using /home/cmccabe/Downloads/ >>>>>>>>>> >>>>>>>>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar >>>>>>>>>>> >>>>>>>>>>> (The "subversion" line here is a little weird -- it now refers >>>>>>>>>>> to> >> git, not >>>>>>>>>>> svn) >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey Sasaki, >>>>>>>>>>>> >>>>>>>>>>>> Yeah, I don't feel too strongly about only supporting -- >>>>>>>>>>>> version. I> >> agree >>>>>>>>>>>> >>>>>>>>>>> it >>>>>>>>>>> >>>>>>>>>>>> may help discoverability given the current approach. On the >>>>>>>>>>>> other> >> hand, >>>>>>>>>>>> >>>>>>>>>>> if >>>>>>>>>>> >>>>>>>>>>>> we refactored all of the tools so that we could use a common >>>>>>>>>>>> set of> >> base >>>>>>>>>>>> options, it might be a little annoying to have to continue >>>>>>>>>>>> >>>>>>>>>>> supporting >>>>>>> >>>>>>>> both >>>>>>>>>>> >>>>>>>>>>>> variations. For example, tool standardization was proposed in >>>>>>>>>>>> KIP-14> >> and >>>>>>>>>>>> I'm still holding out hope that someone will have time to >>>>>>>>>>>> >>>>>>>>>>> pick >> >>> this> >> work >>>>>>>>>>>> back up. It's always easier to add an option than remove one, >>>>>>>>>>>> so I'm> >>>>>> slightly inclined to have only --version for >>>>>>>>>>>> >>>>>>>>>>> now. >>> >>>> What do you >>>>> >>>>>> think?> >>>>> The double dash version is more consistent with >>>>>>>>>>>> >>>>>>>>>>> how >>> >>>> our other >>>>> >>>>>> flags> >> work. >>>>>>>>>>> In general, I feel that if --version is supported, --help >>>>>>>>>>> >>>>>>>>>> should >>> >>>> say> >> so. >>>>>>>>>>> best, >>>>>>>>>>> Colin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>>> Jason >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru < >>>>>>>>>>>> >>>>>>>>>>> sasaki...@oss.nttdata.com >>>>>>>> >>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Jason >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you for helpful comments. I updated wiki based on your >>>>>>>>>>>>> >>>>>>>>>>>> advice. >>>>>>> >>>>>>>> I thought this option was relatively common and making >>>>>>>>>>>>> maintenance> >>>> easy >>>>>>>>>>>>> was also important. >>>>>>>>>>>>> However, as you said, it is not good that version option >>>>>>>>>>>>> >>>>>>>>>>>> won't >>> >>>> be> >>>>> shown up >>>>>>>>>>>>> in help description. >>>>>>>>>>>>> >>>>>>>>>>>>> I thought accepting both single-dash and double-dash will >>>>>>>>>>>>> >>>>>>>>>>>> help >>> >>>> to> >> find >>>>>>>>>>>>> this option. >>>>>>>>>>>>> In my approach this option won't be showed, but most of >>>>>>>>>>>>> software> >> which >>>>>>>>>>>>> >>>>>>>>>>>> has >>>>>>>>>>> >>>>>>>>>>>> this option accepts either single-dash or double-dash. >>>>>>>>>>>>> I guess it doesn't need to support both if we take another >>>>>>>>>>>>> way.> >>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> >>>>>>>>>>>>> @Ted Yeah, you're right. Sorry about the confusion. >>>>>>>>>>>>> >>>>>>>>>>>>>> Since we're here, I think this KIP is a nice improvement. >>>>>>>>>>>>>> It's> >>>>> definitely >>>>>>>>>>>>>> nice to have an easy way to check the version. That said, >>>>>>>>>>>>>> >>>>>>>>>>>>> do >> >>> we> >>>> really >>>>>>>>>>>>>> need >>>>>>>>>>>>>> to support both `-version` and `--version`? The latter is >>>>>>>>>>>>>> >>>>>>>>>>>>> consistent >>>>>>>> >>>>>>>>> with >>>>>>>>>>> >>>>>>>>>>>> our current tools. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also, I think the approach we've taken is basically to >>>>>>>>>>>>>> >>>>>>>>>>>>> build >> >>> the> >>>>> --version >>>>>>>>>>>>>> functionality into the bash script. This is nice because it >>>>>>>>>>>>>> saves> > a >>>>>>>>>>>>>> >>>>>>>>>>>>> lot of >>>>>>>>>>> >>>>>>>>>>>> work to update the commands individually and we don't need >>>>>>>>>>>>>> >>>>>>>>>>>>> to >>> >>>> do> >>>>> anything >>>>>>>>>>>>>> when we add new tools. The downside is that `--version` >>>>>>>>>>>>>> >>>>>>>>>>>>> won't >>> >>>> show> >> up >>>>>>>>>>>>>> >>>>>>>>>>>>> as >>>>>>>>>>> >>>>>>>>>>>> an >>>>>>>>>>>>>> option in any of the --help output. Not sure if that is too >>>>>>>>>>>>>> big of> >> a >>>>>>>>>>>>>> problem, but maybe worth mentioning this in the rejected >>>>>>>>>>>>>> >>>>>>>>>>>>> alternatives >>>>>>>> >>>>>>>>> section. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Jason >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yuzhih...@gmail.com >>>>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>> >>>>>>>> Jason: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Maybe your reply was intended for another KIP ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> KIP-278 is about adding version option, not timeout. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson < >>>>>>>>>>>>>>> >>>>>>>>>>>>>> ja...@confluent.io> >>>>>>>> >>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Sasaki, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for the KIP. I think the timeout controls the >>>>>>>>>>>>>>>> maximum> >>>> allowed >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> time >>>>>>>>>>>>>>> that the consumer will block for the next record. Maybe >>>>>>>>>>>>>>> >>>>>>>>>>>>>> the> >>> >>>> meaning >>>>>>> >>>>>>>> would >>>>>>>>>>>>>>> be clearer with the more concise name `--timeout`? That >>>>>>>>>>>>>>> >>>>>>>>>>>>>> also >>> >>>> fits> >>>>> with >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> old consumer which overrides the `consumer.timeout.ms` >>>>>>>>>>>>>>>> property.> >>>>>>>>>> >>>>>>>>>>>>>>>> By the way, it seems like the default value was >>>>>>>>>>>>>>>> intentionally> > set >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> low >>>>>>>>>>> >>>>>>>>>>>> for >>>>>>>>>>>>>>> both the old and new consumers, but I'm not sure of the >>>>>>>>>>>>>>> reason.> > We >>>>>>>>>>>>>>> >>>>>>>>>>>>>> could >>>>>>>>>>> >>>>>>>>>>>> leave the default as it is if we want to be safe, but >>>>>>>>>>>>>>>> increasing> >> it >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> seems >>>>>>>>>>>>>>> ok to me. Perhaps we could start a little lower, though, >>>>>>>>>>>>>>> >>>>>>>>>>>>>> say >>> >>>> 10> >>>>> seconds? >>>>>>>>>>>>>>> In >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> any case, we should make it clear to the user that the >>>>>>>>>>>>>>>> timeout> >> was >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> reached. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It's surprising to see only the incomplete reported >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> results> >>> >>>> following a >>>>>>>>>> >>>>>>>>>>> timeout. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Jason >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru < >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> sasaki...@oss.nttdata.com> >>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hello everyone, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> you> > >>> >>>> please >>>>> >>>>>> give >>>>>>>>>>> >>>>>>>>>>>> comments and advice ? >>>>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>>>>>>>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 7s+commands> >>>>> >>>>>> JIRA ticket and Pull Request are bellow: >>>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061> >>>>>>>>>>>>>>>>> <https://github.com/apache/kafka/pull/639> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Many thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Sasaki >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA >>>>>>>>>>>>>>>>> CORPORATION> >>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION >>>>>>>>> >>>>>>>>> -- >>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION >>>>>> >>>>>> >>>>> > -- > Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION > >
