+1. Thanks, Sasaki. Colin
On Wed, May 9, 2018, at 09:15, Jason Gustafson wrote: > Hi Sasaki, > > Thanks for the update. The KIP looks good to me. I'd suggest moving to a > vote. > > Thanks, > Jason > > On Mon, May 7, 2018 at 7:08 AM, Sasaki Toru <sasaki...@oss.nttdata.com> > wrote: > > > Hi Manikumar, Colin, > > > > Thank you for your comment. > > > > As Colin said, I proposed adding an option to show version information of > > "local" tool, > > because many software have this option and I think Kafka also needs this > > one. > > > > As you said, the function to show remote Kafka version is useful, > > but I think it is better to create new KIP because this function has some > > points which should be considered. > > > > If you have any better ideas, could you please tell us? > > > > > > Many thanks, > > Sasaki > > > > From: Manikumar <manikumar.re...@gmail.com> > >> Date: 2018-05-03 4:11 GMT+09:00 > >> > >> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's commands > >> To: dev <dev@kafka.apache.org> > >> > >> > >> Hi Colin, > >> > >> Thanks for explanation. It's definitely useful to have --version flag. > >> > >> kafka-broker-api-versions.sh gives the API versions, not Kafka release > >> version. > >> Is not easy to figure out release version from API versions. Currently > >> release version is available via metric/JMX. > >> If required, we can implement this in future. > >> > >> > >> Thanks, > >> > >> On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cmcc...@apache.org> wrote: > >> > >> Hi Manikumar, > >>> > >>> We already have a tool for getting the Kafka broker API versions, > >>> "./bin/kafka-broker-api-versions.sh". It was added as part of KIP-97. > >>> > >>> What Saski is proposing here is having a way of getting the version of > >>> locally installed Kafka software, which may be different from the server > >>> version. Many pieces of software offer a --version flag, and it's always > >>> understood to refer to the local version of the software, not a version > >>> running somewhere else. The user has no way to get this information now, > >>> other than perhaps trying to look at he names of jar files. > >>> > >>> cheers, > >>> Colin > >>> > >>> On Tue, May 1, 2018, at 08:20, Manikumar wrote: > >>> > >>>> I assume the intent of the KIP to find out the Kafka broker version. In > >>>> this case, maybe we should expose > >>>> version using a Kafka request. This will help the remote scripts/tools > >>>> > >>> to > >> > >>> query the Kafka version. > >>>> scripts (kafka-topics.sh, kafka-configs.sh, etc..) may run from remote > >>>> machines and may use > >>>> older Kafka versions. In this case, current proposal prints on the older > >>>> version. > >>>> > >>>> On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cmcc...@apache.org> > >>>> wrote: > >>>> > >>>> Thanks, Sasaki. > >>>>> > >>>>> Colin > >>>>> > >>>>> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote: > >>>>> > >>>>>> Hi Colin, Jason, > >>>>>> > >>>>>> Thank you for your beneficial comment. > >>>>>> I have updated my Pull Request to show git commit hash in version > >>>>>> information.> In my current Pull Request, we cat get the result such > >>>>>> > >>>>> below: > >>>>> > >>>>>> $ bin/kafka-topics.sh --version > >>>>>> (snip) > >>>>>> 2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e) > >>>>>> > >>>>>> > >>>>>> I have also updated to accept double-dash for this option (-- > >>>>>> version) only.> > >>>>>> > >>>>>> Many thanks, > >>>>>> Sasaki > >>>>>> > >>>>>> From: Jason Gustafson <ja...@confluent.io> > >>>>>>> Date: 2018-04-25 9:42 GMT+09:00 > >>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's > >>>>>>> commands> > To: dev <dev@kafka.apache.org> > >>>>>>> > >>>>>>> > >>>>>>> +1 on adding the git commit id to the output. We often encounter > >>>>>>> environments which are testing off of trunk or have modifications > >>>>>>> > >>>>>> on > >>> > >>>> top of> > an existing release. > >>>>>>> > >>>>>>> -Jason > >>>>>>> > >>>>>>> On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmcc...@apache.org > >>>>>>> wrote:> > > >>>>>>> > >>>>>>>> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote: > >>>>>>>> > >>>>>>>>> Hi Jason, Colin, > >>>>>>>>> > >>>>>>>>> Thank you for your comment, and I'm sorry for late reply. > >>>>>>>>> > >>>>>>>>> > we refactored all of the tools so that we could use a common > >>>>>>>>> > set of> >>> base options, > >>>>>>>>> > it might be a little annoying to have to continue supporting > >>>>>>>>> > both> >>> variations. > >>>>>>>>> > >>>>>>>>> I feel KIP-14 is good idea (I'm sorry, I didn't know about > >>>>>>>>> it), and> >>> I think it's no longer necessary both single-dash > >>>>>>>>> > >>>>>>>> and > >>> > >>>> double- > >>>>> > >>>>>> dash are> >>> supported when this KIP will be accepted, as you > >>>>>>>>> > >>>>>>>> said. > >>> > >>>> I revise my Pull Request to support single-dash option only. > >>>>>>>>> > >>>>>>>>> Some my code in kafka-run-class.sh will become unnecessary when > >>>>>>>>> KIP-14> >>> is accepted. > >>>>>>>>> But I will keep this as temporary, because I seem that it's > >>>>>>>>> > >>>>>>>> useful> > >>> > >>>> before KIP-14 accepted. > >>>>>>>> > >>>>>>>>> > >>>>>>>>> > Can you give a little more detail about what would be > >>>>>>>>> > displayed when> >>> the version command was used? > >>>>>>>>> > >>>>>>>>> As Ismael said, the version string is got from > >>>>>>>>> AppInfoParser#getVersion.> >>> > >>>>>>>>> In my Pull Request, we can get the result such as below:: > >>>>>>>>> > >>>>>>>>> $ bin/kafka-topics.sh --version > >>>>>>>>> (snip) > >>>>>>>>> Kafka 1.2.0-SNAPSHOT > >>>>>>>>> > >>>>>>>> Hi Sasaki, > >>>>>>>> > >>>>>>>> Thanks for the info. Can you add this to the KIP? > >>>>>>>> > >>>>>>>> Also, I really think we should include the git hash. > >>>>>>>> > >>>>>>>> best, > >>>>>>>> Colin > >>>>>>>> > >>>>>>>> > >>>>>>>> Many thanks, > >>>>>>>>> Sasaki > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> From: Ismael Juma <ism...@juma.me.uk> > >>>>>>>>>> Date: 2018-04-24 3:44 GMT+09:00 > >>>>>>>>>> Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's > >>>>>>>>>> commands> >>>> To: dev <dev@kafka.apache.org> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> FYI, the injection via the build process that is mentioned here > >>>>>>>>>> > >>>>>>>>> already > >>>>>>> > >>>>>>>> happens. See AppInfoParser. > >>>>>>>>>> > >>>>>>>>>> Ismael > >>>>>>>>>> > >>>>>>>>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe > >>>>>>>>>> <cmcc...@apache.org>> >> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi Sasaki, > >>>>>>>>>>> > >>>>>>>>>>> Thanks for the KIP. I think a version flag is a good idea. > >>>>>>>>>>> > >>>>>>>>>>> Can you give a little more detail about what would be > >>>>>>>>>>> > >>>>>>>>>> displayed > >> > >>> when> >> the > >>>>>>>>>>> version command was used? > >>>>>>>>>>> > >>>>>>>>>>> We clearly want the version number, but we probably also want > >>>>>>>>>>> > >>>>>>>>>> to > >>> > >>>> know> >> if > >>>>>>>>>>> this is an official release, or a random SNAPSHOT from a > >>>>>>>>>>> > >>>>>>>>>> branch. > >>> > >>>> If> >> this > >>>>>>>>>>> is a release candidate, we probably want the RC number as > >>>>>>>>>>> > >>>>>>>>>> well, > >> > >>> like> >>>>> "1.1-rc3" We also want a git hash. This can be > >>>>>>>>>>> > >>>>>>>>>> injected by the> > build > >>>>> > >>>>>> process. In the case of an official release, where the source > >>>>>>>>>>> code> >> is not > >>>>>>>>>>> under git, we can pull it from a file. > >>>>>>>>>>> > >>>>>>>>>>> For example, hadoop's version output looks like this: > >>>>>>>>>>> > >>>>>>>>>>> > cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop > >>>>>>>>>>> > version> >>>>> > Hadoop 2.8.3 > >>>>>>>>>>> > Subversion > >>>>>>>>>>> > https://git-wip-us.apache.org/repos/asf/hadoop.git -r> > >>>>>>>>>>> > >>>>>>>>>> b3fe56402d908019d99af1f1f4fc65cb1d1436a2 > >>>>> > >>>>>> > Compiled by jdu on 2017-12-05T03:43Z > >>>>>>>>>>> > Compiled with protoc 2.5.0 > >>>>>>>>>>> > From source with checksum 9ff4856d824e983fa510d3f843e3f1 > >>>>>>>>>>> > >>>>>>>>>> 9d> > >>> > >>>> > This command was run using /home/cmccabe/Downloads/ > >>>>>>>>>> > >>>>>>>>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar > >>>>>>>>>>> > >>>>>>>>>>> (The "subversion" line here is a little weird -- it now refers > >>>>>>>>>>> to> >> git, not > >>>>>>>>>>> svn) > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hey Sasaki, > >>>>>>>>>>>> > >>>>>>>>>>>> Yeah, I don't feel too strongly about only supporting -- > >>>>>>>>>>>> version. I> >> agree > >>>>>>>>>>>> > >>>>>>>>>>> it > >>>>>>>>>>> > >>>>>>>>>>>> may help discoverability given the current approach. On the > >>>>>>>>>>>> other> >> hand, > >>>>>>>>>>>> > >>>>>>>>>>> if > >>>>>>>>>>> > >>>>>>>>>>>> we refactored all of the tools so that we could use a common > >>>>>>>>>>>> set of> >> base > >>>>>>>>>>>> options, it might be a little annoying to have to continue > >>>>>>>>>>>> > >>>>>>>>>>> supporting > >>>>>>> > >>>>>>>> both > >>>>>>>>>>> > >>>>>>>>>>>> variations. For example, tool standardization was proposed in > >>>>>>>>>>>> KIP-14> >> and > >>>>>>>>>>>> I'm still holding out hope that someone will have time to > >>>>>>>>>>>> > >>>>>>>>>>> pick > >> > >>> this> >> work > >>>>>>>>>>>> back up. It's always easier to add an option than remove one, > >>>>>>>>>>>> so I'm> >>>>>> slightly inclined to have only --version for > >>>>>>>>>>>> > >>>>>>>>>>> now. > >>> > >>>> What do you > >>>>> > >>>>>> think?> >>>>> The double dash version is more consistent with > >>>>>>>>>>>> > >>>>>>>>>>> how > >>> > >>>> our other > >>>>> > >>>>>> flags> >> work. > >>>>>>>>>>> In general, I feel that if --version is supported, --help > >>>>>>>>>>> > >>>>>>>>>> should > >>> > >>>> say> >> so. > >>>>>>>>>>> best, > >>>>>>>>>>> Colin > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>>> Jason > >>>>>>>>>>>> > >>>>>>>>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru < > >>>>>>>>>>>> > >>>>>>>>>>> sasaki...@oss.nttdata.com > >>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Hi Jason > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thank you for helpful comments. I updated wiki based on your > >>>>>>>>>>>>> > >>>>>>>>>>>> advice. > >>>>>>> > >>>>>>>> I thought this option was relatively common and making > >>>>>>>>>>>>> maintenance> >>>> easy > >>>>>>>>>>>>> was also important. > >>>>>>>>>>>>> However, as you said, it is not good that version option > >>>>>>>>>>>>> > >>>>>>>>>>>> won't > >>> > >>>> be> >>>>> shown up > >>>>>>>>>>>>> in help description. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I thought accepting both single-dash and double-dash will > >>>>>>>>>>>>> > >>>>>>>>>>>> help > >>> > >>>> to> >> find > >>>>>>>>>>>>> this option. > >>>>>>>>>>>>> In my approach this option won't be showed, but most of > >>>>>>>>>>>>> software> >> which > >>>>>>>>>>>>> > >>>>>>>>>>>> has > >>>>>>>>>>> > >>>>>>>>>>>> this option accepts either single-dash or double-dash. > >>>>>>>>>>>>> I guess it doesn't need to support both if we take another > >>>>>>>>>>>>> way.> >>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks > >>>>>>>>>>>>> > >>>>>>>>>>>>> @Ted Yeah, you're right. Sorry about the confusion. > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Since we're here, I think this KIP is a nice improvement. > >>>>>>>>>>>>>> It's> >>>>> definitely > >>>>>>>>>>>>>> nice to have an easy way to check the version. That said, > >>>>>>>>>>>>>> > >>>>>>>>>>>>> do > >> > >>> we> >>>> really > >>>>>>>>>>>>>> need > >>>>>>>>>>>>>> to support both `-version` and `--version`? The latter is > >>>>>>>>>>>>>> > >>>>>>>>>>>>> consistent > >>>>>>>> > >>>>>>>>> with > >>>>>>>>>>> > >>>>>>>>>>>> our current tools. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Also, I think the approach we've taken is basically to > >>>>>>>>>>>>>> > >>>>>>>>>>>>> build > >> > >>> the> >>>>> --version > >>>>>>>>>>>>>> functionality into the bash script. This is nice because it > >>>>>>>>>>>>>> saves> > a > >>>>>>>>>>>>>> > >>>>>>>>>>>>> lot of > >>>>>>>>>>> > >>>>>>>>>>>> work to update the commands individually and we don't need > >>>>>>>>>>>>>> > >>>>>>>>>>>>> to > >>> > >>>> do> >>>>> anything > >>>>>>>>>>>>>> when we add new tools. The downside is that `--version` > >>>>>>>>>>>>>> > >>>>>>>>>>>>> won't > >>> > >>>> show> >> up > >>>>>>>>>>>>>> > >>>>>>>>>>>>> as > >>>>>>>>>>> > >>>>>>>>>>>> an > >>>>>>>>>>>>>> option in any of the --help output. Not sure if that is too > >>>>>>>>>>>>>> big of> >> a > >>>>>>>>>>>>>> problem, but maybe worth mentioning this in the rejected > >>>>>>>>>>>>>> > >>>>>>>>>>>>> alternatives > >>>>>>>> > >>>>>>>>> section. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -Jason > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu <yuzhih...@gmail.com > >>>>>>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>> > >>>>>>>> Jason: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Maybe your reply was intended for another KIP ? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> KIP-278 is about adding version option, not timeout. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Cheers > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason Gustafson < > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> ja...@confluent.io> > >>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi Sasaki, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks for the KIP. I think the timeout controls the > >>>>>>>>>>>>>>>> maximum> >>>> allowed > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> time > >>>>>>>>>>>>>>> that the consumer will block for the next record. Maybe > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> the> > >>> > >>>> meaning > >>>>>>> > >>>>>>>> would > >>>>>>>>>>>>>>> be clearer with the more concise name `--timeout`? That > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> also > >>> > >>>> fits> >>>>> with > >>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> old consumer which overrides the `consumer.timeout.ms` > >>>>>>>>>>>>>>>> property.> >>>>>>>>>> > >>>>>>>>>>>>>>>> By the way, it seems like the default value was > >>>>>>>>>>>>>>>> intentionally> > set > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> low > >>>>>>>>>>> > >>>>>>>>>>>> for > >>>>>>>>>>>>>>> both the old and new consumers, but I'm not sure of the > >>>>>>>>>>>>>>> reason.> > We > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> could > >>>>>>>>>>> > >>>>>>>>>>>> leave the default as it is if we want to be safe, but > >>>>>>>>>>>>>>>> increasing> >> it > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> seems > >>>>>>>>>>>>>>> ok to me. Perhaps we could start a little lower, though, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> say > >>> > >>>> 10> >>>>> seconds? > >>>>>>>>>>>>>>> In > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> any case, we should make it clear to the user that the > >>>>>>>>>>>>>>>> timeout> >> was > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> reached. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> It's surprising to see only the incomplete reported > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> results> > >>> > >>>> following a > >>>>>>>>>> > >>>>>>>>>>> timeout. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Jason > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM, Sasaki Toru < > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> sasaki...@oss.nttdata.com> > >>>>>>>>>>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hello everyone, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I would like to start a discussion for KIP 278. Cloud > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> you> > > >>> > >>>> please > >>>>> > >>>>>> give > >>>>>>>>>>> > >>>>>>>>>>>> comments and advice ? > >>>>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP- > >>>>>>>>>>>>>>>>> 278+-> >>>>>>>>>>> +Add+version+option+to+Kafka%2 > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> 7s+commands> > >>>>> > >>>>>> JIRA ticket and Pull Request are bellow: > >>>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061> > >>>>>>>>>>>>>>>>> <https://github.com/apache/kafka/pull/639> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Many thanks, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Sasaki > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA > >>>>>>>>>>>>>>>>> CORPORATION> >>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION > >>>>>>>>> > >>>>>>>>> -- > >>>>>> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION > >>>>>> > >>>>>> > >>>>> > > -- > > Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION > > > >
