-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Thanks for the KIP Sang!
I have a couple of more comments about the wiki page: (1) The "Public Interface" section should only list the new stuff. This KIP does not change anything with regard to the existing options `--input-topic` or `--intermediate-topic` and thus it's just "noise" to have them in this section. Only list the new option `allInputTopicsOption`. (2) Don't post code, ie, the implementation of private methods. KIPs should only describe public interface changes. (3) The KIP should describe that we intend to use `describeConsumerGroups` calls to discover the topic names -- atm, it's unclear from the KIP how the new feature actually works. (4) If the new flag is used, we will discover input and intermediate topics. Hence, the name is miss leading. We could call it `--all-user-topics` and explain in the description that "user topics" are input and intermediate topics for this case (in general, also output topics are "user topics" but there is nothing to be done for output topics). Thoughts? - -Matthias On 1/27/20 6:35 AM, Sang wn Lee wrote: > thank you John Roesle > > It is a good idea "—all-input-topics" > > I agree with you > > I'll update right away > > > On 2020/01/24 14:14:17, "John Roesler" <vvcep...@apache.org> > wrote: > >> Hi all, thanks for the explanation. I was also not sure how the >> kip would be possible to implement. >> >> No that it does seem plausible, my only feedback is that the >> command line option could align better with the existing one. >> That is, the existing option is called “—input-topics”, so it >> seems like the new one should be called “—all-input-topics”. >> >> Thanks, John >> >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote: >>> Thanks Sophie for the explanation! I read Sang's PR and >>> basically he did exactly what you proposed (check it here >>> <https://github.com/apache/kafka/pull/7948/files> in case I'm >>> wrong). >>> >>> I think Sophie's response answers Gwen's question already, >>> while in the meantime for a KIP itself we are not required to >>> mention all the internal details about how to make the changes >>> happen (like how to actually get the external topics), >>> considering the change scope is pretty small as well. But >>> again, it would do no harm if we mention it inside Proposed >>> Change session specifically so that people won't get confused >>> about how. >>> >>> >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman >>> <sop...@confluent.io> wrote: >>> >>>> Hi all, >>>> >>>> I think what Gwen is trying to ask (correct me if I'm wrong) >>>> is how we can infer which topics are associated with Streams >>>> from the admin client's topic list. I agree that this >>>> doesn't seem possible, since as she pointed out the topics >>>> list (or even description) lacks the specific information we >>>> need. >>>> >>>> What we could do instead is use the admin client's >>>> `describeConsumerGroups` API to get the information on the >>>> Streams app's consumer group specifically -- note that the >>>> Streams application.id config is also used as the consumer >>>> group id, so each app forms a group to read from the input >>>> topics. We could compile a list of these topics just by >>>> looking at each member's assignment (and even check for a >>>> StreamsPartitionAssignor to verify that this is indeed a >>>> Streams app group, if we're being paranoid). >>>> >>>> The reset tool actually already gets the consumer group >>>> description, in order to validate there are no active >>>> consumers in the group. We may as well grab the list of >>>> topics from it while it's there. Or did you have something >>>> else in mind? >>>> >>>> On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee >>>> <ssangdd...@gmail.com> wrote: >>>> >>>>> Thank you >>>>> >>>>> I understand you >>>>> >>>>> 1. admin client has topic list 2. applicationId can only >>>>> have one stream, so It won't be a problem! 3. For example, >>>>> --input-topic [reg] Allowing reg solves some inconvenience >>>>> >>>>> >>>>> On 2020/01/18 18:15:23, Gwen Shapira <g...@confluent.io> >>>>> wrote: >>>>>> I am not sure I follow. Afaik: >>>>>> >>>>>> 1. Topics don't include client ID information 2. Even if >>>>>> you did, the same ID could be used for topics that are >>>>>> not >>>>> Kafka >>>>>> Streams input >>>>>> >>>>>> The regex idea sounds doable, but I'm not sure it solves >>>>>> much? >>>>>> >>>>>> >>>>>> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee >>>>>> <ssangdd...@gmail.com> >>>> wrote: >>>>>> >>>>>>> Thank you Gwen Shapira! We'll add a flag to clear all >>>>>>> topics by clientId It is ‘reset-all-external-topics’ >>>>>>> >>>>>>> I also want to use regex on the input topic flag to >>>>>>> clear all >>>> matching >>>>>>> topics. >>>>>>> >>>>>>> On 2020/01/17 19:29:09, Gwen Shapira >>>>>>> <g...@confluent.io> wrote: >>>>>>>> Seem like a very nice improvement to me. But I have >>>>>>>> to admit that I don't understand how this will how - >>>>>>>> how could you infer the input topics? >>>>>>>> >>>>>>>> On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee >>>>>>>> <ssangdd...@gmail.com >>>>> >>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Starting this thread to discuss KIP-560: wiki link >>>>>>>>> : >>>>>>>>> >>>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+i nfer+external+topic+partitions+in+stream+reset+tool >>>>>>>>> >>>>>>>>> >>>> I'm >>>> newbie >>>>>>>>> I would like to receive feedback on the following >>>>>>>>> features! >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEI8mthP+5zxXZZdDSO4miYXKq/OgFAl5j/4sACgkQO4miYXKq /Og4Yw/+LwLEcesv7r9JCVWhtgSe0iDrDFs3vA5p9uCzHfS/e7rm33hvGaaRSikC A5yfayLi/88E6VzZWlY65ry8JPprCxepbalQHPUSBfcx+jwShHQOD7VwDavnAu2s eKhmbstjtrNPtIdzdL0ine88f47ubgQpFbu5sbTFtGPGAkLuCD/QIo+tAghcVNil ZTY/Pw602rH/CI9+jt3lenrm54Cju99bruslIhoP4eBUtiWthns2QwfGPkEOEyv6 Oa0fm8CXJEPtvp6idCMJ9in7GlZvNsSmbHTGKWJtoTx89fdnZNjaZuFdtRK+sNE6 w+fYtQnY7VrLAkQ/zcLgFQE2dqNSR0ZAiC2VjmMoS9Qg5UKJOpmtI0CvIAJ2PVVY o0OnLo41wcG5W7SsumOvtwP38YnflnPjnLR0cZg74R7TVT4VmaKku9unRhUftipQ m8aL4dNfcweB7CqhXthIABCsHUPj3VgA5HP18KmZqnCN5k4JXkJv6zU8ePm3kWGx NowG0gf8Hiy5WbAt/HC2v53xYjjlrf1uI978NWeLqJ7Oi3K4rOi6rhg2IGGl23ym eYt2ElUu+aIU0SB54lHtjOPbXLRKLdhhx5Zrsm5xKU3kKHGTg7kNR1aLE3LeR+M7 i/ux5jA/tZ+l89eRyOCsnEyzTVSZC9f7zjV+acQdR08gBVeXKFE= =LCV/ -----END PGP SIGNATURE-----
