No worry! Could you address Matthias' comments in this mailing thread? Seems we still have some gaps.
On Mon, Jul 6, 2020 at 7:14 AM Sang wn Lee <ssangdd...@gmail.com> wrote: > I'm sorry. > I just modified the KIP! > > On 2020/03/07 20:09:47, "Matthias J. Sax" <mj...@apache.org> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA512 > > > > Thanks for the KIP Sang! > > > > I have a couple of more comments about the wiki page: > > > > (1) The "Public Interface" section should only list the new stuff. This > > KIP does not change anything with regard to the existing options > > `--input-topic` or `--intermediate-topic` and thus it's just "noise" to > > have them in this section. Only list the new option > > `allInputTopicsOption`. > > > > (2) Don't post code, ie, the implementation of private methods. KIPs > > should only describe public interface changes. > > > > (3) The KIP should describe that we intend to use > > `describeConsumerGroups` calls to discover the topic names -- atm, it's > > unclear from the KIP how the new feature actually works. > > > > (4) If the new flag is used, we will discover input and intermediate > > topics. Hence, the name is miss leading. We could call it > > `--all-user-topics` and explain in the description that "user topics" > > are input and intermediate topics for this case (in general, also output > > topics are "user topics" but there is nothing to be done for output > > topics). Thoughts? > > > > > > - -Matthias > > > > > > On 1/27/20 6:35 AM, Sang wn Lee wrote: > > > thank you John Roesle > > > > > > It is a good idea "—all-input-topics" > > > > > > I agree with you > > > > > > I'll update right away > > > > > > > > > On 2020/01/24 14:14:17, "John Roesler" <vvcep...@apache.org> > > > wrote: > > > > > >> Hi all, thanks for the explanation. I was also not sure how the > > >> kip would be possible to implement. > > >> > > >> No that it does seem plausible, my only feedback is that the > > >> command line option could align better with the existing one. > > >> That is, the existing option is called “—input-topics”, so it > > >> seems like the new one should be called “—all-input-topics”. > > >> > > >> Thanks, John > > >> > > >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote: > > >>> Thanks Sophie for the explanation! I read Sang's PR and > > >>> basically he did exactly what you proposed (check it here > > >>> <https://github.com/apache/kafka/pull/7948/files> in case I'm > > >>> wrong). > > >>> > > >>> I think Sophie's response answers Gwen's question already, > > >>> while in the meantime for a KIP itself we are not required to > > >>> mention all the internal details about how to make the changes > > >>> happen (like how to actually get the external topics), > > >>> considering the change scope is pretty small as well. But > > >>> again, it would do no harm if we mention it inside Proposed > > >>> Change session specifically so that people won't get confused > > >>> about how. > > >>> > > >>> > > >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman > > >>> <sop...@confluent.io> wrote: > > >>> > > >>>> Hi all, > > >>>> > > >>>> I think what Gwen is trying to ask (correct me if I'm wrong) > > >>>> is how we can infer which topics are associated with Streams > > >>>> from the admin client's topic list. I agree that this > > >>>> doesn't seem possible, since as she pointed out the topics > > >>>> list (or even description) lacks the specific information we > > >>>> need. > > >>>> > > >>>> What we could do instead is use the admin client's > > >>>> `describeConsumerGroups` API to get the information on the > > >>>> Streams app's consumer group specifically -- note that the > > >>>> Streams application.id config is also used as the consumer > > >>>> group id, so each app forms a group to read from the input > > >>>> topics. We could compile a list of these topics just by > > >>>> looking at each member's assignment (and even check for a > > >>>> StreamsPartitionAssignor to verify that this is indeed a > > >>>> Streams app group, if we're being paranoid). > > >>>> > > >>>> The reset tool actually already gets the consumer group > > >>>> description, in order to validate there are no active > > >>>> consumers in the group. We may as well grab the list of > > >>>> topics from it while it's there. Or did you have something > > >>>> else in mind? > > >>>> > > >>>> On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee > > >>>> <ssangdd...@gmail.com> wrote: > > >>>> > > >>>>> Thank you > > >>>>> > > >>>>> I understand you > > >>>>> > > >>>>> 1. admin client has topic list 2. applicationId can only > > >>>>> have one stream, so It won't be a problem! 3. For example, > > >>>>> --input-topic [reg] Allowing reg solves some inconvenience > > >>>>> > > >>>>> > > >>>>> On 2020/01/18 18:15:23, Gwen Shapira <g...@confluent.io> > > >>>>> wrote: > > >>>>>> I am not sure I follow. Afaik: > > >>>>>> > > >>>>>> 1. Topics don't include client ID information 2. Even if > > >>>>>> you did, the same ID could be used for topics that are > > >>>>>> not > > >>>>> Kafka > > >>>>>> Streams input > > >>>>>> > > >>>>>> The regex idea sounds doable, but I'm not sure it solves > > >>>>>> much? > > >>>>>> > > >>>>>> > > >>>>>> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee > > >>>>>> <ssangdd...@gmail.com> > > >>>> wrote: > > >>>>>> > > >>>>>>> Thank you Gwen Shapira! We'll add a flag to clear all > > >>>>>>> topics by clientId It is ‘reset-all-external-topics’ > > >>>>>>> > > >>>>>>> I also want to use regex on the input topic flag to > > >>>>>>> clear all > > >>>> matching > > >>>>>>> topics. > > >>>>>>> > > >>>>>>> On 2020/01/17 19:29:09, Gwen Shapira > > >>>>>>> <g...@confluent.io> wrote: > > >>>>>>>> Seem like a very nice improvement to me. But I have > > >>>>>>>> to admit that I don't understand how this will how - > > >>>>>>>> how could you infer the input topics? > > >>>>>>>> > > >>>>>>>> On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee > > >>>>>>>> <ssangdd...@gmail.com > > >>>>> > > >>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>> Hello, > > >>>>>>>>> > > >>>>>>>>> Starting this thread to discuss KIP-560: wiki link > > >>>>>>>>> : > > >>>>>>>>> > > >>>>>>> > > >>>>> > > >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+i > > nfer+external+topic+partitions+in+stream+reset+tool > > >>>>>>>>> > > >>>>>>>>> > > >>>> > > I'm > > >>>> > > newbie > > >>>>>>>>> I would like to receive feedback on the following > > >>>>>>>>> features! > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > -----BEGIN PGP SIGNATURE----- > > > > iQIzBAEBCgAdFiEEI8mthP+5zxXZZdDSO4miYXKq/OgFAl5j/4sACgkQO4miYXKq > > /Og4Yw/+LwLEcesv7r9JCVWhtgSe0iDrDFs3vA5p9uCzHfS/e7rm33hvGaaRSikC > > A5yfayLi/88E6VzZWlY65ry8JPprCxepbalQHPUSBfcx+jwShHQOD7VwDavnAu2s > > eKhmbstjtrNPtIdzdL0ine88f47ubgQpFbu5sbTFtGPGAkLuCD/QIo+tAghcVNil > > ZTY/Pw602rH/CI9+jt3lenrm54Cju99bruslIhoP4eBUtiWthns2QwfGPkEOEyv6 > > Oa0fm8CXJEPtvp6idCMJ9in7GlZvNsSmbHTGKWJtoTx89fdnZNjaZuFdtRK+sNE6 > > w+fYtQnY7VrLAkQ/zcLgFQE2dqNSR0ZAiC2VjmMoS9Qg5UKJOpmtI0CvIAJ2PVVY > > o0OnLo41wcG5W7SsumOvtwP38YnflnPjnLR0cZg74R7TVT4VmaKku9unRhUftipQ > > m8aL4dNfcweB7CqhXthIABCsHUPj3VgA5HP18KmZqnCN5k4JXkJv6zU8ePm3kWGx > > NowG0gf8Hiy5WbAt/HC2v53xYjjlrf1uI978NWeLqJ7Oi3K4rOi6rhg2IGGl23ym > > eYt2ElUu+aIU0SB54lHtjOPbXLRKLdhhx5Zrsm5xKU3kKHGTg7kNR1aLE3LeR+M7 > > i/ux5jA/tZ+l89eRyOCsnEyzTVSZC9f7zjV+acQdR08gBVeXKFE= > > =LCV/ > > -----END PGP SIGNATURE----- > > >