Hi Matthias, Bill,
I've updated the KIP with your last feedbacks. However, you have suggested
to rename : `Suppressed#withName(String)`
withName is not a static method like Joined.named was. withName method is
part of the new interface NameOperation.
In addition, I've split the PR in 5 commits so that it will be much easier
to review.
Thanks
Le jeu. 21 févr. 2019 à 23:54, Bill Bejeck <bbej...@gmail.com> a écrit :
> Hi Florian,
>
> Overall the KIP LGTM. Once you've addressed the final comments from
> Matthias I think we can put this up for a vote.
>
> Thanks,
> Bill
>
> On Wed, Feb 20, 2019 at 9:42 PM Matthias J. Sax <matth...@confluent.io>
> wrote:
>
> > Florian,
> >
> > thanks for updating the KIP (and no worries for late reply -- 2.2
> > release kept us busy anyway...). Overall LGTM.
> >
> > Just some nits:
> >
> >
> > KStream-Table:
> >
> > Do we need to list the existing stream-globalTable join methods in the
> > first table (thought it should only contain new/changing methods).
> >
> > typo: `join(GlobalKTbale, KeyValueMapper, ValueJoiner, Named)`
> >
> > `leftJoin(GlobalKTable, KeyValueMapper, ValueJoiner)` is missing the new
> > `Named` parameter.
> >
> > `static Joined#named(final String name)`
> > -> should be `#as(...)` instead of `named(...)`
> >
> > flatTransform() is missing (cf. KIP-313)
> >
> >
> >
> > KTable-table:
> >
> > `Suppressed#withName(String)`
> > -> should we change this to `#as(...)` too (similar to `named()`)
> >
> >
> >
> > -Matthias
> >
> >
> >
> > On 1/25/19 9:49 AM, Matthias J. Sax wrote:
> > > I was reading the KIP again, and there are still some open question and
> > > inconsistencies:
> > >
> > > For example for `KGroupedStream#count(Named)` the KIP says, that only
> > > the processor will be named, while the state store name will be `PREFIX
> > > + COUNT` (ie, an auto-generated name). Additionally, for
> > > `KGroupedStream#count(Named, Materialized)` the processor will be named
> > > according to `Named` and the store will be named according to
> > > `Materialized.as()`. So far so good. It implies that naming the
> > > processor and naming the store are independent. (This pattern is
> applied
> > > to all aggregation functions, for KStream and KTable).
> > >
> > > However, for `KTable#filter(Predicate, Named)` the KIP says, the
> > > processor name and the store name are set. This sound wrong (ie,
> > > inconsistent with the first paragraph from above), because there is
> also
> > > `KTable#filter(Predicate, Named, Materialized)`. Also note, for the
> > > first operator, the store might not be materialized to at all. (This
> > > issue is there for all KTable operators -- stateless and stateful).
> > >
> > > Finally, there is the following statement in the KIP:
> > >
> > >> Also, note that for all methods accepting a Materialized argument, if
> > no state store named is provided then the node named will be used to
> > generate a one. The state store name will be the node name suffixed with
> > "-table".
> > >
> > >
> > > This contradict the non-naming of stores from the very beginning.
> > >
> > >
> > > Also, the KIP still contains the question about `join(GlobalKTable,
> > > KeyValueMapper, ValueJoiner)` and `leftJoin(GlobalKTable,
> > > KeyValueMapper, ValueJoiner)`. I think a consistent approach would be
> to
> > > add one overload each that takes a `Named` parameter.
> > >
> > >
> > > Thoughts?
> > >
> > >
> > > -Matthias
> > >
> > >
> > > On 1/17/19 2:56 PM, Bill Bejeck wrote:
> > >> +1 for me on Guozhang's proposal for changes to Joined.
> > >>
> > >> Thanks,
> > >> Bill
> > >>
> > >> On Thu, Jan 17, 2019 at 5:55 PM Matthias J. Sax <
> matth...@confluent.io>
> > >> wrote:
> > >>
> > >>> Thanks for all the follow up comments!
> > >>>
> > >>> As I mentioned earlier, I am ok with adding overloads instead of
> using
> > >>> Materialized to specify the processor name. Seems this is what the
> > >>> majority of people prefers.
> > >>>
> > >>> I am also +1 on Guozhang's suggestion to deprecate `static
> > >>> Joined#named()` and replace it with `static Joined#as` for
> consistency
> > >>> and to deprecate getter `Joined#name()` for removal and introduce
> > >>> `JoinedInternal` to access the name.
> > >>>
> > >>> @Guozhang: the vote is already up :)
> > >>>
> > >>>
> > >>> -Matthias
> > >>>
> > >>> On 1/17/19 2:45 PM, Guozhang Wang wrote:
> > >>>> Wow that's a lot of discussions in 6 days! :) Just catching up and
> > >>> sharing
> > >>>> my two cents here:
> > >>>>
> > >>>> 1. Materialized: I'm inclined to not let Materialized extending
> Named
> > and
> > >>>> add the overload as well. All the rationales have been very well
> > >>> summarized
> > >>>> before. Just to emphasize on John's points: Materialized is
> > considered as
> > >>>> the control object being leveraged by the optimization framework to
> > >>>> determine if the state store should be physically materialized or
> > not. So
> > >>>> let's say if the user does not want to query the store (hence it can
> > just
> > >>>> be locally materialized), but still want to name the processor, they
> > need
> > >>>> to do either
> > "count(Materialized.as(null).withName("processorName"));" or
> > >>>> "count(Named.as("processorName"));" and neither of it is a bit hard
> to
> > >>>> educate to users, and hence it looks that an overload function with
> > two
> > >>>> parameters are easier to understand.
> > >>>>
> > >>>> 2. As for `NamedOperation`: I've left a comment about it before,
> i.e.
> > "1)
> > >>>> Regarding the interface / function name, I'd propose we call the
> > >>> interface
> > >>>> `NamedOperation` which would be implemented by Produced / Consumed /
> > >>>> Printed / Joined / Grouped / Suppressed (note I intentionally
> exclude
> > >>>> Materialized here since its semantics is quite), and have the
> default
> > >>> class
> > >>>> that implements `NamedOperation` as `Named`, which would be used in
> > our
> > >>>> adding overload functions. The main reason is to have consistency in
> > >>>> naming." And I think I'm on the same page with John with his more
> > >>> detailed
> > >>>> proposal.
> > >>>>
> > >>>> 3. As for `Joined`: I actually would suggest we bite the bullet and
> > >>> remove
> > >>>> it as well, because we are trying to fix some inconsistencies in
> this
> > KIP
> > >>>> anyways (or is that not agreed upon yet?), my thoughts were that we
> > will
> > >>>> have the following breaking renamings as below:
> > >>>>
> > >>>> 3.a) static Joined#named() -> Joined#as()
> > >>>> 3.b) Joined#name() -> "deleted"
> > >>>>
> > >>>>
> > >>>> I also think that we can start the voting thread asap since we are
> > >>>> achieving to an consensus and the KIP deadline is approaching. The
> > wiki
> > >>>> page itself may still need to be updated though with the API
> breaking
> > >>>> changes above.
> > >>>>
> > >>>>
> > >>>> Guozhang
> > >>>>
> > >>>>
> > >>>> On Thu, Jan 17, 2019 at 1:43 PM Florian Hussonnois <
> > >>> fhussonn...@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Sorry, I've sent my previous mail to quickly. Unlike the Consumed,
> > >>> Produced
> > >>>>> and Grouped classes, the Joined class does have getter methods. So
> I
> > >>>>> propose to keep the name() method only for this class.
> > >>>>> For other classes the name will be accessible through XXXInternal
> > >>> classes.
> > >>>>>
> > >>>>> Le jeu. 17 janv. 2019 à 22:39, John Roesler <j...@confluent.io> a
> > >>> écrit :
> > >>>>>
> > >>>>>> Just to chime in regarding NamedInternal. That was my bad mental
> > model
> > >>> to
> > >>>>>> blame. It is indeed coercion, not casting. Even more relevant, I'm
> > not
> > >>> a
> > >>>>>> fan of the XInternal pattern, but it is the pattern we have. It
> > would
> > >>> be
> > >>>>>> worse to start carving out exceptions.
> > >>>>>>
> > >>>>>> So I agree that we should have:
> > >>>>>> * `NamedOperation` interface, declaring only the
> `withName(String)`
> > >>>>> setter
> > >>>>>> member
> > >>>>>> * `Named implements NamedOperation` class with a protected `name`
> > >>> field,
> > >>>>>> set by the `withName` setter (and also other config objects would
> do
> > >>> the
> > >>>>>> same, e.g., `Grouped implements NamedOperation`)
> > >>>>>> * `NamedInternal extends Named` class with a public (but
> internally
> > >>>>>> targeted) `name()` getter to expose the name to the topology
> > builder.
> > >>>>>> Likewise all the other config classes that implement
> NamedOperation
> > >>> would
> > >>>>>> expose a `name()` getter for the same purpose. It's not in the
> > public
> > >>>>> API,
> > >>>>>> but we should take care to make sure the getter method has the
> same
> > >>> name
> > >>>>>> everywhere for minimum confusion.
> > >>>>>>
> > >>>>>> Thanks, everyone!
> > >>>>>> -John
> > >>>>>>
> > >>>>>> On Thu, Jan 17, 2019 at 12:09 PM Bill Bejeck <bbej...@gmail.com>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> I'm getting caught up with the current state of this KIP.
> > >>>>>>>
> > >>>>>>> I agree that the question on what to do with overloads is a
> > difficult
> > >>>>> one
> > >>>>>>> to answer.
> > >>>>>>>
> > >>>>>>> Both John and Matthias have laid out their thoughts thoroughly,
> and
> > >>> the
> > >>>>>>> points made by both resonate with me.
> > >>>>>>>
> > >>>>>>> I've spent some time thinking about this, and while I have a
> > problem
> > >>>>> with
> > >>>>>>> adding overloaded methods, I can't quite get comfortable with the
> > >>>>> notion
> > >>>>>> of
> > >>>>>>> Materialized naming the processing node. For me, it comes down
> to
> > the
> > >>>>>> fact
> > >>>>>>> that Materialized is used to configure the state store for an
> > >>>>> individual
> > >>>>>>> processing node and knows nothing of the operation itself. So
> I'll
> > go
> > >>>>>> with
> > >>>>>>> adding the Named overload to methods taking a Materialized by a
> > narrow
> > >>>>>>> margin.
> > >>>>>>>
> > >>>>>>> As for the name method, I agree with Matthias that it's not
> > consistent
> > >>>>>> with
> > >>>>>>> the approach we've taken so far whether for better or worse, but
> to
> > >>>>> quote
> > >>>>>>> Matthias, "that ship has sailed." IMHO adding the method for
> > making
> > >>>>>>> testing easier doesn't justify it, as there are ways to get the
> > name
> > >>>>> via
> > >>>>>>> NamedInternal class.
> > >>>>>>>
> > >>>>>>> Just my 2 cents.
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Bill
> > >>>>>>>
> > >>>>>>> On Wed, Jan 16, 2019 at 5:40 PM Matthias J. Sax <
> > >>> matth...@confluent.io
> > >>>>>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Thanks for the details John.
> > >>>>>>>>
> > >>>>>>>> While I understand your argument that it is no optimal to use
> > >>>>>>>> `Materialized` to set the processor name, I still slightly
> prefer
> > >>>>> this
> > >>>>>>>> option, because adding more overloads seems to be even worse to
> > me.
> > >>>>>>>>
> > >>>>>>>> But I would also not block this KIP if the majority of people
> > prefer
> > >>>>> to
> > >>>>>>>> add overloads instead of extending `Materialized`.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> However, I cannot follow your argument about
> > `NamedOperation#name()`
> > >>>>>>>> getter method. So far, all configuration classes don't have
> > getters
> > >>>>> and
> > >>>>>>>> it seems to be inconsistent to add a single one now. We also
> don't
> > >>>>> need
> > >>>>>>>> any cast IMHO, as we would use the same construct as we do for
> all
> > >>>>>> other
> > >>>>>>>> config classed via `NamedInternal` to access the name:
> > >>>>>>>>
> > >>>>>>>>> final String name = new NamedInternal(named).name();
> > >>>>>>>>
> > >>>>>>>> Maybe, it would have been better to add getters from the
> > beginning on
> > >>>>>>>> (even if I think it was the right decision to not add getters).
> > >>>>>> However,
> > >>>>>>>> this ship have sailed and if we want to add getters to avoid the
> > >>>>>>>> `XxxInternal()` construct, we should do it for all classes --
> > >>>>> however,
> > >>>>>>>> what would a user gain if we do this? It would just be a lot of
> > >>>>> "noise"
> > >>>>>>>> IMHO.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> @Florian: I would suggest to start a VOTE if you want to get
> this
> > >>>>> into
> > >>>>>>>> 2.2 release. The open questions seem to be minor and I think we
> > can
> > >>>>>>>> resolve them in parallel to the vote.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> -Matthias
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On 1/16/19 12:59 PM, John Roesler wrote:
> > >>>>>>>>> Hi Matthias,
> > >>>>>>>>>
> > >>>>>>>>> One thing that we discussed earlier was to avoid creating
> > ambiguity
> > >>>>>> by
> > >>>>>>>>> conflating config objects that configure an operation (like
> > >>>>> Grouped)
> > >>>>>>> with
> > >>>>>>>>> config objects that configure an aspect of the operation (like
> > >>>>>>>>> Materialized).
> > >>>>>>>>>
> > >>>>>>>>> It is natural for the Grouped config to extend Named, as doing
> so
> > >>>>>>>> indicates
> > >>>>>>>>> that grouping operations can be named (I.e., the name applies
> to
> > >>>>> the
> > >>>>>>>>> operation itself, which in turn makes it reasonable to use the
> > >>>>>>>> operation's
> > >>>>>>>>> name as a component in the related processors' and topics'
> > names).
> > >>>>>>>>>
> > >>>>>>>>> But what would it mean for Materialized to extend Named?
> > >>>>> Materialized
> > >>>>>>>> only
> > >>>>>>>>> configures the materialization of an operation's result, not
> the
> > >>>>>>>> operation
> > >>>>>>>>> itself, so I guess it would mean the name applies to the result
> > of
> > >>>>>> the
> > >>>>>>>>> operation? It doesn't really work.
> > >>>>>>>>>
> > >>>>>>>>> Adding config objects to the DSL was an attempt to avoid
> overload
> > >>>>>> bloat
> > >>>>>>>> as
> > >>>>>>>>> more aspects of operations need to be configured.
> > >>>>>>>>> However, we made a mistake with Materialized, since (as noted)
> it
> > >>>>>>> doesn't
> > >>>>>>>>> configure the operation itself, but just one aspect of it.
> > >>>>>>>>> We basically bagged a bunch of parameters into one, without
> > solving
> > >>>>>> the
> > >>>>>>>>> problem structurally, and this is the result:
> > >>>>>>>>> As soon as we need to configure a *different* aspect of the
> > >>>>>> operation,
> > >>>>>>> we
> > >>>>>>>>> again need to add a new overload, and the cycle begins again.
> > >>>>>>>>>
> > >>>>>>>>> The proper solution here is to add an eponymous config object
> to
> > >>>>> each
> > >>>>>>>>> stateful operation, one which mixes in or composes the
> > Materialized
> > >>>>>>>> aspect
> > >>>>>>>>> config and the Named aspect config. But this is a large API
> > change,
> > >>>>>> and
> > >>>>>>>> we
> > >>>>>>>>> decided on the middle ground of just adding Named as an
> optional
> > >>>>>>>> parameter
> > >>>>>>>>> via new overloads for now.
> > >>>>>>>>>
> > >>>>>>>>> A similar compromise was to go ahead and add a Named overload
> > >>>>>> directly
> > >>>>>>> to
> > >>>>>>>>> all the operators that currently have no config object.
> > >>>>>>>>> Again, the proper thing would be to add a new config class for
> > each
> > >>>>>>>>> individual operation, but it seemed like a drastic change.
> > >>>>>>>>> We basically said that right now, we don't think we'll ever
> need
> > to
> > >>>>>>>>> configure another aspect of those operators than the name, and
> > >>>>> we're
> > >>>>>>>>> acknowledging that if we do, we'll have to created a small mess
> > to
> > >>>>>>> clean
> > >>>>>>>> up.
> > >>>>>>>>> It's really just a generalization of the same problem with
> > >>>>>> Materialized
> > >>>>>>>>> operations.
> > >>>>>>>>>
> > >>>>>>>>> To answer your question about the Named interface:
> > >>>>>>>>> The primary reason is that Named is an aspect that is meant to
> be
> > >>>>>> mixed
> > >>>>>>>> in
> > >>>>>>>>> with other config objects.
> > >>>>>>>>> For example, Grouped can extend Named.
> > >>>>>>>>> If we followed the pattern you've referenced, we would have a
> > >>>>> public
> > >>>>>>>>> interface Named with only the setter and a private class
> > >>>>>> NamedInternal
> > >>>>>>>> with
> > >>>>>>>>> the setter and getter.
> > >>>>>>>>> But would Grouped be a subclass of NamedInternal?
> > >>>>>>>>> Then, we could only have one kind of aspect mixin, since Java
> > >>>>> doesn't
> > >>>>>>>> have
> > >>>>>>>>> multiple class inheritance, or we'd have to decide if the next
> > >>>>> thing
> > >>>>>>>> should
> > >>>>>>>>> be a superclass of Named or a subclass of Named and a
> superclass
> > of
> > >>>>>>>> Grouped.
> > >>>>>>>>> Plus, in the implementation, instead of just casting Grouped to
> > >>>>>>>>> GroupedInternal (which is already unclean design), we'd also be
> > >>>>>> casting
> > >>>>>>>>> Grouped to NamedInternal, which is super confusing.
> > >>>>>>>>>
> > >>>>>>>>> It's far cleaner all around just to use the type system "the
> > right
> > >>>>>>> way",
> > >>>>>>>>> which is what we've proposed.
> > >>>>>>>>> Any config class can mix in the Named aspect, and it inherits a
> > >>>>>>> contract
> > >>>>>>>> to
> > >>>>>>>>> supply both the setter and the getter.
> > >>>>>>>>> Our implementation can actually avoid any casting in this
> usage,
> > >>>>>> since
> > >>>>>>> we
> > >>>>>>>>> can just call grouped.name() to get the name, instead of
> > something
> > >>>>>>> like
> > >>>>>>>>> ((NamedInternal) grouped).name().
> > >>>>>>>>>
> > >>>>>>>>> Plus, what harm does it do to let people get back the
> > configuration
> > >>>>>>>>> property that they *just set* on the config object?
> > >>>>>>>>> It doesn't break encapsulation.
> > >>>>>>>>> It would certainly make writing tests a lot easier for
> everyone.
> > >>>>>>>>>
> > >>>>>>>>> All around, I would advocate for moving toward this design for
> > all
> > >>>>>> the
> > >>>>>>>>> config interfaces, as I've previously demonstrated how we've
> made
> > >>>>> an
> > >>>>>>>>> intractable mess out of the window config hierarchy by trying
> to
> > be
> > >>>>>>>> clever
> > >>>>>>>>> and hiding the getters.
> > >>>>>>>>>
> > >>>>>>>>> I hope this helps,
> > >>>>>>>>> -John
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Jan 16, 2019 at 12:59 AM Matthias J. Sax <
> > >>>>>>> matth...@confluent.io>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> While I understand that it should be possible to specify store
> > >>>>> name
> > >>>>>>> and
> > >>>>>>>>>> processor name independent from each other, it's still unclear
> > to
> > >>>>>> me,
> > >>>>>>>>>> why we cannot use the `Materialized` parameter to specify the
> > >>>>>>> processor
> > >>>>>>>>>> name:
> > >>>>>>>>>>
> > >>>>>>>>>>> // only set the node name
> > >>>>>>>>>>> #count(Named.as("processorName"));
> > >>>>>>>>>>>
> > >>>>>>>>>>> // only set the store name
> > >>>>>>>>>>> #count(Materialized.as("storeName"));
> > >>>>>>>>>>>
> > >>>>>>>>>>> // set both
> > >>>>>>>>>>>
> #count(Materialized.as("storeName").withName("processorName"));
> > >>>>>>>>>>
> > >>>>>>>>>> This this case, it might be good to rename `withName` to
> > >>>>>>>>>> `withProcessorName` to avoid confusion with the store name.
> > >>>>>>>>>>
> > >>>>>>>>>> However, why do we need this:
> > >>>>>>>>>>
> > >>>>>>>>>>> #count(Materialized.as("storeName"),
> > Named.as("processorName"));
> > >>>>>>>>>>
> > >>>>>>>>>> I would prefer to not add this overload.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Strictly, we could also avoid `#count(Named)`, and set the
> > >>>>> processor
> > >>>>>>>>>> name only via:
> > >>>>>>>>>>
> > >>>>>>>>>>> #count(Materialized.as(null).withName("processorName"));
> > >>>>>>>>>>
> > >>>>>>>>>> I admit, it's a little clumsy, but would save us one more
> > >>>>> overload.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> One more comment that I forgot last time: why do we add the
> > getter
> > >>>>>>>>>> `Named#name()`? All other configuration classes only define
> > >>>>> setters
> > >>>>>>> and
> > >>>>>>>>>> we add getters only in the internal implementation.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> -Matthias
> > >>>>>>>>>>
> > >>>>>>>>>> On 1/13/19 4:22 AM, Florian Hussonnois wrote:
> > >>>>>>>>>>> Hi Matthias,
> > >>>>>>>>>>>
> > >>>>>>>>>>> The reason for overloading the methods with Materialized
> > >>>>> parameter
> > >>>>>> is
> > >>>>>>>>>>> regarding the semantic of this class.
> > >>>>>>>>>>> The Materialized class allow to name a queryable store. if a
> > name
> > >>>>>> is
> > >>>>>>>> set
> > >>>>>>>>>>> then it will be used both to name the state-store and the
> > >>>>>>>>>> changelog-topic.
> > >>>>>>>>>>> If no name is given, then the provided Named will be used.
> > >>>>>>>>>>> This allow to name the operation without having a queriable
> > >>>>> store.
> > >>>>>>>>>>>
> > >>>>>>>>>>> So if my analysis is correct, we will end up with :
> > >>>>>>>>>>>
> > >>>>>>>>>>> Generated | Named |
> > Joined /
> > >>>>>>>>>> Grouped
> > >>>>>>>>>>> | Materialized
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> -------------------------------------------------------------------------------------------------
> > >>>>>>>>>>> Node | X | X
> > |
> > >>>>>> X
> > >>>>>>>>>>> |
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> -------------------------------------------------------------------------------------------------
> > >>>>>>>>>>> Repartition Topic | X | |
> > X
> > >>>>>>>>>>> |
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> -------------------------------------------------------------------------------------------------
> > >>>>>>>>>>> Queryable Store | | |
> > >>>>>>>>>>> | X
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> -------------------------------------------------------------------------------------------------
> > >>>>>>>>>>> State store | X | X |
> > >>>>> X
> > >>>>>>>>>>> | X
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> -------------------------------------------------------------------------------------------------
> > >>>>>>>>>>> Changelog Topic | X | X |
> X
> > >>>>>>>>>>> | X
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> >
> -------------------------------------------------------------------------------------------------
> > >>>>>>>>>>>
> > >>>>>>>>>>> Le dim. 13 janv. 2019 à 03:23, Matthias J. Sax <
> > >>>>>>> matth...@confluent.io>
> > >>>>>>>> a
> > >>>>>>>>>>> écrit :
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Just catching up on this KIP again.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> One nit. The KIP says:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> In addition, the generated names have a few disadvantages
> to
> > >>>>>>>> guarantee
> > >>>>>>>>>>>> topology compatibilities. In fact, adding a new operator,
> > using
> > >>>>> a
> > >>>>>>>>>>>> third-library doing some optimization to remove some
> operators
> > >>>>> or
> > >>>>>>>>>> upgrading
> > >>>>>>>>>>>> to a new KafkaStreams version with internal API changes may
> > >>>>>> changed
> > >>>>>>>>>> suffix
> > >>>>>>>>>>>> indexing for a large amount of the processor names. This
> will
> > in
> > >>>>>>> turn
> > >>>>>>>>>>>> change the internal state store names, as well as internal
> > topic
> > >>>>>>> names
> > >>>>>>>>>> as
> > >>>>>>>>>>>> well.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> This is not true any longer (I guess it was true, when the
> KIP
> > >>>>> was
> > >>>>>>>>>>>> initially proposed), because all stores/internal-topics can
> be
> > >>>>>> named
> > >>>>>>>>>>>> since 2.1 release. I would suggest to remove the paragraph.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Overall, I like the Named/NamedOperation design.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> What is unclear to me thought is, why we need new overloads
> > for
> > >>>>>>>> methods
> > >>>>>>>>>>>> that accept `Materialized`. To be more precise, I think it
> > make
> > >>>>>>> sense
> > >>>>>>>> to
> > >>>>>>>>>>>> add an overload that only takes `Named`, but not one that
> > takes
> > >>>>>> both
> > >>>>>>>>>>>> `Named` and `Materialized`. For example:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> KGroupedStream#count() // exists
> > >>>>>>>>>>>> KGroupedStream#count(Materialized) // exits
> > >>>>>>>>>>>> KGroupedStream#count(Named) // added (makes sense to me)
> > >>>>>>>>>>>> KGroupedStream#count(Named, Materialized) // added -- why?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I would prefer to use `Materialized` to name the processor
> for
> > >>>>>> this
> > >>>>>>>>>>>> case, too. Can you elaborate on the motivation?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -Matthias
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 1/11/19 3:39 PM, Florian Hussonnois wrote:
> > >>>>>>>>>>>>> Hi Guozhang,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I have updated the PR as well as the KIP. I should add more
> > >>>>> unit
> > >>>>>>>> tests
> > >>>>>>>>>> to
> > >>>>>>>>>>>>> covers all new methods.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> However, I still have one test in failure. The reason is
> that
> > >>>>>> using
> > >>>>>>>>>>>>> Joined.name() in both potential repartition topic and
> > processor
> > >>>>>>> nodes
> > >>>>>>>>>>>> leads
> > >>>>>>>>>>>>> to topology-incompatible.
> > >>>>>>>>>>>>> How should we deal with that ?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Le jeu. 10 janv. 2019 à 01:21, Guozhang Wang <
> > >>>>> wangg...@gmail.com
> > >>>>>>>
> > >>>>>>> a
> > >>>>>>>>>>>> écrit :
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hello Florian,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Just checking if have read about my previous email and if
> > you
> > >>>>>> feel
> > >>>>>>>>>> happy
> > >>>>>>>>>>>>>> about it. We have the 2.2 KIP freeze deadline at 24th this
> > >>>>>> month,
> > >>>>>>>>>> while
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>> PR itself is getting quite close. So it'll be great if we
> > can
> > >>>>>> get
> > >>>>>>>> the
> > >>>>>>>>>>>>>> agreement on it and get it into 2.2.0 release.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 2:39 PM Guozhang Wang <
> > >>>>>> wangg...@gmail.com
> > >>>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi Florian / John,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Just wanted to throw a couple minor thoughts on the
> current
> > >>>>>>>> proposal:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 1) Regarding the interface / function name, I'd propose
> we
> > >>>>> call
> > >>>>>>> the
> > >>>>>>>>>>>>>>> interface `NamedOperation` which would be implemented by
> > >>>>>>> Produced /
> > >>>>>>>>>>>>>>> Consumed / Printed / Joined / Grouped / Suppressed (note
> I
> > >>>>>>>>>>>> intentionally
> > >>>>>>>>>>>>>>> exclude Materialized here since its semantics is quite),
> > and
> > >>>>>> have
> > >>>>>>>> the
> > >>>>>>>>>>>>>>> default class that implements `NamedOperation` as
> `Named`,
> > >>>>>> which
> > >>>>>>>>>> would
> > >>>>>>>>>>>> be
> > >>>>>>>>>>>>>>> used in our adding overload functions. The main reason is
> > to
> > >>>>>> have
> > >>>>>>>>>>>>>>> consistency in naming.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 2) As a minor tweak, I think it's better to use
> > Joined.name()
> > >>>>>> in
> > >>>>>>>> both
> > >>>>>>>>>>>> its
> > >>>>>>>>>>>>>>> possibly generate repartition topic, as well as the map
> > >>>>>> processor
> > >>>>>>>>>> used
> > >>>>>>>>>>>> for
> > >>>>>>>>>>>>>>> group-by (currently this name is only used for the
> > >>>>> repartition
> > >>>>>>>>>> topic).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Florian: if you think this proposal makes sense, please
> > feel
> > >>>>>> free
> > >>>>>>>> to
> > >>>>>>>>>> go
> > >>>>>>>>>>>>>>> ahead and update the PR; after we made a first pass on it
> > and
> > >>>>>>> feels
> > >>>>>>>>>>>>>>> confident about it, we can go ahead with the VOTING
> > process.
> > >>>>>>> About
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>>> implementation of 2) above, this may be out of your
> > >>>>>>> implementation
> > >>>>>>>>>>>> scope,
> > >>>>>>>>>>>>>>> so feel free to leave it out side your PR while Bill who
> > >>>>>>> originally
> > >>>>>>>>>>>> worked
> > >>>>>>>>>>>>>>> on the Grouped KIP can make a follow-up PR for it.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Fri, Dec 14, 2018 at 9:43 PM Guozhang Wang <
> > >>>>>>> wangg...@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hello Florian,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Really appreciate you for your patience.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I know that we've discussed about the approach to adding
> > >>>>>>>> overloaded
> > >>>>>>>>>>>>>>>> functions and rejected it early on. But looking deeper
> > into
> > >>>>>> the
> > >>>>>>>>>>>> current PR
> > >>>>>>>>>>>>>>>> I realized that this approach has a danger of great API
> > >>>>>>> confusions
> > >>>>>>>>>> to
> > >>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>> (I tried to explain my thoughts in the PR, but it was
> not
> > >>>>> very
> > >>>>>>>>>> clear)
> > >>>>>>>>>>>> ---
> > >>>>>>>>>>>>>>>> the basic idea is that, today we already have a few
> > existing
> > >>>>>>>> control
> > >>>>>>>>>>>>>>>> classes including Grouped, Joined, Suppressed that allow
> > >>>>> users
> > >>>>>>> to
> > >>>>>>>>>>>> specify
> > >>>>>>>>>>>>>>>> serdes etc, while also a "name" which can then be used
> to
> > >>>>>> define
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>> processor name / internal topic names in the topology
> (the
> > >>>>>>> static
> > >>>>>>>>>>>> function
> > >>>>>>>>>>>>>>>> names are not consistent, which I think we should fix as
> > >>>>>> well).
> > >>>>>>>> And
> > >>>>>>>>>>>> Named
> > >>>>>>>>>>>>>>>> interface, by extending the lambda function interfaces
> > like
> > >>>>>>>>>>>> ValueJoiner /
> > >>>>>>>>>>>>>>>> Predicate etc opens the door for another way to specify
> > the
> > >>>>>>> names
> > >>>>>>>>>>>> again.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> So in order to achieve consistency, we are left with
> > >>>>> generally
> > >>>>>>> two
> > >>>>>>>>>>>>>>>> options:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> 1) only allow users to specify names via the lambda
> > >>>>> interfaces
> > >>>>>>>> that
> > >>>>>>>>>>>>>>>> extends Named interface. This means we'd better remove
> the
> > >>>>>>> naming
> > >>>>>>>>>>>> mechanism
> > >>>>>>>>>>>>>>>> from the existing control objects to keep consistency.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> 2) only allow users to specify names via control
> classes,
> > >>>>> and
> > >>>>>> we
> > >>>>>>>>>>>>>>>> introduce a new class (Named) for those which do not
> have
> > >>>>> one
> > >>>>>>> yet
> > >>>>>>>>>> ---
> > >>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>> leads to the overloaded functions.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I did a quick count on the num.of overloaded functions,
> > and
> > >>>>>>>> summing
> > >>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>> KTable (8) / KStream (15) / KGroupedStream (6) /
> > >>>>> KGroupedTable
> > >>>>>>>> (6) /
> > >>>>>>>>>>>>>>>> TimeWindowedKStream (6) / SessionWindowedKStream (6) we
> > got
> > >>>>>>> about
> > >>>>>>>> 47
> > >>>>>>>>>>>>>>>> overloaded functions (our guess was pretty close!) --
> note
> > >>>>>> this
> > >>>>>>> is
> > >>>>>>>>>>>> based on
> > >>>>>>>>>>>>>>>> John's proposal that we can let existing Grouped /
> Joined
> > to
> > >>>>>>>> extend
> > >>>>>>>>>>>> Named
> > >>>>>>>>>>>>>>>> and hence we only need overloaded functions with a
> default
> > >>>>>>>>>>>> NamedOperation
> > >>>>>>>>>>>>>>>> for those operators that do not have a control classes
> > >>>>>> already.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thinking about this approach I feel it is not too bad
> > >>>>> compared
> > >>>>>>>> with
> > >>>>>>>>>>>>>>>> either 1) above, which would require us to deprecate lot
> > of
> > >>>>>>> public
> > >>>>>>>>>>>>>>>> functions around name(), or having a mixed mechanism for
> > >>>>>> naming,
> > >>>>>>>>>> which
> > >>>>>>>>>>>>>>>> could lead to very confusing behavior to users.
> > >>>>> Additionally,
> > >>>>>>> for
> > >>>>>>>>>> most
> > >>>>>>>>>>>>>>>> users who would only want to specify the names for those
> > >>>>>>> stateful
> > >>>>>>>>>>>>>>>> operations which have internal topics / state stores and
> > >>>>> hence
> > >>>>>>> are
> > >>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>> keen to upgrade compatibility, those added overloads
> would
> > >>>>> be
> > >>>>>>>>>>>> not-often
> > >>>>>>>>>>>>>>>> used functions for them anyways. And by letting existing
> > >>>>>> control
> > >>>>>>>>>>>> classes to
> > >>>>>>>>>>>>>>>> extend Named, we can have a unified method name for
> static
> > >>>>>>>>>>>> constructor as
> > >>>>>>>>>>>>>>>> well.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Fri, Dec 14, 2018 at 10:24 AM John Roesler <
> > >>>>>>> j...@confluent.io>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hi Florian,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Sorry about the run-around of rejecting the original
> > >>>>>> proposal,
> > >>>>>>>>>>>>>>>>> only to return to it later on. Hopefully, it's more
> > >>>>>> encouraging
> > >>>>>>>>>>>>>>>>> than frustrating that we're coming around to your
> initial
> > >>>>> way
> > >>>>>>> of
> > >>>>>>>>>>>>>>>>> thinking.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Thu, Dec 13, 2018 at 4:28 PM Florian Hussonnois <
> > >>>>>>>>>>>>>>>>> fhussonn...@gmail.com>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hi all,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks again. I agree with your propositions.
> > >>>>>>>>>>>>>>>>>> Also IMHO, overloading all methods (filter, map) to
> > >>>>> accept a
> > >>>>>>> new
> > >>>>>>>>>>>>>>>>> control
> > >>>>>>>>>>>>>>>>>> object seems to provide a more natural development
> > >>>>>> experience
> > >>>>>>>> for
> > >>>>>>>>>>>>>>>>> users.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Actually, this was the first proposition for this KIP,
> > but
> > >>>>>> we
> > >>>>>>>> have
> > >>>>>>>>>>>>>>>>> rejected
> > >>>>>>>>>>>>>>>>>> it because this solution led to adding a lot of new
> > >>>>> methods.
> > >>>>>>>>>>>>>>>>>> As you mentioned it, the API has evolve since the
> > creation
> > >>>>>> of
> > >>>>>>>> this
> > >>>>>>>>>>>>>>>>> KIP -
> > >>>>>>>>>>>>>>>>>> some existing control objects already allow to
> customize
> > >>>>>>>> internal
> > >>>>>>>>>>>>>>>>> names. We
> > >>>>>>>>>>>>>>>>>> should so keep on that strategy.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> If everyone is OK with that, I will update the KIP and
> > the
> > >>>>>> PR
> > >>>>>>>>>>>>>>>>> accordingly;
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Le jeu. 13 déc. 2018 à 18:08, John Roesler <
> > >>>>>> j...@confluent.io
> > >>>>>>>>
> > >>>>>>>> a
> > >>>>>>>>>>>>>>>>> écrit :
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Hi again, all,
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Matthias, I agree with you.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Florian, thanks for your response.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> I think your proposal is the best way to address the
> > ask
> > >>>>>> for
> > >>>>>>>>>> hiding
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> name() getter. But I'd like to question that ask and
> > >>>>>> instead
> > >>>>>>>>>>>>>>>>> propose that
> > >>>>>>>>>>>>>>>>>>> we just make the name() getter part of the public
> API.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> The desire to "hide" the getters causes a lot of
> > >>>>> complexity
> > >>>>>>> in
> > >>>>>>>>>> our
> > >>>>>>>>>>>>>>>>> code
> > >>>>>>>>>>>>>>>>>>> base, and it will become completely impractical with
> > the
> > >>>>>>> mixin
> > >>>>>>>>>>>>>>>>> strategy
> > >>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>> Named.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> If we were to switch strategies back to mixing Named
> in
> > >>>>> to
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>> control
> > >>>>>>>>>>>>>>>>>>> objects rather than the functions, then the path
> > forward
> > >>>>>>>> becomes
> > >>>>>>>>>>>>>>>>> quite
> > >>>>>>>>>>>>>>>>>>> clear.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On the other hand, it seems harmless for anyone who
> > wants
> > >>>>>> to
> > >>>>>>> be
> > >>>>>>>>>>>>>>>>> able to
> > >>>>>>>>>>>>>>>>>>> query the name from a control object after setting
> it,
> > so
> > >>>>>> my
> > >>>>>>>> vote
> > >>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>> simply to keep the Named interface as:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> public interface Named<T extends Named<T>> {
> > >>>>>>>>>>>>>>>>>>> String name();
> > >>>>>>>>>>>>>>>>>>> T withName(String name);
> > >>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Under this proposal, we only mix Named in to the
> > control
> > >>>>>>>> objects,
> > >>>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>>> means we have no need of default implementations
> > anymore
> > >>>>>>>> (because
> > >>>>>>>>>>>>>>>>> we can
> > >>>>>>>>>>>>>>>>>>> update all the control objects concurrently with
> adding
> > >>>>>> this
> > >>>>>>>>>>>>>>>>> interface to
> > >>>>>>>>>>>>>>>>>>> them).
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> This does hinge on switching over to a
> > >>>>> control-object-only
> > >>>>>>>>>>>> strategy,
> > >>>>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>>> introduces the need to add about 50 new control
> object
> > >>>>>>> classes,
> > >>>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>> only serve to implement Named. As a middle ground,
> > maybe
> > >>>>> we
> > >>>>>>>> could
> > >>>>>>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>>> add
> > >>>>>>>>>>>>>>>>>>> one generic control object class, like:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> public class NamedOperation implements
> > >>>>>> Named<NamedOperation>
> > >>>>>>> {
> > >>>>>>>>>>>>>>>>>>> private final String name;
> > >>>>>>>>>>>>>>>>>>> private NamedOperation(final String name) {
> > this.name
> > >>>>> =
> > >>>>>>>> name;
> > >>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>> public static NamedOperation name(final String
> name)
> > {
> > >>>>>>>>>>>>>>>>>>> return new NamedOperation(name);
> > >>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>> public String name() { return name; }
> > >>>>>>>>>>>>>>>>>>> public NamedOperation withName(final String name) {
> > >>>>>>>>>>>>>>>>>>> return new NamedOperation(name);
> > >>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> And then, we'd add overloads for all the methods that
> > >>>>> don't
> > >>>>>>>> have
> > >>>>>>>>>>>>>>>>> control
> > >>>>>>>>>>>>>>>>>>> objects already (for example, filter() ):
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> // existing
> > >>>>>>>>>>>>>>>>>>> KStream<K, V> filter(Predicate<? super K, ? super V>
> > >>>>>>>> predicate);
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> // new
> > >>>>>>>>>>>>>>>>>>> KStream<K, V> filter(Predicate<? super K, ? super V>
> > >>>>>>> predicate,
> > >>>>>>>>>>>>>>>>>>> NamedOperation named);
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Additionally, in regard to Matthias's point about
> > >>>>> existing
> > >>>>>>>>>> control
> > >>>>>>>>>>>>>>>>>> objects
> > >>>>>>>>>>>>>>>>>>> with naming semantics, they would extend Named (but
> not
> > >>>>>>>>>>>>>>>>> NamedOperation)
> > >>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>> uniformity.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> You provided a good approach to hide the getter with
> > your
> > >>>>>>>>>>>>>>>>> SettableName
> > >>>>>>>>>>>>>>>>>>> class; I think what you proposed is the only way we
> > could
> > >>>>>>> hide
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> name.
> > >>>>>>>>>>>>>>>>>>> In the end, though, it's a lot of complexity added
> > >>>>> (control
> > >>>>>>>>>> object
> > >>>>>>>>>>>>>>>>> class
> > >>>>>>>>>>>>>>>>>>> hierarchy, inheritance, mutable state, internal
> > casting)
> > >>>>>> for
> > >>>>>>>>>>>>>>>>> something of
> > >>>>>>>>>>>>>>>>>>> dubious value: to be able to hide the name from
> someone
> > >>>>>>> *after
> > >>>>>>>>>> they
> > >>>>>>>>>>>>>>>>>>> themselves have set it*.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Although it'll be a pain, perhaps Matthias's
> suggestion
> > >>>>> to
> > >>>>>>>>>>>>>>>>> enumerate all
> > >>>>>>>>>>>>>>>>>>> the API methods is the best way to be sure we all
> agree
> > >>>>> on
> > >>>>>>>> what's
> > >>>>>>>>>>>>>>>>> going
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> happen.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks again for wrangling with this issue,
> > >>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Thu, Dec 13, 2018 at 9:03 AM Matthias J. Sax <
> > >>>>>>>>>>>>>>>>> matth...@confluent.io>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Just catching up on this discussion.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> My overall personal take is, that I am not a big fan
> > of
> > >>>>>> the
> > >>>>>>>>>>>>>>>>> interface
> > >>>>>>>>>>>>>>>>>>>> `Named` that is used as a factory. I would rather
> > prefer
> > >>>>>> to
> > >>>>>>>> add
> > >>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>> control object parameter to all methods that don't
> > have
> > >>>>>> one
> > >>>>>>>> yet.
> > >>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>> KIP was started a while ago, and we added new naming
> > >>>>>>>>>> capabilities
> > >>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> meantime. Guozhang's example in the PR comment about
> > >>>>>> naming
> > >>>>>>> in
> > >>>>>>>>>>>>>>>>>>>> stream-stream join shows, that we might end up in a
> > >>>>>>> confusion
> > >>>>>>>>>>>>>>>>> situation
> > >>>>>>>>>>>>>>>>>>>> for users if we use `Named`. Also, in 2.1, user can
> > >>>>>> already
> > >>>>>>>> name
> > >>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>> repartition-/changelog-topics and stores. Thus,
> > KIP-307
> > >>>>>>> boils
> > >>>>>>>>>>>>>>>>> down to
> > >>>>>>>>>>>>>>>>>>>> provide non-functional naming?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Hence, for all methods that allow to specify names
> > >>>>>> already,
> > >>>>>>> I
> > >>>>>>>>>>>>>>>>> don't see
> > >>>>>>>>>>>>>>>>>>>> any reason to change them, but use the existing API
> to
> > >>>>>> also
> > >>>>>>>> name
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> processor(s) instead of allowing uses to specify a
> new
> > >>>>>> name.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About the inconsistency in method naming. I agree,
> > that
> > >>>>>> `as`
> > >>>>>>>> is
> > >>>>>>>>>>>>>>>>> very
> > >>>>>>>>>>>>>>>>>>>> generic and maybe not the best choice.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I think it might be helpful, to have a table
> overview
> > in
> > >>>>>> the
> > >>>>>>>>>> KIP,
> > >>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>> list all existing static/non-static methods that
> allow
> > >>>>> to
> > >>>>>>>>>> specify
> > >>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>> name, plus a columns with the new suggested naming
> for
> > >>>>>> those
> > >>>>>>>>>>>>>>>>> methods?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Thoughts?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> -Matthias
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On 12/12/18 12:45 AM, Florian Hussonnois wrote:
> > >>>>>>>>>>>>>>>>>>>>> Thank you very much for your feedbacks.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Currently, there is still lot of discussions
> > regarding
> > >>>>>> the
> > >>>>>>>>>> Named
> > >>>>>>>>>>>>>>>>>>>> interface.
> > >>>>>>>>>>>>>>>>>>>>> On the one hand we should provided consistency over
> > the
> > >>>>>>>> stream
> > >>>>>>>>>>>>>>>>> API
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>> the other hand we should not break the semantic as
> > John
> > >>>>>>> point
> > >>>>>>>>>>>>>>>>> it up.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Guozhang, I'm sorry, but I'm little bit confused,
> > >>>>> maybe I
> > >>>>>>>>>> missed
> > >>>>>>>>>>>>>>>>>>>> something.
> > >>>>>>>>>>>>>>>>>>>>> In your comment you have suggested that :
> > >>>>>>>>>>>>>>>>>>>>> * Produced/Consumed/Suppressed should extends Named
> > >>>>>>>>>>>>>>>>>>>>> * Named should have a private-package method to get
> > the
> > >>>>>>>>>>>>>>>>> specified
> > >>>>>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>> name internally (processorName())
> > >>>>>>>>>>>>>>>>>>>>> * Finally we should end up with something like :
> > Named
> > >>>>>> ->
> > >>>>>>>> XXX
> > >>>>>>>>>>>>>>>>> ->
> > >>>>>>>>>>>>>>>>>>>>> XXXInternal or Named -> Produced ->
> ProducedInternal
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> The objective behind that is to :
> > >>>>>>>>>>>>>>>>>>>>> * consolidate the internal method processorName()
> > >>>>>>>>>>>>>>>>>>>>> * consolidate the method withName that exists now
> > >>>>>> existing
> > >>>>>>>> into
> > >>>>>>>>>>>>>>>>>>> Produced,
> > >>>>>>>>>>>>>>>>>>>>> Consumed and Suppressed.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> But, Named is an interface so we can't define a
> > >>>>>>>> private-package
> > >>>>>>>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>> it. Also, for example Produced and ProducedInternal
> > are
> > >>>>>> not
> > >>>>>>>> in
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> same
> > >>>>>>>>>>>>>>>>>>>>> package so having a private-package method doesn't
> > >>>>> really
> > >>>>>>>> help.
> > >>>>>>>>>>>>>>>>>>>>> In addition, if we add the withName method into
> Named
> > >>>>>>>> interface
> > >>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>> become confusing for developers because action
> > >>>>> interfaces
> > >>>>>>>>>>>>>>>>>> (ValueMapper,
> > >>>>>>>>>>>>>>>>>>>>> Reducer, etc) extend it.
> > >>>>>>>>>>>>>>>>>>>>> The interface would look like :
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> public interface Named<T extends Named<T>> {
> > >>>>>>>>>>>>>>>>>>>>> default String name() {
> > >>>>>>>>>>>>>>>>>>>>> return null;
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>> default Named<T> withName(final String name) {
> > >>>>>>>>>>>>>>>>>>>>> return null;
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>> ...
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> So maybe instead of adding another method to Named
> we
> > >>>>>> could
> > >>>>>>>>>>>>>>>>> create a
> > >>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>> package-private class that could be extended by
> > >>>>>>>>>>>>>>>>>>>>> Produced/Consumed/Joined/Suppressed. For exemple,
> > >>>>>>>>>>>>>>>>>>>>> class SettableName<T extends SettableName<T>>
> > >>>>> implements
> > >>>>>>>> Named
> > >>>>>>>>>> {
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> protected String processorName;
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> SettableName(final SettableName settable) {
> > >>>>>>>>>>>>>>>>>>>>> this(Objects.requireNonNull(settable,
> > "settable
> > >>>>>>> can't
> > >>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>> null").name());
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> SettableName(final String processorName) {
> > >>>>>>>>>>>>>>>>>>>>> this.processorName = processorName;
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> @Override
> > >>>>>>>>>>>>>>>>>>>>> public String name() {
> > >>>>>>>>>>>>>>>>>>>>> return processorName;
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>> public T withName(final String processorName) {
> > >>>>>>>>>>>>>>>>>>>>> this.processorName = processorName;
> > >>>>>>>>>>>>>>>>>>>>> return (T)this;
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> In that way, we will get : public class Produced
> > >>>>>> implements
> > >>>>>>>>>>>>>>>>>>>>> SettableName<Produced> { ...
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> WDYT?
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Le mar. 11 déc. 2018 à 02:46, Guozhang Wang <
> > >>>>>>>>>> wangg...@gmail.com>
> > >>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>> écrit :
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> I had one meta comment on the PR:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>> https://github.com/apache/kafka/pull/5909#discussion_r240447153
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Mon, Dec 10, 2018 at 5:22 PM John Roesler <
> > >>>>>>>>>>>>>>>>> j...@confluent.io>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Hi Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> I hope it's ok if I ask a few questions at this
> > late
> > >>>>>>>> stage...
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Comment 1 ======
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> It seems like the proposal is to add a new
> "Named"
> > >>>>>>>> interface
> > >>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>> intended to be mixed in with the existing API
> > objects
> > >>>>>> at
> > >>>>>>>>>>>>>>>>> various
> > >>>>>>>>>>>>>>>>>>>> points.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Just to preface some of my comments, it looks
> like
> > >>>>> your
> > >>>>>>> KIP
> > >>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>> quite a while ago, so the API may have changed
> > >>>>> somewhat
> > >>>>>>>> since
> > >>>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>>>>> started.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> As I see the API, there are a few different kinds
> > of
> > >>>>>> DSL
> > >>>>>>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>>>>>> arguments:
> > >>>>>>>>>>>>>>>>>>>>>>> * functions: things like Initializer, Aggregator,
> > >>>>>>>>>> ValueJoiner,
> > >>>>>>>>>>>>>>>>>>>>>>> ForEachAction... All of these are essentially
> > >>>>>>>>>> Streams-flavored
> > >>>>>>>>>>>>>>>>>>> Function
> > >>>>>>>>>>>>>>>>>>>>>>> interfaces with different arities, type bounds,
> and
> > >>>>>>>>>> semantics.
> > >>>>>>>>>>>>>>>>>>>>>>> * config objects: things like Produced, Consumed,
> > >>>>>> Joined,
> > >>>>>>>>>>>>>>>>>> Grouped...
> > >>>>>>>>>>>>>>>>>>>>>> These
> > >>>>>>>>>>>>>>>>>>>>>>> are containers for configurations, where the
> target
> > >>>>> of
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> configuration
> > >>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>> the operation itself
> > >>>>>>>>>>>>>>>>>>>>>>> * raw configurations: things like a raw
> topic-name
> > >>>>>> string
> > >>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> Materialized:
> > >>>>>>>>>>>>>>>>>>>>>>> These are configurations for operations that have
> > no
> > >>>>>>> config
> > >>>>>>>>>>>>>>>>> object,
> > >>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>> various reasons, we didn't make one. The
> > >>>>> distinguishing
> > >>>>>>>>>>>>>>>>> feature is
> > >>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>> target of the configuration is not the operation
> > >>>>>> itself,
> > >>>>>>>> but
> > >>>>>>>>>>>>>>>>> some
> > >>>>>>>>>>>>>>>>>>>> aspect
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>> it. For example, in Materialized, we are not
> > setting
> > >>>>>> the
> > >>>>>>>>>>>>>>>>> caching
> > >>>>>>>>>>>>>>>>>>>> behavior
> > >>>>>>>>>>>>>>>>>>>>>>> of, for example, an aggregation; we're setting
> the
> > >>>>>>> caching
> > >>>>>>>>>>>>>>>>> behavior
> > >>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>> materialized state store attached to the
> > aggregation.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> It seems like choosing to mix the Named interface
> > in
> > >>>>>> with
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> functions
> > >>>>>>>>>>>>>>>>>>>>>> has
> > >>>>>>>>>>>>>>>>>>>>>>> a couple of unfortunate side-effects:
> > >>>>>>>>>>>>>>>>>>>>>>> * Aggregator is not the only function passed to
> any
> > >>>>> of
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>> relevant
> > >>>>>>>>>>>>>>>>>>>>>>> aggregate methods, so it seems a little arbitrary
> > to
> > >>>>>> pick
> > >>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>> function
> > >>>>>>>>>>>>>>>>>>>>>>> over Initializer or Merger.
> > >>>>>>>>>>>>>>>>>>>>>>> * As you noted, branch() takes an array of
> > Predicate,
> > >>>>>> so
> > >>>>>>> we
> > >>>>>>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>>>> ignore
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>> provided name(s), even though Predicate names are
> > >>>>> used
> > >>>>>>>>>>>>>>>>> elsewhere.
> > >>>>>>>>>>>>>>>>>>>>>>> * Not all things that we want to name have
> function
> > >>>>>>>>>> arguments,
> > >>>>>>>>>>>>>>>>>>> notably
> > >>>>>>>>>>>>>>>>>>>>>>> source and sink, so we'd switch paradigms and use
> > the
> > >>>>>>>> config
> > >>>>>>>>>>>>>>>>> object
> > >>>>>>>>>>>>>>>>>>>>>>> instead.
> > >>>>>>>>>>>>>>>>>>>>>>> * Adding an extra method to the function
> interfaces
> > >>>>>> means
> > >>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> those
> > >>>>>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>>>>>>> longer SAM interfaces. You proposed to add a
> > default
> > >>>>>>>>>>>>>>>>>> implementation,
> > >>>>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>> could still pass a lambda if we don't want to set
> > the
> > >>>>>>> name,
> > >>>>>>>>>>>>>>>>> but if
> > >>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>> *do*
> > >>>>>>>>>>>>>>>>>>>>>>> want to set the name, we can no longer use
> lambdas.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> I think the obvious other choice would be to mix
> > >>>>> Named
> > >>>>>> in
> > >>>>>>>>>>>>>>>>> with the
> > >>>>>>>>>>>>>>>>>>>> config
> > >>>>>>>>>>>>>>>>>>>>>>> objects instead, but this has one main downside
> of
> > >>>>> its
> > >>>>>>>> own...
> > >>>>>>>>>>>>>>>>>>>>>>> * not every operator we wish to name has a config
> > >>>>>>> object. I
> > >>>>>>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>> know
> > >>>>>>>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>>>>>>>> everyone involved is comfortable with adding a
> > config
> > >>>>>>>> object
> > >>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> every
> > >>>>>>>>>>>>>>>>>>>>>>> operator that's missing one.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Personally, I favor moving toward a more
> consistent
> > >>>>>> state
> > >>>>>>>>>>>>>>>>> that's
> > >>>>>>>>>>>>>>>>>>>> forward
> > >>>>>>>>>>>>>>>>>>>>>>> compatible with any further changes we wish to
> > make.
> > >>>>> I
> > >>>>>>>>>>>>>>>>> *think* that
> > >>>>>>>>>>>>>>>>>>>>>> giving
> > >>>>>>>>>>>>>>>>>>>>>>> every operator two forms (one with no config and
> > one
> > >>>>>>> with a
> > >>>>>>>>>>>>>>>>> config
> > >>>>>>>>>>>>>>>>>>>>>> object)
> > >>>>>>>>>>>>>>>>>>>>>>> would be such an API.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Comment 2 =========
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Finally, just a minor comment: the static method
> in
> > >>>>>> Named
> > >>>>>>>>>>>>>>>>> wouldn't
> > >>>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>>>> properly as defined. Assuming that we mix Named
> in
> > >>>>> with
> > >>>>>>>>>>>>>>>>> Produced,
> > >>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>> example, we'd need to be able to use it like:
> > >>>>>>>>>>>>>>>>>>>>>>>> kStream.to("out", Produced.with("myOut"))
> > >>>>>>>>>>>>>>>>>>>>>>> This doesn't work because with() returns a Named,
> > but
> > >>>>>> we
> > >>>>>>>> need
> > >>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>> Produced.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> We can pull off a builder method in the
> interface,
> > >>>>> but
> > >>>>>>> not
> > >>>>>>>> a
> > >>>>>>>>>>>>>>>>> static
> > >>>>>>>>>>>>>>>>>>>>>> method.
> > >>>>>>>>>>>>>>>>>>>>>>> To define a builder method in the interface that
> > >>>>>> returns
> > >>>>>>> an
> > >>>>>>>>>>>>>>>>>> instance
> > >>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>> concrete subtype, you have to use the "curiously
> > >>>>>>> recurring
> > >>>>>>>>>>>>>>>>> generic"
> > >>>>>>>>>>>>>>>>>>>>>>> pattern.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> It would look like:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> public interface Named<N extends Named<N>> {
> > >>>>>>>>>>>>>>>>>>>>>>> String name();
> > >>>>>>>>>>>>>>>>>>>>>>> N withName(String name);
> > >>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> You can see where the name of the pattern comes
> > from
> > >>>>> ;)
> > >>>>>>>>>>>>>>>>>>>>>>> An implementation would then look like:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> public class Produced implements Named<Produced>
> {
> > >>>>>>>>>>>>>>>>>>>>>>> String name() { return name; }
> > >>>>>>>>>>>>>>>>>>>>>>> Produced withName(final String name) {
> this.name
> > =
> > >>>>>>> name;
> > >>>>>>>>>>>>>>>>> return
> > >>>>>>>>>>>>>>>>>>>> this;
> > >>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Note that the generic parameter gets filled in
> > >>>>> properly
> > >>>>>>> in
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> implementing
> > >>>>>>>>>>>>>>>>>>>>>>> class, so that you get the right return type out.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> It doesn't work at all with a static factory
> method
> > >>>>> at
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>>> interface
> > >>>>>>>>>>>>>>>>>>>>>> level,
> > >>>>>>>>>>>>>>>>>>>>>>> so it would be up to Produced to define a static
> > >>>>>> factory
> > >>>>>>> if
> > >>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> wants
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>> present one.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> ======
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Those are my two feedbacks!
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> I hope you find this helpful, rather than
> > >>>>> frustrating.
> > >>>>>>> I'm
> > >>>>>>>>>>>>>>>>> sorry I
> > >>>>>>>>>>>>>>>>>>>> didn't
> > >>>>>>>>>>>>>>>>>>>>>>> get a chance to comment sooner.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP, I think it will be much nicer
> > to
> > >>>>> be
> > >>>>>>>> able
> > >>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> name
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>> processor nodes.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Nov 27, 2018 at 6:34 PM Guozhang Wang <
> > >>>>>>>>>>>>>>>>> wangg...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Hi Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> I've made a pass over the PR. There are some
> > >>>>> comments
> > >>>>>>> that
> > >>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>> related
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>> the function names which may be affecting the
> KIP
> > >>>>> wiki
> > >>>>>>>> page,
> > >>>>>>>>>>>>>>>>> but
> > >>>>>>>>>>>>>>>>>>>>>> overall
> > >>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>> think it looks good already.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Nov 16, 2018 at 4:21 PM Guozhang Wang <
> > >>>>>>>>>>>>>>>>> wangg...@gmail.com
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks Florian! I will take a look at the PR.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Nov 12, 2018 at 2:44 PM Florian
> > Hussonnois
> > >>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>> fhussonn...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Matthias,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I was absent for a while. I have
> started a
> > >>>>> new
> > >>>>>>> PR
> > >>>>>>>>>>>>>>>>> for this
> > >>>>>>>>>>>>>>>>>>>>>> KIP.
> > >>>>>>>>>>>>>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>> still in progress for now. I'm working on it.
> > >>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/kafka/pull/5909
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Le ven. 19 oct. 2018 à 20:13, Matthias J. Sax
> <
> > >>>>>>>>>>>>>>>>>>>>>> matth...@confluent.io>
> > >>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>> écrit :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> What is the status of this KIP?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> -Matthias
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/19/18 5:17 PM, Guozhang Wang wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for being late... Found myself keep
> > >>>>>>> apologizing
> > >>>>>>>>>>>>>>>>> for late
> > >>>>>>>>>>>>>>>>>>>>>>>> replies
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> these days. But I do want to push this KIP's
> > >>>>>>> progress
> > >>>>>>>>>>>>>>>>> forward
> > >>>>>>>>>>>>>>>>>>>>>> as I
> > >>>>>>>>>>>>>>>>>>>>>>>>>> see it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> very important and helpful feature for
> > >>>>>>> extensibility.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> About the exceptions, I've gone through them
> > and
> > >>>>>>>>>>>>>>>>> hopefully it
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> exhaustive list:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. KTable#toStream()
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. KStream#merge(KStream)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 3. KStream#process() / transform() /
> > >>>>>>> transformValues()
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4. KGroupedTable / KGroupedStream#count()
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Here's my reasoning:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * It is okay not letting users to override
> the
> > >>>>>> name
> > >>>>>>>> for
> > >>>>>>>>>>>>>>>>> 1/2,
> > >>>>>>>>>>>>>>>>>>>>>> since
> > >>>>>>>>>>>>>>>>>>>>>>>>>> they
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> too trivial to be useful for debugging, plus
> > >>>>> their
> > >>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>>> names
> > >>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not determine any related topic / store
> names.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * For 3, I'd vote for adding overloaded
> > >>>>> functions
> > >>>>>>> with
> > >>>>>>>>>>>>>>>>> Named.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * For 4, if users really want to name the
> > >>>>>> processor
> > >>>>>>>> she
> > >>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>> call
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> aggregate() instead, so I think it is okay
> to
> > >>>>> skip
> > >>>>>>>> this
> > >>>>>>>>>>>>>>>>> case.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 6, 2018 at 3:06 PM, Florian
> > >>>>>> Hussonnois <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> fhussonn...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The option #3 seems to be a good
> alternative
> > >>>>> and
> > >>>>>> I
> > >>>>>>>> find
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> API
> > >>>>>>>>>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> elegant (thanks John).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> But, we still have the need to overload
> some
> > >>>>>>> methods
> > >>>>>>>>>>>>>>>>> either
> > >>>>>>>>>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> they do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not accept an action instance or because
> they
> > >>>>> are
> > >>>>>>>>>>>>>>>>> translated
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>> multiple
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> processors.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, this is the case for methods
> > >>>>>> branch()
> > >>>>>>>> and
> > >>>>>>>>>>>>>>>>>> merge().
> > >>>>>>>>>>>>>>>>>>>>>>> We
> > >>>>>>>>>>>>>>>>>>>>>>>>>> could
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduce a new interface Named (or maybe a
> > >>>>>>> different
> > >>>>>>>>>>>>>>>>> name ?)
> > >>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> name(). All action interfaces could extend
> > this
> > >>>>>> one
> > >>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> option
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This would result by having the following
> > >>>>>> overloads
> > >>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Stream<K, V> merge(final Named name, final
> > >>>>>>> KStream<K,
> > >>>>>>>>>> V>
> > >>>>>>>>>>>>>>>>>>>>>> stream);
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KStream<K, V>[] branch(final Named name,
> > final
> > >>>>>>>>>>>>>>>>> Predicate<?
> > >>>>>>>>>>>>>>>>>>>>>> super
> > >>>>>>>>>>>>>>>>>>>>>>>> K, ?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> super
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> V>... predicates)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> N.B : The list above is not exhaustive
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---------
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> user's code will become :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KStream<String, Integer> stream =
> > >>>>>>>>>>>>>>>>>>>>>> builder.stream("test");
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KStream<String, Integer>[]
> branches =
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>> stream.branch(Named.with("BRANCH-STREAM-ON-VALUE"),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> Predicate.named("STREAM-PAIR-VALUE",
> > >>>>>>>>>>>>>>>>> (k, v)
> > >>>>>>>>>>>>>>>>>> ->
> > >>>>>>>>>>>>>>>>>>>>>> v
> > >>>>>>>>>>>>>>>>>>>>>>> %
> > >>>>>>>>>>>>>>>>>>>>>>>> 2
> > >>>>>>>>>>>>>>>>>>>>>>>>>> ==
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 0),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>> Predicate.named("STREAM-IMPAIR-VALUE",
> > >>>>>>>>>>>>>>>>> (k, v)
> > >>>>>>>>>>>>>>>>>>>>>> ->
> > >>>>>>>>>>>>>>>>>>>>>>> v
> > >>>>>>>>>>>>>>>>>>>>>>>> %
> > >>>>>>>>>>>>>>>>>>>>>>>>>> 2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> !=
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 0));
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches[0].to("pair");
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches[1].to("impair");
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---------
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is a mix of the options 3) and 1)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Le ven. 6 juil. 2018 à 22:58, Guozhang
> Wang <
> > >>>>>>>>>>>>>>>>>>>>>> wangg...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> écrit :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi folks, just to summarize the options we
> > >>>>> have
> > >>>>>> so
> > >>>>>>>>>> far:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Add a new "as" for KTable / KStream,
> plus
> > >>>>>>> adding
> > >>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>> fields
> > >>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operators-returns-void control objects
> (the
> > >>>>>>> current
> > >>>>>>>>>>>>>>>>> wiki's
> > >>>>>>>>>>>>>>>>>>>>>>>>>> proposal).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pros: no more overloads.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cons: a bit departing with the current
> > >>>>>> high-level
> > >>>>>>>> API
> > >>>>>>>>>>>>>>>>> design
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> DSL,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plus, the inconsistency between
> > >>>>>>>> operators-returns-void
> > >>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operators-not-return-voids.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Add overloaded functions for all
> > operators,
> > >>>>>>> that
> > >>>>>>>>>>>>>>>>> accepts
> > >>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> control
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object "Described".
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pros: consistent with current APIs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cons: lots of overloaded functions to add.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) Add another default function in the
> > >>>>> interface
> > >>>>>>>>>>>>>>>>> (thank you
> > >>>>>>>>>>>>>>>>>>>>>> J8!)
> > >>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proposed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pros: no overloaded functions, no
> > "Described".
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cons: do we lose lambda functions really
> > >>>>> (seems
> > >>>>>>> not
> > >>>>>>>> if
> > >>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>> provide
> > >>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "named"
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for each func)? Plus "Described" may be
> more
> > >>>>>>>>>> extensible
> > >>>>>>>>>>>>>>>>>> than a
> > >>>>>>>>>>>>>>>>>>>>>>>>>> single
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> `String`.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My principle of considering which one is
> > >>>>> better
> > >>>>>>>>>> depends
> > >>>>>>>>>>>>>>>>>>>>>>> primarily
> > >>>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> "how
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to make advanced users easily use the
> > >>>>> additional
> > >>>>>>>> API,
> > >>>>>>>>>>>>>>>>> while
> > >>>>>>>>>>>>>>>>>>>>>>>> keeping
> > >>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hidden from normal users who do not care
> at
> > >>>>>> all".
> > >>>>>>>> For
> > >>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>> purpose I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) > 1) > 2).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One caveat though, is that changing the
> > >>>>>> interface
> > >>>>>>>>>>>>>>>>> would not
> > >>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> binary-compatible though
> source-compatible,
> > >>>>>> right?
> > >>>>>>>>>> I.e.
> > >>>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recompile their code though no changes
> > needed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another note: for 3), if we really want to
> > >>>>> keep
> > >>>>>>>>>>>>>>>>>> extensibility
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Described
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we could do sth. like:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---------
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public interface Predicate<K, V> {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // existing method
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean test(final K key, final V
> > value);
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // new default method adds the ability
> > to
> > >>>>>> name
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> predicate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> default Described described() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return new Described(null);
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----------
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where user's code becomes:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stream.filter(named("key", (k, v) ->
> true));
> > >>>>>> //
> > >>>>>>>> note
> > >>>>>>>>>>>>>>>>>> `named`
> > >>>>>>>>>>>>>>>>>>>>>>> now
> > >>>>>>>>>>>>>>>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sets a Described("key") in "described()".
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> stream.filter(described(Described.as("key",
> > /*
> > >>>>>> any
> > >>>>>>>>>>>>>>>>> other
> > >>>>>>>>>>>>>>>>>> fancy
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the future*/), (k, v) -> true));
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----------
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I feel it is not much likely that we'd
> need
> > to
> > >>>>>>>> extend
> > >>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>> further
> > >>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> future, so just a `String` would be good
> > >>>>> enough.
> > >>>>>>> But
> > >>>>>>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>>>>>>>> listing
> > >>>>>>>>>>>>>>>>>>>>>>>>>> all
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> possibilities here.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 6, 2018 at 8:19 AM, John
> > Roesler <
> > >>>>>>>>>>>>>>>>>>>>>> j...@confluent.io
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm late to the party, but I missed
> > the
> > >>>>>>>> message
> > >>>>>>>>>>>>>>>>>>>>>>> originally.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regarding the names, it's probably a good
> > >>>>> idea
> > >>>>>> to
> > >>>>>>>>>>>>>>>>> stick to
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> same
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> character set we're currently using:
> > letters,
> > >>>>>>>>>>>>>>>>> numbers, and
> > >>>>>>>>>>>>>>>>>>>>>>>> hyphens.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> names are used in Kafka topics, files and
> > >>>>>>> folders,
> > >>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> RocksDB
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> databases,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and we also need them to work with the
> file
> > >>>>>>> systems
> > >>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>> Windows,
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Linux,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MacOS. My opinion is that with a
> situation
> > >>>>> like
> > >>>>>>>> that,
> > >>>>>>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>>>>>>> better
> > >>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> conservative. It might also be a good
> idea
> > to
> > >>>>>>>> impose
> > >>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>> upper
> > >>>>>>>>>>>>>>>>>>>>>>>>>> limit on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> name
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> length to avoid running afoul of any of
> > those
> > >>>>>>>>>> systems.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It seems like there's a small debate
> > between
> > >>>>> 1)
> > >>>>>>>>>>>>>>>>> adding a
> > >>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>> method to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KStream (and maybe KTable) to modify its
> > name
> > >>>>>>> after
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> fact,
> > >>>>>>>>>>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>>>>>>>> 2)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> piggy-backing on the config objects where
> > >>>>> they
> > >>>>>>>> exist
> > >>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> adding
> > >>>>>>>>>>>>>>>>>>>>>>>> one
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> they don't. To me, #2 is the better
> > >>>>> alternative
> > >>>>>>>> even
> > >>>>>>>>>>>>>>>>> though
> > >>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> produces
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more overloads and may be a bit awkward
> in
> > >>>>>>> places.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The reason is simply that #1 is a
> > high-level
> > >>>>>>>>>>>>>>>>> departure from
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> graph-building paradigm we're using in
> the
> > >>>>> DSL.
> > >>>>>>>>>>>>>>>>> Consider:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Graph.node1(config).node2(config)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vs
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Graph.node1().config().node2().config()
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We could have done either, but we picked
> > the
> > >>>>>>>> former.
> > >>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> probably
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a good goal to try and stick to it so
> that
> > >>>>>>>> developers
> > >>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>> develop
> > >>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rely
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on their instincts for how the DSL will
> > >>>>> behave.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I do want to present one alternative to
> > >>>>> adding
> > >>>>>>> new
> > >>>>>>>>>>>>>>>>> config
> > >>>>>>>>>>>>>>>>>>>>>>>> objects:
> > >>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> just add a "name()" method to all our
> > >>>>> "action"
> > >>>>>>>>>>>>>>>>> interfaces.
> > >>>>>>>>>>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> example,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'll demonstrate how we can add a "name"
> to
> > >>>>>>>> Predicate
> > >>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> name a "KStream#filter" DSL operator:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public interface Predicate<K, V> {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // existing method
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean test(final K key, final V
> > value);
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // new default method adds the
> ability
> > to
> > >>>>>>> name
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>> predicate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> default String name() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return null;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // new static factory method adds the
> > >>>>>> ability
> > >>>>>>>> to
> > >>>>>>>>>>>>>>>>> wrap
> > >>>>>>>>>>>>>>>>>>>>>>> lambda
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> predicates
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a named predicate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static <K, V> Predicate<K, V>
> > named(final
> > >>>>>>>> String
> > >>>>>>>>>>>>>>>>> name,
> > >>>>>>>>>>>>>>>>>>>>>>> final
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Predicate<K, V> predicate) {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return new Predicate<K, V>() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Override
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean test(final K
> > key,
> > >>>>>>> final
> > >>>>>>>> V
> > >>>>>>>>>>>>>>>>>> value) {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return
> predicate.test(key,
> > >>>>>>> value);
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Override
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public String name() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return name;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> };
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Then, here's how it would look to use it:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Anonymous predicates continue to work
> > just
> > >>>>>>> fine
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stream.filter((k, v) -> true);
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Devs can swap in a Predicate that
> > >>>>> implements
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>> name()
> > >>>>>>>>>>>>>>>>>>>>>>>> method.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stream.filter(new Predicate<Object,
> > >>>>> Object>() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Override
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean test(final Object key,
> > >>>>> final
> > >>>>>>>>>> Object
> > >>>>>>>>>>>>>>>>>>>>>> value) {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return true;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Override
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public String name() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return "hey";
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> });
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Or they can wrap their existing lambda
> > >>>>> using
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>> static
> > >>>>>>>>>>>>>>>>>>>>>>>> factory
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stream.filter(named("key", (k, v) ->
> > true));
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Just a thought.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Overall, I think it's really valuable to
> be
> > >>>>>> able
> > >>>>>>> to
> > >>>>>>>>>>>>>>>>> name
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> processors,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for all the reasons you mentioned in the
> > KIP.
> > >>>>>> So
> > >>>>>>>>>>>>>>>>> thank you
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introducing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jul 5, 2018 at 4:53 PM Florian
> > >>>>>>> Hussonnois <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> fhussonn...@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, thank you very much for all you
> > >>>>>> suggestions.
> > >>>>>>>>>> I've
> > >>>>>>>>>>>>>>>>>>>>>> started
> > >>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> update
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KIP (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>
> 307%3A+Allow+to+define+custom+processor+names+with+KStreams+DSL
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, I propose to rename the Processed
> > >>>>> class
> > >>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>> Described -
> > >>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more meaningful (but this is just a
> > detail).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm OK to not enforcing uppercase for
> > >>>>> specific
> > >>>>>>>> names
> > >>>>>>>>>>>>>>>>> but
> > >>>>>>>>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> arbitrary names with whitespaces for
> > >>>>> example ?
> > >>>>>>>>>>>>>>>>> Currently,
> > >>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>> can't
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> tell
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this can lead to some side effects ?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Le lun. 11 juin 2018 à 01:31, Matthias
> J.
> > >>>>> Sax
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>> matth...@confluent.io
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> écrit :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Just catching up on this thread.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I like the general idea. Couple of
> > >>>>> comments:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - I think that adding `Processed` (or
> > >>>>> maybe
> > >>>>>> a
> > >>>>>>>>>>>>>>>>> different
> > >>>>>>>>>>>>>>>>>>>>>>> name?)
> > >>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> valid proposal for stateless operators
> > that
> > >>>>>>> only
> > >>>>>>>>>>>>>>>>> have a
> > >>>>>>>>>>>>>>>>>>>>>>> single
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overload
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> atm. It would align with the overall
> API
> > >>>>>>> design.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - for all methods with multiple
> existing
> > >>>>>>>>>>>>>>>>> overloads, we
> > >>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consider
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extend `Consumed`, `Produced`,
> > >>>>> `Materialized`
> > >>>>>>> etc
> > >>>>>>>>>>>>>>>>> to take
> > >>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> additional
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor name (not sure atm how
> elegant
> > >>>>> this
> > >>>>>>> is;
> > >>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "play" with the API a little bit; the
> > >>>>>> advantage
> > >>>>>>>>>>>>>>>>> would be,
> > >>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> add more overloads what seems to be key
> > for
> > >>>>>>> this
> > >>>>>>>>>>>>>>>>> KIP)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - operators return void: while I agree
> > >>>>> that
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>> "name
> > >>>>>>>>>>>>>>>>>>>>>> first"
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chaining
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> idea is not very intuitive, it might
> > still
> > >>>>>>> work,
> > >>>>>>>> if
> > >>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>> name
> > >>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly (again, we would need to
> "play"
> > >>>>>> with
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>>> API a
> > >>>>>>>>>>>>>>>>>>>>>>> little
> > >>>>>>>>>>>>>>>>>>>>>>>>>> bit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> see)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - for DSL operators that are
> translated
> > to
> > >>>>>>>>>> multiple
> > >>>>>>>>>>>>>>>>>> nodes:
> > >>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> might
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make sense to use the specified
> operator
> > >>>>> name
> > >>>>>>> as
> > >>>>>>>>>>>>>>>>> prefix
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>> add
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasonable suffixes. For example, a
> join
> > >>>>>>>> translates
> > >>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>> 5
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> operators
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that could be name
> > >>>>>> "name-left-store-processor",
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "name-left-join-processor",
> > >>>>>>>>>>>>>>>>> "name-right-store-processor",
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "name-right-join-processor", and
> > >>>>>>>>>>>>>>>>>>>>>> "name-join-merge-processor"
> > >>>>>>>>>>>>>>>>>>>>>>>> (or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> similar). Maybe just using numbers
> might
> > >>>>> also
> > >>>>>>>> work.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - I think, we should strip the number
> > >>>>>> suffixes
> > >>>>>>>> if
> > >>>>>>>>>>>>>>>>> a user
> > >>>>>>>>>>>>>>>>>>>>>>>>>> provides
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> names
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - enforcing upper case seems to be
> > tricky:
> > >>>>>> for
> > >>>>>>>>>>>>>>>>> example,
> > >>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>> do
> > >>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enforce upper case for store names and
> we
> > >>>>>>> cannot
> > >>>>>>>>>>>>>>>>> easily
> > >>>>>>>>>>>>>>>>>>>>>>> change
> > >>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would break compatibility -- thus, for
> > >>>>>>>> consistency
> > >>>>>>>>>>>>>>>>>> reasons
> > >>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>> might
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to do this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - for better understand of the impact
> of
> > >>>>> the
> > >>>>>>>> KIP,
> > >>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> quite
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> helpful if you would list all method
> > names
> > >>>>>> that
> > >>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>> affected
> > >>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KIP
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ie, list all newly added overloads)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Matthias
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 5/31/18 6:40 PM, Guozhang Wang
> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Re 1: I think changing the
> KStreamImpl /
> > >>>>>>>>>>>>>>>>> KTableImpl to
> > >>>>>>>>>>>>>>>>>>>>>> allow
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modifying
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor name after the operator is
> > fine
> > >>>>> as
> > >>>>>>>> long
> > >>>>>>>>>>>>>>>>> as we
> > >>>>>>>>>>>>>>>>>> do
> > >>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when modifying that. In fact, we are
> > >>>>> having
> > >>>>>>> some
> > >>>>>>>>>>>>>>>>>> topology
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> optimization
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going on which may modify processor
> > names
> > >>>>> in
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>> final
> > >>>>>>>>>>>>>>>>>>>>>>>> topology
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyways (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > https://github.com/apache/kafka/pull/4983
> > >>>>> ).
> > >>>>>>>>>>>>>>>>>> Semantically
> > >>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> easier to understand to developers
> than
> > >>>>>>>> "deciding
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> name
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the next operator".
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Re 2: Yeah I'm thinking that for
> > operators
> > >>>>>>> that
> > >>>>>>>>>>>>>>>>>> translates
> > >>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor names, we can still use the
> > >>>>>> provided
> > >>>>>>>>>>>>>>>>> "hint" to
> > >>>>>>>>>>>>>>>>>>>>>>> name
> > >>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> names, e.g. for Joins we can name them
> > as
> > >>>>>>>>>>>>>>>>>> `join-foo-this`
> > >>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> `join-foo-that` etc if user calls
> > >>>>>> `as("foo")`.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Re 3: The motivation I had about
> > removing
> > >>>>>> the
> > >>>>>>>>>>>>>>>>> suffix is
> > >>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> has
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> huge
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restrictions on topology
> > compatibilities:
> > >>>>>>>> consider
> > >>>>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>>> user
> > >>>>>>>>>>>>>>>>>>>>>>>> code
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> added a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator, or library does some
> > >>>>> optimization
> > >>>>>> to
> > >>>>>>>>>>>>>>>>> remove
> > >>>>>>>>>>>>>>>>>> some
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operators,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suffix indexing may be changed for a
> > large
> > >>>>>>>> amount
> > >>>>>>>>>>>>>>>>> of the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> names:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this will in turn change the internal
> > >>>>> state
> > >>>>>>>> store
> > >>>>>>>>>>>>>>>>> names,
> > >>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>> well
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internal topic names as well, making
> the
> > >>>>> new
> > >>>>>>>>>>>>>>>>> application
> > >>>>>>>>>>>>>>>>>>>>>>>>>> topology
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible with the ones. One
> > rationale
> > >>>>> I
> > >>>>>>> had
> > >>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>> KIP
> > >>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> aligned this effort, moving forward we
> > can
> > >>>>>>> allow
> > >>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> customize
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internal names so that they can still
> be
> > >>>>>>> reused
> > >>>>>>>>>>>>>>>>> even
> > >>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> topology
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g. KIP-230), so I think removing
> the
> > >>>>>> suffix
> > >>>>>>>>>>>>>>>>> index
> > >>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable in the long run.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, May 31, 2018 at 3:08 PM,
> Florian
> > >>>>>>>>>>>>>>>>> Hussonnois <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fhussonn...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi ,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you very much for your
> feedback.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1/
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree that overloading most of the
> > >>>>>> methods
> > >>>>>>>> with
> > >>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>> Processed
> > >>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ideal.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've started modifying the KStream
> API
> > >>>>> and
> > >>>>>> I
> > >>>>>>>> got
> > >>>>>>>>>>>>>>>>> to the
> > >>>>>>>>>>>>>>>>>>>>>>> same
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> conclusion.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also ading a new method directly to
> > >>>>>>> KStreamImpl
> > >>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>> KTableImpl
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> classes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems to be a better option.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However a processor name cannot be
> > >>>>>> redefined
> > >>>>>>>>>> after
> > >>>>>>>>>>>>>>>>>>>>>> calling
> > >>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maybe I miss something in the code).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From my understanding, this will only
> > set
> > >>>>>> the
> > >>>>>>>>>>>>>>>>> KStream
> > >>>>>>>>>>>>>>>>>>>>>> name
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor name previsouly added to
> the
> > >>>>>>> topology
> > >>>>>>>>>>>>>>>>>> builder -
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> leading
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidTopology exception.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So the new method should actually
> > defines
> > >>>>>> the
> > >>>>>>>>>>>>>>>>> name of
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> next
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Below is an example :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *stream.as <http://stream.as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Processed.name("MAPPE_TO_UPPERCASE")*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * .map( (k, v) ->
> > >>>>> KeyValue.pair(k,
> > >>>>>>>>>>>>>>>>>>>>>>>> v.toUpperCase()))*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think this approach could solve the
> > >>>>> cases
> > >>>>>>> for
> > >>>>>>>>>>>>>>>>> methods
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> returning
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> void ?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regarding this new method we have two
> > >>>>>>> possible
> > >>>>>>>>>>>>>>>>>>>>>>>> implementations
> > >>>>>>>>>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. Adding a method like :
> > >>>>>> withName(String
> > >>>>>>>>>>>>>>>>>>>>>> processorName)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. or adding a method accepting an
> > >>>>>>> Processed
> > >>>>>>>>>>>>>>>>> object
> > >>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as(Processed).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think solution 2. is preferable as
> > the
> > >>>>>>>>>> Processed
> > >>>>>>>>>>>>>>>>>> class
> > >>>>>>>>>>>>>>>>>>>>>>>> could
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enriched
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> further (in futur).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2/
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As Guozhang said some operators add
> > >>>>>> internal
> > >>>>>>>>>>>>>>>>>> processors.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example the branch() method
> create
> > >>>>> one
> > >>>>>>>>>>>>>>>>>> KStreamBranch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> route
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> records and one KStreamPassThrough
> > >>>>>> processor
> > >>>>>>>> for
> > >>>>>>>>>>>>>>>>> each
> > >>>>>>>>>>>>>>>>>>>>>>> branch.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In that situation only the parent
> > >>>>> processor
> > >>>>>>> can
> > >>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>> named.
> > >>>>>>>>>>>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> children
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processors we could keep the current
> > >>>>>>> behaviour
> > >>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> add a
> > >>>>>>>>>>>>>>>>>>>>>>>>>> suffix
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (i.e
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KSTREAM-BRANCHCHILD-)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This also the case for the join()
> > method
> > >>>>>> that
> > >>>>>>>>>>>>>>>>> result to
> > >>>>>>>>>>>>>>>>>>>>>>>> adding
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processors to the topology
> (windowing,
> > >>>>>>>> left/right
> > >>>>>>>>>>>>>>>>> joins
> > >>>>>>>>>>>>>>>>>>>>>>> and a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merge
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think, like for the branch method
> > users
> > >>>>>>> could
> > >>>>>>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>>>>>>> define a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> name prefix.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3/
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we should still added a
> suffix
> > >>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>> "-0000000000"
> > >>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> name and enforce uppercases as this
> > will
> > >>>>>> keep
> > >>>>>>>>>> some
> > >>>>>>>>>>>>>>>>>>>>>>>> consistency
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ones generated by the API.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4/
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, the KTable interface should be
> > >>>>>> modified
> > >>>>>>>> like
> > >>>>>>>>>>>>>>>>>> KStream
> > >>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> custom
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor names definition.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Le jeu. 31 mai 2018 à 19:18, Damian
> > Guy <
> > >>>>>>>>>>>>>>>>>>>>>>>> damian....@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> écrit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. What about
> KTable
> > >>>>> and
> > >>>>>>>> other
> > >>>>>>>>>>>>>>>>> DSL
> > >>>>>>>>>>>>>>>>>>>>>>>>>> interfaces?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> they
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not want to be able to do the same
> > >>>>> thing?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It would be good to see a complete
> set
> > >>>>> of
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>> public
> > >>>>>>>>>>>>>>>>>> API
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damian
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 30 May 2018 at 19:45
> Guozhang
> > >>>>>> Wang <
> > >>>>>>>>>>>>>>>>>>>>>>>>>> wangg...@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. I have some
> meta
> > >>>>>>>> feedbacks
> > >>>>>>>>>>>>>>>>> on the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> proposal:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. You mentioned that this
> > `Processed`
> > >>>>>>> object
> > >>>>>>>>>>>>>>>>> will be
> > >>>>>>>>>>>>>>>>>>>>>>> added
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overloaded variant of all the
> > stateless
> > >>>>>>>>>>>>>>>>> operators,
> > >>>>>>>>>>>>>>>>>> what
> > >>>>>>>>>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stateful
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operators? Would like to hear your
> > >>>>>> opinions
> > >>>>>>>> if
> > >>>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thought
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> note for stateful operators they
> will
> > >>>>>>> usually
> > >>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>> mapped
> > >>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor node names, so we
> probably
> > >>>>> need
> > >>>>>>> to
> > >>>>>>>>>>>>>>>>> come up
> > >>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>> some
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ways
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> define all their names.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. I share the same concern with
> Bill
> > >>>>> as
> > >>>>>>> for
> > >>>>>>>>>>>>>>>>> adding
> > >>>>>>>>>>>>>>>>>>>>>> lots
> > >>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overload
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functions into the stateless
> > operators,
> > >>>>>> as
> > >>>>>>> we
> > >>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>>>>>>>>> spent
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quite
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> effort in trimming them since 1.0.0
> > >>>>>>> release.
> > >>>>>>>> If
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> goal
> > >>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provide
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some "hints" on the generated
> > processor
> > >>>>>>> node
> > >>>>>>>>>>>>>>>>> names,
> > >>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> strictly
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enforcing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the exact names that to be
> generated,
> > >>>>>> then
> > >>>>>>>> how
> > >>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> add a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> function to `KStream` and `KTable`
> > >>>>>> classes
> > >>>>>>>>>> like:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "as(Processed)",
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> semantics as "the latest operators
> > that
> > >>>>>>>>>>>>>>>>> generate this
> > >>>>>>>>>>>>>>>>>>>>>>>> KStream
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> /
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KTable
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be named accordingly to this hint".
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The only caveat, is that for all
> > >>>>>> operators
> > >>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>>>> `KStream#to`
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> `KStream#print` that returns void,
> > this
> > >>>>>>>>>>>>>>>>> alternative
> > >>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current operators:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a. KStream#print,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> b. KStream#foreach,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> c. KStream#to,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> d. KStream#process
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I personally felt that except
> > >>>>>>>> `KStream#process`
> > >>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> usually
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bother to override their names, and
> > for
> > >>>>>>>>>>>>>>>>>>>>>> `KStream#process`
> > >>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> add
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overload variant with the
> additional
> > >>>>>>>> Processed
> > >>>>>>>>>>>>>>>>>> object.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3. In your example, the processor
> > names
> > >>>>>> are
> > >>>>>>>>>>>>>>>>> still
> > >>>>>>>>>>>>>>>>>> added
> > >>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suffix
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -0000000000", is this intentional?
> If
> > >>>>>> yes,
> > >>>>>>>> why
> > >>>>>>>>>>>>>>>>> (I
> > >>>>>>>>>>>>>>>>>>>>>> thought
> > >>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specified processor name hints we
> > will
> > >>>>>> not
> > >>>>>>>> add
> > >>>>>>>>>>>>>>>>> suffix
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> distinguish
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different nodes of the same type
> any
> > >>>>>> more)?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 29, 2018 at 6:47 AM,
> Bill
> > >>>>>>> Bejeck
> > >>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> bbej...@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Florian,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. I think being
> > >>>>> able
> > >>>>>> to
> > >>>>>>>> add
> > >>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>>>>>> context
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processor names would be useful.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I like the idea of adding a
> > >>>>>>>>>>>>>>>>> "withProcessorName" to
> > >>>>>>>>>>>>>>>>>>>>>>>> Produced,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Consumed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Joined.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But instead of adding the
> > "Processed"
> > >>>>>>>>>>>>>>>>> parameter to a
> > >>>>>>>>>>>>>>>>>>>>>>> large
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> percentage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the methods, which would result in
> > >>>>>>>> overloaded
> > >>>>>>>>>>>>>>>>>> methods
> > >>>>>>>>>>>>>>>>>>>>>>>> (which
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quite a bit with KIP-182) what do
> > you
> > >>>>>>> think
> > >>>>>>>> of
> > >>>>>>>>>>>>>>>>>> adding
> > >>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the AbstractStream class
> > >>>>>>> "withName(String
> > >>>>>>>>>>>>>>>>>>>>>>>>>> processorName)"?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BTW
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I"m
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> married to the method name, it's
> the
> > >>>>>> best
> > >>>>>>> I
> > >>>>>>>>>>>>>>>>> can do
> > >>>>>>>>>>>>>>>>>> off
> > >>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> top
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> head.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the methods that return void,
> > we'd
> > >>>>>>> have
> > >>>>>>>> to
> > >>>>>>>>>>>>>>>>> add a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameter,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would at least cut down on the
> > number
> > >>>>> of
> > >>>>>>>>>>>>>>>>> overloaded
> > >>>>>>>>>>>>>>>>>>>>>>>> methods
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> API.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Just my 2 cents.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bill
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, May 27, 2018 at 4:13 PM,
> > >>>>> Florian
> > >>>>>>>>>>>>>>>>> Hussonnois
> > >>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fhussonn...@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to start a new
> > >>>>> discussion
> > >>>>>> on
> > >>>>>>>>>>>>>>>>> following
> > >>>>>>>>>>>>>>>>>>>>>>> KIP :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>
> 307%3A+Allow+to+define+custom+processor+names+with+KStreams+DSL
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is still a draft.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking forward for your
> feedback.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Florian HUSSONNOIS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Florian HUSSONNOIS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Florian HUSSONNOIS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Florian HUSSONNOIS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Florian HUSSONNOIS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>> Florian HUSSONNOIS
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> -- Guozhang
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Florian HUSSONNOIS
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> >
> >
>
--
Florian HUSSONNOIS
