Thanks for the discussion, Bill, John, Guozhang, and Matthias!
It sounds like consensus is that we would like to redesign the join
interfaces more broadly to have fewer overloads, but the scope of this
redesign is larger than makes sense for this particular KIP.
As for the interfaces proposed in
The current proposal to replace the `Named` overload with `TableJoined`
overload does not increase the number of method we have. Thus, I don't
share you concern, and I also think that even if an API redesign make be
good in general, it not the right thing to piggyback on this KIP.
I also think
Hi Victoria,
Earlier in the thread, I proposed rolling `Function
foreignKeyExtractor` and `Materialized>` into TableJoined as well. From that perspective, I partially
agree with John's latest alternative, but I think we should still retain
the `leftJoin`, `join`, and `outerJoin` methods on the KTa
Hi Victoria,
Here's the other conern I mentioned. I didn't bother
bringing it up before because it would be obviated by the
last proposal.
I'm wondering about the need to add four new overloads and a
new config object. We already have 20 KTable#join overloads,
out of a total of 47 KTable operatio
Thanks, Guozhang!
I agree it would be bigger in scope to approach it that way.
We certainly would need to implement a way for the
partitioner to flow down the topology, like we do with
serdes and grace period information. It's not super complex,
but not trivial either.
I think we've always been m
Thanks Victoria for writing the KIP! I think this is a miss when we
designed KIP-213 and should be fixed in syntax. Regarding how large its
scope should be, here's my thoughts:
1) Today Streams does not take any indicator on how the input stream/table
are partitioned, instead it simply assumes tha
Thanks for the KIP, Victoria!
This is a great catch. As fas as my memory serves, we
totally missed this case in the design of FK joins.
tl;dr: I'm wondering if it might be better to instead
introduce a way to specify the partitioning scheme when we
create a KTable and then just use it when we joi
Hi Victoria,
Thanks for the KIP; this is a beneficial addition.
I'm a +1 on the KIP and the changes made:
- Using a config object TableJoined
- Limiting the static methods to two
I have an additional "wild" thought about rolling the `Function
foreignKeyExtractor` and `Materialized>` into
Thanks, Matthias.
> > The existing methods which accept Named will be marked for deprecation
in 4.0.
> We can skip `in 4.0`. (1) The next release will be 3.1 (not 4.0) and (2)
a KIP could always slip into a future release.
This was actually a typo -- I meant that the methods will be marked for
d
Thanks for updating the KIP.
One nit:
The existing methods which accept Named will be marked for deprecation in 4.0.
We can skip `in 4.0`. (1) The next release will be 3.1 (not 4.0) and (2)
a KIP could always slip into a future release.
About `TableJoined`: It seems you propose to add sta
Hi Matthias,
Thanks for having a look at the KIP! I've updated it with your suggestion
to introduce a new `TableJoined` object with partitioners of type
`StreamPartitioner` and `StreamPartitioner`, and to
deprecate the existing FK join methods which accept a `Named` object
accordingly. I agree it
Thanks for the KIP Victoria.
As pointed out on the Jira ticket by you, using `` and `` as
partitioner types does not really work, because we don't have access to
the right value on the left side nor have we access to the left value on
the right hand side. -- I like your idea to use `Void` as v
Hi,
I've opened a small KIP for adding Kafka Streams support for foreign key
joins on tables with custom partitioners:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-775%3A+Custom+partitioners+in+foreign+key+joins
Feedback appreciated. Thanks!
- Victoria
13 matches
Mail list logo
