Moved this KIP into status "inactive". Feel free to resume and any time.
-Matthias
On 7/1/18 9:47 PM, Guozhang Wang wrote:
> Hello Adam,
>
> Sorry for being late on this thread. I've read through your updated wiki
> and here are some thoughts:
>
> * I agree with your assessed impediments. In fa
Hello Adam,
Sorry for being late on this thread. I've read through your updated wiki
and here are some thoughts:
* I agree with your assessed impediments. In fact, today although the
Global KTable have its own checkpoint files, and restoration process,
during its restoration it will always try to
Thanks for your help so far guys.
While I do think that I have a fairly reasonable way forward for
restructuring the topologies and threads, there is, unfortunately, what I
believe is a fatal flaw that cannot be easily resolved. I have updated the
page (
https://cwiki.apache.org/confluence/display
Hello Adam,
Please see my comments inline.
On Thu, Jun 21, 2018 at 8:14 AM, Adam Bellemare
wrote:
> Hi Guozhang
>
> *Re: Questions*
> *1)* I do not yet have a solution to this, but I also did not look that
> closely at it when I begun this KIP. I admit that I was unaware of exactly
> how the Gl
Hi Guozhang
*Re: Questions*
*1)* I do not yet have a solution to this, but I also did not look that
closely at it when I begun this KIP. I admit that I was unaware of exactly
how the GlobalKTable worked alongside the KTable/KStream topologies. You
mention "It means the two topologies will be merge
Hello Adam,
Thanks for proposing the KIP. A few meta comments:
1. As Matthias mentioned, the current GlobalKTable is designed to be
read-only, and not driving any computations (btw the global store backing a
GlobalKTable should also be read-only). Behind the scene the global store
updating task a
Matthias
Thanks for the links. I have seen those before but I will dig deeper into
them, especially around the CombinedKey and the flush + cache + rangescan
functionality. I believe Jan had a PR with many of the changes in there,
perhaps I can use some of the work that was done there to help lever
Adam,
thanks a lot for the KIP. I agree that this would be a valuable feature
to add. It's a very complex one though. You correctly pointed out, that
the GlobalKTable (or global stores in general) cannot be the "driver"
atm and are passively updated only. This is by design. Are you familiar
with t
Hi All
I created KIP-314 and I would like to initiate a discussion on it.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-314%3A+KTable+to+GlobalKTable+Bi-directional+Join
The primary goal of this KIP is to improve the way that Kafka can deal with
relational data at scale. This KIP would a
