at a segment-level,
instead of at an individual record level.
On Tue, Sep 25, 2018 at 5:18 PM Jan Filipiak
wrote:
Will that work? I expected it to blow up with ClassCastException or
similar.
You either would have to specify the window you fetch/put or iterate
across all windows the key was found
Hi,
Currently MirrorMaker is usually run collocated with the target cluster.
This is all nice and good. But one big obstacle in this was
always that group coordination happened on the source cluster. So when
then network was congested, you sometimes loose group membership and
have to rebalance
will also cause (a series) of MirrorMaker consumer
rebalances.
On Tue, Oct 16, 2018 at 7:58 AM Jan Filipiak
wrote:
Hi,
Currently MirrorMaker is usually run collocated with the target cluster.
This is all nice and good. But one big obstacle in this was
always that group coordination
no worries,
glad i could clarify
On 16.10.2018 15:14, Andrew Otto wrote:
> O ok apologies. Interesting!
>
> On Tue, Oct 16, 2018 at 9:06 AM Jan Filipiak
> wrote:
>
>> Hi Andrew,
>>
>> thanks for your message, you missed my point.
>>
>> Mirrormake
pport on a subsequent KIP that
> addresses consumer coordination and rebalance issues. Stay tuned!
>
> Ryanne
>
> On Tue, Oct 16, 2018 at 6:58 AM Jan Filipiak <mailto:jan.filip...@trivago.com>> wrote:
>
> Hi,
>
> Currently MirrorMaker is usually run collocat
Hi Zahari,
would you be willing to scan through the KIP-349 discussion a little?
I think it has suggestions that could be interesting for you
Best Jan
On 16.10.2018 09:29, Zahari Dichev wrote:
Hi there Kafka developers,
I am currently trying to find a solution to an issue that has been
manife
especially my suggestions ;)
On 18.10.2018 08:30, Jan Filipiak wrote:
Hi Zahari,
would you be willing to scan through the KIP-349 discussion a little?
I think it has suggestions that could be interesting for you
Best Jan
On 16.10.2018 09:29, Zahari Dichev wrote:
Hi there Kafka developers
Ryanne
On Wed, Oct 17, 2018 at 7:09 AM Jan Filipiak mailto:jan.filip...@trivago.com>> wrote:
This is not a performance optimisation. Its a fundamental design choice.
I never really took a look how streams does exactly once. (its a trap
anyways and you usually can deal wi
t
> an actual problem with the use os pause/resume ? Not sure how to explain
> the situation in a better way..
>
> Zahari
>
>
> On Thu, Oct 18, 2018 at 9:46 AM Zahari Dichev
> wrote:
>
>> Thanks a lot Jan,
>>
>> I will read it.
>>
>> Zaha
rg/mod_mbox/kafka-dev/201810.mbox/browser
> for previous discussion thread.
>
> I would also like to give a shout-out to Jan Filipiak who helped me out
> greatly in this project, and who led the initial work into this problem.
> Without Jan's help and insight I do not think this would have been possible
> to get to this point.
>
> Adam
>
/kafka/pull/5527
>
> See http://mail-archives.apache.org/mod_mbox/kafka-dev/201810.mbox/browser
> for previous discussion thread.
>
> I would also like to give a shout-out to Jan Filipiak who helped me out
> greatly in this project, and who led the initial work into this problem.
> With
On 07.11.2018 22:24, Adam Bellemare wrote:
> Bumping this thread, as per convention - 1
>
> On Fri, Nov 2, 2018 at 8:22 AM Adam Bellemare
> wrote:
>
>> As expected :) But still, thanks none-the-less!
>>
>> On Fri, Nov 2, 2018 at 3:36 AM Jan Filipiak
>
; bound on the number of lhs records that can reference rhs records. Offhand,
> I'd say we could page the values, so when one row is past the threshold, we
> append the key for the next page. Then in most cases, it would be a single
> key lookup, but for large fan-out updates, it would be on
>>> probably missed something here, but I'm wondering if we can avoid the
>>>>> HWM
>>>>>> tracking at all and resolve out-of-order during a final join
>>> instead...
>>>>>>
>>>>>> Let's say w
ect output should at least be decided on the offset of
the original message.
>
>
> That's all I have in mind now. Again, great appreciation to Adam to make
> such HUGE progress on this KIP!
>
>
> Guozhang
>
> On Wed, Dec 5, 2018 at 2:40 PM Jan Filipiak
> wrote:
On 02.01.2019 23:44, John Roesler wrote:
> However, you seem to have a strong intuition that the scatter/gather
> approach is better.
> Is this informed by your actual applications at work? Perhaps you can
> provide an example
> data set and sequence of operations so we can all do the math and ag
On 11.01.2019 21:29, John Roesler wrote:
> Hi Jan,
>
> Thanks for the reply.
>
> It sounds like your larger point is that if we provide a building block
> instead of the whole operation, then it's not too hard for users to
> implement the whole operation, and maybe the building block is
> independ
On 14.01.2019 02:48, n...@afshartous.com wrote:
>
> On reflection, it would be hard to describe the semantics of an API that
> tried to address starvation by temporarily disabling prioritization, and then
> oscillating back and forth.
> Thus I agree that it makes sense not to try and address sta
I made this for fun :) maybe you like it
https://imgflip.com/i/2r2y15
> The connection queue for Processors will be changed to ArrayBlockingQueue
> with a fixed size of 20. Acceptor will use round-robin allocation to allocate
> each new connection to the next available Processor to which the connection
> can be added without blocking. If a Processor's queue is f
ssors are full, we assign a Processor and block until the connection
> can be added. There is currently no timeout for this. The PR is here:
> https://github.com/apache/kafka/pull/6022
>
> Thanks,
>
> Rajini
>
> On Tue, Jan 15, 2019 at 12:02 PM Jan Filipiak
> wrote:
>
On 16.01.2019 14:05, Thomas Becker wrote:
> I'm going to bow out of this discussion since it's been made clear that
> the feature is not targeted at streams. But for the record, my desire is
> to have an alternative to the timestamp based message choosing strategy
> streams currently imposes, and
ate feedback on all the
points
Best Jan
On 27.10.2017 06:38, Jan Filipiak wrote:
Hello everyone,
this is the new discussion thread after the ID-clash.
Best
Jan
__
Hello Kafka-users,
I want to continue with the development of KAFKA-3705, which allows
the Streams DSL to perform KTa
Hi,
just want to throw my though in. In general the functionality is very
usefull, we should though not try to find the architecture to hard while
implementing.
The manual steps would be to
create a new topic
the mirrormake from the new old topic to the new topic
wait for mirror making to ca
ubscribe to? It will be helpful for discussion if you can provide more
detail on the interface change for this solution.
Thanks,
Dong
On Mon, Feb 26, 2018 at 12:48 AM, Jan Filipiak
wrote:
Hi,
just want to throw my though in. In general the functionality is very
usefull, we should tho
userland, it will be much more valuable. Do you have ideas on how this can
be done?
Not sure why the current KIP not help people who depend on log compaction.
Could you elaborate more on this point?
Thanks,
Dong
On Wed, Feb 28, 2018 at 10:55 PM, Jan Filipiak
wrote:
Hi Dong,
I tried to focus on w
cular implementation which never has
any epoch dependencies. To implement this, we would need some way for the
consumer to find out how many partitions there were in each epoch, but
maybe that's not too unreasonable.
Thanks,
Jason
On Mon, Mar 5, 2018 at 4:51 AM, Jan Filipiak
wrote:
Hi Don
Broker support is defenitly the preferred way here. I have nothing
against broker support.
I tried to say that for what I would preffer - copying the data over, at
least for log compacted topics -
I would require more broker support than the KIP currently offers.
Jun
On Tue, Mar 6, 2018
Map> dependencies(int
numPartitionsBeforeEpochBump,
int numPartitionsAfterEpochBump)
The unordered case then is just a particular implementation
which
never
has
any epoch dependencies. To implement this, we would need some
way
for
the
consumer to find out how many partitions there were in each
ot possible to
start a new application on? I think this is an overall flaw of the
discussed idea here. Not playing attention to the overall architecture."
Could you explain a bit more when one can't start a new application?
Jun
On Sat, Mar 10, 2018 at 1:40 AM, Jan Filipiak
wrote:
Hi
In
that case, in place partition expansion seems more desirable.
I understand your concern about adding complexity in KStreams. But, perhaps
we could iterate on that a bit more to see if it can be simplified.
Jun
On Mon, Mar 12, 2018 at 11:21 PM, Jan Filipiak
wrote:
Hi Jun,
I will focus o
pic needs to be reshuffled more than once
in different consumer groups during the transition phase.
What do you think?
Thanks,
Jun
On Thu, Mar 15, 2018 at 1:04 AM, Jan Filipiak
wrote:
Hi Jun,
thank you for following me on these thoughts. It was important to me to
feel that kind of under
t that this statestore RPC is a bad idea
and it was only invented to allow for optimisations that are better not
done.Not to say they are premature ;)
I hope it makes it clear
best jan
Thanks,
Jun
On Wed, Mar 21, 2018 at 6:57 AM, Jan Filipiak
wrote:
Hi Jun,
I was really seeing progress
o increase
consumer number, the existing consumer can backfill the existing data in
the change log topic to the same change log topic with the new partition
number, before the new set of consumers bootstrap state from the new
partitions of the change log topic. If this solution works, then coul
increase
consumers, the current KIP already supports in-order delivery
without
the
overhead of copying the data. For stream use-case that needs to
increase
consumer number, the existing consumer can backfill the existing
data
in
the change log topic to the same change log topic with the n
case. For core
Kafka
use-case, and for the stream use-case that does not need to
increase
consumers, the current KIP already supports in-order delivery
without
the
overhead of copying the data. For stream use-case that needs
to
increase
consumer number, the existing consumer can backfill t
, Apr 4, 2018 at 1:25 AM, Jan Filipiak
wrote:
Want to quickly step in here again because it is going places again.
The last part of the discussion is just a pain to read and completely
diverged from what I suggested without making the reasons clear to me.
I don't know why this happens her
Hi,
Happy to see that you want to make an effort here.
Regarding the ProcessSuppliers I couldn't find a way to not rewrite the
joiners + the merger.
The re-partitioners can be reused in theory. I don't know if repartition
is optimized in 2.0 now.
I made this
https://cwiki.apache.org/confluen
Sorry for missing the discussion
-1 nonbinding
see
https://samza.apache.org/learn/documentation/0.7.0/api/javadocs/org/apache/samza/system/chooser/MessageChooser.html
Best Jan
On 14.08.2018 03:19, n...@afshartous.com wrote:
Hi All,
Calling for a vote on KIP-349
https://cwiki.apache.org/co
ta would be helpful.
Thanks
Adam
On Mon, Aug 13, 2018 at 3:34 AM, Jan Filipiak
wrote:
Hi,
Happy to see that you want to make an effort here.
Regarding the ProcessSuppliers I couldn't find a way to not rewrite the
joiners + the merger.
The re-partitioners can be reused in theory. I don'
hash-join).
---
Back to your questions to KIP-213, I think Jan has summarized it
pretty-well. Note that back then we do not have headers support so we have
to do such "key-widening" approach to ensure ordering.
Guozhang
On M
7
Do I continue on this thread or do we begin a new one for discussion?
On Thu, Aug 16, 2018 at 1:40 AM, Jan Filipiak
wrote:
even before message headers, the option for me always existed to just wrap
the messages into my own custom envelop.
So I of course thought this through. One sentence in
On 20.08.2018 00:19, Matthias J. Sax wrote:
@Nick: A KIP is only accepted if it got 3 binding votes, ie, votes from
committers. If you close the vote before that, the KIP would not be
accepted. Note that committers need to pay attention to a lot of KIPs
and it can take a while until people can
Congrats Dong!
On 20.08.2018 16:35, Attila Sasvári wrote:
Congratulations Dong! Well deserved.
Regards,
Attila
Paolo Patierno ezt írta (időpont: 2018. aug. 20., H
15:09):
Congratulations Dong!
Paolo Patierno
Principal Software Engineer (IoT) @ Red Hat
Microsoft MVP on Azure & IoT
Microso
Still havent completly grabbed it.
sorry will read more
On 17.08.2018 21:23, Jan Filipiak wrote:
Cool stuff.
I made some random remarks. Did not touch the core of the algorithm yet.
Will do Monday 100%
I don't see Interactive Queries :) like that!
On 17.08.2018 20:28, Adam Bell
streams.
-Tommy
On Mon, 2018-08-20 at 12:52 +0200, Jan Filipiak wrote:
On 20.08.2018 00:19, Matthias J. Sax wrote:
@Nick: A KIP is only accepted if it got 3 binding votes, ie, votes from
committers. If you close the vote before that, the KIP would not be
accepted. Note that committers need to pay
On 30.08.2018 15:17, Matthias J. Sax wrote:
Nick,
Would be good to understand the difference between the current approach
and what Jan suggested. If we believe that the current proposal is too
limited in functionality and also hard to extend later on, it might make
sense to work on a more gen
.
Now, it is not possible to collide with user-defined headers.
That said, I'd also be fine with just reserving "__" as a header
prefix
and
not worrying about collisions.
Thanks for the KIP,
-John
On Tue, Aug 21, 2018 at 9:48 AM Jan Filipiak <
jan.filip...@trivago.com
wrote:
Hi Nick,
sorry for not beeing so helpfull. I don't quite understand what _this_
would be in your email.
Is this the part in question?
/interface TopicPrioritizer {
List prioritize(List topicPriorities);
}
//
//public void registerTopicPrioritizer(TopicPrioritizer topicPrioritizer);//
/
this
s in parallel across different nodes within the processor API, there
are no ordering guarantees and everything is simple first-come,
first-served(processed). If this is not the case then I am unaware of that
fact.
Thanks
Adam
On Mon, Sep 3, 2018 at 8:38 AM, Jan Filipiak
wrote:
Finished my
Just on remark here.
The high-watermark could be disregarded. The decision about the forward
depends on the size of the aggregated map.
Only 1 element long maps would be unpacked and forwarded. 0 element maps
would be published as delete. Any other count
of map entries is in "waiting for correc
On 05.09.2018 02:38, n...@afshartous.com wrote:
On Sep 4, 2018, at 4:20 PM, Jan Filipiak wrote:
what I meant is litterally this interface:
https://samza.apache.org/learn/documentation/0.7.0/api/javadocs/org/apache/samza/system/chooser/MessageChooser.html
<https://samza.apache.org/le
of the input tables as key of the output table. This
makes the keys of the output table unique and we can store both in the
output table:
Result would be ,
Thoughts?
-Matthias
On 9/4/18 1:36 PM, Jan Filipiak wrote:
Just on remark here.
The high-watermark could be disregarded. The decision ab
ly needed.
There might be some use-cases that need priorities eventually, but I'm
concerned that we're jumping the gun by trying to implement this before we know
what they are.
best,
Colin
On Wed, Sep 5, 2018, at 01:06, Jan Filipiak wrote:
On 05.09.2018 02:38, n...@afshartou
cs in fetch requests.
Overall, I get the impression that topic prioritization and
MessageChosser are orthogonal (or complementary) to each other.
-Matthias
On 9/6/18 5:24 AM, Jan Filipiak wrote:
On 05.09.2018 17:18, Colin McCabe wrote:
Hi all,
I agree that DISCUSS is more appropriate than VOTE at
currentStateAsMap.remove(toModifyKey);
if(currentStateAsMap.isEmpty()){
return null;
}
}
retrun asAggregateType(currentStateAsMap)
}
Thanks,
Adam
On Wed, Sep 5, 2018 at 3:35 PM, Jan Filipiak
wrote:
Thanks Adam for bri
wo streams? Is there somewhere I can
see the original requirement or proposal?
On Sep 7, 2018, at 8:13 AM, Jan Filipiak wrote:
On 05.09.2018 22:17, Adam Bellemare wrote:
I'm currently testing using a Windowed Store to store the highwater mark.
By all indications this should work fine, with
ng to guess.
Adam
On Sat, Sep 8, 2018 at 8:00 AM, Jan Filipiak
wrote:
Hi James,
nice to see you beeing interested. kafka streams at this point supports
all sorts of joins as long as both streams have the same key.
Adam is currently implementing a join where a KTable and a KTable can h
On Tue, Sep 11, 2018 at 10:07 AM, Prajakta Dumbre
mailto:dumbreprajakta...@gmail.com>> wrote:
please remove me from this group
On Tue, Sep 11, 2018 at 1:29 PM Jan Filipiak
mailto:jan.filip...@trivago.com>>
wrote:
> Hi Adam,
>
> give me some tim
t consider non-windowed KTable-KTable non-key joins for
now. In which case, KIP-258 should help.
Guozhang
On Wed, Sep 12, 2018 at 9:20 PM, Jan Filipiak
wrote:
On 11.09.2018 18:00, Adam Bellemare wrote:
Hi Guozhang
Current highwater mark implementation would grow endlessly based on
primary
store, not
the ProcessorSupplier.
Thanks,
Adam
On Mon, Sep 24, 2018 at 2:47 PM, Jan Filipiak
wrote:
On 24.09.2018 16:26, Adam Bellemare wrote:
@Guozhang
Thanks for the information. This is indeed something that will be
extremely
useful for this KIP.
@Jan
Thanks for your explanations. That being s
Hello,
to get KAFKA-3705 moving a little bit more I am thinking of starting a
KIP to discuss the finaly API design.
My confluence id is: "jfilipiak"
Best Jan
Hello everyone,
this is the new discussion thread after the ID-clash.
Best
Jan
__
Hello Kafka-users,
I want to continue with the development of KAFKA-3705, which allows the
Streams DSL to perform KTableKTable-Joins when the KTables have a
one-to-many relationship.
To make sure we cover
sing my previous comments ?
http://search-hadoop.com/m/Kafka/uyzND1hzF8SRzUqb?subj=Re+DISCUSS+KIP+213+Support+non+key+joining+in+KTable
On Thu, Oct 26, 2017 at 9:38 PM, Jan Filipiak
wrote:
Hello everyone,
this is the new discussion thread after the ID-clash.
Best
Jan
__
Hello Kafka-users,
es. Its a nightmare for JSON serdes.
Thanks,
Damian
On Fri, 27 Oct 2017 at 10:27 Ted Yu wrote:
I think if you explain what A and B are in the beginning, it makes sense to
use them since readers would know who they reference.
Cheers
On Thu, Oct 26, 2017 at 11:04 PM, Jan Filipiak
wrote:
-1 non binding
I don't get the motivation.
In 80% of my DSL processors there is no such thing as a reasonable
RecordContext.
After a join the record I am processing belongs to at least 2 topics.
After a Group by the record I am processing was created from multiple
offsets.
The API Design do
I think:
1 If there are many records with null, the partition they ended up in
might run hot.
2. we need a way to indicate to the iterator that this key is not to be
found. empty byte array would join against all records.
Null might work but no Serdes gonna give us null, they all produces some
Hi.
Im not 100 % up to date what version 1.0 DSL looks like ATM.
I just would argue that repartitioning should be an own API call like
through or something.
One can use through or to already to get this. I would argue one should
look there instead of overloads
Best Jan
On 04.11.2017 16:01,
do we drop it at the first sub-topology (i.e the
"Repartition by A's key" block)?
Guozhang
On Wed, Nov 1, 2017 at 12:18 PM, Jan Filipiak
wrote:
Hi thanks for the feedback
On 01.11.2017 12:58, Damian Guy wrote:
Hi Jan, Thanks for the KIP!
In both alternatives the API will ne
s. However, users gets some more control about
topic parameters like number of partitions (we should discuss what other
configs would be useful).
Does this make sense?
-Matthias
On 11/5/17 1:21 AM, Jan Filipiak wrote:
Hi.
Im not 100 % up to date what version 1.0 DSL looks like ATM.
I just w
Id Say this is trivial enough for users to implement on their own?
-1 I guess
On 06.11.2017 10:53, Jakub Scholz wrote:
Hi all,
Just a friendly reminder that this is still up for vote. If you think this
is a good fetrue, please give it your vote.
Regards
Jakub
On Tue, Oct 3, 2017 at 11:24 PM,
non-binding of course
On 06.11.2017 12:42, Jan Filipiak wrote:
Id Say this is trivial enough for users to implement on their own?
-1 I guess
On 06.11.2017 10:53, Jakub Scholz wrote:
Hi all,
Just a friendly reminder that this is still up for vote. If you think
this
is a good fetrue, please
that
using RecordContext on a KTable that is the result of an aggregation is
questionable. But I don't see a reason to down vote the KIP for this reason.
WDYT about this?
-Matthias
On 11/1/17 10:19 PM, Jan Filipiak wrote:
-1 non binding
I don't get the motivation.
In 80% of m
lly, thus, it's still an
internal topic and Streams create the topic name automatically as for
all other internal topics. However, users gets some more control about
topic parameters like number of partitions (we should discuss what other
configs would be useful).
Does this make sense?
-Matt
11/5/17 2:09 AM, Jan Filipiak wrote:
Hi Gouzhang
I hope the wikipage looks better now. made a little more effort into the
diagram. Still not ideal but I think it serves its purpose.
On 02.11.2017 01:17, Guozhang Wang wrote:
Thanks for the KIP writeup Jan. I made a first pass and here are
the joining
task serializes the effects.
Best Jan
On 06.11.2017 21:20, Jan Filipiak wrote:
Will do! Need to do it carefully. One mistake in this detailed
approach and confusion is perfect ;)
Hope I can deliver this week.
Best Jan
On 06.11.2017 17:21, Matthias J. Sax wrote:
Jan,
thanks a lot
12:31 PM, Jan Filipiak
wrote:
I created an example Table in the WIKI page
Can you quickly check if that would be a good format?
I tried todo it ~like the unit tests but with the information of what
state is there _AFTER_
processing happend.
I make the first 2 columns exclusive even though the in
direct solution.
Perhaps we should continue this discussion in DISCUSS thread.
Cheers,
Jeyhun
On Mon, Nov 6, 2017 at 9:14 PM Jan Filipiak
wrote:
Hi.
I do understand that it might come in Handy.
From my POV in any relational algebra this is only a projection.
Currently we hide these "fi
which users
still need to do manual customizing.
-Matthias
On 11/7/17 5:54 AM, Jan Filipiak wrote:
I Aggree completely.
Exposing this information in a place where it has no _natural_ belonging
might really be a bad blocker in the long run.
Concerning your first point. I would argue its not
On 07.11.2017 12:59, Jan Filipiak wrote:
On 07.11.2017 11:20, Matthias J. Sax wrote:
About implementation if we do the KIP as proposed: I agree with Guozhang
that we would need to use the currently processed record's metadata in
the context. This does leak some implementation details,
good
time to double check on such stages to make sure we are not introducing any
regressions.
Guozhang
On Mon, Nov 6, 2017 at 8:54 PM, Jan Filipiak
wrote:
I Aggree completely.
Exposing this information in a place where it has no _natural_ belonging
might really be a bad blocker in th
11/9/17 9:13 PM, Jan Filipiak wrote:
Okay,
looks like it would _at least work_ for Cached KTableSources .
But we make it harder to the user to make mistakes by putting
features into places where they don't make sense and don't
help anyone.
I once again think that my suggestion is
ore
of through() (add more overloads) instead of introducing a separate set
of
overloads to other methods.
I will update the KIP soon based on the discussion and inform.
Cheers,
Jeyhun
On Mon, Nov 6, 2017 at 9:18 PM Jan Filipiak
wrote:
Sorry for not beeing 100% up to date.
Back then we had
Feel free to continue the discussion w/o the table. I want
to finish the table during next week.
Best Jan thank you for your interest!
_ Jira Quote ______
Jan Filipiak
<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jfilipiak>
Please bear with me while I try to get cau
.
On 16.11.2017 18:02, Trevor Huey wrote:
Ah, I think I see the problem now. Thanks for the explanation. That is
tricky. As you said, it seems the easiest solution would just be to
flush the cache. I wonder how big of a performance hit that'd be...
On Thu, Nov 16, 2017 at 9:07 AM Jan Fil
Nov 16, 2017 at 1:03 PM, Matthias J. Sax
wrote:
Any thoughts about my latest proposal?
-Matthias
On 11/10/17 10:02 PM, Jan Filipiak wrote:
Hi,
i think this is the better way. Naming is always tricky Source is kinda
taken
I had TopicBackedK[Source|Table] in mind
but for the user its way bett
the resulted records and claim "we cannot infer its traced
context anymore"?
Guozhang
On Thu, Nov 16, 2017 at 1:03 PM, Matthias J. Sax
wrote:
Any thoughts about my latest proposal?
-Matthias
On 11/10/17 10:02 PM, Jan Filipiak wrote:
Hi,
i think this is the better way. Naming
completely.
Cheers,
Jeyhun
On Sat 18. Nov 2017 at 07:49, Jan Filipiak wrote:
Yes, the mail said only join so I wanted to clarify.
On 17.11.2017 19:05, Matthias J. Sax wrote:
Yes. But I think an aggregation is an many-to-one operation, too.
For the stripping off part: internally, we can just
think hard to figure it out)
will add this
Last but not least:
But noone is really interested.
This was the first time some took the effort to address the most
pressuring issue moving this forward.
I counted this as not beeing interested before.
Don't understand this statement...
between the different used types, K0,K1,KO
should be explains explicitly (all information is there implicitly, but
one need to think hard to figure it out)
Last but not least:
But noone is really interested.
Don't understand this statement...
-Matthias
On 11/16/17 9:05 AM, Jan
hard to figure it out)
Last but not least:
But noone is really interested.
Don't understand this statement...
-Matthias
On 11/16/17 9:05 AM, Jan Filipiak wrote:
We are running this perfectly fine. for us the smaller table changes
rather infrequent say. only a few times per day. The p
ld be way smaller. Also widely escalates the context
of the KIP
Just wanted to lay the ground for discussions so we are all on the same
page before chatting more.
Guozhang
On Sat, Nov 18, 2017 at 3:10 AM, Jan Filipiak
wrote:
Hi,
not an issue at all. IMO
the approach that is on the tab
a remark of mine that got missed during migration:
There is this problem that even though we have source.table.filter.join
the state-fullness happens at the table step not a the join step. In a
filter
we still going to present change.oldValue to the filter even though the
record context() is fo
Value". Are you referring to `KTableFilter#process()`? If yes
could you point to me which LOC are you concerning about?
Guozhang
On Mon, Nov 20, 2017 at 9:29 PM, Jan Filipiak
wrote:
a remark of mine that got missed during migration:
There is this problem that even thou
this
related to KIP-159?
Btw: with KIP-182, I am wondering if this would not be possible, by
putting a custom dummy store into the table and materialize the filter
result afterwards? It's not a nice way to do, but seems to be possible.
-Matthias
On 11/23/17 4:56 AM, Jan Filipiak wrote:
on the internal topic: I think many readers may
not get the schema of this topic, so it is better to indicate that what
would be the key of this internal topic used for compaction, and what would
be used as the partition-key.
Guozhang
On Sat, Nov 18, 2017 at 2:30 PM, Jan Filipiak
wrote:
->
Hi Colin, thank you for this KIP, it can become a really useful thing.
I just scanned through the discussion so far and wanted to start a
thread to make as decision about keeping the
cache with the Connection / Session or having some sort of UUID indexed
global Map.
Sorry if that has been se
tract, then they may need to get this themselves (embed such information
in the value payload, for example).
If people do not like the second idea, I'd suggest we hold on pursuing the
first direction since to me its beneficial scope is too limited compared to
its cost.
Guozhang
On Fri, Nov 24, 20
k it may be useful if the KIP can include the example workflow
of how this feature will be used in case of partition change and so on.
Yeah, that might help.
best,
Colin
Thanks,
Dong
On Wed, Nov 29, 2017 at 12:13 PM, Colin McCabe
wrote:
I updated the KIP with the ideas we've been discu
nts to the server. Not sure if this
will introduce some complexity to the broker. Intuitively it seems fine.
The logic would basically be similar to the draining logic in the
RecordAccumulator of the producer.
Thanks,
Jiangjie (Becket) Qin
On Thu, Nov 30, 2017 at 11:29 PM, Jan Filipiak
wrote:
1 - 100 of 138 matches
Mail list logo
