/jira/browse/KAFKA-5488
PR#6164: https://github.com/apache/kafka/pull/6164
Regards,
Ivan Ponomarev
Congratulations, Bill!
Your 'Kafka Streams in Action' is a great book. These months it is
always travelling with me in my backpack with my laptop ))
Regards,
Ivan
14.02.2019 3:56, Guozhang Wang пишет:
Hello all,
The PMC of Apache Kafka is happy to announce that we've added Bill Bejeck
as o
-418%3A+A+method-chaining+way+to+branch+KStream
JIRA KAFKA-5488: https://issues.apache.org/jira/browse/KAFKA-5488
PR#6164: https://github.com/apache/kafka/pull/6164
Regards,
Ivan Ponomarev
dynamically build of branches off of KBranchedStreams if desired.
Thanks,
Paul
On Sat, Mar 23, 2019 at 4:28 PM Ivan Ponomarev
wrote:
Hi Bill,
Thank you for your reply!
This is how I usually do it:
void handleFirstCase(KStream ks){
ks.filter().mapValues(...)
}
vo
ariable so you can call as many operations on it as you desire.
Thoughts?
Best,
Paul
On Sun, Mar 24, 2019 at 2:02 PM Ivan Ponomarev wrote:
Hello Paul,
I afraid this won't work because we do not always need the
defaultBranch. And without a terminal operation we don't know when to
an exception in runtime.
Colleagues, what are your thoughts? Can we do better?
Regards,
Ivan
27.03.2019 18:46, Ivan Ponomarev пишет:
25.03.2019 17:43, Ivan Ponomarev пишет:
Paul,
I see your point when you are talking about
stream..branch..branch...default..
Still, I believe that this c
l this means is that for either solution,
deprecating the existing branch(...) is not on the table.
Thanks,
Paul
On Wed, Mar 27, 2019 at 12:08 PM Ivan Ponomarev wrote:
OK, let me summarize what we have discussed up to this point.
First, it seems that it's commonly agreed that branch API n
round though.
[Also, great to hear additional interest in 401, I’m excited to hear your
thoughts!]
Paul
On Mar 28, 2019, at 4:00 AM, Ivan Ponomarev wrote:
Hi Paul!
The idea to postpone the wiring of branches to the streamsBuilder.build() also
looked great for me at first glance, but ---
the ne
Hi all!
I was about to write another KIP, but found out that KIP-401 addresses
exactly the problem I faced. So let me jump into your discussion and ask
you to assess another idea.
I fully agree with the KIP-401's motivation part. E. g in my project I
had to invent a wrapper class that hides
eneral structure would work, it's just a
little sloppy overall in terms of naming and clarity. In particular,
passing in the "predicates" and "children" lists which get modified in
KBranchedStream but read from all the way KStreamLazyBranch is a bit
complicated to follow.
f
`defaultBranch()` is.
You did not comment on the idea to add a `KBranchedStream#get(int
index)
-> KStream` method to get the individually branched-KStreams. Would
be
nice to get your feedback about it. It seems you suggest that users
would need to write custom utility code otherwise, to
lcate.jo...@gmail.com> wrote:
This will be a great addition. Thanks Ivan!
+1 (non-binding)
On Fri, Jun 26, 2020 at 7:07 PM John Roesler
wrote:
Thanks, Ivan!
I’m +1 (binding)
-John
On Thu, May 28, 2020, at 17:24, Ivan Ponomarev wrote:
Hello all!
I'd like to start the vote for KIP-41
/pull/9210
Regards,
Ivan Ponomarev
Sorry, I forgot to add [DISCUSS] tag to the topic
24.08.2020 2:27, Ivan Ponomarev пишет:
Hello,
I'd like to start a discussion for KIP-655.
KIP-655:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-655%3A+Windowed+Distinct+Operation+for+Kafka+Streams+API
I also opened a pro
? It might be better to have one overload
with all require parameters, and add optional parameters using the
builder pattern? This seems to follow the DSL Grammer proposal.
(5) Even if it might be an implementation detail (and maybe the KIP
itself does not need to mention it), can you give a hig
r) solution, I don’t see any real drawbacks
for the dynamic case.
IMO the one trade off to consider at this point is the scope
question. I don’t know if I totally agree that “we rarely need them
in the same scope” since merging the branches back together later
seems like a pe
bject to name operators. It seems reasonable to me to build on this.
Furthermore, John did a writeup about "DSL design principles" that we
want to follow:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+DSL+Grammar
-- might be worth to checkout.
-Matthias
On 4/17/20 4:3
Hello everyone,
will someone please take a look at the reworked KIP?
I believe that now it follows design principles and takes into account
all the arguments discussed here.
Regards,
Ivan
23.04.2020 2:45, Ivan Ponomarev пишет:
Hi,
I have read the John's "DSL design principles
Sink` has the issue that it might be confused with a "sink
node", ie., writing the KStream to a topic.
Maybe `withJavaConsumer` would make it less ambiguous?
-Matthias
On 5/8/20 7:13 AM, John Roesler wrote:
Hi Ivan,
It looks like you missed my reply on Apr 23rd. I think it’s close,
eam in the result). Furthermore, we might need to allow `return
null` in your prosal to allow uses to "hide" a sub-stream in the Map.
I guess I can be convinced either way. However, if we follow your
proposal, I am wondering if we need `withJavaConsumer` at all? Its
benefit seems to
, but I'm not sure it will actually work. You might want to give it a
try. In the past, we've run into soe truly strange interactions between the
Java type inferencer and lambdas (and/or anonymous inner classes) in
combination with nested covariant types.
Another issue is that the
Hello all!
I'd like to start the vote for KIP-418 which proposes deprecation of
current `branch` method and provides a method-chaining based API for
branching.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-418%3A+A+method-chaining+way+to+branch+KStream
Regards,
Ivan
rV>> etc. instead
I have updated the KIP.
Does anyone want to object these changes?
Regards,
Ivan
06.07.2020 21:43, Matthias J. Sax пишет:
I am late, but I am also +1 (binding).
-Matthias
On 7/6/20 2:16 AM, Ivan Ponomarev wrote:
Wow!
So excited to hear that!
Thanks for your collabo
s
or
#aggregate. I get that there's some precedent for 'distinct' specifically,
but
maybe something like 'deduplicate' would be more appropriate for the Streams
API.
WDYT?
On Mon, Sep 14, 2020 at 10:04 AM Ivan Ponomarev
wrote:
Hi Matthias,
Thanks for your review! It ma
indowing function without having to carve out
a separate domain just for `distinct()`. All the rest of the KStream
operations would also benefit.
What do you think?
Thanks,
John
On Sun, May 23, 2021, at 08:09, Ivan Ponomarev wrote:
Hello everyone,
let me revive the discussion for KIP-655. No
adding distinct() to the sliding window interface, but hopping
windows are just a different parameterization of epoch-aligned windows. It
seems we can’t do much about that except document the issue.
Thanks,
John
On Wed, May 26, 2021, at 10:14, Ivan Ponomarev wrote:
Hi John!
I think that your p
Hello,
I'd like to start a discussion for KIP-759:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-759%3A+Unneeded+repartition+canceling
This is an offshoot of the discussion of KIP-655 for a `distinct`
operator, which turned out to be a separate proposal.
The proposal is quite trivia
fect. We can avoid adding distinct() to the sliding window
interface, but hopping windows are just a different parameterization
of epoch-aligned windows. It seems we can’t do much about that except
document the issue.
Thanks,
John
On Wed, May 26, 2021, at 10:14, Ivan Ponomarev wrote:
Hi John!
I th
ummarize, you are now only proposing the zero-arg distict() method to be
added to TimeWindowedKStream and SessionWindowedKStream, right?
I’m in favor of this proposal.
Thanks,
John
On Sat, Jul 10, 2021, at 10:18, Ivan Ponomarev wrote:
Hello everyone,
I would like to remind you about KIP-655 and
Hello all!
I'd like to start the vote for KIP-655 which proposes the zero-arg
distict() method to be added to TimeWindowedKStream and
SessionWindowedKStream.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-655%3A+Windowed+Distinct+Operation+for+Kafka+Streams+API
Regards,
Ivan
Thanks,
John
On Sat, Jul 10, 2021, at 10:18, Ivan Ponomarev wrote:
Hello everyone,
I would like to remind you about KIP-655 and KIP-759 just in case they
got lost in your inbox.
Now the initial proposal is split into two independent and smaller ones,
so it must be easier to review them. Of cour
ed to let you all hash it out.
Thanks,
-John
On Tue, 2021-07-20 at 11:42 +0300, Ivan Ponomarev wrote:
Hi all,
1. Actually I always thought about the serialized byte array only -- at
least this is what local stores depend upon, and what Kafka itself
depends upon when doing log compaction.
I can
be
stateful, right? In that case, I agree that comparing
serialized values seems to be way to do it.
2. Thanks for the confirmation
3. I continue to be satisfied to let you all hash it out.
Thanks,
-John
On Tue, 2021-07-20 at 11:42 +0300, Ivan Ponomarev wrote:
Hi all,
1. Actually I alway
osed named though (the good old "naming
things" discussion :))
Did you consider alternatives? What about
- markAsPartitioned()
- markAsKeyed()
- skipRepartition()
Not sure if there are other idea on a good name?
-Matthias
On 6/24/21 7:45 AM, Ivan Ponomarev wrote:
Hello,
uot;cousin" should
yield the same result? -- Given your example it seems you want to keep
the first record base on offset order. Wondering why?
While I agree that deduplication for overlapping window is questionable,
I am still wondering if you plan to disallow it (by adding a runtime
check an
ver, because we can now
stick any additional operator properties into that same config object.
WDYT? By the way, the above idea (introducing a single config object to
wrap all operator properties) was also
raised by John Roesler a while back. Let's hope he hasn't changed his mind
s
runtime check that applies
the Partitioner
associated with that subtopology to the key being processed and the mapped
key result
to assert that they do indeed match. Imo this is probably overkill, just
putting it out there.
On Sat, Aug 7, 2021 at 1:42 PM Ivan Ponomarev
wrote:
Hi Sophie,
than
ble similar to
KStream-KStream state stores...).
To recap: Could it be that the idea to apply a DISTINCT-aggregation is
for a different use-case than to remove duplicate messages from a KStream?
-Matthias
On 8/6/21 4:12 AM, Ivan Ponomarev wrote:
- Why restrict de-duplication for the key o
Ivan Ponomarev created KAFKA-10369:
--
Summary: Introduce Distinct operation in KStream
Key: KAFKA-10369
URL: https://issues.apache.org/jira/browse/KAFKA-10369
Project: Kafka
Issue Type
Ivan Ponomarev created KAFKA-12230:
--
Summary: Some Kafka TopologyTestDriver-based unit tests can't be
run on Windows file system
Key: KAFKA-12230
URL: https://issues.apache.org/jira/browse/KAFKA-
[
https://issues.apache.org/jira/browse/KAFKA-12230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Ponomarev resolved KAFKA-12230.
Resolution: Duplicate
Duplicate of KAFKA-12190
> Some Kafka TopologyTestDriver-ba
41 matches
Mail list logo
