Andre,
Back to your original question I think KIP-138 could help, as it allows you
to punctuate based on processing time as well.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-138%3A+Change+punctuate+semantics
Guozhang
On Mon, Jun 19, 2017 at 2:02 PM, Andre Eriksson wrote:
> So I'm tr
Two comments:
(1) This approach is basically "within" the processor, as it does use
ProcessorContext -- "outside" would mean to use IQ API.
(2) As reported, this does not work and fails with an exception and this
is by design and thus we won't change it.
-Matthias
On 6/20/17 10:10 AM, Michal
Thanks, maybe I miss-read it:
I then tried implementing my own scheduling that periodically sends/clears out
messages using the ProcessorContext provided to the aforementioned transform ste
I understood it to say it inspects a state store, sends the messages
that should be sent and removes the
>> I didn't know you could write to state stores from outside a
>> processor/transformer.
You can't. And as far as I understand this thread, nobody said you can.
Did I miss something?
-Matthias
On 6/20/17 1:02 AM, Michal Borowiecki wrote:
> I didn't know you could write to state stores from ou
I didn't know you could write to state stores from outside a
processor/transformer. Interesting to hear that it is working although
I'd be careful as KIP-67 warns it can introduce undefined behaviour:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-67%3A+Queryable+state+for+Kafka+Streams
> On Jun 19, 2017, at 2:02 PM, Andre Eriksson wrote:
>
> I then tried implementing my own scheduling that periodically sends/clears
> out messages using the ProcessorContext provided to the aforementioned
> transform step. However, it seems that when I call forward() from my
> scheduler (i.e.
So I'm trying to implement a rate limiting processing step using Kafka Streams
(0.10.2.1).
Basically, this step should just let messages through, unless similar messages
have already been seen in the last N seconds, in which case it should aggregate
them into a single message and then send them
