Moving this very old KIP into "discarded" state.
On 10/9/20 4:03 AM, Javier Freire Riobo wrote:
I see. It's great hidden functionality and all you have to do is define a
good API and expose it. I think the easiest thing would be to expose a
first operator, and then add more operators if they are claimed. I think it
is hidden for not complicating the API.
Thank you
El vie., 9 oct. 2020 a las 7:26, Matthias J. Sax (<mj...@apache.org>)
escribió:
I agree that there are cases when it is useful to get the old and new
value. In fact, the DSL internally often tracks old and new value via a
`Change` value type.
We did have some discussion that it might be useful to expose this
currently internal `Change` type in the public API. But if we do this,
it would not be limited to a single operator.
-Matthias
On 10/8/20 1:41 PM, Javier Freire Riobo wrote:
You're right. The behavior is correct with the cache disabled.
Anyway I think the operator I propose can be useful. The need to
generate a
value from the previous and current value of a record can be quite
common.
I think the only way to implement it is through an aggregate using a
helper
class. It is simpler and more natural to be able to receive the previous
and current values in a function.
Anyway thank you very much. I have been working with Kafka for a short
time, but I find it an amazing tool. Congratulations.
El jue., 8 oct. 2020 a las 21:10, Matthias J. Sax (<mj...@apache.org>)
escribió:
I guess I understand now.
However, it seems to be an "issue" with record caching. Setting the
commit interval to zero would flush the cache each time, but it is not
the "right" config change. You should just disable the `KTable` cache
instead.
You can disable caching globally by setting `cache.max.bytes.buffering`
configuration parameter to zero.
Or you can disable caching for an individual KTable via
`Materialized#withCachingDisabled()` that you can pass into your
`aggregation()` operator.
Thus, overall, I don't see the need for a new operator.
-Matthias
On 10/7/20 1:51 PM, Javier Freire Riobo wrote:
I have done a small demo example. I hope it serves as a clarification.
https://github.com/javierfreire/KTableToKStreamTest
Thank you very much
El mié., 7 oct. 2020 a las 3:01, Matthias J. Sax (<mj...@apache.org>)
escribió:
Thanks for the KIP.
I am not sure if I understand the motivation. In particular the KIP
says:
The main problem, apart from needing more code, is that if the same
event is received twice at the same time and the commit time is not 0,
the
difference is deleted and nothing is emitted.
Can you elaborate? Maybe you can provide a concrete example? I don't
understand the relationship between "the same event is received twice"
and a "non-zero commit time".
-Matthias
On 10/6/20 6:25 AM, Javier Freire Riobo wrote:
Hi all,
I'd like to propose these changes to the Kafka Streams API.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-675%3A+Convert+KTable+to+a+KStream+using+the+previous+value
This is a proposal to convert a KTable to a KStream knowing the
previous
value of the registry.
I also opened a proof-of-concept PR:
PR#9321: https://github.com/apache/kafka/pull/9381
What do you think?
Cheers,
Javier Freire
