I can just second was Lucas and Bill said already.
1. We cannot break compatibility
2. BiFunction sounds like a good alternative
3. I would personally deprecate the existing method, but don't feel
strong about it.
-Matthias
On 11/12/24 8:33 AM, Bill Bejeck wrote:
Hi Peter,
It's important that we don't break compatibility.
We faced a similar situation in KIP-149
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner>
when we
provided access to the key in mapping and joining. I think it's worth
exploring Lucas's suggestion of using a `BiFunction`.
But if for some reason it won't work (I don't see why it wouldn't) we'll
need to maintain compatibly and go with the renaming the new method.
Thanks,
Bill
On Tue, Nov 12, 2024 at 8:06 AM Lucas Brutschy
<lbruts...@confluent.io.invalid> wrote:
Hi,
1. I don't think we can/should break backwards compatibility.
2. Have you considered using `BiFunction<K,V,KO> foreignKeyExtractor`
? This should work without renaming the method.
3. I don't see the benefit of deprecating it. I agree, we wouldn't add
both overloads normally, but the value-only overload is useful /
correct as it is, I wouldn't break users code just to "clean" things
very slighly.
Cheers,
Lucas
On Mon, Nov 11, 2024 at 6:43 PM Chu Cheng Li <peterx...@gmail.com> wrote:
Hi all,
I'm working on enhancing the KTable foreign key join API to allow
extracting foreign keys from both key and value. However, I've
encountered
a Java type erasure issue that affects our API design choices.
Current Situation:
- We want to allow foreign key extraction from both key and value
- We'd like to maintain backward compatibility with existing value-only
extractors
- Initial attempt was to use method overloading
The Issue:
Due to Java type erasure, we can't have both:
- Function<V, KO> foreignKeyExtractor
- Function<KeyValue<K, V>, KO> foreignKeyExtractor
in method overloads as they become identical at runtime.
Options:
1. Breaking Change Approach
- Replace value-only extractor with key-value extractor
- Clean API but breaks backward compatibility
2. Compatibility Approach
- Keep both capabilities through different method names (e.g.,
join/joinWithKey)
- Maintains compatibility but less elegant API
Questions for Discussion:
1. Should we prioritize API elegance or backward compatibility?
2. If we choose compatibility, which naming convention should we use for
the new methods?
3. Should we consider deprecating the value-only extractor in a future
release?
Looking forward to your thoughts.
Best regards,
Peter
On Thu, Oct 31, 2024 at 1:08 PM Chu Cheng Li <peterx...@gmail.com>
wrote:
Hi Everyone,
I would like to start a discussion on KIP-1104:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1103%3A+Additional+metrics+for+cooperative+consumption
<https://bitli.pro/2h2om_f8e74a0c>
This KIP allow foreign key extraction from both key and value in KTable
Joins, before this KIP user can only extract foreign from record's
value,
this KIP provides more flexibility on it.
Regards,
Peter Lee
