Hi Peter, It's important that we don't break compatibility. We faced a similar situation in KIP-149 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner> when we provided access to the key in mapping and joining. I think it's worth exploring Lucas's suggestion of using a `BiFunction`. But if for some reason it won't work (I don't see why it wouldn't) we'll need to maintain compatibly and go with the renaming the new method.
Thanks, Bill On Tue, Nov 12, 2024 at 8:06 AM Lucas Brutschy <lbruts...@confluent.io.invalid> wrote: > Hi, > > 1. I don't think we can/should break backwards compatibility. > 2. Have you considered using `BiFunction<K,V,KO> foreignKeyExtractor` > ? This should work without renaming the method. > 3. I don't see the benefit of deprecating it. I agree, we wouldn't add > both overloads normally, but the value-only overload is useful / > correct as it is, I wouldn't break users code just to "clean" things > very slighly. > > Cheers, > Lucas > > On Mon, Nov 11, 2024 at 6:43 PM Chu Cheng Li <peterx...@gmail.com> wrote: > > > > Hi all, > > > > I'm working on enhancing the KTable foreign key join API to allow > > extracting foreign keys from both key and value. However, I've > encountered > > a Java type erasure issue that affects our API design choices. > > > > Current Situation: > > - We want to allow foreign key extraction from both key and value > > - We'd like to maintain backward compatibility with existing value-only > > extractors > > - Initial attempt was to use method overloading > > > > The Issue: > > Due to Java type erasure, we can't have both: > > - Function<V, KO> foreignKeyExtractor > > - Function<KeyValue<K, V>, KO> foreignKeyExtractor > > in method overloads as they become identical at runtime. > > > > Options: > > 1. Breaking Change Approach > > - Replace value-only extractor with key-value extractor > > - Clean API but breaks backward compatibility > > > > 2. Compatibility Approach > > - Keep both capabilities through different method names (e.g., > > join/joinWithKey) > > - Maintains compatibility but less elegant API > > > > Questions for Discussion: > > 1. Should we prioritize API elegance or backward compatibility? > > 2. If we choose compatibility, which naming convention should we use for > > the new methods? > > 3. Should we consider deprecating the value-only extractor in a future > > release? > > > > Looking forward to your thoughts. > > > > Best regards, > > Peter > > > > On Thu, Oct 31, 2024 at 1:08 PM Chu Cheng Li <peterx...@gmail.com> > wrote: > > > > > Hi Everyone, > > > I would like to start a discussion on KIP-1104: > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1103%3A+Additional+metrics+for+cooperative+consumption > > > <https://bitli.pro/2h2om_f8e74a0c> > > > > > > This KIP allow foreign key extraction from both key and value in KTable > > > Joins, before this KIP user can only extract foreign from record's > value, > > > this KIP provides more flexibility on it. > > > > > > Regards, > > > Peter Lee > > > > > > > > > >
