Thanks guys ! BiFunction seems promising! KIP updated! https://cwiki.apache.org/confluence/display/KAFKA/KIP-1104%3A+Allow+Foreign+Key+Extraction+from+Both+Key+and+Value+in+KTable+Joins <https://bitli.pro/2i7Ja_1c6b6a68>
On Wed, Nov 13, 2024 at 8:30 AM Matthias J. Sax <mj...@apache.org> wrote: > I can just second was Lucas and Bill said already. > > 1. We cannot break compatibility > 2. BiFunction sounds like a good alternative > 3. I would personally deprecate the existing method, but don't feel > strong about it. > > > -Matthias > > > On 11/12/24 8:33 AM, Bill Bejeck wrote: > > Hi Peter, > > > > It's important that we don't break compatibility. > > We faced a similar situation in KIP-149 > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner > > > > when we > > provided access to the key in mapping and joining. I think it's worth > > exploring Lucas's suggestion of using a `BiFunction`. > > But if for some reason it won't work (I don't see why it wouldn't) we'll > > need to maintain compatibly and go with the renaming the new method. > > > > Thanks, > > Bill > > > > On Tue, Nov 12, 2024 at 8:06 AM Lucas Brutschy > > <lbruts...@confluent.io.invalid> wrote: > > > >> Hi, > >> > >> 1. I don't think we can/should break backwards compatibility. > >> 2. Have you considered using `BiFunction<K,V,KO> foreignKeyExtractor` > >> ? This should work without renaming the method. > >> 3. I don't see the benefit of deprecating it. I agree, we wouldn't add > >> both overloads normally, but the value-only overload is useful / > >> correct as it is, I wouldn't break users code just to "clean" things > >> very slighly. > >> > >> Cheers, > >> Lucas > >> > >> On Mon, Nov 11, 2024 at 6:43 PM Chu Cheng Li <peterx...@gmail.com> > wrote: > >>> > >>> Hi all, > >>> > >>> I'm working on enhancing the KTable foreign key join API to allow > >>> extracting foreign keys from both key and value. However, I've > >> encountered > >>> a Java type erasure issue that affects our API design choices. > >>> > >>> Current Situation: > >>> - We want to allow foreign key extraction from both key and value > >>> - We'd like to maintain backward compatibility with existing value-only > >>> extractors > >>> - Initial attempt was to use method overloading > >>> > >>> The Issue: > >>> Due to Java type erasure, we can't have both: > >>> - Function<V, KO> foreignKeyExtractor > >>> - Function<KeyValue<K, V>, KO> foreignKeyExtractor > >>> in method overloads as they become identical at runtime. > >>> > >>> Options: > >>> 1. Breaking Change Approach > >>> - Replace value-only extractor with key-value extractor > >>> - Clean API but breaks backward compatibility > >>> > >>> 2. Compatibility Approach > >>> - Keep both capabilities through different method names (e.g., > >>> join/joinWithKey) > >>> - Maintains compatibility but less elegant API > >>> > >>> Questions for Discussion: > >>> 1. Should we prioritize API elegance or backward compatibility? > >>> 2. If we choose compatibility, which naming convention should we use > for > >>> the new methods? > >>> 3. Should we consider deprecating the value-only extractor in a future > >>> release? > >>> > >>> Looking forward to your thoughts. > >>> > >>> Best regards, > >>> Peter > >>> > >>> On Thu, Oct 31, 2024 at 1:08 PM Chu Cheng Li <peterx...@gmail.com> > >> wrote: > >>> > >>>> Hi Everyone, > >>>> I would like to start a discussion on KIP-1104: > >>>> > >>>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1103%3A+Additional+metrics+for+cooperative+consumption > >>>> <https://bitli.pro/2h2om_f8e74a0c> > >>>> > >>>> This KIP allow foreign key extraction from both key and value in > KTable > >>>> Joins, before this KIP user can only extract foreign from record's > >> value, > >>>> this KIP provides more flexibility on it. > >>>> > >>>> Regards, > >>>> Peter Lee > >>>> > >>>> > >>>> > >> > > >
