Hi all, I'm working on enhancing the KTable foreign key join API to allow extracting foreign keys from both key and value. However, I've encountered a Java type erasure issue that affects our API design choices.
Current Situation: - We want to allow foreign key extraction from both key and value - We'd like to maintain backward compatibility with existing value-only extractors - Initial attempt was to use method overloading The Issue: Due to Java type erasure, we can't have both: - Function<V, KO> foreignKeyExtractor - Function<KeyValue<K, V>, KO> foreignKeyExtractor in method overloads as they become identical at runtime. Options: 1. Breaking Change Approach - Replace value-only extractor with key-value extractor - Clean API but breaks backward compatibility 2. Compatibility Approach - Keep both capabilities through different method names (e.g., join/joinWithKey) - Maintains compatibility but less elegant API Questions for Discussion: 1. Should we prioritize API elegance or backward compatibility? 2. If we choose compatibility, which naming convention should we use for the new methods? 3. Should we consider deprecating the value-only extractor in a future release? Looking forward to your thoughts. Best regards, Peter On Thu, Oct 31, 2024 at 1:08 PM Chu Cheng Li <peterx...@gmail.com> wrote: > Hi Everyone, > I would like to start a discussion on KIP-1104: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1103%3A+Additional+metrics+for+cooperative+consumption > <https://bitli.pro/2h2om_f8e74a0c> > > This KIP allow foreign key extraction from both key and value in KTable > Joins, before this KIP user can only extract foreign from record's value, > this KIP provides more flexibility on it. > > Regards, > Peter Lee > > >
