Hi Kumud, Thanks for the KIP. I'm a little bit skeptical about the necessity for this predicate but I think we may be able to satisfy your requirements with a slightly different approach. The motivation section deals largely with skipping the invocation of SMTs that expect a certain field to be present in a record, and will fail if it is not present. This seems like a reasonable use case; even when your data has a fixed schema, optional fields are still possible, and preventing SMTs from being used on optional fields seems unnecessarily restrictive. In fact, it seems like such a reasonable use case that I wonder if it'd be worth investing in new SMT-level properties to handle this case, instead of requiring users to configure a predicate separately alongside them? Something like an "on.field.missing" property with options of "skip", "fail", and/or "warn" could do the trick.
It's also worth noting that the proposed syntax for the "field.path" property in the HasField predicate would make it impossible to refer to field names that have dots in them. It hasn't been finalized yet but if we do end up going this route, we should probably use the same syntax that's agreed upon for KIP-821 [1]. [1] - https://cwiki.apache.org/confluence/display/KAFKA/KIP-821%3A+Connect+Transforms+support+for+nested+structures Cheers, Chris On Thu, May 26, 2022 at 5:28 AM Sagar <sagarmeansoc...@gmail.com> wrote: > Hi Kumud, > > Thanks for that. I don't have any other comments at this point on the KIP. > LGTM overall. > > Thanks! > Sagar. > > On Wed, May 25, 2022 at 5:14 PM Sagar <sagarmeansoc...@gmail.com> wrote: > > > Thanks for the KIP Kumud. > > > > Can you please add a couple of examples on how this would behave with > > different combinations. I think that way it would be easier to > understand. > > > > Thanks! > > Sagar. > > > > On Wed, May 25, 2022 at 4:59 PM Kumud Kumar Srivatsava Tirupati < > > kumudkumartirup...@gmail.com> wrote: > > > >> Hi all, > >> I have written a KIP for having a new HasField predicate for kafka > connect > >> transforms and would like to start a discussion. Please share your > >> thoughts > >> on the same. > >> > >> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-845%3A+%27HasField%27+predicate+for+kafka+connect > >> > >> *---* > >> *Thanks and Regards,* > >> *Kumud Kumar Srivatsava Tirupati* > >> > > >
