Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-06-03 Thread Kumud Kumar Srivatsava Tirupati
Hi Chris, Thanks for your comment. I might have misunderstood the filter SMT. Makes sense. Dropping this KIP for now. I will look at improving the existing SMTs separately. *---* *Thanks and Regards,* *Kumud Kumar Srivatsava Tirupati* On Sat, 4 Jun 2022 at 03:57, Chris Egerton wrote: > Hi Kumu

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-06-03 Thread Kumud Kumar Srivatsava Tirupati
Hi Chris, Thanks for the explanation. This clears my thoughts. I can now agree that the concerns are totally different for SMTs and predicates. I also do agree with you that this might encourage SMTs to be poorly designed. Do you see this worth considering just for the filter use case? ['errors.to

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-06-03 Thread Chris Egerton
Hi Kumud, Responses inline. > But, I still believe this logic of predicate checking shouldn't be made a part of each individual SMT. After all, that is what the predicates are for right? I don't quite agree. I think the benefit of predicates is that they can allow you to selectively apply a tran

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-06-02 Thread Kumud Kumar Srivatsava Tirupati
Hi Chris, Thanks for your comments. I am totally aligned with your comment on nested field names which include dots. I will incorporate the same based on how the KIP-821 discussion goes (maybe this parser could be a utility that can be reused in other areas as well). But, I still believe this log

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-05-31 Thread Chris Egerton
Hi Kumud, Thanks for the KIP. I'm a little bit skeptical about the necessity for this predicate but I think we may be able to satisfy your requirements with a slightly different approach. The motivation section deals largely with skipping the invocation of SMTs that expect a certain field to be pr

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-05-26 Thread Sagar
Hi Kumud, Thanks for that. I don't have any other comments at this point on the KIP. LGTM overall. Thanks! Sagar. On Wed, May 25, 2022 at 5:14 PM Sagar wrote: > Thanks for the KIP Kumud. > > Can you please add a couple of examples on how this would behave with > different combinations. I think

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-05-25 Thread Kumud Kumar Srivatsava Tirupati
Hi Sagar, Added the examples to the KIP wiki as suggested. *---* *Thanks and Regards,* *Kumud Kumar Srivatsava Tirupati* On Wed, 25 May 2022 at 17:14, Sagar wrote: > Thanks for the KIP Kumud. > > Can you please add a couple of examples on how this would behave with > different combinations. I

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

2022-05-25 Thread Sagar
Thanks for the KIP Kumud. Can you please add a couple of examples on how this would behave with different combinations. I think that way it would be easier to understand. Thanks! Sagar. On Wed, May 25, 2022 at 4:59 PM Kumud Kumar Srivatsava Tirupati < kumudkumartirup...@gmail.com> wrote: > Hi a

[DISCUSS] KIP-845: HasField predicate for kafka connect

2022-05-25 Thread Kumud Kumar Srivatsava Tirupati
Hi all, I have written a KIP for having a new HasField predicate for kafka connect transforms and would like to start a discussion. Please share your thoughts on the same. https://cwiki.apache.org/confluence/display/KAFKA/KIP-845%3A+%27HasField%27+predicate+for+kafka+connect *---* *Thanks and Reg