There's also this https://forum.confluent.io/t/kafka-connect-jmespath-expressive-content-based-record-filtering/1104 which is worth checking out - I came across it after recording that video unfortunately but it looks like a useful predicate implementation.
-- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 7 May 2021 at 11:30, Tom Bentley <tbent...@redhat.com> wrote: > Just to be clear: It's only the Predicate implementation for filtering on a > JSON key that you'd need to write. MM2 already supports SMTs and > predicates. > > Alternatively > > https://rmoff.net/2020/12/22/twelve-days-of-smt-day-11-predicate-and-filter/#_filtering_based_on_the_contents_of_a_message > might work for you, but it's not an Apache Kafka feature currently. I'm not > aware that anyone is currently planning on contributing a JSONPath-based > Predicate to Apache Kafka. > > On Fri, May 7, 2021 at 10:06 AM Anup Tiwari <anupsdtiw...@gmail.com> > wrote: > > > Hi Tom, > > > > Thanks for quick reply. As per your comments, it seems we will have to > > build something of our own together to work with MM2. > > So just wanted to confirm that it is not an inbuilt feature.. right? Like > > confluent replicator provide this. > > Also are we planning to add this in near future? > > > > > > On Fri, 7 May 2021 13:33 Tom Bentley, <tbent...@redhat.com> wrote: > > > > > Hi Anup, > > > > > > This should be possible using the Filter SMT in Kafka Connect together > > with > > > a custom Predicate which you'd have to write yourself and provide on > the > > > plugin path. > > > > > > See http://kafka.apache.org/documentation.html#connect_predicates and > > > > > > > > > https://rmoff.net/2020/12/22/twelve-days-of-smt-day-11-predicate-and-filter/ > > > for info about Filter and predicates, and > > > > > > > > > https://stackoverflow.com/questions/61742209/smt-does-not-work-when-added-to-connect-mirror-maker-properties-mirrormaker-2 > > > for a MM2-specific example of SMTs > > > > > > Kind regards, > > > > > > Tom > > > > > > On Fri, May 7, 2021 at 8:49 AM Anup Tiwari <anupsdtiw...@gmail.com> > > wrote: > > > > > > > Hi Team, > > > > > > > > I have a topic which contains JSON data and multiple user click > > events. I > > > > just wanted to know if we can filter out events based on keys of JSON > > > > before replicating it to some other kafka. > > > > > > > > Regards, > > > > Anup Tiwari > > > > > > > > > >
