Hi all, Any other feedback on this? I'll take the discussion open for another week and then start the vote.
Thanks, Mario. On Fri, Jan 10, 2025 at 10:30 AM Mario Fiore Vitale <mvit...@redhat.com> wrote: > Hi Greg, thanks for giving a look. > > > It's a closed type system which cannot have new "physical" types added by > third-party developers > > Do you have some examples for this? In my experience I never had the need > to add a new type. > > > There are carve-outs in the implementation for framework logical types > (java.util.Date etc) > > Yeah, java.util.Date is quite old these days. > > > Compatibility between components isn't managed: custom types can be > misinterpreted as dumb physical types which is undesirable to users, who > may prefer to fail-fast instead. > > Any examples for this? > > > I think that we should evolve the Connect data model in the future to > resolve some or all of these problems. > > Yes, this for sure can have a positive impact. > > > This will be easier and less risky > if we change the data model first, before expanding it to non-Connect > users. > > understandable but I have then the feeling that the change to the data > model will be not so easy and fast with the risk to block the > expansion outside connect. > Maybe starting with the expansion and if there will be more traction also > outside the Connect community we will have more motivation to improve the > data model. > > > If there was a suitably general data model out there, we could promote > it within the Connect ecosystem and deprecate our current model. > > Honestly, I don't have anything in mind? Do you? > > Thanks, > Mario. > > On Thu, Jan 9, 2025 at 6:14 PM Greg Harris <greg.har...@aiven.io.invalid> > wrote: > >> Hi Mario, >> >> Thanks for the KIP! I think that using the same data model inside and >> outside of connect is valuable, and is a good motivation for this effort. >> >> However, I have seen many situations where the existing data model is >> insufficient for Connect plugin developers. >> * It's a closed type system which cannot have new "physical" types added >> by >> third-party developers >> * There are carve-outs in the implementation for framework logical types >> (java.util.Date etc) >> * Compatibility between components isn't managed: custom types can be >> misinterpreted as dumb physical types which is undesirable to users, who >> may prefer to fail-fast instead. >> >> I think that we should evolve the Connect data model in the future to >> resolve some or all of these problems. This will be easier and less risky >> if we change the data model first, before expanding it to non-Connect >> users. >> >> Another way to satisfy the requirements of your KIP would be to adopt an >> existing external data model in Connect. If there was a suitably general >> data model out there, we could promote it within the Connect ecosystem and >> deprecate our current model. >> >> Thanks, >> Greg >> >> >> On Thu, Jan 9, 2025 at 12:19 AM Mario Fiore Vitale <mvit...@redhat.com> >> wrote: >> >> > Hi all, >> > >> > I just want to bring the discussion up since it was open during the >> > Christmas holidays. >> > >> > Any other considerations? >> > >> > Thanks, >> > Mario. >> > >> > On Sat, Dec 28, 2024 at 3:04 PM Mario Fiore Vitale <mvital...@gmail.com >> > >> > wrote: >> > >> > > Hey Hector, >> > > >> > > the aim of this KIP is to separate the Connect data (Struct, Schema, >> > etc.) >> > > so it can be used without having a dependency con connect-api. >> > > >> > > So the proposal is more related to economics. >> > > >> > > As Kafka itself became an API during the years, maybe also the Connect >> > data >> > > can be used as a general data format not only for Connect. >> > > >> > > Is it more clear now? >> > > >> > > Il ven 20 dic 2024, 22:20 Hector Geraldino (BLOOMBERG/ 919 3RD A) < >> > > hgerald...@bloomberg.net> ha scritto: >> > > >> > > > Hey Mario, >> > > > >> > > > From a quick read of this KIP, it's not super clear to me what >> problem >> > it >> > > > is trying to solve. The motivation states that "current >> implementation >> > > > restricts the usage of data classes", and I'm not sure what you >> mean by >> > > > that. >> > > > >> > > > Can you maybe add one or two examples of things that are not >> possible >> > to >> > > > do today and could be achieved by this refactor. If this proposal is >> > more >> > > > about improving the ergonomics of users of these classes, that's >> also >> > > > valuable IMO. >> > > > >> > > > From: dev@kafka.apache.org At: 12/18/24 08:32:28 UTC-5:00To: >> > > > dev@kafka.apache.org >> > > > Subject: [DISCUSS] KIP-1122: Create a dedicated data module for >> Kafka >> > > > Connect data classes >> > > > >> > > > Hi Everyone, >> > > > >> > > > I would like to start a discussion on KIP-1122: Create a dedicated >> data >> > > > module for Kafka Connect data classes [1]. >> > > > >> > > > [1] >> > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1122%3A+Create+a+dedicated >> > > > +data+module+for+Kafka+Connect+data+classes >> > > > >> > > > Regards, >> > > > -- >> > > > Mario Fiore Vitale >> > > > >> > > > >> > > > >> > > >> > >> > >> > -- >> > >> > Mario Fiore Vitale >> > >> > Senior Software Engineer >> > >> > Red Hat <https://www.redhat.com/> >> > <https://www.redhat.com/> >> > >> > > > -- > > Mario Fiore Vitale > > Senior Software Engineer > > Red Hat <https://www.redhat.com/> > <https://www.redhat.com/> > -- Mario Fiore Vitale Senior Software Engineer Red Hat <https://www.redhat.com/> <https://www.redhat.com/>
