Hi all, we have opened the VOTE thread a few weeks ago as we hoped that this DISCUSS thread exchange had been exhaustive. If so, we would you like any interested party to. cast a vote there. Of course we're happy to further progress the KIP discussion if needed.
Thanks Edo & Adrian On Wed, 5 Jul 2023 at 16:55, Edoardo Comar <edoardli...@gmail.com> wrote: > > Hi Jorge! > > On Fri, 30 Jun 2023 at 15:47, Jorge Esteban Quilcate Otoya > <quilcate.jo...@gmail.com> wrote: > > > > Thank you both for the replies! A couple more comments: > > > > The current proposal is to have ‘record.validation.policy’ per topic > > (default null). A flag would be something like > > ‘record.validation.policy.enable’ (default=false) may be simpler to > > configure from the user perspective. > > > > Also, at the moment, is a bit unclear to me what value the topic config > > ‘record.validation.policy’ should contain: is the policy class name? How is > > the policy expected to use the name received? > > > > The 'record.validation.policy' will typically contain a value that is > meaningful to the policy implementation. > For example, a schema registry might support different strategies to > associate a schema with a topic. > The policy could use this property to determine which strategy is in > use and then evaluate whether the record is valid. > We decided to reserve the 'null' value to mean disable validation for > this topic to avoid the need for introducing a second inter-dependent > boolean property. > > > > > Thanks! I think adding a simple example of a Policy implementation and how > > plugin developer may use this hints (and metadata as well) may bring some > > clarity to the proposal. > > > > We've added a sample to the KIP, hope this helps. > > We expect the RecordIntrospectionHints to be a declaration the policy makes, > which the implementation of the KIP may use to optimise record > iteration avoiding a full decompression in the case where a message is > received with compression type matching the topic compression config. > Currently Kafka optimizes that case by supplying an iterator that does > not provide access to the record data, only answers hasKey/hasValue > checks. > > HTH, > best > Edo & Adrian
