Hi Chris,
Thank you for your feedback and insights on the KIP.
Let me address your queries
1 . The KIP will work with existing config providers , we have tested with the
Directory config .I have updated the examples in KIP .
2. We have tested it out with inline schema and we should not need any
backslashes. However, your point about the ergonomics is true - as this needs
the schema to be provided in a single line, it would be difficult to read
unless the schema is very small and simple. I think using a config provider
would be the answer for this.
Thanks
Priyanka
From: Chris Egerton <fearthecel...@gmail.com>
Date: Saturday, 29 June 2024 at 12:37 PM
To: dev@kafka.apache.org <dev@kafka.apache.org>
Subject: [EXTERNAL] Re: [VOTE] KIP-1054: Support external schemas in
JSONConverter
Hi Priyanka,
I think allowing a raw schema string, instead of introducing a property
that gives users access to the file system on the worker, is a good idea. I
have two small thoughts:
1) The KIP provides an example
of schema.content:${directory:/schema/schema.json} for how a config
provider might be used to read a schema from the file system. Is this an
accurate example corresponding to the file config provider or the directory
config provider? If not, can an example be provided for either of these
out-of-the-box providers? I'm asking because I think it'd make sense to
ensure that readily-accessible config providers can be used to achieve what
we need for this KIP, and if there's a gap, we should try to fill it.
2) Schemas for the JSON converter are, unsurprisingly, encoded in JSON.
This is fine if someone uses a config provider to fetch that JSON string
from somewhere else (like a file on the worker), but if someone tries to
provide a schema directly in a connector config, it'll be a mess of
backslashes and harder to read. Do you have any thoughts on a way we might
improve the ergonomics here?
Cheers,
Chris
On Fri, Jun 28, 2024 at 5:24 AM Priyanka K U <priyanka....@ibm.com.invalid>
wrote:
> Hello Everyone,
>
> I'd like to start a vote on KIP-1054, which aims to Support external
> schemas in JSONConverter to Kafka Connect:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1054%3A+Support+external+schemas+in+JSONConverter
>
>
> Thank you,
>
> Priyanka
>
>
