Re: Avro JSON Encoding

2024-04-18 Thread Clemens Vasters via user
The discriminator is "const". I added "displayname" because I also have other scenarios for it and it appears like a good workaround for alias names that do not fit the "name", e.g. "$type". I am not passionate about "displayname", but "alias" is taken and it's going to be a user-supplied name

Re: Avro JSON Encoding

2024-04-18 Thread Oscar Westra van Holthe - Kind
Thank you Clemens, This is a very detailed set of proposals, and it looks like it would work. I do however, feel we'd need to define a way to unions with records. Your proposal lists various options, of which the discriminatory option seems most portable to me. You mention the "displayName" prop

RE: Avro JSON Encoding

2024-04-18 Thread Clemens Vasters via user
I literally do the “FWIW” here: https://github.com/clemensv/avrotize?tab=readme-ov-file#convert-json-schema-to-avro-schema From: Andrew Otto Sent: Thursday, April 18, 2024 2:24 PM To: user@avro.apache.org Cc: Clemens Vasters Subject: Re: Avro JSON Encoding Sie erhalten nicht oft eine E-Mail vo

Re: Avro JSON Encoding

2024-04-18 Thread Andrew Otto
This is a great proposal. At the Wikimedia Foundation, we've explicitly chosen to use JSON as our streaming serialization format . We considered using Avro JSON, but the need to use an Avro specific serial

"Avrotize" tool

2024-04-18 Thread Clemens Vasters via user
Hi everyone, I'm interested in feedback on the "Avrotize" tool: Git: https://github.com/clemensv/avrotize PyPI: https://pypi.org/project/avrotize/ Avrotize is a command-line tool for converting data structure definitions between different schema formats, using Apache Avro Schema as the integr

Re: Avro JSON Encoding

2024-04-18 Thread Jean-Baptiste Onofré
Hi Clemens, I propose to wait a bit to give a chance to the community to review your email and points. Then, we will create the Jira accordingly. Regards JB On Thu, Apr 18, 2024 at 9:20 AM Clemens Vasters wrote: > > Hi JB, > > > > I have not done that yet. I’m happy to break that up into items

RE: Avro JSON Encoding

2024-04-18 Thread Clemens Vasters via user
Hi JB, I have not done that yet. I'm happy to break that up into items once I get the sense that this is a direction we can get to a consensus on. Shall I file the whole email as a "New Feature" issue first? Thanks Clemens From: Jean-Baptiste Onofré Sent: Thursday, April 18, 2024 10:17 AM To:

Re: Avro JSON Encoding

2024-04-18 Thread Jean-Baptiste Onofré
Hi Clemens Thanks for the detailed email. Quick question : did you already create Jira about each improvements/issues ? I will take the time to read asap. Thanks Regards JB Le jeu. 18 avr. 2024 à 09:12, Clemens Vasters via user a écrit : > Hi everyone, > > > > the current JSON Encoding appro

Avro JSON Encoding

2024-04-18 Thread Clemens Vasters via user
Hi everyone, the current JSON Encoding approach severely limits interoperability with other JSON serialization frameworks. In my view, the JSON Encoding is only really useful if it acts as a bridge into and from JSON-centric applications and it currently gets in its own way. The current encodi