Hi Martin, I am saying that the specification of the schema is currently entangled with the specification of the serialization framework. Avro Schema is useful and usable even if you never touch the Avro binaries (the framework, an implementation using the spec).
I am indeed proposing to separate the schema spec from the specs of the Avro binary encoding and the Avro JSON encoding, which also avoids strange entanglements like the JSON encoding pointing to the schema description’s default values section, which is in itself rather lacking in precision, i.e. the encoding rule for binary or fixed is “defined” with a rather terse example: "\u00ff" Microsoft would like to propose Avro and Avro Schema in several standardization efforts, but we need a spec that works in those contexts and that can stand on its own. I would also like to see “application/avro” as a formal media type, but the route towards that only goes via formal standardization of both schema and encodings. I believe the Avro project’s reach and importance is such that schema and encodings should have formal specs that can stand on their own as JSON and CBOR and AMQP and XML and OPC/Binary and other serialization schemas/formats do. I don’t think existence of a formal spec gets in the way of progress and Avro is so mature that the spec captures a fairly stable state. Best Regards Clemens From: Martin Grigorov <mgrigo...@apache.org> Sent: Wednesday, May 15, 2024 10:54 AM To: dev@avro.apache.org Cc: u...@avro.apache.org Subject: Re: Formal spec for Avro Schema Hi Clemens, On Wed, May 15, 2024 at 11:18 AM Clemens Vasters <cleme...@microsoft.com.invalid<mailto:cleme...@microsoft.com.invalid>> wrote: Hi Martin, we find Avro Schema to be a great fit for describing application data structures in general and even independent of wire-serialization scenarios. Therefore, I would like to have a spec that focuses specifically on the schema format, is grounded in the IETF RFC specs, and which follows the conventions set by IETF, so that folks who need a sane schema format to describe data structures independent of implementation can use that. Do you say that the specification document is implementation dependent ? If this is the case then maybe we should work on improving it instead of duplicating it. The benefit for the Avro serialization framework of having such a formal spec that is untangled from the wire-serialization specs is that all schemas defined by that schema model are compatible with the framework. What do you mean by "framework" here ? The differences are organization, scope, and language style (including keywords etc.). The expressed ruleset is the same. I don't think it is a good idea to add a second document that is very similar to the specification but uses a different language style. To me this looks like a duplication. IMO it would be better to suggest (many) (smaller) improvements for the existing document. Best Regards Clemens -----Original Message----- From: Martin Grigorov <mgrigo...@apache.org<mailto:mgrigo...@apache.org>> Sent: Wednesday, May 15, 2024 9:26 AM To: dev@avro.apache.org<mailto:dev@avro.apache.org> Cc: u...@avro.apache.org<mailto:u...@avro.apache.org> Subject: Re: Formal spec for Avro Schema [Sie erhalten nicht häufig E-Mails von mgrigo...@apache.org<mailto:mgrigo...@apache.org>. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ] Hi Clemens, What is the difference between your document and the specification [1] ? I haven't read it completely but it looks very similar to the specification to me. 1. https://avro.apache.org/docs/1.11.1/specification/ 2. https://github.com/apache/avro/tree/main/doc/content/en/docs/%2B%2Bversion%2B%2B/Specification - sources of the specification On Wed, May 15, 2024 at 9:28 AM Clemens Vasters <cleme...@microsoft.com<mailto:cleme...@microsoft.com>.invalid> wrote: > I wrote a formal spec for the Avro Schema format. > > > > https://gist/ > .github.com<http://github.com/>%2Fclemensv%2F498c481965c425b218ee156b38b49333&data=05%7C02 > %7Cclemensv%40microsoft.com<http://40microsoft.com/>%7C5cd57d6ebe504e02e6dd08dc74b06a33%7C72f98 > 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C638513548275308005%7CUnknown%7CT > WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI > 6Mn0%3D%7C0%7C%7C%7C&sdata=n24LJspeNxYRKjlD0tgJzxQh3CzuILK%2FRe30gbarB > ec%3D&reserved=0 > > > > Where would that go in the repo? > > > > > > > <http://www/. > microsoft.com<http://microsoft.com/>%2Fen-us%2Fnews%2FImageDetail.aspx%3Fid%3D4DABA54CBB4D25A > 9E9905BC59E4A6D44E33694EA&data=05%7C02%7Cclemensv%40microsoft.com<http://40microsoft.com/>%7C5c > d57d6ebe504e02e6dd08dc74b06a33%7C72f988bf86f141af91ab2d7cd011db47%7C1% > 7C0%7C638513548275312403%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL > CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=x6ZAZ > YEAjqkSVznt3N%2FKGjZzE%2BJietvHZuaiqVQYuDY%3D&reserved=0> > > *Clemens Vasters* > > Messaging Platform Architect > > Microsoft Azure > > È+49 151 44063557 > > * cleme...@microsoft.com<mailto:cleme...@microsoft.com> > European Microsoft Innovation Center GmbH | Gewürzmühlstrasse 11 | > 80539 > Munich| Germany > Geschäftsführer/General Managers: Keith Dolliver, Benjamin O. Orndorff > Amtsgericht Aachen, HRB 12066 > > > > >
