On Tue, Dec 10, 2019 at 3:35 PM Jonah H. Harris <jonah.har...@gmail.com> wrote:
> On Tue, Dec 10, 2019 at 10:24 AM roger peppe <rogpe...@gmail.com> wrote: > >> On Tue, 10 Dec 2019 at 10:28, Ryan Skraba <r...@skraba.com> wrote: >> >>> Attributes outside of the spec should be OK to use as metadata, and >>> >> that seems like the right fit for your use case (such as the >>> interesting obfuscation attribute in lenses). Are the avro tools that >>> strip non-spec-attributes/metadata doing something wrong? I can see >>> this happening if they are relying on the Parsing Canonical Form or >>> the fingerprint (based on canonical form), but that is deliberate to >>> remove all differences between two schemas that can be used to parse >>> the same binary data. Note that PCF also removes doc attributes. >>> >> >> Yeah, I hadn't noticed that arbitrary extra attributes are allowed. >> That's a pity from my p.o.v. because it makes any validating schema >> substantially less useful, because it can't report misspelled fields, but >> the schema should nonetheless allow it given that the spec does. >> > > Agreed. As all attributes listed in the spec are mentioned as "valid" > attributes, I'd assume all non-listed attributes are invalid. Especially as > there is no mention in the spec about handling/validity of > non-type-specific attributes. Perhaps other encoder/decoder authors assumed > the same. Could the spec be updated to make this clear? > > While it actually makes it easier to specify custom attributes, I think it > actually makes it harder to differentiate spec vs. non-spec attributes in a > schema. It would be nice if there were a way to do that clearly. > In this way I mean something like a guideline. Something like this added to the spec: Arbitrary/Custom Attributes: In addition to Avro standard type attributes, this specification also permits custom, arbitrary, attributes within type definitions. As to avoid possible naming collisions with official Avro type attributes, however, it is recommended any custom attribute is prefixed with an underscore. In this way, custom attributes are both easier to identify and assured not to collide with the Avro specification. -- Jonah H. Harris
