davidgm0 opened a new pull request, #3317: URL: https://github.com/apache/avro/pull/3317
I wasn't able to create an issue in JIRA, since I don't seem to be able to request a user. I am a first time contributor, apologies in case something is not precise enough ## What is the purpose of the change This is a second take on https://github.com/apache/avro/pull/2062 (hopefully better) Improving validation errors for unions of records When no valid union is found, this line selects the **first** error that it finds. If there would be 3 different possible schemas, there are 3 `failures`, however only the first one will be considered. https://github.com/apache/avro/blob/fc2a4e0e5d88755c776b97fa1872e70ffbe0d4bb/lang/ruby/lib/avro/schema_validator.rb#L199 The returned error also does not give any context of the error, only about the invalid fields. As an example, consider we have 3 types of adresses `PersonalAddress`, `WorkAddress`, `SecondAddress` and we made a typo: We used the field `compan` instead of `company`, which belongs to `WorkAddress`. No type will work for the union. `{ "compan" => "acme inc.", "some_other_field" => "something else" }` Since `PersonalAddress` is the first type that fails, the errors of `PersonalAddress` are the ones that will be returned. Since `PersonalAddress` has different structure, all fields that aren't the same as in `WorkAddress` will be shown as errors. The error will be something like ``` at .address extra field 'compan' - not in schema at .address extra field 'some_other_field' - not in schema ``` This gets very confusing because 1) The issue is that no union matched 2) It is not clear because only one of the error is displayed. I'd be happy if this is fix in some other fashion, this is just an example. With we replace `detect` with `select` and we add the schema name to the error. Then the error will be like: ``` PersonalAddress at .address extra field 'compan' - not in schema PersonalAddress at .address extra field 'some_other_field' - not in schema WorkAddress at .address extra field 'compan' - not in schema SecondAddress at .address extra field 'compan' - not in schema SecondAddress at .address extra field 'some_other_field' - not in schema ``` I don't know the internals very well, but the existing behaviour also seems flawed because if any of the types is complex, only the complex type errors are listed. I've changed this to list all errors. The error format isn't perfect and I'm happy to accept changes or improve it myself, but I would very much like this to get eventually merged, since the existing validation behaviour for unions is not correct. ## Verifying this change This change added tests and can be verified as follows: - Modified existing test, existing tests pass ## Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
