Ok, thanks for clarifying! I'll leave this for now then. I'll be happy to help if needed, whenever this work starts. Oscar, could you update the PR and/or Jira issue to spread this information? Maybe the PR should be closed even?
Best regards, Katrin On 2024-08-20, 19:56, "Oscar Westra van Holthe - Kind" <os...@westravanholthe.nl <mailto:os...@westravanholthe.nl>> wrote: EXTERNAL EMAIL: Do not click any links or open any attachments unless you trust the sender and know the content is safe. On tue 20 aug. 2024 16:23, Katrin Skoglund <katrin.skogl...@avanza.se.inva <mailto:katrin.skogl...@avanza.se.inva>lid> wrote: > Ok, just so I understand, you're basically saying that the Parsing > Canonical Form doesn't currently include information on logical types and > that's why it doesn't make sense to merge this now? Correct. Because the logical type info in the writer schema might have been stripped > away and in that case the validation in this PR won't work? Yes: to correctly resolve the evolution between logical write type and logical read type, both need to be known. As e.g. the single object encoding identifies schemata by fingerprint of the Parsing Canonical Form, the PCF must include this information. Or is it that we want to solve more cases of logical type > evolution/validation than the one in this PR while we're at it, and for all > of that to work the Parsing Canonical Form must be extended? > > If it is the latter, I would argue for maybe separating this into two > different issues - one for solving the (IMO) more pressing problem with > following the spec for decimal types and another for looking at the entire > scope of possible evolution between logical types. > Thwe are indeed two variations of the same issue. I prefer to tackle them at once, as it really should not be difficult. The most tricky thing is to identify how to work with multiple Avro/PCF versions simultaneously, and document the limits/caveats one needs to take into account. Kind regards, Oscar -- Oscar Westra van Holthe - Kind <opw...@apache.org <mailto:opw...@apache.org>> >
