Re: Status of AVRO-2078 - enforcement of Avro schema resolution rules for Decimal type? (java)
Ok, thanks for clarifying! I'll leave this for now then. I'll be happy to help if needed, whenever this work starts. Oscar, could you update the PR and/or Jira issue to spread this information? Maybe the PR should be closed even? Best regards, Katrin On 2024-08-20, 19:56, "Oscar Westra van Holthe - Kind" mailto:os...@westravanholthe.nl>> wrote: EXTERNAL EMAIL: Do not click any links or open any attachments unless you trust the sender and know the content is safe. On tue 20 aug. 2024 16:23, Katrin Skoglund mailto:katrin.skogl...@avanza.se.inva>lid> wrote: > Ok, just so I understand, you're basically saying that the Parsing > Canonical Form doesn't currently include information on logical types and > that's why it doesn't make sense to merge this now? Correct. Because the logical type info in the writer schema might have been stripped > away and in that case the validation in this PR won't work? Yes: to correctly resolve the evolution between logical write type and logical read type, both need to be known. As e.g. the single object encoding identifies schemata by fingerprint of the Parsing Canonical Form, the PCF must include this information. Or is it that we want to solve more cases of logical type > evolution/validation than the one in this PR while we're at it, and for all > of that to work the Parsing Canonical Form must be extended? > > If it is the latter, I would argue for maybe separating this into two > different issues - one for solving the (IMO) more pressing problem with > following the spec for decimal types and another for looking at the entire > scope of possible evolution between logical types. > Thwe are indeed two variations of the same issue. I prefer to tackle them at once, as it really should not be difficult. The most tricky thing is to identify how to work with multiple Avro/PCF versions simultaneously, and document the limits/caveats one needs to take into account. Kind regards, Oscar -- Oscar Westra van Holthe - Kind mailto:opw...@apache.org>> >
Re: [PR] AVRO-4038: [C++] Add local-timestamp-nanos and timestamp-nanos [avro]
martin-g merged PR #3102: URL: https://github.com/apache/avro/pull/3102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (AVRO-4039) ClassCastException when deserializing an array of logical types
William McNutt created AVRO-4039:
Summary: ClassCastException when deserializing an array of logical
types
Key: AVRO-4039
URL: https://issues.apache.org/jira/browse/AVRO-4039
Project: Apache Avro
Issue Type: Bug
Components: java
Affects Versions: 1.12.0
Reporter: William McNutt
With a schema
{code:java}
{"type":"record","name":"Root","fields":[
{"name":"f_date_repeated","type":{"type":"array","items:
{"type":int",logicalType:"date"}
}},
{"name":"f_time_repeated","type":{"type":"array","items":
{"type":long",logicalType:"time-micros"}
}},
{"name":"f_datetime_repeated","type":{"type":"array","items":
{"type":string",logicalType:"datetime"}
}}]}]
{code}
and converters added:
{code:java}
static{
GenericData.get().addLogicalTypeConversion(new Conversions.DecimalConversion());
GenericData.get().addLogicalTypeConversion(new
TimeConversions.DateConversion());
GenericData.get().addLogicalTypeConversion(new
TimeConversions.TimeMillisConversion());
GenericData.get().addLogicalTypeConversion(new
TimeConversions.TimeMicrosConversion());
GenericData.get().addLogicalTypeConversion(new
TimeConversions.TimestampMillisConversion());
GenericData.get().addLogicalTypeConversion(new
TimeConversions.TimestampMicrosConversion());
GenericData.get().addLogicalTypeConversion(new
TimeConversions.LocalTimestampMicrosConversion());
GenericData.get().addLogicalTypeConversion(new
TimeConversions.LocalTimestampMillisConversion());
}
{code}
we are get an exception (and similarly Instant cannot be cast to Long)
{code:java}
ava.lang.ClassCastException: class java.time.LocalDate cannot be cast to class
java.lang.Integer (java.time.LocalDate and java.lang.Integer are in module
java.base of loader 'bootstrap')
at
org.apache.avro.generic.PrimitivesArrays$IntArray.add(PrimitivesArrays.java:28)
at java.base/java.util.AbstractList.add(Unknown Source)
at
org.apache.avro.generic.GenericDatumReader.addToArray(GenericDatumReader.java:333)
at
org.apache.avro.generic.GenericDatumReader.readArray(GenericDatumReader.java:294)
at
org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:184)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:161)
at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:260)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:248)
at
org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:180)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:161)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:154)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] Docs build fixes [avro]
martin-g opened a new pull request, #3108: URL: https://github.com/apache/avro/pull/3108 Follow-up of https://github.com/apache/avro/pull/3092 ## What is the purpose of the change * Fix the build of the docs: https://github.com/apache/avro/actions/runs/10488520943/job/29051171217 ## Verifying this change * The new Github Actions workflow for building the docs should pass https://github.com/apache/avro/actions/workflows/deploy-docs.yml ## Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
