[
https://issues.apache.org/jira/browse/AVRO-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547441#comment-14547441
]
Ryan Blue commented on AVRO-1497:
---------------------------------
[~tomwhite], could you take another look at this?
I think I've gotten the performance back to normal. There were a couple of
major issues that I've now fixed:
* Looking up Schema properties was too slow, even for just one (logicalType).
To work around that, I needed to add it to the schema so properties are read
when the schema is parsed. Because modifying the Schema is required for
performance, I went ahead and cached the logical type rather than just the
name. That allows us to do validation on the logical type later and makes the
performance when there is a conversion much better.
* Logical type conversions are now cached by the converion's class. This means
a conversion works with a certain class, not its children.
* I also started interning persistent strings in the Schema that are used in
comparisons a lot, like record names, field names, enum symbols, and
properties. That makes common lookups faster.
I've updated the performance doc with the latest run. The second sheet compares
the current PR code against trunk. There are a couple of anomalous tests, like
GenericNested_Write. I've left the data from run 16 in one of the sheets, which
shows the anomalous tests with normal performance. That was the run just before
I fixed arrays on the read side, so the write side should not have that large
of a performance problem unless it is due to something else. Another data point
supporting that is the fact that RecordWrite shows the same behavior: in run 16
it was fine, but with no changes it is way off for run 17. I think this is due
to some external factor because all of the anomalous tests seem to do this at
the same time.
> Add LogicalType support to Java
> -------------------------------
>
> Key: AVRO-1497
> URL: https://issues.apache.org/jira/browse/AVRO-1497
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Ryan Blue
> Assignee: Zoltan Farkas
> Fix For: 1.7.8
>
> Attachments: Logical Types Performance Comparison - Sheet1.csv
>
>
> AVRO-1402 updates the Avro spec to include logical types, which are stored as
> existing types but interpreted as others. The next step is to implement
> LogicalType (in Java) that provides tools to attach a logical type to a
> schema, read a logical type, and validate logical types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
