Yes, definitely. AVRO-1760 is one that is waiting on this commit to do
basically the same fix. I think we should remove the custom Identity
maps and replace them with Guava equivalents.
rb
On 01/13/2016 10:55 AM, Doug Cutting wrote:
Should we (in separate issues) remove other uses of
WeakIdentityHashMap? Are there other caches that might be improved
with Guava? For example, GenericDatumReader's RESOLVER_CACHE might
avoid ThreadLocal.
Doug
On Wed, Jan 13, 2016 at 10:18 AM, Ryan Blue (JIRA) <j...@apache.org> wrote:
[
https://issues.apache.org/jira/browse/AVRO-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096719#comment-15096719
]
Ryan Blue commented on AVRO-1781:
---------------------------------
Thanks, Tom! I've added the comment and committed this.
Schema.parse is not thread safe
-------------------------------
Key: AVRO-1781
URL: https://issues.apache.org/jira/browse/AVRO-1781
Project: Avro
Issue Type: Bug
Components: java
Affects Versions: 1.8.0
Reporter: Sean Busbey
Priority: Blocker
Fix For: 1.8.0
Attachments: AVRO-1781.1.patch, AVRO-1781.2.patch
Post AVRO-1497, Schema.parse calls {{LogicalTypes.fromSchemaIgnoreInvalid}} on
any schema that is expressed as a JSON object (anything except bare primitives).
That static method relies on a static cache based on WeakIdentityHashMap (WIHM).
WIHM clearly states that it isn't threadsafe
[ref|https://github.com/apache/avro/blob/branch-1.8/lang/java/avro/src/main/java/org/apache/avro/util/WeakIdentityHashMap.java#L42]
{code}
*
* Note that this implementation is not synchronized.
* </b>
*/
public class WeakIdentityHashMap<K, V> implements Map<K, V> {
{code}
All of the Schema.Parser instances use that same static Schema.parse method.
The end result is that as-is it's only safe to have a single thread parsing
schemas in a given JVM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
--
Ryan Blue
Software Engineer
Cloudera, Inc.
