Kellen Dye created BEAM-13782:
---------------------------------
Summary: NPE
Key: BEAM-13782
URL: https://issues.apache.org/jira/browse/BEAM-13782
Project: Beam
Issue Type: Bug
Components: sdk-java-core
Affects Versions: 2.35.0
Reporter: Kellen Dye
Discussed here:
[https://lists.apache.org/thread/rcto0yggqygfk0ljnd8mg69h83rhvvs5]
---
We (Spotify) are experiencing flaky tests on beam 2.35.0 as a result of
NullPointerExceptions during pipeline construction.
Stacktrace [1]. Root cause appears to be multiple threads accessing
DeserializationContext from ValueProvider.Deserializer.deserialize [2] which,
according to the javadocs "is guaranteed to only be used from single-threaded
context" [3]
In the context internals, the mutable _currentType is set [4] then accessed a
few lines later [5], but in a multithreaded situation the writes clobber each
other and a call to next() can fail with a NPE.
I think synchronizing on ctxt in [2] should be sufficient to avoid this issue,
but I'm not sure if the NPEs are due to beam misuse or something on the jackson
side that should be changed.
Failing (scala) test here, scalatest runs tests in parallel:
[https://gist.github.com/kellen/124185463c16a66167a7fa704147c510#file-fail-scala]
Test stacktrace:
[https://gist.github.com/kellen/124185463c16a66167a7fa704147c510#file-test_stacktrace-txt]
[1]
[https://gist.github.com/kellen/124185463c16a66167a7fa704147c510#file-stacktrace-txt]
[2]
[https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ValueProvider.java#L359]
[3]
[https://github.com/joansmith/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/DeserializationContext.java#L36]
[4]
[https://github.com/joansmith/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/DeserializationContext.java#L652]
[5]
[https://github.com/joansmith/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/DeserializationContext.java#L656]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
