Hi,
I have come across something that looks like a bug to me, but I'm not
sure of that. If I understand it correctly,
RestrictionTracker.trySplit() and RestrictionTracker.tryClaim() methods
are necessarily called from different threads. That implies, that
modifying some fields inside these methods might require
synchronization. Looking here [1], I didn't find anything that should
ensure atomicity and consistency of these methods. If anything I'd
expect the lastClaimedOffset and lastAttemptedOffset be volatile. But
probably the problem is deeper. Is this a bug, or am I missing something?
Jan
[1]
https://github.com/apache/beam/blob/939fa99ce943a30da46cb3d67c924d524fbf1be4/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/OffsetRangeTracker.java#L44
- Synchronization of RestrictionTrackers Jan Lukavský
-