----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40934/ -----------------------------------------------------------
(Updated May 11, 2016, 6:25 p.m.) Review request for samza. Summary (updated) ----------------- SAMZA-827: null offsets when writing state store OFFSET file cause container stalls Repository: samza Description ------- Currently, we don't handle null offsets when writing state store OFFSET file. Impact: 1.Without the fix, writing a null offset to a file will fail Samza in its shutdown sequence when taskStorageManager.stop is invoked. 2.An exception in the shut-down sequence will invoke the UncaughtExceptionHandler calling System.exit. 3.However, the Samzacontainer's JVM shutdown hook waits for task.shutdown.ms for all tasks to shutdown. 4.It turns out that the shutdown hook, and the System.exit (called by *1*) lock the same object as a part of the JVM shutdown sequence. (causing an unnecessary stall of task.shutdown.ms even if there was no exception in user code. This stall is effectively downtime for critical jobs which need to be always up.) 5.Often people configure very large values of task.shutdown.ms aggravating the problem. Diffs ----- samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 9588dcde2e698942097eb8ae1e3afcb8192dcfb5 samza-core/src/test/scala/org/apache/samza/storage/TestTaskStorageManager.scala 6491d095461b9181d5bf371ab12956086ff695d4 Diff: https://reviews.apache.org/r/40934/diff/ Testing ------- Thanks, Jagadish Venkatraman
