----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48080/ -----------------------------------------------------------
(Updated June 20, 2016, 10:04 p.m.) Review request for samza. Changes ------- Update for Navina's comments. Repository: samza Description ------- This change introduces a ThrottlingExecutor which is used to control the rate of execution in the main run loop. The DiskQuotaEnforcer houses the rules for switching from one DiskQuotaPolicy to the next as new disk usage samples arrive. By default, no throttling will occur. New policies can be added using the following form: ``` container.disk.quota.bytes=XXX container.disk.quota.policy.count=2 container.disk.quota.policy.0.lowWaterMark=0.4 container.disk.quota.policy.0.highWaterMark=0.5 container.disk.quota.policy.0.workFactor=0.5 container.disk.quota.policy.1.lowWaterMark=0.05 container.disk.quota.policy.1.highWaterMark=0.1 container.disk.quota.policy.1.workFactor=0.05 ``` See ThrottlingExecutor for details about how the work factor works and DiskQuotaPolicy for details about how the low and high water mark configuration work. Diffs (updated) ----- samza-core/src/main/java/org/apache/samza/container/disk/DiskQuotaPolicy.java PRE-CREATION samza-core/src/main/java/org/apache/samza/container/disk/DiskQuotaPolicyFactory.java PRE-CREATION samza-core/src/main/java/org/apache/samza/container/disk/DiskSpaceMonitor.java 2a565be samza-core/src/main/java/org/apache/samza/container/disk/NoThrottlingDiskQuotaPolicy.java PRE-CREATION samza-core/src/main/java/org/apache/samza/container/disk/NoThrottlingDiskQuotaPolicyFactory.java PRE-CREATION samza-core/src/main/java/org/apache/samza/container/disk/WatermarkDiskQuotaPolicy.java PRE-CREATION samza-core/src/main/java/org/apache/samza/container/disk/WatermarkDiskQuotaPolicyFactory.java PRE-CREATION samza-core/src/main/java/org/apache/samza/util/HighResolutionClock.java PRE-CREATION samza-core/src/main/java/org/apache/samza/util/SystemHighResolutionClock.java PRE-CREATION samza-core/src/main/java/org/apache/samza/util/ThrottlingExecutor.java PRE-CREATION samza-core/src/main/scala/org/apache/samza/container/RunLoop.scala 3f25eca samza-core/src/main/scala/org/apache/samza/container/SameThreadExecutor.scala PRE-CREATION samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 2e8a500 samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala.orig PRE-CREATION samza-core/src/main/scala/org/apache/samza/container/SamzaContainerMetrics.scala 9e6641c samza-core/src/main/scala/org/apache/samza/util/Util.scala c77d929 samza-core/src/test/java/org/apache/samza/container/disk/TestDiskQuotaPolicyEntry.java PRE-CREATION samza-core/src/test/java/org/apache/samza/container/disk/TestWatermarkDiskQuotaPolicy.java PRE-CREATION samza-core/src/test/java/org/apache/samza/util/TestThrottlingExecutor.java PRE-CREATION samza-core/src/test/scala/org/apache/samza/util/TestUtil.scala 05b4e5c Diff: https://reviews.apache.org/r/48080/diff/ Testing ------- - Added new unit tests - Ran existing tests with gradle test - Verified throttling behavior and instrumentation with local deployment - Verified average latency impact of feature to be < 150ns for Linux and OSX Thanks, Chris Pettitt
