----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48080/#review137376 -----------------------------------------------------------
Ship it! Really like this latest diff. Addresses all my comments, well-tested, and reads nicely. Thanks! - Jake Maes On June 10, 2016, 7:39 p.m., Chris Pettitt wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48080/ > ----------------------------------------------------------- > > (Updated June 10, 2016, 7:39 p.m.) > > > Review request for samza. > > > Repository: samza > > > Description > ------- > > This change introduces a ThrottlingExecutor which is used to control the > rate of execution in the main run loop. The DiskQuotaEnforcer houses the > rules for switching from one DiskQuotaPolicy to the next as new disk > usage samples arrive. > > By default, no throttling will occur. New policies can be added using > the following form: > > ``` > container.disk.quota.bytes=XXX > container.disk.quota.policy.count=2 > container.disk.quota.policy.0.lowWaterMark=0.4 > container.disk.quota.policy.0.highWaterMark=0.5 > container.disk.quota.policy.0.workFactor=0.5 > container.disk.quota.policy.1.lowWaterMark=0.05 > container.disk.quota.policy.1.highWaterMark=0.1 > container.disk.quota.policy.1.workFactor=0.05 > ``` > > See ThrottlingExecutor for details about how the work factor works and > DiskQuotaPolicy for details about how the low and high water mark > configuration work. > > > Diffs > ----- > > > samza-core/src/main/java/org/apache/samza/container/disk/DiskQuotaPolicy.java > PRE-CREATION > > samza-core/src/main/java/org/apache/samza/container/disk/DiskQuotaPolicyFactory.java > PRE-CREATION > > samza-core/src/main/java/org/apache/samza/container/disk/DiskSpaceMonitor.java > 2a565be7858a4d3a6adbc49989b43b71ca3f6721 > > samza-core/src/main/java/org/apache/samza/container/disk/NoThrottlingDiskQuotaPolicy.java > PRE-CREATION > > samza-core/src/main/java/org/apache/samza/container/disk/NoThrottlingDiskQuotaPolicyFactory.java > PRE-CREATION > > samza-core/src/main/java/org/apache/samza/container/disk/WatermarkDiskQuotaPolicy.java > PRE-CREATION > > samza-core/src/main/java/org/apache/samza/container/disk/WatermarkDiskQuotaPolicyFactory.java > PRE-CREATION > samza-core/src/main/java/org/apache/samza/util/HighResolutionClock.java > PRE-CREATION > > samza-core/src/main/java/org/apache/samza/util/SystemHighResolutionClock.java > PRE-CREATION > samza-core/src/main/java/org/apache/samza/util/ThrottlingExecutor.java > PRE-CREATION > samza-core/src/main/scala/org/apache/samza/container/RunLoop.scala > 6916c5c71e479d43a7435fa4987565d93ed437ac > > samza-core/src/main/scala/org/apache/samza/container/SameThreadExecutor.scala > PRE-CREATION > samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala > cf3c4c0ab08a59760bc899c6f2027755e933b350 > > samza-core/src/main/scala/org/apache/samza/container/SamzaContainerMetrics.scala > 9e6641c3628290dc05e1eb5537e86bff9d37f92c > samza-core/src/main/scala/org/apache/samza/util/Util.scala > fc3d085d7fff9f7dcec766ba48e550eb0052e99d > > samza-core/src/test/java/org/apache/samza/container/disk/TestDiskQuotaPolicyEntry.java > PRE-CREATION > > samza-core/src/test/java/org/apache/samza/container/disk/TestWatermarkDiskQuotaPolicy.java > PRE-CREATION > samza-core/src/test/java/org/apache/samza/util/TestThrottlingExecutor.java > PRE-CREATION > samza-core/src/test/scala/org/apache/samza/util/TestUtil.scala > 05b4e5c37578340eefe6d412f5a76f540bec6fa6 > > Diff: https://reviews.apache.org/r/48080/diff/ > > > Testing > ------- > > - Added new unit tests > - Ran existing tests with gradle test > - Verified throttling behavior and instrumentation with local deployment > - Verified average latency impact of feature to be < 150ns for Linux and OSX > > > Thanks, > > Chris Pettitt > >
