----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41912/#review125828 -----------------------------------------------------------
Ship it! +1 looks good to me! Only 1 nit: Can you add some documentation to the website? The tool will go unused if we don't document it in the website :) - Navina Ramesh On Jan. 5, 2016, 1:21 a.m., Xinyu Liu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/41912/ > ----------------------------------------------------------- > > (Updated Jan. 5, 2016, 1:21 a.m.) > > > Review request for samza, Boris Shkolnik, Navina Ramesh, and Yi Pan (Data > Infrastructure). > > > Repository: samza > > > Description > ------- > > We've seen a strong need for validating running samza jobs programmatically. > Consider the following scenarios: > 1) running samza as a service (as in SAMZA-849), which requires automated job > validation when upgrading the framework for large number of jobs; > 2) production monitoring, automated validation can run periodically and > trigger alerts; > 3) integration testing, which can use validation tool to check the job status. > > So a tool is needed to provide the checkout of job health. Minimally it will > check: > - job submission > - app running > - container count > > It should also be able to extend and validate the job metrics through metrics > system, such as JMX. The user will be able to validate against important > metrics such as process_count or message_behind_high_watermark. > > > Diffs > ----- > > checkstyle/import-control.xml 53cb8b447240fea08d98ccfb12ed24bec6cbf67c > samza-api/src/main/java/org/apache/samza/metrics/MetricsAccessor.java > PRE-CREATION > > samza-api/src/main/java/org/apache/samza/metrics/MetricsValidationFailureException.java > PRE-CREATION > samza-api/src/main/java/org/apache/samza/metrics/MetricsValidator.java > PRE-CREATION > samza-core/src/main/java/org/apache/samza/job/model/JobModel.java > 9445a30c9e605a3623873ed09eedfd5140af98f7 > samza-core/src/main/java/org/apache/samza/metrics/JmxMetricsAccessor.java > PRE-CREATION > samza-core/src/main/java/org/apache/samza/metrics/JmxUtil.java PRE-CREATION > > samza-core/src/main/scala/org/apache/samza/metrics/reporter/JmxReporter.scala > e9661023a04f39d059d879fea2140cb57af3b546 > > samza-core/src/test/java/org/apache/samza/metrics/TestJmxMetricsAccessor.java > PRE-CREATION > samza-shell/src/main/bash/validate-yarn-job.sh PRE-CREATION > > samza-yarn/src/main/java/org/apache/samza/validation/YarnJobValidationTool.java > PRE-CREATION > > samza-yarn/src/test/java/org/apache/samza/validation/MockMetricsValidator.java > PRE-CREATION > > samza-yarn/src/test/java/org/apache/samza/validation/TestYarnJobValidationTool.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/41912/diff/ > > > Testing > ------- > > Unit tests added to cover all the new methods/classes. > > > Thanks, > > Xinyu Liu > >
