----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36006/#review90929 -----------------------------------------------------------
checkstyle/import-control.xml (line 126) <https://reviews.apache.org/r/36006/#comment144096> Remove trailing whitespaces - Gustavo Anatoly F. V. Solís On Julho 2, 2015, 7:20 p.m., Shadi A. Noghabi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36006/ > ----------------------------------------------------------- > > (Updated Julho 2, 2015, 7:20 p.m.) > > > Review request for samza, Yi Pan (Data Infrastructure), Navina Ramesh, and > Naveen Somasundaram. > > > Repository: samza > > > Description > ------- > > After a job is submitted, it might need some configuration change, > specifically it might need more containers. In SAMZA-704 a tool is being > added to write to the coordinator stream (CoordinatorStreamWriter). This > tool can be used to write new configurations to the coordinator stream. > However, another tool (ConfigManager) is needed to read the config changes > and react to them, which is the goal of this task. This tool should be > brought up after the job is submitted and read any config changes added to > the coordinator stream, and react to each accordingly. > > This tool, called the Config Manager, is focusing on handling container > changs by reacting to cset-config massages with key "yarn.container.count". > > If the config manager is enabled, it will come up after the JobRunner submits > the job, and listen to config messages added there after. To enable the > ConfigManager use the flag "--config-manager-is-active=true" when submitting > the job. The config manger will periodically poll the coordinator stream to > see if there are any new messages. This period is set to 100 ms by deafualt. > However, it can be configured using "--config-manager-polling-interval=200" > when submitting the job. Thus, overal the command to run the config manager > along with the job would be: > > > <path to samza deployment>/bin/run-job.sh --config-factory=<config factory> > --config-path=<path to config file of a job> > --config-manager-is-active=<false,true> > --config-manager-polling-interval=<polling-interval> > > And this command should be used to run a job instead. > > > Diffs > ----- > > checkstyle/import-control.xml 3374f0c432e61ac4cda275377604cfd481f0cddf > > samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamMessage.java > 6c1e488d00d8593d59c89b57e673e0b6b90fd7d2 > > samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamSystemConsumer.java > b1078bdf7bddd16c9ccc6559b9efd40ca5ae67bc > > samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamWriter.java > PRE-CREATION > samza-core/src/main/java/org/apache/samza/job/ConfigManager.java > PRE-CREATION > samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala > 1c178a661e449c6bdfc4ce431aef9bb2d261a6c2 > samza-core/src/main/scala/org/apache/samza/util/CommandLine.scala > f26501b2820b99d1ad2964c6f7833ef7eaddba97 > > samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamSystemFactory.java > 647cadb3a4e51bec8204197d77ad35a6b29afcec > > samza-core/src/test/java/org/apache/samza/coordinator/stream/TestCoordinatorStreamSystemProducer.java > 68e32554c18f443565284b807f43f4a5b05afbce > > samza-core/src/test/java/org/apache/samza/coordinator/stream/TestCoordinatorStreamWriter.java > PRE-CREATION > > samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala > ea702a919348305ff95ce0b4ca1996a13aff04ec > samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMaster.scala > 20aa3736a842c462449682f3fc54c7f66c356d7d > > samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMaster.scala > 7b7d86a43c69e72c47eaa91f68be24e0f4022891 > > Diff: https://reviews.apache.org/r/36006/diff/ > > > Testing > ------- > > > Thanks, > > Shadi A. Noghabi > >
