[GitHub] samza pull request #146: SAMZA-1224 : Revert job coordinator factory config ...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/146 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] samza pull request #161: SAMZA-1259: LocalApplicationRunner throws exception...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/161 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] samza pull request #162: SAMZA-1228 : StreamProcessor should stop JmxServer

2017-05-03 Thread navina
GitHub user navina opened a pull request: https://github.com/apache/samza/pull/162 SAMZA-1228 : StreamProcessor should stop JmxServer This is not the solution posted in SAMZA-1228. For now, we are moving jmxserver lifecycle to be within the container. Ideally, it should be within t

Re: [VOTE] SEP 3 : Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Navina Ramesh
+1 (binding) Awesome work. Cheers! Navina On Wed, May 3, 2017 at 12:01 PM, Jagadish Venkatraman < jagadish1...@gmail.com> wrote: > +1 from my side (as described in my previous email). > > Thanks for incorporating all feedback from my previous review. > > Nice work! > > On Wed, May 3, 2017 at 11

Re: [DISCUSS] SEP-3: Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Navina Ramesh (Apache)
Abhishek, Thanks for clarifying and updating the SEP. Cheers! Navina On Wed, May 3, 2017 at 8:20 PM, Jagadish Venkatraman wrote: > Navina, > > > >> The ContainerHeartbeatMonitor and the ContainerHeartbeatClient are both > internal > APIs and have a concrete implementations. > > More specificall

Re: [DISCUSS] SEP-3: Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Navina Ramesh
Abhishek, Thanks for clarifying and updating the SEP. Cheers! Navina On Wed, May 3, 2017 at 8:20 PM, Jagadish Venkatraman wrote: > Navina, > > > >> The ContainerHeartbeatMonitor and the ContainerHeartbeatClient are both > internal > APIs and have a concrete implementations. > > More specificall

Re: [DISCUSS] SEP-3: Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Jagadish Venkatraman
Navina, >> The ContainerHeartbeatMonitor and the ContainerHeartbeatClient are both >> internal APIs and have a concrete implementations. More specifically, both of these are purely internal implementation classes (and have nothing to do with any pluggable public API that we expose) Best, Jagad

Re: [DISCUSS] SEP-3: Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Abhishek Shivanna
Hey Navina, Thank you for reviewing the SEP. > Are you planning on exposing this monitor class as a public api? What is the significance of doing so? Sorry for the confusion of having implementation details under "public interfaces". The ContainerHeartbeatMonitor and the ContainerHeartbeatClient

[GitHub] samza pull request #161: SAMZA-1259: LocalApplicationRunner throws exception...

2017-05-03 Thread smccauliff
GitHub user smccauliff opened a pull request: https://github.com/apache/samza/pull/161 SAMZA-1259: LocalApplicationRunner throws exception when configured with a ProcessorIdGenerator Minor fix. Unit tests were updated and passed. You can merge this pull request into a Git reposito

[GitHub] samza pull request #160: SAMZA-1155: Validate users configure window.ms when...

2017-05-03 Thread vjagadish1989
GitHub user vjagadish1989 opened a pull request: https://github.com/apache/samza/pull/160 SAMZA-1155: Validate users configure window.ms when using the fluent API You can merge this pull request into a Git repository by running: $ git pull https://github.com/vjagadish1989/samz

[GitHub] samza pull request #159: SAMZA-1247: MessageStreamImpl#merge shouldn't mutat...

2017-05-03 Thread prateekm
GitHub user prateekm opened a pull request: https://github.com/apache/samza/pull/159 SAMZA-1247: MessageStreamImpl#merge shouldn't mutate input collection Also fixes SAMZA-1253: MessageStream.merge operator broken for nested types You can merge this pull request into a Git repos

[GitHub] samza pull request #156: SAMZA-1257: make sure a dataChange listener in Lead...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/156 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] samza pull request #54: SAMZA-1084 - User thread does not see errors from th...

2017-05-03 Thread navina
Github user navina closed the pull request at: https://github.com/apache/samza/pull/54 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] samza pull request #148: SAMZA-1212 - Refactor interaction between StreamPro...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/148 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] samza pull request #158: Disable flaky tests.

2017-05-03 Thread shanthoosh
GitHub user shanthoosh opened a pull request: https://github.com/apache/samza/pull/158 Disable flaky tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/shanthoosh/samza master Alternatively you can review and apply these cha

Re: [DISCUSS] SEP-3: Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Navina Ramesh (Apache)
Hi Abhishek, I checked your latest proposal in SEP and it looks good to me. QQ: > A new ContainerHeartbeatMonitor class that accepts a ContainerHeartbeatClient (which has the business logic to make heartbeat checks on the JC endpoint) and a callback. Are you planning on exposing this monitor clas

[GitHub] samza pull request #155: SAMZA-1256: Improve trace logging for troubleshooti...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/155 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] samza pull request #154: SAMZA-1246: ApplicatonRunner.stats() should include...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/154 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [VOTE] SEP 3 : Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Jagadish Venkatraman
+1 from my side (as described in my previous email). Thanks for incorporating all feedback from my previous review. Nice work! On Wed, May 3, 2017 at 11:46 AM, Abhishek Shivanna wrote: > Hey everyone, > > This is the voting thread for SEP 3: Heart-beat mechanism between > JobCoordinator and al

Re: [VOTE] SEP 3 : Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Jacob Maes
+1 non-binding I think this will go a long way toward improving samza's reliability on YARN. On Wed, May 3, 2017 at 11:46 AM, Abhishek Shivanna wrote: > Hey everyone, > > This is the voting thread for SEP 3: Heart-beat mechanism between > JobCoordinator and all running containers > The Wiki pag

[VOTE] SEP 3 : Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Abhishek Shivanna
Hey everyone, This is the voting thread for SEP 3: Heart-beat mechanism between JobCoordinator and all running containers The Wiki page that discusses the SEP is: https://cwiki.apache.org/confluence/display/SAMZA/SEP-3%3A+Heart-beat+mechanism+between+JobCoordinator+and+all+running+containers Plea

Re: [DISCUSS] SEP-3: Heart-beat mechanism between JobCoordinator and all running containers

2017-05-03 Thread Abhishek Shivanna
Hey Jagadish, Thank you for taking the time to review the design. I agree with moving the heartbeat into the the LocalContainerRunner instead of fitting it into the SamzaContainer. I will update the SEP with the new design changes. Also agree with the changes to the configuration and choosing suit

Re: [DISCUSS] SEP-2: ApplicationRunner Design

2017-05-03 Thread Chris Pettitt
Hi Xinyu, I took a second look at the registerStore API. Would it be possible to call register storeDirectly on the app, similar to what we're doing with app.input (possible with the restriction registerStore must be called before we add an operator that uses the store)? Otherwise we'll end up hav