Re: [DISCUSS] SEP-2: ApplicationRunner Design

2017-04-28 Thread xinyu liu
Right, option #2 seems redundant for defining streams after further discussion here. StreamSpec itself is flexible enough to achieve both static and programmatic specification of the stream. Agree it's not convenient for now (pretty obvious after looking at your bsr beam.runners.samza.wrapper), and

Re: Review Request 58866: fixed SAMZA-1248. use processor id for stand alone barrier

2017-04-28 Thread Navina Ramesh via Review Board
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58866/#review173409 --- samza-core/src/main/java/org/apache/samza/zk/ZkJobCoordinator.jav

Re: Review Request 58866: fixed SAMZA-1248. use processor id for stand alone barrier

2017-04-28 Thread Boris Shkolnik
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58866/ --- (Updated April 28, 2017, 11:54 p.m.) Review request for samza. Bugs: SAMZA-12

Review Request 58866: fixed SAMZA-1248. use processor id for stand alone barrier

2017-04-28 Thread Boris Shkolnik
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58866/ --- Review request for samza. Bugs: SAMZA-1248 https://issues.apache.org/jira/b

Re: Review Request 58851: SAMZA-1212 - Refactor interaction between StreamProcessor, JobCoordinator and SamzaContainer

2017-04-28 Thread Navina Ramesh via Review Board
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58851/ --- (Updated April 28, 2017, 6:50 p.m.) Review request for samza and Prateek Mahesh

Review Request 58851: SAMZA-1212 - Refactor interaction between StreamProcessor, JobCoordinator and SamzaContainer

2017-04-28 Thread Navina Ramesh via Review Board
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58851/ --- Review request for samza and Prateek Maheshwari. Repository: samza Descriptio

Re: [DISCUSS] SEP-2: ApplicationRunner Design

2017-04-28 Thread Chris Pettitt
Your proposal for #1 looks good. I'm not quite how to reconcile the proposals for #1 and #2. In #1 you add the stream spec straight onto the runner while in #2 you do it in a callback. If it is either-or, #1 looks a lot better for my purposes. For #4 what mechanism are you using to distribute the

Re: [DISCUSS] SEP-2: ApplicationRunner Design

2017-04-28 Thread Prateek Maheshwari
Thanks for summarizing and looping me back into the discussion Xinyu. Apparently I've been missing some emails from this list at my work address and some of my replies from personal email aren't being delivered either. Agree that we should replace the callback-based graph building in StreamApplica

[GitHub] samza pull request #148: SAMZA-1212 - Refactor interaction between StreamPro...

2017-04-28 Thread navina
GitHub user navina opened a pull request: https://github.com/apache/samza/pull/148 SAMZA-1212 - Refactor interaction between StreamProcessor, JobCoordinator and SamzaContainer See SAMZA-1212 for motivation toward this refactoring. Changes here are: * Removed awaitStart (bloc