Re: Review Request 30667: Samza Calcite Integration Prototype (SAMZA-483)

2015-02-08 Thread Milinda Pathirage
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30667/ --- (Updated Feb. 6, 2015, 9:24 p.m.) Review request for samza, Chris Riccomini, Ju

How to create a single task per container

2015-02-08 Thread Bae, Jae Hyeon
Our current main purpose of samza is for data pipeline, so we don't want to create multiple tasks in the single SamzaContainer. As I read Samza implementation, it will create as many tasks as the number of partitions assigned in the container, right? The problem of that approach is, each task will

Re: [DISCUSS] SQL workflow

2015-02-08 Thread Jakob Homan
+1. On 6 February 2015 at 16:38, Chris Riccomini wrote: > Hey all, > > Are we +1 on this? I think Jakob was the only one who was curious about it. > > Cheers, > Chris > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan wrote: > >> Hi, Jakob, >> >> >> > > Eh? Not sure what this means... >> > >> > I mean S

Re: Review Request 30758: samza-435

2015-02-08 Thread Ruslan Khafizov
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30758/#review71552 --- Please ignore. Haven't noticed it's been merged. - Ruslan Khafizov

Re: [DISCUSS] SQL workflow

2015-02-08 Thread Yi Pan
+1 On Sat, Feb 7, 2015 at 8:38 AM, Chris Riccomini wrote: > Hey all, > > Are we +1 on this? I think Jakob was the only one who was curious about it. > > Cheers, > Chris > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan wrote: > > > Hi, Jakob, > > > > > > > > Eh? Not sure what this means... > > > > > >

Re: How to create a single task per container

2015-02-08 Thread Bae, Jae Hyeon
I am reading JobCoordinator and now I can understand why multiple containers were not launched. I need to create multiple tasks, which are grouped again based on containerCount. On Fri, Feb 6, 2015 at 1:26 PM, Bae, Jae Hyeon wrote: > Our current main purpose of samza is for data pipeline, so we

Review Request 30758: samza-435

2015-02-08 Thread Ruslan Khafizov
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30758/ --- Review request for samza. Repository: samza Description --- Removed leve

Re: Question on nullEnvelop

2015-02-08 Thread Zach Cox
I just added a comment to https://issues.apache.org/jira/browse/SAMZA-506 with details on our current approach to clean shutdown in Samza 0.8.0, hopefully it's useful to others. On Fri, Feb 6, 2015 at 2:39 PM, Chris Riccomini wrote: > Hey Jae, > > > If so, what's the best way to shutdown the co

Review Request 30781: SAMZA-548

2015-02-08 Thread Chris Riccomini
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30781/ --- Review request for samza. Bugs: SAMZA-548 https://issues.apache.org/jira/br

Re: container concurrency and pipelining

2015-02-08 Thread Chris Riccomini
Hey Jordan, I've put up a perf test on: https://issues.apache.org/jira/browse/SAMZA-548 The JIRA describes the test implementation, observed performance, and noted deficiencies in the test. I'm getting much more than 2mb/s. Cheers, Chris On Fri, Feb 6, 2015 at 8:34 AM, Chris Riccomini wrote

Re: container concurrency and pipelining

2015-02-08 Thread Jordan Shaw
Hey Chris, Sorry for the delayed response, did a Tahoe 3 day weekend. Could you post your configs, and version of Samza that you're running? https://gist.github.com/jshaw86/02dbca21ae32d1a9a24e. We were running 0.8 Samza the latest stable release. We upgraded to the 0.9 branch on Friday and Kafka