Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-05 Thread Robert Zuljevic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36163/ --- (Updated Aug. 6, 2015, 6:55 a.m.) Review request for samza. Repository: samza

Re: How should Samza be run on AWS?

2015-08-05 Thread Milinda Pathirage
Hi all, Looks like github deployment used by my university doesn't allow public access. I moved it to github (https://github.com/milinda/samza-ec2-ansible). Thanks Milinda On Wed, Aug 5, 2015 at 2:03 PM, Milinda Pathirage wrote: > I wrote several Ansible playbooks to deploy YARN (without HDFS

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-08-05 Thread Milinda Pathirage
> On Aug. 5, 2015, 5:34 p.m., Milinda Pathirage wrote: > > I went through old discussions and also went through Calcite's RelBuilder > > (https://github.com/milinda/incubator-calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java) > > to look at our TopologyBuilder from

Re: Missing a change log offset for SystemStreamPartition

2015-08-05 Thread Yan Fang
Hi Jordi, I wonder, the reason of your first exception is that, you changed the task number (partition number of your input stream), but still were using the same changelog stream. It is trying to send to the partition 2, which does not exist? Can you reproduce this exception in a new job? (new s

Re: How should Samza be run on AWS?

2015-08-05 Thread Milinda Pathirage
I wrote several Ansible playbooks to deploy YARN (without HDFS), Zookeeper and Kafka to EC2 for deploying Samza jobs. If you know ansible those scripts may be helpful. You can find them at https://github.iu.edu/mpathira/samza-ec2-ansible. I was planning to add document describing these scripts but

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-08-05 Thread Milinda Pathirage
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34500/#review94271 --- I went through old discussions and also went through Calcite's RelBu

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-05 Thread Yi Pan (Data Infrastructure)
> On July 27, 2015, 7:36 a.m., Yi Pan (Data Infrastructure) wrote: > > The code LGTM. For testing, if we can verify this fix w/ a stateful > > StreamTask w/ changelog enabled with some partition numbers that are > > different from the default auto-creation partition number (i.e. 8) in > > Kafk

Re: How should Samza be run on AWS?

2015-08-05 Thread Gian Merlino
I don't know of any tutorials, but the order to tackle things would be: 1) Set up ZK- this could be a single node install for a PoC or a 3 or 5 node install for production. m3.medium is a reasonable node type. 2) Set up Kafka- could be a single instance without replication for a PoC. For producti

Re: Review Request 37069: SAMZA-738 Samza Timer based metrics does not have enough precision

2015-08-05 Thread Aleksandar Pejakovic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37069/ --- (Updated Aug. 5, 2015, 2:58 p.m.) Review request for samza. Changes ---

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-05 Thread Robert Zuljevic
> On July 27, 2015, 7:36 a.m., Yi Pan (Data Infrastructure) wrote: > > The code LGTM. For testing, if we can verify this fix w/ a stateful > > StreamTask w/ changelog enabled with some partition numbers that are > > different from the default auto-creation partition number (i.e. 8) in > > Kafk

Re: Review Request 37069: SAMZA-738 Samza Timer based metrics does not have enough precision

2015-08-05 Thread Aleksandar Pejakovic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37069/ --- (Updated Aug. 5, 2015, 12:18 p.m.) Review request for samza. Changes ---

Missing a change log offset for SystemStreamPartition

2015-08-05 Thread Jordi Blasi Uribarri
Hi, I am trying to use the Keystore to manage some state information. Basically this is the code I am using. As long as I have tested, the rest is working correctly. private KeyValueStore storestp; public void init(Config config, TaskContext context) { this.storestp = (KeyValu

Re: Review Request 36903: SAMZA-744: shutdown stores before shutdown producers

2015-08-05 Thread Yan Fang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36903/#review94193 --- samza-test/src/test/scala/org/apache/samza/test/integration/StreamT