Re: Review Request 37817: SAMZA-619 - Modify SamzaAppMaster to enable host-affinity

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37817/ --- (Updated Sept. 23, 2015, 3:59 a.m.) Review request for samza, Yan Fang, Chinmay

Re: Review Request 37817: SAMZA-619 - Modify SamzaAppMaster to enable host-affinity

> On Sept. 23, 2015, 1:38 a.m., Yi Pan (Data Infrastructure) wrote: > > samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContainerAllocator.java, > > line 57 > > > > > > nit: wouldn't it make more sense to

Re: Review Request 38614: SAMZA-615: Add auto-checkpoint migration for Kafka checkpoints

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38614/#review99969 --- Ship it! Looks good to me. I hope you were able to test with a samp

Re: Review Request 37817: SAMZA-619 - Modify SamzaAppMaster to enable host-affinity

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37817/#review100063 --- samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContain

Re: [Discuss/Vote] upgrade to Yarn 2.6.0

Hi, Yan, If there is no more objections, can we close this vote w/ the conclusion that Samza 0.10 will drop the support for YARN 2.4 and 2.5? Thanks! On Mon, Aug 24, 2015 at 6:32 PM, Roger Hoover wrote: > Jordan, > > Thanks for sharing that. > > Roger > > On Mon, Aug 24, 2015 at 10:17 AM, Jord

Re: Review Request 38614: SAMZA-615: Add auto-checkpoint migration for Kafka checkpoints

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38614/ --- (Updated Sept. 22, 2015, 11 p.m.) Review request for samza, Yan Fang and Navina

Re: Review Request 38614: SAMZA-615: Add auto-checkpoint migration for Kafka checkpoints

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38614/ --- (Updated Sept. 22, 2015, 6:53 p.m.) Review request for samza, Yan Fang and Navi

Re: Review Request 38614: SAMZA-615: Add auto-checkpoint migration for Kafka checkpoints

> On Sept. 22, 2015, 10 a.m., Navina Ramesh wrote: > > samza-core/src/main/java/org/apache/samza/coordinator/stream/messages/CoordinatorStreamMessage.java, > > line 295 > > > > > > I separated out this change for SA

Re: Review Request 38614: SAMZA-615: Add auto-checkpoint migration for Kafka checkpoints

> On Sept. 22, 2015, 10 a.m., Navina Ramesh wrote: > > samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamSystemFactory.java, > > line 26 > > > > > > Can you please import only the nece

Re: Review Request 38395: SAMZA-731: CoordinatorStream Equality not working

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38395/#review100024 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On Sept. 15, 2

Re: job versioning

Using “job.version”+1, Samza would automatically assign job version. However, as Navina pointed out, this is probably not a good way to go since you might want to skip a version. Thus, I think you are right in this regard. Sorry if I made a confusion. > On Sep 22, 2015, at 4:18 AM, Richard Lee

Re: Review Request 38296: SAMZA-341: Support metrics report via Ganglia

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38296/ --- (Updated Sept. 22, 2015, 2:05 p.m.) Review request for samza. Changes ---

RE: container is running beyond virtual memory limits

I am seeing that I can not get even a single job running. I have recovered the original configuration of yarn-site.xml and capacity-scheduler.xml and that does not work. I am thinking that maybe there is some kind of information related to old jobs that have not been correctly cleaned when killi

Re: Review Request 38614: SAMZA-615: Add auto-checkpoint migration for Kafka checkpoints

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38614/#review99963 --- samza-core/src/main/java/org/apache/samza/coordinator/stream/messag

Re: job versioning

I am not following what this ‘job.version’ + 1 business is about. I have build id for my code, likely a build timestamp or similar. I want to be able to determine if the currently running samza job has that version of the code. If not, I want to kill it, and launch a new job, with the same jo

Re: Review Request 38485: SAMZA-683 add meter and histogram support in the metrics reporter

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38485/ --- (Updated Sept. 22, 2015, 8:47 a.m.) Review request for samza. Changes ---

Review Request 38614: SAMZA-615: Add auto-checkpoint migration for Kafka checkpoints

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38614/ --- Review request for samza, Yan Fang and Navina Ramesh. Bugs: SAMZA-615 https

container is running beyond virtual memory limits

Hi, I am not really sure If this is related to any of the previous questions so I am asking it in a new message. I am running three different samza jobs that perform different actions and interchange information. As I found limits in the memory that were preventing the jobs to get from Accepted

RE: process killing

Hi, I have two machines running yarn and samza. They are samza 0.9.1 and hadoop 2.6.0. I run the kill-all.sh I recently wrote and calls the kill-yarn-job.sh. This is the output: java version "1.7.0_79" OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-1~deb7u1) OpenJDK 64-Bit Server VM (

Re: process killing

Hi Jordi, 1. Are you running the job in one machine yarn? or in a cluster? 2. what kind of the java process do you see after killing the yarn application? Because usually, when we run kill-yarn-job applicationId, we do kill all the processes (this is actually done by the Yarn). 3. Which version

process killing

Hi, I am currently developing solution using samza and in the development process I need to constantly change the code and test in the system. What I am seeing is that most of the times I kill a job using the kill-yarn-job script the job gets killed according to the web interface but I see the