Re: Review Request 47542: SAMZA-950 - Add backward-compatible KafkaSystemProducer constructor for Java clients

2016-05-18 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47542/#review133846 --- Ship it! Ship It! - Navina Ramesh On May 18, 2016, 6:39 p.m.

Re: Review Request 47251: SAMZA-852 - Better logging when system can not be created Always log the exception when we cant instantiate a producer or consumer

2016-05-18 Thread Boris Shkolnik
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47251/#review133835 --- Ship it! - Boris Shkolnik On May 11, 2016, 6:53 p.m., Jake Ma

Re: Samza job killed by left orphaned on YARN

2016-05-18 Thread Jacob Maes
Hmm, could there be something in your job holding up the container shutdown process? Perhaps something ignoring SIGTERM/Thread.interrupt, by chance? Also, I think there's a YARN property specifying the amount of time the NM waits between sending a SIGTERM and a SIGKILL, though I can't find it at t

Re: Review Request 47542: SAMZA-950 - Add backward-compatible KafkaSystemProducer constructor for Java clients

2016-05-18 Thread Joshua Hartman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47542/#review133790 --- Ship it! Ship It! - Joshua Hartman On May 18, 2016, 6:39 p.m

Re: Review Request 47542: SAMZA-950 - Add backward-compatible KafkaSystemProducer constructor for Java clients

2016-05-18 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47542/ --- (Updated May 18, 2016, 6:39 p.m.) Review request for samza, Boris Shkolnik, Chr

RE: Enabling Core Dumps

2016-05-18 Thread Nick Quinn
Thanks Chris and Jagadish! Nick -Original Message- From: Chris Pettitt [mailto:cpett...@linkedin.com.INVALID] Sent: Wednesday, May 18, 2016 11:30 AM To: dev@samza.apache.org Subject: Re: Enabling Core Dumps ulimit sets limits for the shell and subprocesses, so you'd want to call it from

Re: Enabling Core Dumps

2016-05-18 Thread Chris Pettitt
ulimit sets limits for the shell and subprocesses, so you'd want to call it from the startup script for the samza container. On Wed, May 18, 2016 at 2:25 PM, Jagadish Venkatraman < jagadish1...@gmail.com> wrote: > Can this be enabled by passing a command line option to the jvm? > > If so, you can

Re: Enabling Core Dumps

2016-05-18 Thread Jagadish Venkatraman
Can this be enabled by passing a command line option to the jvm? If so, you can pass the option to task.opts. They will be passed on during spawning the container. On Wednesday, May 18, 2016, Nick Quinn wrote: > I am getting an intermittent crash on one of my samza tasks. How would I > go abou

Re: Samza job killed by left orphaned on YARN

2016-05-18 Thread David Yu
>From the NM log, I'm seeing: 2016-05-18 06:29:06,248 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_e01_1463512986427_0007_01_022016-05-18 06:29:06,265 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ap

Re: Samza job killed by left orphaned on YARN

2016-05-18 Thread David Yu
Jacob, I have checked and made sure that NM is running on the node: $ ps aux | grep java ... yarn 25623 0.5 0.8 2366536 275488 ? Sl May17 7:04 /usr/java/jdk1.8.0_51/bin/java -Dproc_nodemanager ... org.apache.hadoop.yarn.server.nodemanager.NodeManager Thanks, David On Wed, May

Re: Review Request 47542: SAMZA-950 - Add backward-compatible KafkaSystemProducer constructor for Java clients

2016-05-18 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47542/ --- (Updated May 18, 2016, 5:19 p.m.) Review request for samza, Boris Shkolnik, Chr

Review Request 47542: SAMZA-950 - Add backward-compatible KafkaSystemProducer constructor for Java clients

2016-05-18 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47542/ --- Review request for samza, Boris Shkolnik, Chris Pettitt, Jake Maes, Navina Rames

Enabling Core Dumps

2016-05-18 Thread Nick Quinn
I am getting an intermittent crash on one of my samza tasks. How would I go about enabling a core dump? Do I just add "ulimit -c unlimited" to the java configuration in the properties file? Any assistance would be appreciated. Thanks! Nick

Re: Java 8, RocksDB and Samza 0.10.0

2016-05-18 Thread Jason Erickson
We have had this issue, too. Did you only see the issue when you migrated to Java 8, or did you migrate to Java 8 and Samza 0.10 at the same time? On Tue, May 17, 2016 at 12:09 PM Louis Calisi wrote: > Hi, > > We are upgrading our pre-production environment to Java 8 and Samza 0.10. > We are us

Re: Java 8, RocksDB and Samza 0.10.0

2016-05-18 Thread Louis Calisi
We tried downgrading our environment back to Java 7 and we get the same error message. The same error happens with running Java 8 or Java 7. This works find (Java 8 or Java 7) on our Mac’s. It’s possibly an issue with Centos. Or it could be an issue with running inside a CDH 5.4.4 container. I’d

Re: Samza job killed by left orphaned on YARN

2016-05-18 Thread Jacob Maes
Hey David, The only time I've seen orphaned containers is when the NM dies. If the NM isn't running, the RM has no means to kill the containers on a node. Can you verify that the NM was healthy at the time of the shut down? If it wasn't healthy and/or it was restarted, one option that may help is