Yes, I think switching to ThreadJobFactory is a good solution. I think the
reasons why I switched to ProcessJobFactory earlier no longer hold true.
Thanks.
Lukas
-----Original Message-----
From: Yi Pan
Sent: Friday, May 29, 2015 2:05 PM
To: dev@samza.apache.org
Subject: Re: ProcessJobFactory parent process
Hi, Lukas,
Yes. That's exactly part of the feature to allow
health-check/failure-detection of containers. Another short-term option is
trying to use ThreadJobFactory, which has the JobCoordinator and containers
in the same process. Does that work for your use case?
-Yi
On Fri, May 29, 2015 at 12:59 PM, Lukas Steiblys <lu...@doubledutch.me>
wrote:
Yes, I'm talking about the child process crashing. I'd like the parent to
die as well if the child crashes so Docker can understand that the process
failed and restart the container.
Lukas
-----Original Message----- From: Yi Pan
Sent: Friday, May 29, 2015 12:47 PM
To: dev@samza.apache.org
Subject: Re: ProcessJobFactory parent process
Hi, Lukas,
I assume that when you say "the job crashes", you were referring to the
child process running the container, not the parent process? If yes, we
were actually talking about adding container
health-check/failure-detection
in the JobCoordinator. SAMZA-680 would be the good place to start these
kind of discussion.
Thanks!
-Yi
On Fri, May 29, 2015 at 8:39 AM, Lukas Steiblys <lu...@doubledutch.me>
wrote:
Hi Yan,
The memory usage is not very high, but I'm trying to cut the usage any
way
I can.
The bigger problem is when the job crashes and the parent process stays
active preventing an auto restart by the Docker supervisor.
Lukas
On Thursday, May 28, 2015, Yan Fang <yanfang...@gmail.com> wrote:
> Hi Lukas,
>
> The parent process is used to manage the lifecycle of the actual
process. I
> am curious how much memory the parent process takes?
>
> Thanks,
>
> Fang, Yan
> yanfang...@gmail.com <javascript:;>
>
> On Thu, May 28, 2015 at 2:30 PM, Lukas Steiblys <lu...@doubledutch.me
> <javascript:;>>
> wrote:
>
> > Hello,
> >
> > I’m running Samza tasks using ProcessJobFactory and after I start the
> job,
> > the initial process spawns a new process that is the actual process
where
> > the code is run. The problem is that the parent process stays active
even
> > after the job is started and that messes with the way I deploy Samza
(in
> > Docker containers) and consumes memory while not doing anything.
> >
> > My question: is it possible to kill the parent process while still
> leaving
> > the Samza tasks to process messages?
> >
> > Lukas
>
