Re: How to gracefully stop samza job

Hi, After killing the app, all the containers don’t hang, all the processes were gone, so I can’t make a thread dump. Here I only need to write data to HDFS, so my implementation of SystemFactory return null for both getConsumer and getAdmin method, only provide a valid SystemProducer. Is thi

Re: How to gracefully stop samza job

You probably should return a valid SystemAdmin object, but returning null for SystemConsumer should be OK. Again, two questions: 1) Did the container hangs during the shutdown? Or it just crashes w/ exception? Since stderr does not show anything, I was assuming that the container hangs??? 2) If the

Re: How to gracefully stop samza job

Hi, My SystemFactory implementation return null for both 『getConsumer』 and 『getAdmin』, is this the cause of the problem? Thanks. 舒琦 地址：长沙市岳麓区文轩路27号麓谷企业广场A4栋1单元6F 网址：http://www.eefung.com 微博：http://weibo.com/eefung 邮编：410013 电话：400-677-0986 传真：0731-88519609 > 在 2017年1月17日，17:18，Yi Pan

Re: How to gracefully stop samza job

Hi, Qi, In your log, the log line stops at "closing simple consumer...". It is part of the shutdownConsumers() method in the shutdown sequence. Are you sure that the container process actually proceed further in the shutdown sequence? If the container process does not proceed further (i.e. somehow

Re: How to gracefully stop samza job

Sorry, forget the log file. 舒琦 地址：长沙市岳麓区文轩路27号麓谷企业广场A4栋1单元6F 网址：http://www.eefung.com 微博：http://weibo.com/eefung 邮编：410013 电话：400-677-0986 传真：0731-88519609 > 在 2017年1月17日，10:40，舒琦 写道： > > Hi, > > Actually I check the log by using“tail" on the yarn local data dir on which > the contai

Re: How to gracefully stop samza job

Hi, I used AggregatedLogFormat to read the log file, it is the same as I uses “tail” on yarn local data dir, please check it in attachment. In the log file , you can find somethings like“Open file: /eefung/shuqi/samza-test”, but can not find lines start with“Begin to close files”. And another

Re: How to gracefully stop samza job

Hi, Actually I check the log by using“tail" on the yarn local data dir on which the container is running, the container log I found in hdfs already, but can’t tell the format for log. 舒琦 地址：长沙市岳麓区文轩路27号麓谷企业广场A4栋1单元6F 网址：http://www.eefung.com 微博：http://weibo.com/eefung 邮编：410013 电话：400-

Re: How to gracefully stop samza job

Hi I don't think you can view samza container logs in the web as mr job history, try to check the dump folder at HDFS. There should be one aggregated log file per machine in the folder named according to the job_id On 16 January 2017 at 15:30, 舒琦 wrote: > Hi, > > Thanks for your help. > > Here

Re: How to gracefully stop samza job

Hi, *container log will be removed automatically,* you can turn on yarn log aggregation, so that terminated yarn jobs' log will be dumped to HDFS On 14 January 2017 at 07:44, Yi Pan wrote: > Hi, Qi, > > Sorry to reply late. I am curious on your comment that the close and stop > methods are not

Re: How to gracefully stop samza job

Hi, Qi, Sorry to reply late. I am curious on your comment that the close and stop methods are not called. When user initiated a kill request, the graceful shutdown sequence is triggered by the shutdown hook added to SamzaContainer. The shutdown sequence is the following in the code: {code} in

How to gracefully stop samza job

Hi Guys, How can I stop running samza job gracefully except killing it? Because when samza job was killed, the close and stop method in BaseMessageChooser and SystemProducer will not be called and the container log will be removed automatically, how can resolve this? Th