I don't think your solution works, as after more than 4 minutes I could still see logs of my job showing that it was running. Do you have a way to check that even if the job was running, it was not being killed by Hive ? Or another solution ?
Thanks for your help, Loïc Loïc CHANEL Engineering student at TELECOM Nancy Trainee at Worldline - Villeurbanne 2015-07-29 16:26 GMT+02:00 Loïc Chanel <loic.cha...@telecomnancy.net>: > Yes, I set it to negative 60. > > It's not a problem if the session is killed. That's actually what I try to > do, because I can't allow to a user to try to end an infinite request. > Therefore I'll try your solution :) > > Thanks, > > > Loïc > > Loïc CHANEL > Engineering student at TELECOM Nancy > Trainee at Worldline - Villeurbanne > > 2015-07-29 16:14 GMT+02:00 Xuefu Zhang <xzh...@cloudera.com>: > >> Okay. To confirm, you set it to negative 60s? >> >> The next thing you can try is to set >> hive.server2.idle.session.timeou=60000 (60sec) and >> hive.server2.idle.session.check.operation=false. I'm pretty sure this >> works, but the user's session will be killed though. >> >> --Xuefu >> >> On Wed, Jul 29, 2015 at 7:02 AM, Loïc Chanel < >> loic.cha...@telecomnancy.net> wrote: >> >>> I confirm : I just tried hive.server2.idle.operation.timeout setting it >>> to -60 (seconds), but my veeeeeery slow job have not been killed. The issue >>> here is "what if another user come and try to submit a MapReduce job but >>> the cluster is stuck in an infinite loop ?". >>> >>> Do you or anyone else have another idea ? >>> Thanks, >>> >>> >>> Loïc >>> >>> Loïc CHANEL >>> Engineering student at TELECOM Nancy >>> Trainee at Worldline - Villeurbanne >>> >>> 2015-07-29 15:34 GMT+02:00 Loïc Chanel <loic.cha...@telecomnancy.net>: >>> >>>> No, because I thought the idea of infinite operation was not very >>>> compatible with the "idle" word (as the operation will not stop running), >>>> but I'll try :-) >>>> Thanks for the idea, >>>> >>>> >>>> Loïc >>>> >>>> Loïc CHANEL >>>> Engineering student at TELECOM Nancy >>>> Trainee at Worldline - Villeurbanne >>>> >>>> 2015-07-29 15:27 GMT+02:00 Xuefu Zhang <xzh...@cloudera.com>: >>>> >>>>> Have you tried hive.server2.idle.operation.timeout? >>>>> >>>>> --Xuefu >>>>> >>>>> On Wed, Jul 29, 2015 at 5:52 AM, Loïc Chanel < >>>>> loic.cha...@telecomnancy.net> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> As I'm trying to build a secured and multi-tenant Hadoop cluster with >>>>>> Hive, I am desperately trying to set a timeout to Hive requests. >>>>>> My idea is that some users can make mistakes such as a join with >>>>>> wrong keys, and therefore start an infinite loop believing that they are >>>>>> just launching a very heavy job. Therefore, I'd like to set a limit to >>>>>> the >>>>>> time a request should take, in order to kill the job automatically if it >>>>>> exceeds it. >>>>>> >>>>>> As such a notion cannot be set directly in YARN, I saw that >>>>>> MapReduce2 provides with its own native timeout property, and I would >>>>>> like >>>>>> to know if Hive provides with the same property someway. >>>>>> >>>>>> Did anyone heard about such a thing ? >>>>>> >>>>>> Thanks in advance for your help, >>>>>> >>>>>> >>>>>> Loïc >>>>>> >>>>>> Loïc CHANEL >>>>>> Engineering student at TELECOM Nancy >>>>>> Trainee at Worldline - Villeurbanne >>>>>> >>>>> >>>>> >>>> >>> >> >
