Re: Quota for rogue ad-hoc queries

Hi, I am trying to add this feature in hive ( HIVE-11735 ). But hit a road block while setting the quota during session folder creation as the quota can be only set by super user in HDFS. Any thoughts how to avoid this issue? Thanks, Ravi On Fri

Re: Quota for rogue ad-hoc queries

Hi Gopal, We are using MR not Tez. I feel since the adhoc queries data output size is something we can determine, rather than the time the job takes, I was wondering more from output size/number of rows quota. Thanks, Ravi On Fri, Sep 2, 2016 at 2:57 AM, Gopal Vijayaraghavan wrote: > > > Are t

Re: Quota for rogue ad-hoc queries

> Are there any other ways? Are you running Tez? Tez heartbeats counters back to the AppMaster every few seconds, so the AppMaster has an accurate (but delayed) count of HDFS_BYTES_WRITTEN. Cheers, Gopal

Re: Quota for rogue ad-hoc queries

I have written nagios scripts that watch the job tracker UI and report when things take too long. On Thu, Sep 1, 2016 at 11:08 AM, Loïc Chanel wrote: > On the topic of timeout, if I may say, they are a dangerous way to deal > with requests as a "good" request may last longer than an "evil" one.

Re: Quota for rogue ad-hoc queries

On the topic of timeout, if I may say, they are a dangerous way to deal with requests as a "good" request may last longer than an "evil" one. Be sure timeouts won't kill any important job before putting them into place. You can set these things on in the components (Tez, MapReduce ...) parameters,

Re: Quota for rogue ad-hoc queries

> rogue queries so this really isn't limited to just hive is it? any dbms system perhaps has to contend with this. even malicious rogue queries as a matter of fact. timeouts are cheap way systems handle this - assuming time is related to resource. i'm sure beeline or whatever client you use has

Re: Quota for rogue ad-hoc queries

Thanks Mich, Unfortunately we have many insert queries. Are there any other ways? Thanks, Ravi On Wed, Aug 31, 2016 at 9:45 PM, Mich Talebzadeh wrote: > Trt this > > hive.limit.optimize.fetch.max > >- Default Value: 5 >- Added In: Hive 0.8.0 > > Maximum number of rows allowed for a

Re: Quota for rogue ad-hoc queries

Trt this hive.limit.optimize.fetch.max - Default Value: 5 - Added In: Hive 0.8.0 Maximum number of rows allowed for a smaller subset of data for simple LIMIT, if it is a fetch query. Insert queries are not restricted by this limit. HTH Dr Mich Talebzadeh LinkedIn * https://www.l