RE: inconsistent results when doing a select over a join

2012-01-10 Thread David Ginzburg
gt; Subject: Re: inconsistent results when doing a select over a join > > Hi guys, > I spent the day today investigating this issue, it seems like the > differences occur when there are many killed tasks. > > We are using the fair scheduler, I ran the queries on large data and >

Re: inconsistent results when doing a select over a join

2012-01-10 Thread Guy Doulberg
Hi guys, I spent the day today investigating this issue, it seems like the differences occur when there are many killed tasks. We are using the fair scheduler, I ran the queries on large data and with low priority which caused the tasks of this job to be preempt(killed) many times. After I

Re: inconsistent results when doing a select over a join

2012-01-10 Thread Guy Doulberg
Hi, Sorry for the late answer, I ran the query on small data, but couldn't reproduce, I can reproduce it at the moment on data that takes about 1.5 hour to process, I am trying to narrow the amount of data as much as I can, and still reproduce it... But I think it is clear to me, that the sca

Re: inconsistent results when doing a select over a join

2012-01-09 Thread Edward Capriolo
Create table, query , and some small data set to reproduce On Monday, January 9, 2012, Guy Doulberg wrote: > Thanks, I am trying to reproduce it again, > > But what should I send the ML? > > > > > On Mon 09 Jan 2012 07:54:24 PM IST, Edward Capriolo wrote: >> >> Can you reproduce the issue? possib

Re: inconsistent results when doing a select over a join

2012-01-09 Thread Guy Doulberg
Thanks, I am trying to reproduce it again, But what should I send the ML? On Mon 09 Jan 2012 07:54:24 PM IST, Edward Capriolo wrote: Can you reproduce the issue? possibly with the smaller tables and send that to the ML? Edward On Mon, Jan 9, 2012 at 12:46 PM, Guy Doulberg mailto:guy.dou

Re: inconsistent results when doing a select over a join

2012-01-09 Thread Edward Capriolo
Can you reproduce the issue? possibly with the smaller tables and send that to the ML? Edward On Mon, Jan 9, 2012 at 12:46 PM, Guy Doulberg wrote: > Hey Dave, > I didn't understand your question, > > The Inconsistant is slightly different, about 2% of differences, > > Thanks > > Guy > > On 01/0

Re: inconsistent results when doing a select over a join

2012-01-09 Thread Bejoy Ks
code on a data quality issue. It could mostly be a data quality issue. Regards Bejoy.K.S From: Guy Doulberg To: user@hive.apache.org Sent: Monday, January 9, 2012 11:16 PM Subject: Re: inconsistent results when doing a select over a join Hey Dave, I didn&#

Re: inconsistent results when doing a select over a join

2012-01-09 Thread Guy Doulberg
Hey Dave, I didn't understand your question, The Inconsistant is slightly different, about 2% of differences, Thanks Guy On 01/09/2012 07:05 PM, David Houston wrote: Hi Guy, Inconsistant by way of the results are total off or the order is different? Thanks Dave On Jan 9, 2012 5:03 PM,

Re: inconsistent results when doing a select over a join

2012-01-09 Thread David Houston
Hi Guy, Inconsistant by way of the results are total off or the order is different? Thanks Dave On Jan 9, 2012 5:03 PM, "Guy Doulberg" wrote: > Hi guys, > > We are using hive for a while now, and recently we have encountered an > issue we just can't understand, > > We are selecting(the select

inconsistent results when doing a select over a join

2012-01-09 Thread Guy Doulberg
Hi guys, We are using hive for a while now, and recently we have encountered an issue we just can't understand, We are selecting(the select includes count(*)) over a join of two big tables. We ran the same query twice consequently over the same two tables , and each time the result were sl