Re: Review Request 23270: Wrong results when union all of grouping followed by group by with correlation optimization

2014-07-13 Thread Yin Huai
r.java <https://reviews.apache.org/r/23270/#comment83869> Do we need this? ql/src/test/queries/clientpositive/correlationoptimizer16.q <https://reviews.apache.org/r/23270/#comment83870> I think correlationoptimizer8 is for cases with UNION ALL. Can we add test queries i

Do we support "==" operator?

2014-07-22 Thread Yin Huai
Hi, Based on our language manual ( https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF), seems we do not support "==" operator. However, in FunctionRegistry, we treat "==" as "=". I guess we want to make "==" invalid and throw an exception when a user uses it? Thanks, Yin

Re: using the Hive SQL parser in Spark

2015-12-18 Thread Yin Huai
Let me add Reynold to the thread. On Fri, Dec 18, 2015 at 12:36 PM, Gopal Vijayaraghavan wrote: > > >We have looked into various options, and it looks like the best option is > >to copy the ANTLR grammar file from Hive into Spark. Because the grammar > >file is tightly coupled with Hive's semant

Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2011-09-21 Thread Yin Huai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- Review request for hive. Summary --- This optimizer exploits intra-query co

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2011-12-05 Thread Yin Huai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2011-12-05 19:12:23.087778) Review request for hive. Changes ---

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2011-12-29 Thread Yin Huai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2011-12-29 18:50:12.277210) Review request for hive. Summary ---

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-01-29 Thread Yin Huai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2012-01-29 17:56:48.704757) Review request for hive. Changes ---

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread Yin Huai
gt; > findNextChildReduceSinkOperator can return null, do you need to check > > for this? findNextChildReduceSinkOperator will not return null since its input will not be the last ReduceSinkOperator before the FileSinkOperator. For example, suppose that we have

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread Yin Huai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2012-02-10 20:49:01.177796) Review request for hive. Changes ---

configuring log4j for unit tests

2012-09-12 Thread Yin Huai
Hello All, When I run hive unit tests, I always can find an error saying the class of org.apache.hadoop.log.metrics.EventCounter can not be found. Seems the reason is that in hadoop 0.20.2, the location of EventCounter is org.apache.hadoop.metrics.jvm.EventCounter, so log4j should load the old cla

Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-15 Thread Yin Huai
relevant to the patch. Please see https://issues.apache.org/jira/browse/HIVE-2206?focusedCommentId=13456171&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13456171 for details. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-18 Thread Yin Huai
://issues.apache.org/jira/browse/HIVE-2206?focusedCommentId=13456171&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13456171 for details. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-19 Thread Yin Huai
skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Review Request: HIVE-3495 elements in aggParameters passed to SemanticAnalyzer.getGenericUDAFEvaluator are generated in two different ways

2012-09-24 Thread Yin Huai
skewjoinopt17.q, and skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

2012-09-24 Thread Yin Huai
skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-24 Thread Yin Huai
skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-24 Thread Yin Huai
skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-25 Thread Yin Huai
iew11858 ------- On Sept. 24, 2012, 3:53 p.m., Yin Huai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/7126/ > -

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-25 Thread Yin Huai
pt5.q, skewjoinopt10.q, skewjoinopt15.q to skewjoinopt17.q, and skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-09-26 Thread Yin Huai
://reviews.apache.org/r/7126/diff/ Testing (updated) --- All tests pass. Thanks, Yin Huai

Re: Review Request: For UDFAs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

2012-09-26 Thread Yin Huai
/clientpositive/nullgroup4.q.out 3dd3c66 ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc Diff: https://reviews.apache.org/r/7221/diff/ Testing (updated) --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

2012-09-26 Thread Yin Huai
) --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDFAs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

2012-09-26 Thread Yin Huai
/results/clientpositive/nullgroup4.q.out 3dd3c66 ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

2012-09-26 Thread Yin Huai
/results/clientpositive/nullgroup4.q.out 3dd3c66 ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

2012-09-29 Thread Yin Huai
/clientpositive/create_view.q 0af3442 ql/src/test/results/clientpositive/create_view.q.out 3b78f1e Diff: https://reviews.apache.org/r/7222/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

2012-09-30 Thread Yin Huai
Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 3f2 ql/src/test/queries/clientpositive/create_view.q 0af3442 ql/src/test/results/clientpositive/create_view.q.out 3b78f1e Diff: https://reviews.apache.org/r/7222/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

2012-09-30 Thread Yin Huai
/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc ql/src/test/results/clientpositive/udaf_percentile_approx.q.out 4f85a82 Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

2012-10-01 Thread Yin Huai
l/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc ql/src/test/results/clientpositive/udaf_percentile_approx.q.out 4f85a82 Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Brock Noland

2013-07-17 Thread Yin Huai
Congrats Brock! On Tue, Jul 16, 2013 at 1:26 PM, Lefty Leverenz wrote: > Congratulations & hooray, another committer to share the load! > > – Lefty > > > > On Tue, Jul 16, 2013 at 12:49 PM, Mark Grover > wrote: > > > Congrats, Brock! > > > > On Tue, Jul 16, 2013 at 12:31 PM, Thejas Nair > > wro

Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

2013-07-19 Thread Yin Huai
mac and tests passed. Thanks, Yin Huai

Re: Trying to run TestRCFile.java

2013-07-19 Thread Yin Huai
Hi Mainak, Can you try 'ant test -Dtestcase=TestRCFile'? It should work. Thanks, Yin On Fri, Jul 19, 2013 at 10:16 AM, Mainak Ghosh wrote: > > > Hello, > > I am new to Hive codebase and was hoping to run some of its test code > specifically TestRCFile.java. Unfortunately I am not sure what a

Re: Trying to run TestRCFile.java

2013-07-19 Thread Yin Huai
> > Any idea what these are? How to resolve them? > > I personally do not have a lot of idea on remote debugging Hive. So a step > by step manual if present would be of great help. > > Thanks and Regards, > Mainak. > > > [image: Inactive hide details for Yin Huai ---07

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

2013-07-19 Thread Yin Huai
94 > > <https://reviews.apache.org/r/12767/diff/1/?file=323664#file323664line194> > > > > I think this should read as: > > // remove the tag from key coming out of reducer and store it in > > separate variable. Done - Yin --- This is an automatic

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

2013-07-19 Thread Yin Huai
s are hbase_stats_empty_partition.q and ppd_key_ranges.q in TestHBaseCliDriver. I manually tested these two in my mac and tests passed. Thanks, Yin Huai

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

2013-07-19 Thread Yin Huai
Errors Success rateTime 26882 0 99.93% 43249.945 Two failures are hbase_stats_empty_partition.q and ppd_key_ranges.q in TestHBaseCliDriver. I manually tested these two in my mac and tests passed. Thanks, Yin Huai

Re: Trying to run TestRCFile.java

2013-07-19 Thread Yin Huai
a debugger with breakpoints inside the code. Any idea how I can > achieve that? > > The problem here is my debugger attaches to the live instance of hive and > testcase runs using ant. > > Thanks and Regards, > Mainak. > > [image: Inactive hide details for Yin Huai ---07/19/2

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

2013-07-19 Thread Yin Huai
------ On July 19, 2013, 5 p.m., Yin Huai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12767/ >

Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-20 Thread Yin Huai
, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Gunther Hagleitner

2013-07-21 Thread Yin Huai
Congratulations Gunther! On Sun, Jul 21, 2013 at 7:55 AM, Jarek Jarcec Cecho wrote: > Congratulations Gunther! > > Jarcec > > On Sun, Jul 21, 2013 at 01:00:31AM -0700, Carl Steinbach wrote: > > The Apache Hive PMC has voted to make Gunther Hagleitner a > > committer on the Apache Hive project. >

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-21 Thread Yin Huai
Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-23 Thread Yin Huai
Why is this? Should work regardless, no? This part is from trunk. I will take a look and see why we need it - Yin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12795/#review23671 --- On

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-24 Thread Yin Huai
---- On July 22, 2013, 4:19 a.m., Yin Huai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12795/ > --- >

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-25 Thread Yin Huai
. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-26 Thread Yin Huai
/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-26 Thread Yin Huai
/clientpositive/multiMapJoin2.q.out PRE-CREATION ql/src/test/results/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-30 Thread Yin Huai
/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-30 Thread Yin Huai
/test/results/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

2013-07-30 Thread Yin Huai
/multiMapJoin2.q.out PRE-CREATION ql/src/test/results/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Does the trunk still have an eclipse formatter?

2013-12-04 Thread Yin Huai
Hi all, After importing the trunk to eclipse, I found that eclipse did not load the hive formatter. Does trunk still have one? If so, where is the location of it? Thanks, Yin

Re: Review Request 16172: ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask also sums those tables which are not used in the child of this conditional task.

2013-12-17 Thread Yin Huai
/apache/hadoop/hive/ql/plan/ConditionalResolverCommonJoin.java <https://reviews.apache.org/r/16172/#comment58602> Let's change it to log the exception instead of printing the stack trace. - Yin Huai On Dec. 11, 2013, 2:12 a.m., Navis Ryu wrote: > >

Re: Review Request 16172: ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask also sums those tables which are not used in the child of this conditional task.

2013-12-18 Thread Yin Huai
tps://reviews.apache.org/r/16172/#comment58622> Seems it is not an error? If so, let's not put it in the ErrorMsg. ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverCommonJoin.java <https://reviews.apache.org/r/16172/#comment58623> Is this one necessary? - Yin Huai

Re: Review Request 16422: HIVE-5891: Alias conflict when merging multiple mapjoin tasks into their common child mapred task

2013-12-20 Thread Yin Huai
de in GenMapRedUtils, we do not expect this check (aliases.contains(mapJoinAlias)) will be true, right? If so, let's add a comment at here tell other code readers that they should not expect to find any conflict on aliases. - Yin Huai On Dec. 20, 2013,

Re: [ANNOUNCE] New Hive PMC Member - Gunther Hagleitner

2013-12-27 Thread Yin Huai
Congrats Gunther! On Fri, Dec 27, 2013 at 10:13 AM, Brock Noland wrote: > Congratulations Gunther! > > > On Fri, Dec 27, 2013 at 9:00 AM, Xuefu Zhang wrote: > > > Many congratulations, Gunther! > > > > --Xuefu > > > > > > On Thu, Dec 26, 2013 at 8:59 PM, Carl Steinbach wrote: > > > > > I am p

Re: Review Request 16172: ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask also sums those tables which are not used in the child of this conditional task.

2013-12-30 Thread Yin Huai
/ConditionalResolverCommonJoin.java <https://reviews.apache.org/r/16172/#comment59277> For the example in the description of HIVE-5945, the same alias "$INTNAME" can actually refer to different intermediate tables. So, at here, we will not update the correct size for an alias "$INTNAME".

Review Request 16531: HIVE-6083: User provided table properties are not assigned to the TableDesc of the FileSinkDesc in a CTAS query

2013-12-30 Thread Yin Huai
/ Testing --- Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-10-02 Thread Yin Huai
/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-10-19 Thread Yin Huai
://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-10-19 Thread Yin Huai
://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-11-02 Thread Yin Huai
/groupby1.q.xml e1eb209 ql/src/test/results/compiler/plan/groupby2.q.xml 268eb90 ql/src/test/results/compiler/plan/groupby3.q.xml 3e02d15 ql/src/test/results/compiler/plan/groupby5.q.xml 89e1207 Diff: https://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-11-12 Thread Yin Huai
/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-11-19 Thread Yin Huai
/plan/groupby2.q.xml 7b07f02 ql/src/test/results/compiler/plan/groupby3.q.xml a6a1986 ql/src/test/results/compiler/plan/groupby5.q.xml 25e3583 Diff: https://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2013-01-10 Thread Yin Huai
t; implementation and how endGroup() is dealt with in processOp() done > On Jan. 10, 2013, 2:24 a.m., Ashutosh Chauhan wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java, > > line 294 > > <https

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2013-01-15 Thread Yin Huai
Does this imply CLSRSOperator cannot have more than one child operator > > in any case. If so, can you please add comments stating that along with > > small description why is that? > > Yin Huai wrote: > I thought that, in the original plan, a ReduceSinkOperator

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2013-01-16 Thread Yin Huai
/groupby5.q.xml 632d88e Diff: https://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Custom SerDe: Initialize() passes a null configuration to my Custom SerDe

2013-10-14 Thread Yin Huai
Can you try to set serde properties? https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AddSerDeProperties I have not tried it, but seems it is the right way to pass configurations to serde class. Thanks, Yin On Mon, Oct 14, 2013 at 8:20 AM, Rui Martins wrot

Re: VOTE: Remove phabricator instructions from hive-development guide (wiki), officially only support Apache's review board.

2013-10-17 Thread Yin Huai
+1 On Thu, Oct 17, 2013 at 5:51 PM, Gunther Hagleitner < ghagleit...@hortonworks.com> wrote: > +1 > > Thanks, > Gunther. > > > On Thu, Oct 17, 2013 at 2:18 PM, Owen O'Malley wrote: > > > Ed, > > I didn't remember being unable to see revisions without a login. That > is > > uncool. I'll change

Review Request 15101: HIVE-5697: Correlation Optimizer may generate wrong plans for cases involving outer join

2013-10-30 Thread Yin Huai
/src/test/results/clientpositive/correlationoptimizer1.q.out dc46a77 Diff: https://reviews.apache.org/r/15101/diff/ Testing --- HiveQA Thanks, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Xuefu Zhang

2013-11-04 Thread Yin Huai
Congrats Xuefu! On Mon, Nov 4, 2013 at 7:11 AM, Lianhui Wang wrote: > Congrats Xuefu! > > > 2013/11/4 Thejas Nair > > > Congrats Xuefu! > > > > On Sun, Nov 3, 2013 at 11:11 PM, Mohammad Islam > > wrote: > > > Congrats Xuefu! > > > > > > --Mohammad > > > > > > > > > On Sunday, November 3, 2013

Re: Timeline for the Hive 0.13 release?

2014-03-14 Thread Yin Huai
Guys, Seems ConditionalResolverCommonJoin is not working correctly? I created https://issues.apache.org/jira/browse/HIVE-6668 and set it as a blocker. thanks, Yin On Fri, Mar 14, 2014 at 11:34 AM, Thejas Nair wrote: > Can you also add HIVE-6647 < > https://issues.apache.org/jira/browse/HIVE-6

Re: [ANNOUNCE] New Hive Committer - Thejas Nair

2013-08-20 Thread Yin Huai
Congrats Thejas!!! On Tue, Aug 20, 2013 at 4:54 PM, Thejas Nair wrote: > Thanks everybody! > This is a great honor. I hope to live up to the expectations! > > It is great to see the fresh momentum in hive and I am glad to be able to > work with the community on it. > Now I will also do my best

Re: DISTRIBUTE BY works incorrectly in Hive 0.11 in some cases

2013-08-25 Thread Yin Huai
Seems ReduceSinkDeDuplication picked the wrong partitioning columns. On Fri, Aug 23, 2013 at 9:15 PM, Shahansad KP wrote: > I think the problem lies with in the group by operation. For this > optimization to work the group bys partitioning should be on the column 1 > only. > > It wont effect th

Re: DISTRIBUTE BY works incorrectly in Hive 0.11 in some cases

2013-08-25 Thread Yin Huai
Created a jira https://issues.apache.org/jira/browse/HIVE-5149 On Sun, Aug 25, 2013 at 9:11 PM, Yin Huai wrote: > Seems ReduceSinkDeDuplication picked the wrong partitioning columns. > > > On Fri, Aug 23, 2013 at 9:15 PM, Shahansad KP wrote: > >> I think the problem lies

Re: DISTRIBUTE BY works incorrectly in Hive 0.11 in some cases

2013-08-26 Thread Yin Huai
forgot to add in my last reply To generate correct results, you can set hive.optimize.reducededuplication to false to turn off ReduceSinkDeDuplication On Sun, Aug 25, 2013 at 9:35 PM, Yin Huai wrote: > Created a jira https://issues.apache.org/jira/browse/HIVE-5149 > > > On

Review Request 13862: ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-08-27 Thread Yin Huai
Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 13862: ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-08-28 Thread Yin Huai
ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-08-30 Thread Yin Huai
a52f4eb ql/src/test/results/clientpositive/groupby_rollup1.q.out f120471 ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Review Request 13910: [HIVE-5186] Remove JoinReducerProc from ReduceSinkDeDuplication

2013-08-30 Thread Yin Huai
c380a2d ql/src/test/queries/clientpositive/reduce_deduplicate_extended.q a5e9cdf ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13910/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai
On Sept. 2, 2013, 5:39 a.m., Yin Huai wrote: > > Another sanity check. Done. - Yin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13862/#revie

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai
On Sept. 2, 2013, 5:29 a.m., Yin Huai wrote: > > Thanks for adding comments! We can have a query like ... explain select * from (select * from src1 cluster by key) tmp sort by key, value; In this case, at first, we have two MR jobs. Since the second job is used for "sort by&qu

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-02 Thread Yin Huai
llup1.q.out f120471 ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Yin Huai

2013-09-04 Thread Yin Huai
; > > >>>> -- Lefty > > >>>> > > >>>> > > >>>> On Wed, Sep 4, 2013 at 4:17 AM, Sushanth Sowmyan < > khorg...@gmail.com> > > >>> wrote: > > >>>> > > >>>>> Congrats, Yin!

Re: [jira] [Commented] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-04 Thread Yin Huai
Sure. Let me take a look at 0.11 branch. On Wed, Sep 4, 2013 at 2:53 PM, Eric Chu wrote: > Hi Yin Huai, > > This patch doesn't apply cleanly in Hive 11. For example, > ReduceSinkDeDuplication.java is in > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ in Hive 11 while the

Re: [jira] [Commented] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns

2013-09-04 Thread Yin Huai
Hi Eric, Can you try https://issues.apache.org/jira/secure/attachment/12601446/HIVE-5149-branch-0.11.patch? Thanks, Yin On Wed, Sep 4, 2013 at 3:14 PM, Yin Huai wrote: > Sure. Let me take a look at 0.11 branch. > > > On Wed, Sep 4, 2013 at 2:53 PM, Eric Chu wrote: >

Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-18 Thread Yin Huai
/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-18 Thread Yin Huai
/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-19 Thread Yin Huai
--- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-19 Thread Yin Huai
nnotation or have an assertion? serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#comment51371> ok will remove it. serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java <https://reviews.apache.org/r/14221/#

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-19 Thread Yin Huai
erde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-20 Thread Yin Huai
/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-20 Thread Yin Huai
/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-20 Thread Yin Huai
/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-20 Thread Yin Huai
/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-21 Thread Yin Huai
/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-21 Thread Yin Huai
/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

2013-09-21 Thread Yin Huai
/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

[jira] [Commented] (HIVE-7205) Wrong results when union all of grouping followed by group by with correlation optimization

2014-10-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173804#comment-14173804 ] Yin Huai commented on HIVE-7205: [~navis] Can you update the review board? I will ta

[jira] [Created] (HIVE-7362) Enabling Correlation Optimizer by default.

2014-07-07 Thread Yin Huai (JIRA)
Yin Huai created HIVE-7362: -- Summary: Enabling Correlation Optimizer by default. Key: HIVE-7362 URL: https://issues.apache.org/jira/browse/HIVE-7362 Project: Hive Issue Type: Task

[jira] [Commented] (HIVE-7205) Wrong results when union all of grouping followed by group by with correlation optimization

2014-07-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054303#comment-14054303 ] Yin Huai commented on HIVE-7205: Sure. I will take a look at it. Seems the issue is

[jira] [Commented] (HIVE-7205) Wrong results when union all of grouping followed by group by with correlation optimization

2014-07-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054454#comment-14054454 ] Yin Huai commented on HIVE-7205: [~navis] Simplifying interactions between operator

[jira] [Commented] (HIVE-5130) Document Correlation Optimizer in Hive wiki

2014-07-13 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060292#comment-14060292 ] Yin Huai commented on HIVE-5130: Design doc in Hive wiki: https://cwiki.apache

  1   2   3   4   5   6   7   >