Re: [VOTE] officially stop supporting hadoop 0.20.x in hive 0.14 ?

2014-10-07 Thread Gopal V
On 10/7/14, 3:53 PM, Thejas Nair wrote: +1 Cheers, Gopal I think it is time to revisit the support for hive support for hadoop 0.20. Trying to maintain support for it puts additional burden on hive contributors. The last hadoop 0.20.x version was released on Feb 2010. Hadoop 1.0 was released

Re: GC/OOM fix when writing large/many columns

2014-10-14 Thread Gopal V
On 10/13/14, 10:53 PM, Sean McNamara wrote: I’ve found a condition where the MemoryManager will wait too long before notifying writers to check their memory and flush. ... This issue affects anyone who is writing a lot of columns, very large columns, or worst of all: both. I have tested and

Re: [ANNOUNCE] New Apache Tez Committer - Jeff Zhang

2014-10-17 Thread Gopal V
On 10/16/14, 3:42 PM, Hitesh Shah wrote: Welcome aboard, Jeff. Congrats Jeff! Cheers, Gopal

Re: Review Request 24830: HIVE-7548: Precondition checks should not fail the merge task in case of automatic trigger

2014-08-25 Thread Gopal V
tps://reviews.apache.org/r/24830/#comment89740> Use named capture in java as much as possible. (?[0-9]+) etc. ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java <https://reviews.apache.org/r/24830/#comment89741> What about LOAD DATA INPATH? - Gopal V On Aug.

Re: Patches to release branches

2014-09-11 Thread Gopal V
On 9/9/14, 1:52 PM, Mithun Radhakrishnan wrote: 1. For P1 bugs (i.e. involving data corruption, service unavailability, or serious failures without reasonable workarounds), along with a fix for trunk, I move that the current stable release branch also be patched. This will be much easier to ac

Re: Review Request 25711: Create root scratch dir with 733 instead of 777 perms

2014-09-16 Thread Gopal V
777 today from continuing to use it. ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java <https://reviews.apache.org/r/25711/#comment93298> ws - nit - Gopal V On Sept. 16, 2014, 8:36 p.m., Vaibhav Gumashta wrote: > >

Re: Review Request 25711: Create root scratch dir with 733 instead of 777 perms

2014-09-16 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25711/#review53644 --- Ship it! Ship It! - Gopal V On Sept. 16, 2014, 10:20 p.m

Re: Review Request 67420: HIVE-19772: Streaming ingest V2 API can generate invalid orc file if interrupted

2018-06-05 Thread Gopal V
atched) <https://reviews.apache.org/r/67420/#comment286770> This might not be necessary to do here. - Gopal V On June 5, 2018, 6:39 a.m., Prasanth_J wrote: > > --- > This is an automatically generated e-mail. To

Re: Review Request 67800: HIVE-20039

2018-07-02 Thread Gopal V
tables with new bucketing version will never allow an ALTER TABLE clustered by (id) buckets from changing a partition's bucket count. - Gopal V On July 3, 2018, 12:23 a.m., Deepak Jaiswal wrote: > > ---

Re: Review Request 67800: HIVE-20039

2018-07-03 Thread Gopal V
/FixedBucketPruningOptimizer.java Lines 46 (patched) <https://reviews.apache.org/r/67800/#comment288585> minor nit - Gopal V On July 3, 2018, 8:20 p.m., Deepak Jaiswal wrote: > > --- > This is an automatically generated e-mail.

Re: Review Request 67800: HIVE-20039

2018-07-03 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67800/#review205687 --- Ship it! Ship It! - Gopal V On July 3, 2018, 8:20 p.m

Re: Review Request 67966: HIVE-20202: Add profiler endpoint to httpserver

2018-07-18 Thread Gopal V
ck + timeout - fail with an error if the GET request can't get a lock after a while. - Gopal V On July 18, 2018, 6:42 p.m., Prasanth_J wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https

Re: Review Request 67966: HIVE-20202: Add profiler endpoint to httpserver

2018-07-18 Thread Gopal V
(patched) <https://reviews.apache.org/r/67966/#comment289108> Enable this only if the "async.profiler.home" property is setup - so that this is not enabled accidentally for anyone. - Gopal V On July 18, 2018, 6:42 p.m.,

Re: Review Request 67966: HIVE-20202: Add profiler endpoint to httpserver

2018-07-18 Thread Gopal V
/ProfileServlet.java Lines 211 (patched) <https://reviews.apache.org/r/67966/#comment289122> final boolean variables for "thread", "simple" and "reverse" - Gopal V On July 19, 2018, 1:08 a.m., Prasanth_J wrote: > >

Re: Review Request 68115: HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List object inspectors

2018-07-30 Thread Gopal V
check). Of course, I'm not sure that's a "real" problem unless a query takes > 5 minutes to compile. If that doesn't show up in the leaks, it might better to leave that one alone. - Gopal V

Re: Review Request 68281: HIVE-20354

2018-08-09 Thread Gopal V
/UpdateDeleteSemanticAnalyzer.java Lines 1000 (patched) <https://reviews.apache.org/r/68281/#comment290233> why not save it directly into setHintList()? - Gopal V On Aug. 9, 2018, 5:44 p.m., Deepak Jaiswal wrote: > > --- > This is a

Re: Review Request 70286: HIVE-21305: LLAP: Option to skip cache for ETL queries

2019-03-24 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70286/#review213954 --- Ship it! Ship It! - Gopal V On March 23, 2019, 2:47 a.m

Re: Review Request 71995: TopN Key optimizer should use array instead of priority queue

2020-01-21 Thread Gopal V
Line 42 (original), 44 (patched) <https://reviews.apache.org/r/71995/#comment307492> Add a counter for metrics of this - Gopal V On Jan. 14, 2020, 3:38 p.m., Attila Magyar wrote: > > --- > This is an automatically gener

Re: [VOTE] Should we release Hive Storage API 2.7.2-rc1?

2020-05-08 Thread Gopal V
Hi, Validated checksums, signatures, built and verified against latest orc. Since I was lazy enough to automate this, here's a script for others who might not have voted (or want to add things to this). https://github.com/t3rmin4t0r/verify-asf-releases + make -f Makefile.storage-api should

Re: Time to Remove Hive-on-Spark

2020-06-03 Thread Gopal V
+1 Cheers, Gopal On 6/3/20 7:48 PM, Jesus Camacho Rodriguez wrote: +1 -Jesús On Wed, Jun 3, 2020 at 1:58 PM Alan Gates wrote: +1. Alan. On Wed, Jun 3, 2020 at 1:40 PM Prasanth Jayachandran wrote: +1 On Jun 3, 2020, at 1:38 PM, Ashutosh Chauhan wrote: +1 On Wed, Jun 3, 2020 at

Re: Review Request 23351: Support direct fetch for lateral views, sub queries, etc.

2014-07-31 Thread Gopal V
/SimpleFetchOptimizer.java <https://reviews.apache.org/r/23351/#comment86301> This needs some comments, because I'm lost on what this exactly does. - Gopal V On July 9, 2014, 6:55 a.m., Navis Ryu wrote: > > --- > This is an autom

Re: Review Request 23351: Support direct fetch for lateral views, sub queries, etc.

2014-08-04 Thread Gopal V
> On Aug. 1, 2014, 6:21 a.m., Gopal V wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java, > > line 161 > > <https://reviews.apache.org/r/23351/diff/1/?file=626500#file626500line161> > > > > This needs some comments

Re: Review Request 23351: Support direct fetch for lateral views, sub queries, etc.

2014-08-04 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23351/#review49569 --- Ship it! Ship It! - Gopal V On July 9, 2014, 6:55 a.m., Navis

Re: Review Request 24834: HIVE-7771: ORC PPD fails for some decimal predicates

2014-08-18 Thread Gopal V
<https://reviews.apache.org/r/24834/#comment88843> Does this imply an immediate return? Is there are reason we go through the type coercion codepaths after getting the value out of the object? - Gopal V On Aug. 19, 2014, 1:32 a.m., Prasanth_J

Re: OOM with Hive on Tez

2014-11-06 Thread Gopal V
On 11/6/14, 11:09 AM, Kostas Tzoumas wrote: I am running into the same error [1] with plain Tez (not Hive): Any advice on what configuration parameters I should start looking at? Both issues are related to the Tez memory distributor (InitialMemoryAllocator) impl used. http://tez.apache.org/

Re: hive 0.14.1 + tez 0.4 sometimes throws NPE on small data

2015-01-09 Thread Gopal V
On 1/9/15, 11:53 AM, Alexander Pivovarov wrote: I noticed that if amount of data which should be sent to reducers is small then tez might throw NPE. I noticed that you sent it to the tez lists as well. This is a hive specific issue. https://issues.apache.org/jira/browse/HIVE-8361 to be prec

Re: Created branch 1.0

2015-01-21 Thread Gopal V
On 1/21/15, 7:09 PM, Brock Noland wrote: Too be clear I strongly feel creating 1.0 from 0.14 will be confusing. In fact it's already crrated confusion amongst folks on this list. Furthermore 1.0 should be created from trunk and be a superset of previous releases. I don't think there is any con

Re: Created branch 1.0

2015-01-23 Thread Gopal V
On 1/23/15, 6:59 AM, Xuefu Zhang wrote: While it's true that a release isn't going to include everything from trunk, proposed 1.0 release is branched off 0.14, which was again branched from trunk long time ago. If you compare the code base, you will see the huge difference. From the stability a

Re: Review Request 65342: HIVE-18546

2018-02-01 Thread Gopal V
st the txns which are currently writing and may be committed in the future. If there are 100k aborted txns, it is relevant for the reader query, but not for the mview txn state. - Gopal V On Jan. 31, 2018, 12:07 p.m., Jesús Camacho Rodr

Re: Review Request 65431: HIVE-16605 : NOT NULL constraint enforcement

2018-02-05 Thread Gopal V
uot;explain vectorization detail" qtest to verify. - Gopal V On Feb. 4, 2018, 2:36 a.m., Vineet Garg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https:

Re: Review Request 66222: HIVE-18909: Metrics for results cache

2018-03-22 Thread Gopal V
/QueryResultsCache.java Lines 277 (patched) <https://reviews.apache.org/r/66222/#comment280296> Guava has compat issues specifically for StopWatch - System.nanoTime is likely to be better than the StopWatch - Gopal V On March 22, 2018, 6:08 p.m., Jason Dere

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

2018-04-05 Thread Gopal V
/CompactorMR.java Lines 334 (patched) <https://reviews.apache.org/r/66485/#comment281391> That's a REPL event & the trouble with IOW is that it also destroys commits in progress with the new base_n files, where n > all previous open txns. - Gopal V On April 6, 2018, 1:54 a.m., Serg

Re: Review Request 66516: HIVE-19138: Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails

2018-04-09 Thread Gopal V
/SemanticAnalyzer.java Line 14642 (original), 14643 (patched) <https://reviews.apache.org/r/66516/#comment281625> Does the loop only exit if cacheEntry is non-null? - Gopal V On April 9, 2018, 9:53 p.m., Jason Dere wrote: > > ---

Re: Review Request 66533: HIVE-19154 Poll notification events to invalidate the results cache

2018-04-10 Thread Gopal V
/QueryResultsCache.java Lines 470 (patched) <https://reviews.apache.org/r/66533/#comment281792> SessionState.get().getQueryCurrentTimestamp() Possibly pass it in via QueryInfo? - Gopal V On April 10, 2018, 7:19 p.m., Jason Dere

Re: Review Request 66533: HIVE-19154 Poll notification events to invalidate the results cache

2018-04-11 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66533/#review200927 --- Ship it! Ship It! - Gopal V On April 11, 2018, 6:01 p.m

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

2018-04-16 Thread Gopal V
le that adds this commit"). storage-api/src/java/org/apache/hadoop/hive/common/ValidReaderWriteIdList.java Lines 255 (patched) <https://reviews.apache.org/r/66485/#comment282439> Should probably return a new Object here (for sane debugging). - Gopal V On April 16, 2018, 10:35

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

2018-04-22 Thread Gopal V
//reviews.apache.org/r/66485/#comment283277> This + the next look looks a bit odd - Gopal V On April 20, 2018, 11:15 p.m., Sergey Shelukhin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: >

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

2018-04-23 Thread Gopal V
/CompactorMR.java Lines 356 (patched) <https://reviews.apache.org/r/66485/#comment283344> Is that supposed to be a "," or a +? - Gopal V On April 23, 2018, 7:03 p.m., Sergey Shelukhin wrote: > > --- > This is an automatic

Re: Review Request 66485: HIVE-19124 implement a basic major compactor for MM tables

2018-04-24 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66485/#review201849 --- Ship it! Ship It! - Gopal V On April 23, 2018, 7:03 p.m

Re: Review Request 30789: HIVE-9619 Uninitialized read of numBitVectors in NumDistinctValueEstimator

2015-02-13 Thread Gopal V
/NumDistinctValueEstimator.java <https://reviews.apache.org/r/30789/#comment118534> Slight behaviour change, but mod 2 == 1 vs mod 2 != 0 matters only for -ve numbers. But that is already taken care of in generateHash. - Gopal V On Feb. 11, 2015, 9:11 p.m., Alexander Pivovarov

Re: Review Request 31386: HIVE-9555 assorted ORC refactorings for LLAP on trunk

2015-03-18 Thread Gopal V
gt; Bad behaviour - the original DiskRange was written with final variables for easier debugging. ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java <https://reviews.apache.org/r/31386/#comment124582> for loop? - Gopal V On March 11, 2015, 12:50 a.m., Serg

Re: Review Request 32918: HIVE-10180 Loop optimization for SIMD in ColumnArithmeticColumn.txt

2015-04-06 Thread Gopal V
<https://reviews.apache.org/r/32918/#comment128252> Final vars for the other case - Gopal V On April 7, 2015, 5:46 a.m., chengxiang li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 33814: Fold case udf

2015-05-06 Thread Gopal V
lect 1 from lineitem_ix will return 3024 rows). Is this a stats mismatch due to this patch? - Gopal V On May 6, 2015, 6:35 a.m., Ashutosh Chauhan wrote: > > --- > This is an automatically generated e-mail. To

Re: Review Request 54236: HIVE-15296 AM may lose task failures and not reschedule when scheduling to LLAP

2016-12-06 Thread Gopal V
/LlapTaskUmbilicalExternalClient.java (line 191) <https://reviews.apache.org/r/54236/#comment229028> This is async update to an object in the pending events queue. Possible sync issues? - Gopal V On Nov. 30, 2016, 11:39 p.m., Sergey Shelukhin

Re: Review Request 60728: HIVE-16989: fix some alerts found by lgtm.com

2017-07-11 Thread Gopal V
Line 63 (original) <https://reviews.apache.org/r/60728/#comment255206> That's probably a bug? - Gopal V On July 11, 2017, 7:13 a.m., Malcolm Taylor wrote: > > --- > This is an automatically generated e-mail. To r

Re: Review Request 61087: HIVE-16965 SMB join may produce incorrect results

2017-07-24 Thread Gopal V
hed) <https://reviews.apache.org/r/61087/#comment256770> Clear the prev and IOContext refs - interrupts do leave leaky state behind sometimes - Gopal V On July 24, 2017, 6:47 p.m., Deepak Jaiswal wrote: > > --- > This is an auto

Re: Review Request 61087: HIVE-16965 SMB join may produce incorrect results

2017-07-25 Thread Gopal V
/KeyValueInputMerger.java Lines 86 (patched) <https://reviews.apache.org/r/61087/#comment256909> That is not always true. splits.get(1) could have a different path. You might want to add a loop + assert there. - Gopal V On July 25, 2017, 8:01 p.m., Deepak Jaiswal

Re: Review Request 61374: HIVE-17220: Bloomfilter probing in semijoin reduction is thrashing L1 dcache

2017-08-02 Thread Gopal V
apache.org/r/61374/#comment257846> This won't unroll easily - better to use 1 loop + Arrays fill() - Gopal V On Aug. 2, 2017, 9:57 a.m., Prasanth_J wrote: > > --- > This is an automatically generated e-mail. To

Re: Review Request 61374: HIVE-17220: Bloomfilter probing in semijoin reduction is thrashing L1 dcache

2017-08-02 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61374/#review182078 --- Ship it! Ship It! - Gopal V On Aug. 3, 2017, 3:09 a.m

Re: Review Request 61543: HIVE-17283

2017-08-09 Thread Gopal V
(patched) <https://reviews.apache.org/r/61543/#comment258460> minor nit - semijoins twice - Gopal V On Aug. 9, 2017, 11:15 p.m., Deepak Jaiswal wrote: > > --- > This is an automatically generated e-mail. To reply, visit:

Re: Review Request 61543: HIVE-17283

2017-08-10 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61543/#review182560 --- Ship it! Ship It! - Gopal V On Aug. 10, 2017, 6:54 a.m

Re: Review Request 61956: HIVE-17323

2017-08-29 Thread Gopal V
/dynamic_semijoin_reduction.q.out Line 2667 (original), 2893 (patched) <https://reviews.apache.org/r/61956/#comment260076> Result change? - Gopal V On Aug. 29, 2017, 6:24 p.m., Deepak Jaiswal wrote: > > --- > This is an automatically gener

Re: Review Request 61985: HIVE-17399

2017-08-29 Thread Gopal V
equeue? ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q Lines 115 (patched) <https://reviews.apache.org/r/61985/#comment260179> Pick a query with a non-zero result, so that we can see when it has false negatives (i.e loses rows it is meant to have?). - Gopal V On Aug.

Re: Review Request 34576: Bucketized Table feature fails in some cases

2015-05-26 Thread Gopal V
> On May 24, 2015, 2:03 a.m., Xuefu Zhang wrote: > > Have you thought of what if the client is not interactive, such as JDBC or > > thrift? > > pengcheng xiong wrote: > I am sorry that we have not thought about it yet. We admitted that the > patch will not cover the case when the client is

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-06 Thread Gopal V
g/r/34666/#comment143696> Enum ordering nit - this needs to move down to the end for b/c. - Gopal V On July 3, 2015, 10:45 p.m., Chao Sun wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: Review Request 38493: HIVE-11132

2015-09-21 Thread Gopal V
(a,b) != (b,a) ... but one of the mismatched RS entries can be rewritten if they're both hash partitioned. - Gopal V On Sept. 18, 2015, 3:01 p.m., Ashutosh Chauhan wrote: > > --- > This is an automatically gener

Re: Review Request 38493: HIVE-11132

2015-09-24 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38493/#review100513 --- Ship it! Ship It! - Gopal V On Sept. 25, 2015, 1:37 a.m

Re: Review Request 38702: HIVE-11553 use basic file metadata cache in ETLSplitStrategy-related paths

2015-09-29 Thread Gopal V
/HiveMetaStore.java (line 5751) <https://reviews.apache.org/r/38702/#comment158357> Wrap this with a ByteBuffer impl of your own instead of doing Reflection. - Gopal V On Sept. 24, 2015, 1:03 a.m., Sergey Shelukhin wrote: > > ---

Re: Review Request 39522: HIVE-12220 LLAP: Usability issues with hive.llap.io.cache.orc.size

2015-10-21 Thread Gopal V
2291) <https://reviews.apache.org/r/39522/#comment161526> Remove the .ll.? - Gopal V On Oct. 21, 2015, 8:04 p.m., Sergey Shelukhin wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: Review Request 43070: HIVE-12976 MetaStoreDirectSql doesn't batch IN lists in all cases

2016-02-02 Thread Gopal V
/MetaStoreDirectSql.java (line 1789) <https://reviews.apache.org/r/43070/#comment178798> Very suspicious line. If the query doesn't return any results, it will get re-run old stuff? - Gopal V On Feb. 3, 2016, 2:39 a.m., Sergey Shel

Re: Review Request 43921: HIVE-13120 propagate doAs when generating ORC splits

2016-02-23 Thread Gopal V
(line 731) <https://reviews.apache.org/r/43921/#comment181852> Earlier, the FileSystem object used to handle the UGI issues - did we start passing NULL here for some reason? - Gopal V On Feb. 24, 2016, 12:12 a.m., Sergey Shelukhin

Re: Review Request 44248: HIVE-12995 LLAP: Synthetic file ids need collision checks

2016-03-07 Thread Gopal V
pache.org/r/44248/#comment184462> fileKey? - Gopal V On March 3, 2016, 11:20 p.m., Sergey Shelukhin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 44248: HIVE-12995 LLAP: Synthetic file ids need collision checks

2016-03-08 Thread Gopal V
/OrcBatchKey.java (line 48) <https://reviews.apache.org/r/44248/#comment184743> equals() need to change to handle the long to Long. - Gopal V On March 3, 2016, 11:20 p.m., Sergey Shelukhin wrote: > > --- > This is an automatica

Re: Review Request 45062: HIVE-13241 LLAP: Incremental Caching marks some small chunks as "incomplete CB"

2016-03-21 Thread Gopal V
/OrcEncodedDataReader.java (line 844) <https://reviews.apache.org/r/45062/#comment187381> is Data ever non-null here? - Gopal V On March 18, 2016, 11:18 p.m., Sergey Shelukhin wrote: > > --- > This is an automatically gener

Re: Review Request 49728: HIVE-14172 LLAP: force evict blocks by size to handle memory fragmentation

2016-07-13 Thread Gopal V
/cache/LowLevelLrfuCachePolicy.java (line 228) <https://reviews.apache.org/r/49728/#comment207572> Race condition? - Gopal V On July 6, 2016, 9:31 p.m., Sergey Shelukhin wrote: > > --- > This is an automatically generated e-

Re: Review Request 49728: HIVE-14172 LLAP: force evict blocks by size to handle memory fragmentation

2016-07-13 Thread Gopal V
ich is how this does not have a race condition. - Gopal V On July 6, 2016, 9:31 p.m., Sergey Shelukhin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Review Request: Vectorized Timestamp functions for long nanosecond based timestamps

2013-05-29 Thread Gopal V
against its non-vectorized one's output, with random data and year boundary data (+1,0,-1). Thanks, Gopal V

Re: Review Request: Vectorized Timestamp functions for long nanosecond based timestamps

2013-05-31 Thread Gopal V
gt; <https://reviews.apache.org/r/11530/diff/1/?file=298368#file298368line97> > > > > Can you comment this function to explain how you are using long[] > > inputs? I think I understand but a comment would help. > > > > The input

Re: Review Request: Vectorized Timestamp functions for long nanosecond based timestamps

2013-05-31 Thread Gopal V
/TestVectorTimestampExpressions.java PRE-CREATION Diff: https://reviews.apache.org/r/11530/diff/ Testing --- Unit tests included which compare each UDF against its non-vectorized one's output, with random data and year boundary data (+1,0,-1). Thanks, Gopal V

Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead

2014-04-29 Thread Gopal V
e/src/java/org/apache/hadoop/hive/serde2/WriteBuffers.java <https://reviews.apache.org/r/18936/#comment75421> please submit patch to hadoop as well. serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java <https://reviews.apache.org/r/18936/#comment75422>

Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead

2014-05-01 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18936/#review41935 --- Ship it! Ship It! - Gopal V On May 1, 2014, 2:29 a.m., Sergey

Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead

2014-05-01 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18936/#review41936 --- Ship it! Ship It! - Gopal V On May 1, 2014, 2:29 a.m., Sergey

Re: [ANNOUNCE] New Hive Committers - Gopal Vijayaraghavan and Szehon Ho

2014-06-22 Thread Gopal V
On 6/22/14, 8:42 PM, Carl Steinbach wrote: The Apache Hive PMC has voted to make Gopal Vijayaraghavan and Szehon Ho committers on the Apache Hive Project. Thanks everyone! And congrats Szehon! Cheers, Gopal

Review Request 17632: HDFS ZeroCopy Shims for Hive

2014-01-31 Thread Gopal V
/hive/shims/ShimLoader.java bf9c84f shims/pom.xml 9843836 Diff: https://reviews.apache.org/r/17632/diff/ Testing (updated) --- TPC-DS queries. Thanks, Gopal V

Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files

2014-01-31 Thread Gopal V
ueries on scale=2 flat ORC data. Thanks, Gopal V

Re: Review Request 17632: HDFS ZeroCopy Shims for Hive

2014-02-03 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17632/#review33519 --- On Feb. 1, 2014, 3:05 a.m., Gopal V wrote: > > --

Re: Review Request 17632: HDFS ZeroCopy Shims for Hive

2014-02-03 Thread Gopal V
shims/Hadoop23Shims.java 54c38ee shims/0.23/src/main/java/org/apache/hadoop/hive/shims/ZeroCopyShims.java PRE-CREATION shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 2b3c6c1 Diff: https://reviews.apache.org/r/17632/diff/ Testing --- TPC-DS queries. Thanks, Gopal V

Re: Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files

2014-02-05 Thread Gopal V
tests to test with zero-copy reads on. TPC-DS queries on scale=2 flat ORC data. Thanks, Gopal V

Review Request 15097: Remove .array() assumptions from ORC RecordReaderImpl and InStream

2013-10-30 Thread Gopal V
tcase=TestCliDriver -Dqfile_regex=.*orc.* Thanks, Gopal V

Re: Review Request 15095: Make the VectorGroupByOperator memory pressure factors configurable

2013-11-11 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15095/#review28670 --- Ship it! Ship It! - Gopal V On Oct. 30, 2013, 6:10 p.m., Remus

Re: Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files

2014-02-14 Thread Gopal V
a 03fc705 Diff: https://reviews.apache.org/r/17634/diff/ Testing --- Updated seek tests to test with zero-copy reads on. TPC-DS queries on scale=2 flat ORC data. Thanks, Gopal V

Re: Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files

2014-02-17 Thread Gopal V
ql/io/orc/TestOrcFile.java 03fc705 Diff: https://reviews.apache.org/r/17634/diff/ Testing --- Updated seek tests to test with zero-copy reads on. TPC-DS queries on scale=2 flat ORC data. Thanks, Gopal V

Re: Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files

2014-02-25 Thread Gopal V
7634/diff/ Testing --- Updated seek tests to test with zero-copy reads on. TPC-DS queries on scale=2 flat ORC data. Thanks, Gopal V

Re: Timeline for the Hive 0.13 release?

2014-03-10 Thread Gopal V
Can I add HIVE-6518 as well to the merge queue on https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status It is a relatively simple OOM safety patch to vectorized group-by. Tests pass locally for vec group-by, but the pre-commit tests haven't fired eventhough it's been PA for

Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead

2014-03-10 Thread Gopal V
67748> why is there an init()? serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java <https://reviews.apache.org/r/18936/#comment67736> Comment eaten up in diff? - Gopal V On March 8,

Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead

2014-03-12 Thread Gopal V
r/18936/#comment68252> Really? :) ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java <https://reviews.apache.org/r/18936/#comment68250> Rehashing involves calculating hashCodes of all keys - this is very expensive CPU wise. - Gopal V On Mar

Review Request: Improve RCFile::sync(long) by 10x

2013-04-26 Thread Gopal V
Thanks, Gopal V

[jira] [Resolved] (HIVE-7293) Hive-trunk does not build against JDK8 with generic class checks

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved HIVE-7293. --- Resolution: Not a Problem Builds are succeeding on JDK8. > Hive-trunk does not build against JDK8 w

[jira] [Commented] (HIVE-8296) Tez ReduceShuffle Vectorization needs 2 data buffers (key and value) for adding rows

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153580#comment-14153580 ] Gopal V commented on HIVE-8296: --- LGTM - +1. [~vikram.dixit]: this is necessary for

[jira] [Updated] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8304: -- Status: Patch Available (was: Open) > Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys c

[jira] [Updated] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8304: -- Attachment: HIVE-8304.2.patch Reupload for the unit tests to pick up the right file. > Tez Reduce-Side GROUP

[jira] [Commented] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154230#comment-14154230 ] Gopal V commented on HIVE-8304: --- Patch LGTM, but it is confusing to read the {

[jira] [Commented] (HIVE-8304) Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly

2014-09-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154234#comment-14154234 ] Gopal V commented on HIVE-8304: --- +1, tests pending - [~vikram.dixit], this is aimed at

[jira] [Commented] (HIVE-7664) VectorizedBatchUtil.addRowToBatchFrom is not optimized for Vectorized execution and takes 25% CPU

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155043#comment-14155043 ] Gopal V commented on HIVE-7664: --- Nope. Functional issues need to be all resolved before

[jira] [Updated] (HIVE-8236) VectorHashKeyWrapper allocates too many zero sized arrays

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8236: -- Resolution: Fixed Release Note: HIVE-8236: VectorHashKeyWrapper allocates too many zero sized arrays

[jira] [Commented] (HIVE-8236) VectorHashKeyWrapper allocates too many zero sized arrays

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155107#comment-14155107 ] Gopal V commented on HIVE-8236: --- Committed to trunk and hive-14, thanks [~prasan

[jira] [Updated] (HIVE-8271) Jackson incompatibility between hadoop-2.4 and hive-14

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-8271: -- Resolution: Fixed Release Note: "HIVE-8271: Relocate jackson within hive-exec.jar for hadoop-2.4 c

[jira] [Commented] (HIVE-7156) Group-By operator stat-annotation only uses distinct approx to generate rollups

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155450#comment-14155450 ] Gopal V commented on HIVE-7156: --- [~xuefuz]: That variable defaults to map.container.

[jira] [Commented] (HIVE-8240) VectorColumnAssignFactory throws "Incompatible Bytes vector column and primitive category VARCHAR"

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155504#comment-14155504 ] Gopal V commented on HIVE-8240: --- [~mmccline]: Can you reupload this patch, the Jenkins

[jira] [Commented] (HIVE-7156) Group-By operator stat-annotation only uses distinct approx to generate rollups

2014-10-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1414#comment-1414 ] Gopal V commented on HIVE-7156: --- bq. My point is, it's probably better if we have c

  1   2   3   4   5   6   7   8   9   10   >