[jira] [Updated] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3552:
-

Attachment: hive.3552.3.patch

> HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a 
> high number of grouping set keys
> -
>
> Key: HIVE-3552
> URL: https://issues.apache.org/jira/browse/HIVE-3552
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch
>
>
> This is a follow up for HIVE-3433.
> Had a offline discussion with Sambavi - she pointed out a scenario where the
> implementation in HIVE-3433 will not scale. Assume that the user is performing
> a cube on many columns, say '8' columns. So, each row would generate 256 rows
> for the hash table, which may kill the current group by implementation.
> A better implementation would be to add an additional mr job - in the first 
> mr job perform the group by assuming there was no cube. Add another mr job, 
> where
> you would perform the cube. The assumption is that the group by would have 
> decreased the output data significantly, and the rows would appear in the 
> order of
> grouping keys which has a higher probability of hitting the hash table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3074) Create a new API which gets a new empty partition object.

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3074:
-

Status: Open  (was: Patch Available)

> Create a new API which gets a new empty partition object.
> -
>
> Key: HIVE-3074
> URL: https://issues.apache.org/jira/browse/HIVE-3074
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: Nadeem Moidu
>Assignee: Namit Jain
> Attachments: hive.3074.1.patch, HIVE-3074.1.patch
>
>
> Gets a new partition object with the given parameters. Similar to 
> append_partition, but does not add the partition to metastore. It only works 
> for tables; does not work for other objects like views. The table is fetched 
> from the metastore using the db name and the table name.  However, the actual 
> partition is not fetched from the metastore. It does not matter whether the 
> partition exists or not.The partition values are used to construct a new 
> partition.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529808#comment-13529808
 ] 

Namit Jain commented on HIVE-3774:
--

comments addressed

> Sort merge join should work if join cols are a prefix of sort columns for 
> each partition
> 
>
> Key: HIVE-3774
> URL: https://issues.apache.org/jira/browse/HIVE-3774
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3774.1.patch, hive.3774.2.patch
>
>
> Currently, a join is converted into a sort-merge join only if the join cols 
> exactly matches the sort cols.
> This constraint can definitely be relaxed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3774:
-

Attachment: hive.3774.2.patch

> Sort merge join should work if join cols are a prefix of sort columns for 
> each partition
> 
>
> Key: HIVE-3774
> URL: https://issues.apache.org/jira/browse/HIVE-3774
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3774.1.patch, hive.3774.2.patch
>
>
> Currently, a join is converted into a sort-merge join only if the join cols 
> exactly matches the sort cols.
> This constraint can definitely be relaxed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3784) de-emphasize mapjoin hint

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3784:
-

Attachment: hive.3784.1.patch

> de-emphasize mapjoin hint
> -
>
> Key: HIVE-3784
> URL: https://issues.apache.org/jira/browse/HIVE-3784
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3784.1.patch
>
>
> hive.auto.convert.join has been around for a long time, and is pretty stable.
> When mapjoin hint was created, the above parameter did not exist.
> The only reason for the user to specify a mapjoin currently is if they want
> it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin.
> Eventually, that should also go away, but that may take some time to 
> stabilize.
> There are many rules in SemanticAnalyzer to handle the following trees:
> ReduceSink -> MapJoin
> Union  -> MapJoin
> MapJoin-> MapJoin
> This should not be supported anymore. In any of the above scenarios, the
> user can get the mapjoin behavior by setting hive.auto.convert.join to true
> and not specifying the hint. This will simplify the code a lot.
> What does everyone think ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3784) de-emphasize mapjoin hint

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3784:
-

Attachment: hive.3784.2.patch

> de-emphasize mapjoin hint
> -
>
> Key: HIVE-3784
> URL: https://issues.apache.org/jira/browse/HIVE-3784
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3784.1.patch, hive.3784.2.patch
>
>
> hive.auto.convert.join has been around for a long time, and is pretty stable.
> When mapjoin hint was created, the above parameter did not exist.
> The only reason for the user to specify a mapjoin currently is if they want
> it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin.
> Eventually, that should also go away, but that may take some time to 
> stabilize.
> There are many rules in SemanticAnalyzer to handle the following trees:
> ReduceSink -> MapJoin
> Union  -> MapJoin
> MapJoin-> MapJoin
> This should not be supported anymore. In any of the above scenarios, the
> user can get the mapjoin behavior by setting hive.auto.convert.join to true
> and not specifying the hint. This will simplify the code a lot.
> What does everyone think ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3793) Print number of fetched rows after query in CliDriver

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529876#comment-13529876
 ] 

Phabricator commented on HIVE-3793:
---

njain has accepted the revision "HIVE-3793 [jira] Print number of fetched rows 
after query in CliDriver".

REVISION DETAIL
  https://reviews.facebook.net/D7305

BRANCH
  DPAL-1942

To: JIRA, njain, navis


> Print number of fetched rows after query in CliDriver
> -
>
> Key: HIVE-3793
> URL: https://issues.apache.org/jira/browse/HIVE-3793
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3793.D7305.1.patch
>
>
> Currently shows time taken only. But it would be useful showing number of 
> rows fetched also. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3793) Print number of fetched rows after query in CliDriver

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529877#comment-13529877
 ] 

Namit Jain commented on HIVE-3793:
--

+1

> Print number of fetched rows after query in CliDriver
> -
>
> Key: HIVE-3793
> URL: https://issues.apache.org/jira/browse/HIVE-3793
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3793.D7305.1.patch
>
>
> Currently shows time taken only. But it would be useful showing number of 
> rows fetched also. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530021#comment-13530021
 ] 

Namit Jain commented on HIVE-3552:
--

comments addressed + tests passed

> HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a 
> high number of grouping set keys
> -
>
> Key: HIVE-3552
> URL: https://issues.apache.org/jira/browse/HIVE-3552
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch, 
> hive.3552.4.patch
>
>
> This is a follow up for HIVE-3433.
> Had a offline discussion with Sambavi - she pointed out a scenario where the
> implementation in HIVE-3433 will not scale. Assume that the user is performing
> a cube on many columns, say '8' columns. So, each row would generate 256 rows
> for the hash table, which may kill the current group by implementation.
> A better implementation would be to add an additional mr job - in the first 
> mr job perform the group by assuming there was no cube. Add another mr job, 
> where
> you would perform the cube. The assumption is that the group by would have 
> decreased the output data significantly, and the rows would appear in the 
> order of
> grouping keys which has a higher probability of hitting the hash table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3552:
-

Attachment: hive.3552.4.patch

> HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a 
> high number of grouping set keys
> -
>
> Key: HIVE-3552
> URL: https://issues.apache.org/jira/browse/HIVE-3552
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch, 
> hive.3552.4.patch
>
>
> This is a follow up for HIVE-3433.
> Had a offline discussion with Sambavi - she pointed out a scenario where the
> implementation in HIVE-3433 will not scale. Assume that the user is performing
> a cube on many columns, say '8' columns. So, each row would generate 256 rows
> for the hash table, which may kill the current group by implementation.
> A better implementation would be to add an additional mr job - in the first 
> mr job perform the group by assuming there was no cube. Add another mr job, 
> where
> you would perform the cube. The assumption is that the group by would have 
> decreased the output data significantly, and the rows would appear in the 
> order of
> grouping keys which has a higher probability of hitting the hash table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530075#comment-13530075
 ] 

Ashutosh Chauhan commented on HIVE-3783:


TestStatsPublisherEnhanced failures on apache jenkins machine. 
https://builds.apache.org/job/Hive-trunk-h0.21/1849/testReport/ 

> stats19.q is failing on trunk
> -
>
> Key: HIVE-3783
> URL: https://issues.apache.org/jira/browse/HIVE-3783
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.11
>Reporter: Ashutosh Chauhan
>Assignee: Kevin Wilfong
> Attachments: HIVE-3783.1.patch.txt
>
>
> This test-case was introduced in HIVE-3750 and is failing since as soon as it 
> was introduced. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530082#comment-13530082
 ] 

Namit Jain commented on HIVE-3774:
--

tests passed

> Sort merge join should work if join cols are a prefix of sort columns for 
> each partition
> 
>
> Key: HIVE-3774
> URL: https://issues.apache.org/jira/browse/HIVE-3774
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3774.1.patch, hive.3774.2.patch, hive.3774.3.patch
>
>
> Currently, a join is converted into a sort-merge join only if the join cols 
> exactly matches the sort cols.
> This constraint can definitely be relaxed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3774:
-

Attachment: hive.3774.3.patch

> Sort merge join should work if join cols are a prefix of sort columns for 
> each partition
> 
>
> Key: HIVE-3774
> URL: https://issues.apache.org/jira/browse/HIVE-3774
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3774.1.patch, hive.3774.2.patch, hive.3774.3.patch
>
>
> Currently, a join is converted into a sort-merge join only if the join cols 
> exactly matches the sort cols.
> This constraint can definitely be relaxed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #227

[Apache Jenkins Server]

See 


--
[...truncated 10010 lines...]

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 

[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 

 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 

[ivy:report] Processing 

 to 


ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 


test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 

 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 

[ivy:report] Processing 

 to 


ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 

 against hadoop 0.20.2 
(

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 


ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling 

 against hadoop 1.0.0 
(

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 


ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.23

build_shims:
 [echo] Project: shims
 [echo] Compiling 

 against hadoop 0.23.3 
(

[jira] [Updated] (HIVE-3793) Print number of fetched rows after query in CliDriver

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3793:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Navis

> Print number of fetched rows after query in CliDriver
> -
>
> Key: HIVE-3793
> URL: https://issues.apache.org/jira/browse/HIVE-3793
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3793.D7305.1.patch
>
>
> Currently shows time taken only. But it would be useful showing number of 
> rows fetched also. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3794) Oracle upgrade script for Hive is broken

[Deepesh Khandelwal (JIRA)]

Deepesh Khandelwal created HIVE-3794:


 Summary: Oracle upgrade script for Hive is broken
 Key: HIVE-3794
 URL: https://issues.apache.org/jira/browse/HIVE-3794
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.10.0
 Environment: Oracle 11g r2
Reporter: Deepesh Khandelwal
Priority: Critical


As part of Hive configuration for Oracle I ran the schema creation script for 
Oracle. Here is what I observed when ran the script:
% sqlplus hive/hive@xe

SQL*Plus: Release 11.2.0.2.0 Production on Mon Dec 10 18:47:11 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

SQL> @scripts/metastore/upgrade/oracle/hive-schema-0.10.0.oracle.sql;
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT 
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES 
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED

   *
ERROR at line 1:
{color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT 
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES 
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED

   *
ERROR at line 1:
{color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}



Table created.


Table altered.


Table altered.

CREATE TABLE SKEWED_COL_VALUE_LOCATION_MAPPING
 *
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}



Table created.


Table created.

ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
SKEWED_COL_VALUE_LOCATION_MAPPING_PK PRIMARY KEY (SD_ID,STRING_LIST_ID_KID)
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}


ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
SKEWED_COL_VALUE_LOCATION_MAPPING_FK1 FOREIGN KEY (STRING_LIST_ID_KID) 
REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}


ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
SKEWED_COL_VALUE_LOCATION_MAPPING_FK2 FOREIGN KEY (SD_ID) REFERENCES SDS 
(SD_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}



Table created.


Table altered.

ALTER TABLE SKEWED_VALUES ADD CONSTRAINT SKEWED_VALUES_FK1 FOREIGN KEY 
(STRING_LIST_ID_EID) REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY 
DEFERRED

   *
ERROR at line 1:
{color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}


Basically there are two issues here with the Oracle sql script:

(1) Table "SKEWED_STRING_LIST" is created with the column "SD_ID". Later the 
script tries to reference "STRING_LIST_ID" column in "SKEWED_STRING_LIST" which 
is obviously not there. Comparing the sql with that for other flavors it seems 
it should be "STRING_LIST_ID".

(2) Table name "SKEWED_COL_VALUE_LOCATION_MAPPING" is too long for Oracle which 
limits identifier names to 30 characters. Also impacted are identifiers 
"SKEWED_COL_VALUE_LOCATION_MAPPING_PK" and 
"SKEWED_COL_VALUE_LOCATION_MAPPING_FK1".


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3794) Oracle upgrade script for Hive is broken

[Deepesh Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepesh Khandelwal updated HIVE-3794:
-

Fix Version/s: 0.10.0
   Status: Patch Available  (was: Open)

I have a patch which I successfully tested on Oracle. It does impact other 
flavors.

> Oracle upgrade script for Hive is broken
> 
>
> Key: HIVE-3794
> URL: https://issues.apache.org/jira/browse/HIVE-3794
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.10.0
> Environment: Oracle 11g r2
>Reporter: Deepesh Khandelwal
>Priority: Critical
> Fix For: 0.10.0
>
>
> As part of Hive configuration for Oracle I ran the schema creation script for 
> Oracle. Here is what I observed when ran the script:
> % sqlplus hive/hive@xe
> SQL*Plus: Release 11.2.0.2.0 Production on Mon Dec 10 18:47:11 2012
> Copyright (c) 1982, 2011, Oracle.  All rights reserved.
> Connected to:
> Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
> SQL> @scripts/metastore/upgrade/oracle/hive-schema-0.10.0.oracle.sql;
> .
> ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT 
> SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES 
> SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
>   
>  *
> ERROR at line 1:
> {color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}
> .
> ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT 
> SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES 
> SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
>   
>  *
> ERROR at line 1:
> {color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}
> Table created.
> Table altered.
> Table altered.
> CREATE TABLE SKEWED_COL_VALUE_LOCATION_MAPPING
>  *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> Table created.
> Table created.
> ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
> SKEWED_COL_VALUE_LOCATION_MAPPING_PK PRIMARY KEY (SD_ID,STRING_LIST_ID_KID)
> *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
> SKEWED_COL_VALUE_LOCATION_MAPPING_FK1 FOREIGN KEY (STRING_LIST_ID_KID) 
> REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
> *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
> SKEWED_COL_VALUE_LOCATION_MAPPING_FK2 FOREIGN KEY (SD_ID) REFERENCES SDS 
> (SD_ID) INITIALLY DEFERRED
> *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> Table created.
> Table altered.
> ALTER TABLE SKEWED_VALUES ADD CONSTRAINT SKEWED_VALUES_FK1 FOREIGN KEY 
> (STRING_LIST_ID_EID) REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY 
> DEFERRED
>   
>  *
> ERROR at line 1:
> {color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}
> Basically there are two issues here with the Oracle sql script:
> (1) Table "SKEWED_STRING_LIST" is created with the column "SD_ID". Later the 
> script tries to reference "STRING_LIST_ID" column in "SKEWED_STRING_LIST" 
> which is obviously not there. Comparing the sql with that for other flavors 
> it seems it should be "STRING_LIST_ID".
> (2) Table name "SKEWED_COL_VALUE_LOCATION_MAPPING" is too long for Oracle 
> which limits identifier names to 30 characters. Also impacted are identifiers 
> "SKEWED_COL_VALUE_LOCATION_MAPPING_PK" and 
> "SKEWED_COL_VALUE_LOCATION_MAPPING_FK1".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3794) Oracle upgrade script for Hive is broken

[Deepesh Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepesh Khandelwal updated HIVE-3794:
-

Attachment: HIVE-3794.patch

> Oracle upgrade script for Hive is broken
> 
>
> Key: HIVE-3794
> URL: https://issues.apache.org/jira/browse/HIVE-3794
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.10.0
> Environment: Oracle 11g r2
>Reporter: Deepesh Khandelwal
>Priority: Critical
> Fix For: 0.10.0
>
> Attachments: HIVE-3794.patch
>
>
> As part of Hive configuration for Oracle I ran the schema creation script for 
> Oracle. Here is what I observed when ran the script:
> % sqlplus hive/hive@xe
> SQL*Plus: Release 11.2.0.2.0 Production on Mon Dec 10 18:47:11 2012
> Copyright (c) 1982, 2011, Oracle.  All rights reserved.
> Connected to:
> Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
> SQL> @scripts/metastore/upgrade/oracle/hive-schema-0.10.0.oracle.sql;
> .
> ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT 
> SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES 
> SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
>   
>  *
> ERROR at line 1:
> {color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}
> .
> ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT 
> SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES 
> SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
>   
>  *
> ERROR at line 1:
> {color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}
> Table created.
> Table altered.
> Table altered.
> CREATE TABLE SKEWED_COL_VALUE_LOCATION_MAPPING
>  *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> Table created.
> Table created.
> ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
> SKEWED_COL_VALUE_LOCATION_MAPPING_PK PRIMARY KEY (SD_ID,STRING_LIST_ID_KID)
> *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
> SKEWED_COL_VALUE_LOCATION_MAPPING_FK1 FOREIGN KEY (STRING_LIST_ID_KID) 
> REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
> *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT 
> SKEWED_COL_VALUE_LOCATION_MAPPING_FK2 FOREIGN KEY (SD_ID) REFERENCES SDS 
> (SD_ID) INITIALLY DEFERRED
> *
> ERROR at line 1:
> {color:red}ORA-00972: identifier is too long{color}
> Table created.
> Table altered.
> ALTER TABLE SKEWED_VALUES ADD CONSTRAINT SKEWED_VALUES_FK1 FOREIGN KEY 
> (STRING_LIST_ID_EID) REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY 
> DEFERRED
>   
>  *
> ERROR at line 1:
> {color:red}ORA-00904: "STRING_LIST_ID": invalid identifier{color}
> Basically there are two issues here with the Oracle sql script:
> (1) Table "SKEWED_STRING_LIST" is created with the column "SD_ID". Later the 
> script tries to reference "STRING_LIST_ID" column in "SKEWED_STRING_LIST" 
> which is obviously not there. Comparing the sql with that for other flavors 
> it seems it should be "STRING_LIST_ID".
> (2) Table name "SKEWED_COL_VALUE_LOCATION_MAPPING" is too long for Oracle 
> which limits identifier names to 30 characters. Also impacted are identifiers 
> "SKEWED_COL_VALUE_LOCATION_MAPPING_PK" and 
> "SKEWED_COL_VALUE_LOCATION_MAPPING_FK1".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3795) NPE in SELECT when WHERE-clause is an and/or/not operation involving null

[Xiao Jiang (JIRA)]

Xiao Jiang created HIVE-3795:


 Summary: NPE in SELECT when WHERE-clause is an and/or/not 
operation involving null
 Key: HIVE-3795
 URL: https://issues.apache.org/jira/browse/HIVE-3795
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Xiao Jiang


Sometimes users forget to quote date constants in queries. For example, SELECT 
* FROM some_table WHERE ds >= 2012-12-10 and ds <= 2012-12-12; . In such cases, 
if the WHERE-clause contains and/or/not operation, it would throw NPE 
exception. That's because PcrExprProcFactory in ql/optimizer forgot to check 
null. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3795) NPE in SELECT when WHERE-clause is an and/or/not operation involving null

[Xiao Jiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiao Jiang updated HIVE-3795:
-

Priority: Trivial  (was: Major)

> NPE in SELECT when WHERE-clause is an and/or/not operation involving null
> -
>
> Key: HIVE-3795
> URL: https://issues.apache.org/jira/browse/HIVE-3795
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Xiao Jiang
>Priority: Trivial
>
> Sometimes users forget to quote date constants in queries. For example, 
> SELECT * FROM some_table WHERE ds >= 2012-12-10 and ds <= 2012-12-12; . In 
> such cases, if the WHERE-clause contains and/or/not operation, it would throw 
> NPE exception. That's because PcrExprProcFactory in ql/optimizer forgot to 
> check null. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530152#comment-13530152
 ] 

Kevin Wilfong commented on HIVE-3783:
-

The failures in TestStatsPublisherEnhanced are coming from the init method of 
the stats Publisher.

> stats19.q is failing on trunk
> -
>
> Key: HIVE-3783
> URL: https://issues.apache.org/jira/browse/HIVE-3783
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.11
>Reporter: Ashutosh Chauhan
>Assignee: Kevin Wilfong
> Attachments: HIVE-3783.1.patch.txt
>
>
> This test-case was introduced in HIVE-3750 and is failing since as soon as it 
> was introduced. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530157#comment-13530157
 ] 

Kevin Wilfong commented on HIVE-3783:
-

I suspect the two are unrelated.

> stats19.q is failing on trunk
> -
>
> Key: HIVE-3783
> URL: https://issues.apache.org/jira/browse/HIVE-3783
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.11
>Reporter: Ashutosh Chauhan
>Assignee: Kevin Wilfong
> Attachments: HIVE-3783.1.patch.txt
>
>
> This test-case was introduced in HIVE-3750 and is failing since as soon as it 
> was introduced. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530167#comment-13530167
 ] 

Namit Jain commented on HIVE-3783:
--

+1

> stats19.q is failing on trunk
> -
>
> Key: HIVE-3783
> URL: https://issues.apache.org/jira/browse/HIVE-3783
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.11
>Reporter: Ashutosh Chauhan
>Assignee: Kevin Wilfong
> Attachments: HIVE-3783.1.patch.txt
>
>
> This test-case was introduced in HIVE-3750 and is failing since as soon as it 
> was introduced. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3537) release locks at the end of move tasks

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530176#comment-13530176
 ] 

Kevin Wilfong commented on HIVE-3537:
-

+1

> release locks at the end of move tasks
> --
>
> Key: HIVE-3537
> URL: https://issues.apache.org/jira/browse/HIVE-3537
> Project: Hive
>  Issue Type: Bug
>  Components: Locking, Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3537.1.patch, hive.3537.2.patch, hive.3537.3.patch
>
>
> Look at HIVE-3106 for details.
> In order to make sure that concurrency is not an issue for multi-table 
> inserts, the current option is to introduce a dependency task, which thereby
> delays the creation of all partitions. It would be desirable to release the
> locks for the outputs as soon as the move task is completed. That way, for
> multi-table inserts, the concurrency can be enabled without delaying any 
> table.
> Currently, the movetask contains a input/output, but they do not seem to be
> populated correctly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-3795) NPE in SELECT when WHERE-clause is an and/or/not operation involving null

[Xiao Jiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiao Jiang reassigned HIVE-3795:


Assignee: Xiao Jiang

> NPE in SELECT when WHERE-clause is an and/or/not operation involving null
> -
>
> Key: HIVE-3795
> URL: https://issues.apache.org/jira/browse/HIVE-3795
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Xiao Jiang
>Assignee: Xiao Jiang
>Priority: Trivial
>
> Sometimes users forget to quote date constants in queries. For example, 
> SELECT * FROM some_table WHERE ds >= 2012-12-10 and ds <= 2012-12-12; . In 
> such cases, if the WHERE-clause contains and/or/not operation, it would throw 
> NPE exception. That's because PcrExprProcFactory in ql/optimizer forgot to 
> check null. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs

[Kevin Wilfong (JIRA)]

Kevin Wilfong created HIVE-3796:
---

 Summary: Multi-insert involving bucketed/sorted table turns off 
merging on all outputs
 Key: HIVE-3796
 URL: https://issues.apache.org/jira/browse/HIVE-3796
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Affects Versions: 0.11
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong


When a multi-insert query has at least one output that is bucketed, merging is 
turned off for all outputs, rather than just the bucketed ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3785) Core hive changes for HiveServer2 implementation

[Mark Grover (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530287#comment-13530287
 ] 

Mark Grover commented on HIVE-3785:
---

FWIW, this patch doesn't apply cleanly on trunk. There is a conflict with 
conf/hive-default.xml.template
{code}
***
*** 1581,1590 
  
  
  
-hive.decode.partition.name
-false
-Whether to show the unquoted partition names in query 
results.
  
  
  
  
--- 1580,1660 
  
  
  
+   hive.server2.thrift.min.worker.threads
+   5
+   Minimum number of Thrift worker threads
+ 
+ 
+ 
+   hive.server2.thrift.max.worker.threads
+   100
+   Maximum number of Thrift worker threads
+ 
+ 
{code}
I don't it's necessary to refresh the patch now (but it wouldn't hurt) since we 
are still waiting for review but it would need to be refreshed before 
submission.

> Core hive changes for HiveServer2 implementation
> 
>
> Key: HIVE-3785
> URL: https://issues.apache.org/jira/browse/HIVE-3785
> Project: Hive
>  Issue Type: Sub-task
>  Components: Authentication, Build Infrastructure, Configuration, 
> Thrift API
>Affects Versions: 0.10.0
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
> Attachments: HS2-changed-files-only.patch
>
>
> The subtask to track changes in the core hive components for HiveServer2 
> implementation

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530288#comment-13530288
 ] 

Phabricator commented on HIVE-3384:
---

mbautin has abandoned the revision "[jira] [HIVE-3384] HIVE JDBC module won't 
compile under JDK1.7 as new methods added in JDBC specification".

  Committed into Hive trunk and 0.9 branches.

REVISION DETAIL
  https://reviews.facebook.net/D6873

To: cdrome, weidongbian, JIRA, ashutoshc, mbautin


> HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC 
> specification
> --
>
> Key: HIVE-3384
> URL: https://issues.apache.org/jira/browse/HIVE-3384
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Reporter: Weidong Bian
>Assignee: Mikhail Bautin
>Priority: Minor
> Fix For: 0.11
>
> Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, 
> D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, 
> HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, 
> HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch
>
>
> jdbc module couldn't be compiled with jdk7 as it adds some abstract method in 
> the JDBC specification 
> some error info:
>  error: HiveCallableStatement is not abstract and does not override abstract
> method getObject(String,Class) in CallableStatement
> .
> .
> .

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530320#comment-13530320
 ] 

Kevin Wilfong commented on HIVE-3796:
-

https://reviews.facebook.net/D7335

> Multi-insert involving bucketed/sorted table turns off merging on all outputs
> -
>
> Key: HIVE-3796
> URL: https://issues.apache.org/jira/browse/HIVE-3796
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.11
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-3796.1.patch.txt
>
>
> When a multi-insert query has at least one output that is bucketed, merging 
> is turned off for all outputs, rather than just the bucketed ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-3796:


Attachment: HIVE-3796.1.patch.txt

> Multi-insert involving bucketed/sorted table turns off merging on all outputs
> -
>
> Key: HIVE-3796
> URL: https://issues.apache.org/jira/browse/HIVE-3796
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.11
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-3796.1.patch.txt
>
>
> When a multi-insert query has at least one output that is bucketed, merging 
> is turned off for all outputs, rather than just the bucketed ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-3796:


Status: Patch Available  (was: Open)

> Multi-insert involving bucketed/sorted table turns off merging on all outputs
> -
>
> Key: HIVE-3796
> URL: https://issues.apache.org/jira/browse/HIVE-3796
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.11
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-3796.1.patch.txt
>
>
> When a multi-insert query has at least one output that is bucketed, merging 
> is turned off for all outputs, rather than just the bucketed ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1851 - Still Failing

[Apache Jenkins Server]

Changes for Build #1844
[hashutosh] HIVE-3705 : Adding authorization capability to the metastore 
(Sushanth Sowmyan via Ashutosh Chauhan)


Changes for Build #1845
[hashutosh] HIVE-3231 : msck repair should find partitions already containing 
data files (Keegan Mosley via Ashutosh Chauhan)

[hashutosh] HIVE-2691 : Specify location of log4j configuration files via 
configuration properties (Zhenxiao Luo via Ashutosh Chauhan)

[hashutosh] HIVE-2794 : Aggregations without grouping should return NULL when 
applied to partitioning column of a partitionless table (Zhenxiao Luo via 
Ashutosh Chauhan)

[hashutosh] HIVE-3780 : RetryingMetaStoreClient Should Log the Caught Exception 
(Bhushan Mandhani via Ashutosh Chauhan)

[hashutosh] HIVE-3084 : Hive CI failing due to script_broken_pipe1.q (Gunther 
Hagleitner via Ashutosh Chauhan)

[hashutosh] HIVE-3760 : TestNegativeMinimrCliDriver_mapreduce_stack_trace.q 
fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan)


Changes for Build #1846

Changes for Build #1847
[hashutosh] HIVE-3714 : Patch: Hive's ivy internal resolvers need to use 
sourceforge for sqlline (Gopal V via Ashutosh Chauhan)


Changes for Build #1848
[hashutosh] HIVE-3782 : testCliDriver_sample_islocalmode_hook fails on hadoop-1 
(Gunther Hagleitner via Ashutosh Chauhan)

[hashutosh] HIVE-2288 : Adding the oracle nvl function to the UDF (Ed Capriolo, 
Guy Doulberg via Ashutosh Chauhan)

[hashutosh] HIVE-2689 : ObjectInspectorConverters cannot convert Void types to 
Array/Map/Struct types. (Jonathan Chang via Ashutosh Chauhan)


Changes for Build #1849
[hashutosh] HIVE-3622 : reflect udf cannot find method which has arguments of 
primitive types and String, Binary, Timestamp types mixed (Navis via Ashutosh 
Chauhan)

[namit] HIVE-3401 Diversify grammar for split sampling
(Navis via namit)


Changes for Build #1850

Changes for Build #1851



7 tests failed.
REGRESSION:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1

Error Message:
Forked Java VM exited abnormally. Please note the time in the report does not 
reflect the time until the VM exit.

Stack Trace:
junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please 
note the time in the report does not reflect the time until the VM exit.
at 
net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259)
at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268)
at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324)
at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244)


REGRESSION:  
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat

Error Message:
null

Stack Trace:
junit.framework.AssertionFailedError: null
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at 
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat(TestStatsPublisherEnhanced.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79)
at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)


REGRESSION:  
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher

Error Message:
null

Stack Trace:
junit.framework.AssertionFailedError: null
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at 
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher(TestStatsPublisherEnhanced.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.inv

[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling

[Lars Francke (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530505#comment-13530505
 ] 

Lars Francke commented on HIVE-3401:


I  know Namit already asked about it but I think this'd be the relevant Wiki 
page to update: 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Sampling

> Diversify grammar for split sampling
> 
>
> Key: HIVE-3401
> URL: https://issues.apache.org/jira/browse/HIVE-3401
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, 
> HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, 
> HIVE-3401.D4821.7.patch
>
>
> Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But 
> some users wants to specify just the size of input. It can be easily 
> calculated with a few commands but it seemed good to support more grammars 
> something like TABLESAMPLE(500M). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling

[Navis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530508#comment-13530508
 ] 

Navis commented on HIVE-3401:
-

How can I get the permission?

> Diversify grammar for split sampling
> 
>
> Key: HIVE-3401
> URL: https://issues.apache.org/jira/browse/HIVE-3401
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, 
> HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, 
> HIVE-3401.D4821.7.patch
>
>
> Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But 
> some users wants to specify just the size of input. It can be easily 
> calculated with a few commands but it seemed good to support more grammars 
> something like TABLESAMPLE(500M). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3789) Patch HIVE-3648 causing the majority of unit tests to fail on branch 0.9

[Arup Malakar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530545#comment-13530545
 ] 

Arup Malakar commented on HIVE-3789:


The exception gets eaten up and doesn't show up in console. Here is the exact 
stacktrace which I caught and logged, before rethrowing.

{code}
[junit] Something wrong happened while moving to trash Wrong FS: 
pfile:/Users/malakar/code/oss/hive_09/hive/build/metastore/test/data/warehouse/testtablefilter.db/table1,
 expected: file:///[junit] 
org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:581)
[junit] org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:680)
[junit] 
org.apache.hadoop.fs.FilterFileSystem.resolvePath(FilterFileSystem.java:139)
[junit] 
org.apache.hadoop.fs.FilterFileSystem.resolvePath(FilterFileSystem.java:139)
[junit] 
org.apache.hadoop.fs.FilterFileSystem.resolvePath(FilterFileSystem.java:139)
[junit] org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:70)
[junit] 
org.apache.hadoop.hive.shims.Hadoop23Shims.moveToAppropriateTrash(Hadoop23Shims.java:133)
[junit] 
org.apache.hadoop.hive.metastore.HiveMetaStoreFsImpl.deleteDir(HiveMetaStoreFsImpl.java:45)
[junit] 
org.apache.hadoop.hive.metastore.Warehouse.deleteDir(Warehouse.java:201)
[junit] 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:929)
[junit] 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table(HiveMetaStore.java:944)
[junit] 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table.getResult(ThriftHiveMetastore.java:4955)
[junit] 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table.getResult(ThriftHiveMetastore.java:4943)
[junit] org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
[junit] org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
[junit] 
org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)
[junit] 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
[junit] 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[junit] 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[junit] java.lang.Thread.run(Thread.java:680)
{code}

> Patch HIVE-3648 causing the majority of unit tests to fail on branch 0.9
> 
>
> Key: HIVE-3789
> URL: https://issues.apache.org/jira/browse/HIVE-3789
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Tests
>Affects Versions: 0.9.0
> Environment: Hadooop 0.23.5, JDK 1.6.0_31
>Reporter: Chris Drome
>
> Rolling back to before this patch shows that the unit tests are passing, 
> after the patch, the majority of the unit tests are failing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2379) Hive/HBase integration could be improved

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis reassigned HIVE-2379:
---

Assignee: Navis

> Hive/HBase integration could be improved
> 
>
> Key: HIVE-2379
> URL: https://issues.apache.org/jira/browse/HIVE-2379
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, Clients, HBase Handler
>Affects Versions: 0.7.1, 0.8.0, 0.9.0
>Reporter: Roman Shaposhnik
>Assignee: Navis
>Priority: Minor
>
> For now any Hive/HBase queries would require the following jars to be 
> explicitly added via hive's add jar command:
> add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar;
> add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar;
> add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar;
> add jar /usr/lib/hive/lib/guava-r06.jar;
> the longer term solution, perhaps, should be to have the code at submit time 
> call hbase's 
> TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship 
> it in distributedcache.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2379) Hive/HBase integration could be improved

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-2379:


Status: Patch Available  (was: Open)

> Hive/HBase integration could be improved
> 
>
> Key: HIVE-2379
> URL: https://issues.apache.org/jira/browse/HIVE-2379
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, Clients, HBase Handler
>Affects Versions: 0.9.0, 0.8.0, 0.7.1
>Reporter: Roman Shaposhnik
>Assignee: Navis
>Priority: Minor
>
> For now any Hive/HBase queries would require the following jars to be 
> explicitly added via hive's add jar command:
> add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar;
> add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar;
> add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar;
> add jar /usr/lib/hive/lib/guava-r06.jar;
> the longer term solution, perhaps, should be to have the code at submit time 
> call hbase's 
> TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship 
> it in distributedcache.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2379) Hive/HBase integration could be improved

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2379:
--

Attachment: HIVE-2379.D7347.1.patch

navis requested code review of "HIVE-2379 [jira] Hive/HBase integration could 
be improved".
Reviewers: JIRA

  DPAL-1936 Add libraries for hbase handler automatically

  For now any Hive/HBase queries would require the following jars to be 
explicitly added via hive's add jar command:

  add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar;
  add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar;
  add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar;
  add jar /usr/lib/hive/lib/guava-r06.jar;

  the longer term solution, perhaps, should be to have the code at submit time 
call hbase's
  TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship 
it in distributedcache.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D7347

AFFECTED FILES
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
  ql/src/java/org/apache/hadoop/hive/ql/metadata/DefaultStorageHandler.java
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/17559/

To: JIRA, navis


> Hive/HBase integration could be improved
> 
>
> Key: HIVE-2379
> URL: https://issues.apache.org/jira/browse/HIVE-2379
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, Clients, HBase Handler
>Affects Versions: 0.7.1, 0.8.0, 0.9.0
>Reporter: Roman Shaposhnik
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-2379.D7347.1.patch
>
>
> For now any Hive/HBase queries would require the following jars to be 
> explicitly added via hive's add jar command:
> add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar;
> add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar;
> add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar;
> add jar /usr/lib/hive/lib/guava-r06.jar;
> the longer term solution, perhaps, should be to have the code at submit time 
> call hbase's 
> TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship 
> it in distributedcache.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530656#comment-13530656
 ] 

Namit Jain commented on HIVE-3401:
--

Navis, can you tell me your wiki id. ?
I can add you.

> Diversify grammar for split sampling
> 
>
> Key: HIVE-3401
> URL: https://issues.apache.org/jira/browse/HIVE-3401
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, 
> HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, 
> HIVE-3401.D4821.7.patch
>
>
> Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But 
> some users wants to specify just the size of input. It can be easily 
> calculated with a few commands but it seemed good to support more grammars 
> something like TABLESAMPLE(500M). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3790) UDF to introduce an OFFSET(day,month or year) for a given date or timestamp

[Jithin John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530658#comment-13530658
 ] 

Jithin John commented on HIVE-3790:
---

hi Sun Rui,
currently we could acheive the scenario by nesting the fuction like 
date_offset(date_offset(date,1,'year'),3,'month')
we could consider whether SQL inteval type is possible or not?

> UDF to introduce an OFFSET(day,month or year) for a given date or timestamp 
> 
>
> Key: HIVE-3790
> URL: https://issues.apache.org/jira/browse/HIVE-3790
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Affects Versions: 0.9.0
>Reporter: Jithin John
>
> Current releases of Hive lacks a  generic function which would find the date 
> offset to a date / timestamp. Current releases have date_add (date) and 
> date_sub(date) which allows user to add or substract days only.But we could 
> not use year or month as a unit.
> 
> The Function DATE_OFFSET(date,offset,unit) returns the date offset value from 
> start_date according to the unit. Here the unit can be year , month and day.
> The function could be used for date range queries and is more flexible than 
> the existing functions.
> Functionality :-
> Function Name: DATE_OFFSET(date,offset,unit)
>
> Add a offset value to the unit part of the date/timestamp.
> Returns  the date in the format of "-MM-dd" .
> Example: hive> select date_offset('2009-07-29', -1 ,'MONTH' ) FROM src LIMIT 1
> -> 2009-06-29
> Usage :-
> Case  : To calculate the expiry date of a item from manufacturing date 
> Table :- ITEM_TAB
>  Manufacturing_date  |item id|store id|value|unit|price
>   2012-12-01|110001|00003|0.99|1.00|0.99
>   2012-12-02|110001|00008|0.99|0.00|0.00
>   2012-12-03|110001|00009|0.99|0.00|0.00
>   2012-12-04|110001|001112002|0.99|0.00|0.00
>   2012-12-05|110001|001112003|0.99|0.00|0.00
>   2012-12-06|110001|001112006|0.99|1.00|0.99
>   2012-12-07|110001|001112007|0.99|0.00|0.00
>   2012-12-08|110001|001112008|0.99|0.00|0.00
>   2012-12-09|110001|001112009|0.99|0.00|0.00
>   2012-12-10|110001|001112010|0.99|0.00|0.00
>   2012-12-11|110001|001113003|0.99|0.00|0.00
>   2012-12-12|110001|001113006|0.99|0.00|0.00
>   2012-12-13|110001|001113008|0.99|0.00|0.00
>   2012-12-14|110001|001113010|0.99|0.00|0.00
>   2012-12-15|110001|001114002|0.99|0.00|0.00
>   2012-12-16|110001|001114004|0.99|1.00|0.99
>   2012-12-17|110001|001114005|0.99|0.00|0.00
>   2012-12-18|110001|001121004|0.99|0.00|0.00 
> QUERY:
> select man_date , date_offset(man_date ,5 ,'year') as expiry_date from 
> item_tab;
> RESULT:
> 2012-12-01  2017-12-01
> 2012-12-02  2017-12-02
> 2012-12-03  2017-12-03
> 2012-12-04  2017-12-04
> 2012-12-05  2017-12-05
> 2012-12-06  2017-12-06
> 2012-12-07  2017-12-07
> 2012-12-08  2017-12-08
> 2012-12-09  2017-12-09
> 2012-12-10  2017-12-10
> 2012-12-11  2017-12-11
> 2012-12-12  2017-12-12
> 2012-12-13  2017-12-13
> 2012-12-14  2017-12-14
> 2012-12-15  2017-12-15
> 2012-12-16  2017-12-16
> 2012-12-17  2017-12-17
> 2012-12-18  2017-12-18

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling

[Navis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530659#comment-13530659
 ] 

Navis commented on HIVE-3401:
-

It's 'navis' and..  as you know by reviewing my codes/comments, I'm really bad 
at English writing.

> Diversify grammar for split sampling
> 
>
> Key: HIVE-3401
> URL: https://issues.apache.org/jira/browse/HIVE-3401
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, 
> HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, 
> HIVE-3401.D4821.7.patch
>
>
> Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But 
> some users wants to specify just the size of input. It can be easily 
> calculated with a few commands but it seemed good to support more grammars 
> something like TABLESAMPLE(500M). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3790) UDF to introduce an OFFSET(day,month or year) for a given date or timestamp

[Jithin John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530666#comment-13530666
 ] 

Jithin John commented on HIVE-3790:
---

hi georgy,
since we are using java calendar , leap year scenario will work.


> UDF to introduce an OFFSET(day,month or year) for a given date or timestamp 
> 
>
> Key: HIVE-3790
> URL: https://issues.apache.org/jira/browse/HIVE-3790
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Affects Versions: 0.9.0
>Reporter: Jithin John
>
> Current releases of Hive lacks a  generic function which would find the date 
> offset to a date / timestamp. Current releases have date_add (date) and 
> date_sub(date) which allows user to add or substract days only.But we could 
> not use year or month as a unit.
> 
> The Function DATE_OFFSET(date,offset,unit) returns the date offset value from 
> start_date according to the unit. Here the unit can be year , month and day.
> The function could be used for date range queries and is more flexible than 
> the existing functions.
> Functionality :-
> Function Name: DATE_OFFSET(date,offset,unit)
>
> Add a offset value to the unit part of the date/timestamp.
> Returns  the date in the format of "-MM-dd" .
> Example: hive> select date_offset('2009-07-29', -1 ,'MONTH' ) FROM src LIMIT 1
> -> 2009-06-29
> Usage :-
> Case  : To calculate the expiry date of a item from manufacturing date 
> Table :- ITEM_TAB
>  Manufacturing_date  |item id|store id|value|unit|price
>   2012-12-01|110001|00003|0.99|1.00|0.99
>   2012-12-02|110001|00008|0.99|0.00|0.00
>   2012-12-03|110001|00009|0.99|0.00|0.00
>   2012-12-04|110001|001112002|0.99|0.00|0.00
>   2012-12-05|110001|001112003|0.99|0.00|0.00
>   2012-12-06|110001|001112006|0.99|1.00|0.99
>   2012-12-07|110001|001112007|0.99|0.00|0.00
>   2012-12-08|110001|001112008|0.99|0.00|0.00
>   2012-12-09|110001|001112009|0.99|0.00|0.00
>   2012-12-10|110001|001112010|0.99|0.00|0.00
>   2012-12-11|110001|001113003|0.99|0.00|0.00
>   2012-12-12|110001|001113006|0.99|0.00|0.00
>   2012-12-13|110001|001113008|0.99|0.00|0.00
>   2012-12-14|110001|001113010|0.99|0.00|0.00
>   2012-12-15|110001|001114002|0.99|0.00|0.00
>   2012-12-16|110001|001114004|0.99|1.00|0.99
>   2012-12-17|110001|001114005|0.99|0.00|0.00
>   2012-12-18|110001|001121004|0.99|0.00|0.00 
> QUERY:
> select man_date , date_offset(man_date ,5 ,'year') as expiry_date from 
> item_tab;
> RESULT:
> 2012-12-01  2017-12-01
> 2012-12-02  2017-12-02
> 2012-12-03  2017-12-03
> 2012-12-04  2017-12-04
> 2012-12-05  2017-12-05
> 2012-12-06  2017-12-06
> 2012-12-07  2017-12-07
> 2012-12-08  2017-12-08
> 2012-12-09  2017-12-09
> 2012-12-10  2017-12-10
> 2012-12-11  2017-12-11
> 2012-12-12  2017-12-12
> 2012-12-13  2017-12-13
> 2012-12-14  2017-12-14
> 2012-12-15  2017-12-15
> 2012-12-16  2017-12-16
> 2012-12-17  2017-12-17
> 2012-12-18  2017-12-18

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1955) Support non-constant expressions for array indexes.

[Navis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530678#comment-13530678
 ] 

Navis commented on HIVE-1955:
-

Is there any reason index value should be constant?

> Support non-constant expressions for array indexes.
> ---
>
> Key: HIVE-1955
> URL: https://issues.apache.org/jira/browse/HIVE-1955
> Project: Hive
>  Issue Type: Improvement
>Reporter: Adam Kramer
>
> FAILED: Error in semantic analysis: line 4:8 Non Constant Expressions for 
> Array Indexes not Supported dut
> ...just wrote my own UDF to do this, and it is trivial. We should support 
> this natively.
> Let foo have these rows:
> arr   i
> [1,2,3]   1
> [3,4,5]   2
> [5,4,3]   2
> [0,0,1]   0
> Then,
> SELECT arr[i] FROM foo
> should return:
> 2
> 5
> 3
> 1
> Similarly, for the same table,
> SELECT 3 IN arr FROM foo
> should return:
> true
> true
> true
> false
> ...these use cases are needless limitations of functionality. We shouldn't 
> need UDFs to accomplish these goals.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3383) MinimrCliDriver test hangs when run on 20S (aka 1.0) hadoop profile

[Mark Grover (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530682#comment-13530682
 ] 

Mark Grover commented on HIVE-3383:
---

I ran into the same problem today with simple {{ant test}} on trunk.

> MinimrCliDriver test hangs when run on 20S (aka 1.0) hadoop profile
> ---
>
> Key: HIVE-3383
> URL: https://issues.apache.org/jira/browse/HIVE-3383
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Reporter: Ashutosh Chauhan
>
> I ran 
> {code} 
> ant clean package test -Dtestcase=TestMinimrCliDriver -Dhadoop.mr.rev=20S
> {code}
> This hangs after printing following on console:
> {code}
> test:
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is 
> deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the 
> log4j.properties files.
> [junit] Starting DataNode 0 with dfs.data.dir: 
> build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
> [junit] Starting DataNode 1 with dfs.data.dir: 
> build/test/data/dfs/data/data3,build/test/data/dfs/data/data4
> [junit] Starting DataNode 2 with dfs.data.dir: 
> build/test/data/dfs/data/data5,build/test/data/dfs/data/data6
> [junit] Starting DataNode 3 with dfs.data.dir: 
> build/test/data/dfs/data/data7,build/test/data/dfs/data/data8
> [junit] Generating rack names for tasktrackers
> [junit] Generating host names for tasktrackers
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3784) de-emphasize mapjoin hint

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3784:
-

Attachment: hive.3784.3.patch

> de-emphasize mapjoin hint
> -
>
> Key: HIVE-3784
> URL: https://issues.apache.org/jira/browse/HIVE-3784
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3784.1.patch, hive.3784.2.patch, hive.3784.3.patch
>
>
> hive.auto.convert.join has been around for a long time, and is pretty stable.
> When mapjoin hint was created, the above parameter did not exist.
> The only reason for the user to specify a mapjoin currently is if they want
> it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin.
> Eventually, that should also go away, but that may take some time to 
> stabilize.
> There are many rules in SemanticAnalyzer to handle the following trees:
> ReduceSink -> MapJoin
> Union  -> MapJoin
> MapJoin-> MapJoin
> This should not be supported anymore. In any of the above scenarios, the
> user can get the mapjoin behavior by setting hive.auto.convert.join to true
> and not specifying the hint. This will simplify the code a lot.
> What does everyone think ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2991) Integrate Clover with Hive

[Ilya Katsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Katsov updated HIVE-2991:
--

Attachment: hive-trunk-clover-html-report.zip

Latest report for trunk is attached (class-level info was removed to reduce the 
archive size).

> Integrate Clover with Hive
> --
>
> Key: HIVE-2991
> URL: https://issues.apache.org/jira/browse/HIVE-2991
> Project: Hive
>  Issue Type: Test
>  Components: Testing Infrastructure
>Affects Versions: 0.9.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2991.D2985.1.patch, 
> hive.2991.1.branch-0.10.patch, hive.2991.1.branch-0.9.patch, 
> hive.2991.1.trunk.patch, hive-trunk-clover-html-report.zip
>
>
> Atlassian has donated license of their code coverage tool Clover to ASF. Lets 
> make use of it to generate code coverage report to figure out which areas of 
> Hive are well tested and which ones are not. More information about license 
> can be found in Hadoop jira HADOOP-1718 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3793) Print number of fetched rows after query in CliDriver

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530747#comment-13530747
 ] 

Hudson commented on HIVE-3793:
--

Integrated in Hive-trunk-h0.21 #1852 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1852/])
HIVE-3793 Print number of fetched rows after query in CliDriver
(Navis via namit) (Revision 1420817)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1420817
Files : 
* /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java


> Print number of fetched rows after query in CliDriver
> -
>
> Key: HIVE-3793
> URL: https://issues.apache.org/jira/browse/HIVE-3793
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3793.D7305.1.patch
>
>
> Currently shows time taken only. But it would be useful showing number of 
> rows fetched also. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1852 - Still Failing

[Apache Jenkins Server]

Changes for Build #1844
[hashutosh] HIVE-3705 : Adding authorization capability to the metastore 
(Sushanth Sowmyan via Ashutosh Chauhan)


Changes for Build #1845
[hashutosh] HIVE-3231 : msck repair should find partitions already containing 
data files (Keegan Mosley via Ashutosh Chauhan)

[hashutosh] HIVE-2691 : Specify location of log4j configuration files via 
configuration properties (Zhenxiao Luo via Ashutosh Chauhan)

[hashutosh] HIVE-2794 : Aggregations without grouping should return NULL when 
applied to partitioning column of a partitionless table (Zhenxiao Luo via 
Ashutosh Chauhan)

[hashutosh] HIVE-3780 : RetryingMetaStoreClient Should Log the Caught Exception 
(Bhushan Mandhani via Ashutosh Chauhan)

[hashutosh] HIVE-3084 : Hive CI failing due to script_broken_pipe1.q (Gunther 
Hagleitner via Ashutosh Chauhan)

[hashutosh] HIVE-3760 : TestNegativeMinimrCliDriver_mapreduce_stack_trace.q 
fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan)


Changes for Build #1846

Changes for Build #1847
[hashutosh] HIVE-3714 : Patch: Hive's ivy internal resolvers need to use 
sourceforge for sqlline (Gopal V via Ashutosh Chauhan)


Changes for Build #1848
[hashutosh] HIVE-3782 : testCliDriver_sample_islocalmode_hook fails on hadoop-1 
(Gunther Hagleitner via Ashutosh Chauhan)

[hashutosh] HIVE-2288 : Adding the oracle nvl function to the UDF (Ed Capriolo, 
Guy Doulberg via Ashutosh Chauhan)

[hashutosh] HIVE-2689 : ObjectInspectorConverters cannot convert Void types to 
Array/Map/Struct types. (Jonathan Chang via Ashutosh Chauhan)


Changes for Build #1849
[hashutosh] HIVE-3622 : reflect udf cannot find method which has arguments of 
primitive types and String, Binary, Timestamp types mixed (Navis via Ashutosh 
Chauhan)

[namit] HIVE-3401 Diversify grammar for split sampling
(Navis via namit)


Changes for Build #1850

Changes for Build #1851

Changes for Build #1852
[namit] HIVE-3793 Print number of fetched rows after query in CliDriver
(Navis via namit)




2 tests failed.
FAILED:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1

Error Message:
Forked Java VM exited abnormally. Please note the time in the report does not 
reflect the time until the VM exit.

Stack Trace:
junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please 
note the time in the report does not reflect the time until the VM exit.
at 
net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259)
at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268)
at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324)
at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244)


FAILED:  org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats19

Error Message:
Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... 
-Dtest.silent=false" to get more logs.

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get 
more logs.
at junit.framework.Assert.fail(Assert.java:47)
at 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats19(TestCliDriver.java:41417)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1852)

Status: Still Failing

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1852/ to 
view the results.