Some queries re locking

2011-02-16 Thread Krishna Kumar
Hello, While looking into some of the tangential issues encountered while doing the export/import related work, I have some questions: 1. Should "CREATE TABLE" lock (shared) the database? I think so from the discussions, but I do not think it happens now. 2. Similarly "LOAD" should a

RCFile - some queries

2011-03-18 Thread Krishna Kumar
Hello, I was looking into the RCFile format, esp when used with compression; a picture of the file layout as I understand it in this case is attached. Some queries/potential issues: 1. RCFile makes a claim of being sequence file compatible; but the recordLength is not the actual on-d

Re: RCFile - some queries

2011-03-18 Thread Krishna Kumar
emselves. It is initially designed for Hive. Thread safety is there at first time, and then removed because Hive does not need that, and 'synchronized' may need extra overhead >>3.1 Reader.nextBlock() is later added for file merge. So the normal reader should not use this method.

Re Stats Publishing /Aggregation

2011-06-20 Thread Krishna Kumar
Any reason why persistent stores such as jdbc and hbase are supported for temporary stats storage IIUC, but hadoop counters were not used for the tasks to 'publish' their stats for the aggregation task to pick it up from? Cheers, Krishna

[jira] Created: (HIVE-1918) Add export/import facilities to the hive system

2011-01-17 Thread Krishna Kumar (JIRA)
Reporter: Krishna Kumar This is an enhancement request to add export/import features to hive. With this language extension, the user can export the data of the table - which may be located in different hdfs locations in case of a partitioned table - as well as the metadata of the

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-01-18 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Status: Patch Available (was: Open) Patch for adding export/import. > Add export/imp

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-01-18 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: HIVE-1918.patch.txt > Add export/import facilities to the hive sys

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-01-18 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983532#action_12983532 ] Krishna Kumar commented on HIVE-1918: - Design notes: - Export/Import modele

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-01-20 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12984128#action_12984128 ] Krishna Kumar commented on HIVE-1918: - Ok. Will take of this via a delegating ctor

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-01-20 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12984563#action_12984563 ] Krishna Kumar commented on HIVE-1918: - Why export/import needs this change: It is

[jira] Created: (HIVE-1924) checkformat implementations leak handles

2011-01-23 Thread Krishna Kumar (JIRA)
: Krishna Kumar In validateInput, Reader constructors of SequenceFile and RCFile throw exceptions to indicate that the format is incorrect, but the close is not called in a finally block. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the

[jira] Updated: (HIVE-1924) checkformat implementations leak handles

2011-01-23 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1924: Status: Patch Available (was: Open) Not sure how to test this automatically, since the readers are

[jira] Updated: (HIVE-1924) checkformat implementations leak handles

2011-01-23 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1924: Attachment: HIVE.1924.patch.txt Tested only manually with hive.fileformat.check set to true

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-01-24 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985556#action_12985556 ] Krishna Kumar commented on HIVE-1918: - @Edward: Both the existing data m

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-01-24 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985557#action_12985557 ] Krishna Kumar commented on HIVE-1918: - @Carl: 1. Taken care in the new patch.

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-01-24 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: HIVE-1918.patch.1.txt A quick summary of the second derivative (difference between

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-01-24 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: hive-metastore-er.pdf Prettified ER diagram of the existing data model > Add exp

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-01-24 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985565#action_12985565 ] Krishna Kumar commented on HIVE-1918: - @Namit: 1. Do you have any ideas re how we

[jira] Updated: (HIVE-1924) checkformat implementations leak handles

2011-01-24 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1924: Status: Open (was: Patch Available) Hmm, The patch is not correct. Setting hive.checkformat=false

[jira] Resolved: (HIVE-1924) checkformat implementations leak handles

2011-01-24 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar resolved HIVE-1924. - Resolution: Duplicate Problem referenced in HADOOP-5476 and HIVE-1185 > checkfor

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-01-31 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: HIVE-1918.patch.2.txt Patch including - no changes to ql.metadata.Partition as per

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-01-31 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988693#comment-12988693 ] Krishna Kumar commented on HIVE-1918: - @Namit: Attached patch now uses relative p

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-02-02 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: HIVE-1918.patch.3.txt Patch with all open issues addressed > Add export/imp

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-02-02 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Status: Patch Available (was: Open) > Add export/import facilities to the hive sys

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-02-02 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989747#comment-12989747 ] Krishna Kumar commented on HIVE-1918: - With this patch, I think all above issues

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-02-08 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992036#comment-12992036 ] Krishna Kumar commented on HIVE-1918: - Thanks, Namit, for the comments. 1. O

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-02-09 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992373#comment-12992373 ] Krishna Kumar commented on HIVE-1918: - Importing into existing tables is now suppo

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-02-09 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992606#comment-12992606 ] Krishna Kumar commented on HIVE-1918: - Hmm. LoadSemanticAnalyzer (which knows

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-02-11 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Status: Patch Available (was: Open) Please review. Will try and see if I can update the

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-02-11 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: HIVE-1918.patch.4.txt Patch with - metadata ser/deser methods moved from HiveUtils to

[jira] [Commented] (HIVE-2417) Merging of compressed rcfiles fails to write the valuebuffer part correctly

2011-08-30 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094294#comment-13094294 ] Krishna Kumar commented on HIVE-2417: - Yes, the test is designed to produce the e

[jira] [Updated] (HIVE-2417) Merging of compressed rcfiles fails to write the valuebuffer part correctly

2011-08-30 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2417: Attachment: HIVE-2417.v1.patch Test changed after review comments - default codec instead of bzip2

[jira] [Updated] (HIVE-2413) BlockMergeTask ignores client-specified jars

2011-09-01 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2413: Attachment: HIVE-2413.v1.patch Empty string not handled correctly in JC so handling it here

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-02-17 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995767#comment-12995767 ] Krishna Kumar commented on HIVE-1918: - https://reviews.apache.org/r/430/ added (

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-02-21 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997296#comment-12997296 ] Krishna Kumar commented on HIVE-1918: - There are a few reasons why I took

[jira] Created: (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-02-23 Thread Krishna Kumar (JIRA)
URL: https://issues.apache.org/jira/browse/HIVE-2003 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Krishna Kumar Assignee: Krishna Kumar Priority: Minor The table/partition being loaded is not being added to

[jira] Updated: (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-02-23 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2003: Attachment: HIVE-2003.patch.txt Patch attached. 1. LoadSemanticAnalyzer adds the table/partition

[jira] Commented: (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-02-23 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998276#comment-12998276 ] Krishna Kumar commented on HIVE-2003: - 6. Loading a partitioned table wit

[jira] Updated: (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-02-23 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2003: Attachment: (was: HIVE-2003.patch.txt) > LOAD compilation does not set the outputs dur

[jira] Updated: (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-02-23 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2003: Attachment: HIVE-2003.patch.txt One results file was diffed as binary so patch regenerated with

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

2011-02-23 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998442#comment-12998442 ] Krishna Kumar commented on HIVE-1918: - Thanks Paul. [Your comments are

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-02-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: HIVE-1918.patch.5.txt - Nested ternaries expanded - thrift-based serialization for

[jira] Updated: (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-02-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2003: Status: Patch Available (was: Open) > LOAD compilation does not set the outputs during seman

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-03-14 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: HIVE-1918.patch.5.txt Merged with trunk > Add export/import facilities to the h

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

2011-03-14 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-1918: Attachment: (was: HIVE-1918.patch.5.txt) > Add export/import facilities to the hive sys

[jira] Created: (HIVE-2065) RCFile issues

2011-03-18 Thread Krishna Kumar (JIRA)
RCFile issues - Key: HIVE-2065 URL: https://issues.apache.org/jira/browse/HIVE-2065 Project: Hive Issue Type: Bug Reporter: Krishna Kumar Priority: Minor Some potential issues with RCFile 1. Remove

[jira] Updated: (HIVE-2065) RCFile issues

2011-03-18 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2065: Attachment: Slide1.png Compressed RCFile Layout > RCFile iss

[jira] Assigned: (HIVE-2065) RCFile issues

2011-03-18 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar reassigned HIVE-2065: --- Assignee: Krishna Kumar > RCFile issues > - > > Ke

[jira] Commented: (HIVE-2065) RCFile issues

2011-03-19 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008741#comment-13008741 ] Krishna Kumar commented on HIVE-2065: - So should I go ahead and fix #2 and #3 as

[jira] Updated: (HIVE-2065) RCFile issues

2011-03-19 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2065: Attachment: proposal.png Notation : Length bracket inside the dashed box means it is the

[jira] [Updated] (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-03-21 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2003: Attachment: HIVE-2003.patch.1.txt Regenerated the patch. > LOAD compilation does not set

[jira] [Commented] (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-03-21 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009202#comment-13009202 ] Krishna Kumar commented on HIVE-2003: - Added to review board: h

[jira] [Commented] (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-03-21 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009203#comment-13009203 ] Krishna Kumar commented on HIVE-2003: - For #4 above, I have taken now the

[jira] [Updated] (HIVE-2003) LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it.

2011-03-22 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2003: Status: Patch Available (was: Open) Please review asap as there are lots of changes to q.out files

[jira] [Commented] (HIVE-2065) RCFile issues

2011-03-22 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009981#comment-13009981 ] Krishna Kumar commented on HIVE-2065: - Hmm. #3 is taking me a bit too far th

[jira] [Updated] (HIVE-2065) RCFile issues

2011-03-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2065: Attachment: proposal.png The layout as implemented in the attached patch > RCFile iss

[jira] [Updated] (HIVE-2065) RCFile issues

2011-03-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2065: Attachment: (was: proposal.png) > RCFile issues > - > >

[jira] [Updated] (HIVE-2065) RCFile issues

2011-03-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2065: Attachment: HIVE.2065.patch.0.txt Patch. Since it contains binary files, it was generated with

[jira] [Commented] (HIVE-2065) RCFile issues

2011-03-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012125#comment-13012125 ] Krishna Kumar commented on HIVE-2065: - Review Board Entry at h

[jira] [Commented] (HIVE-2071) enforcereadonlytables hook should not check a configuration variable

2011-03-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012293#comment-13012293 ] Krishna Kumar commented on HIVE-2071: - Namit, While we can read the config vari

[jira] [Updated] (HIVE-2065) RCFile issues

2011-03-28 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2065: Status: Patch Available (was: Open) > RCFile issues > - > >

[jira] [Commented] (HIVE-2065) RCFile issues

2011-03-31 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014065#comment-13014065 ] Krishna Kumar commented on HIVE-2065: - The RCFile layout seems to have been desi

[jira] [Commented] (HIVE-2065) RCFile issues

2011-03-31 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014351#comment-13014351 ] Krishna Kumar commented on HIVE-2065: - As I indicated, the reason I made changes

[jira] [Updated] (HIVE-2065) RCFile issues

2011-04-06 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2065: Attachment: HIVE.2065.patch.1.txt Updated patch where sequence file compliance is not addressed but

[jira] [Commented] (HIVE-2065) RCFile issues

2011-04-06 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016444#comment-13016444 ] Krishna Kumar commented on HIVE-2065: - Updated patch where sequence file complianc

[jira] [Created] (HIVE-2097) Explore mechanisms for better compression with RC Files

2011-04-06 Thread Krishna Kumar (JIRA)
: Query Processor, Serializers/Deserializers Reporter: Krishna Kumar Priority: Minor Optimization of the compression mechanisms used by RC File to be explored. Some initial ideas 1. More efficient serialization/deserialization based on type-specific and storage-specific

[jira] [Assigned] (HIVE-2097) Explore mechanisms for better compression with RC Files

2011-04-06 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar reassigned HIVE-2097: --- Assignee: Krishna Kumar > Explore mechanisms for better compression with RC Fi

[jira] [Commented] (HIVE-2097) Explore mechanisms for better compression with RC Files

2011-04-06 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016451#comment-13016451 ] Krishna Kumar commented on HIVE-2097: - Comment hijacked from HIVE-2065: He Yongq

[jira] [Updated] (HIVE-2071) enforcereadonlytables hook should not check a configuration variable

2011-04-06 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2071: Attachment: HIVE.2071.patch.0.txt Tentative patch to show the implementation for the sessionstate

[jira] [Created] (HIVE-2098) Make couple of convenience methods in EximUtil public

2011-04-07 Thread Krishna Kumar (JIRA)
Make couple of convenience methods in EximUtil public - Key: HIVE-2098 URL: https://issues.apache.org/jira/browse/HIVE-2098 Project: Hive Issue Type: Bug Reporter: Krishna

[jira] [Assigned] (HIVE-2098) Make couple of convenience methods in EximUtil public

2011-04-07 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar reassigned HIVE-2098: --- Assignee: Krishna Kumar > Make couple of convenience methods in EximUtil pub

[jira] [Updated] (HIVE-2098) Make couple of convenience methods in EximUtil public

2011-04-07 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2098: Attachment: HIVE.2098.patch.0.txt Couple of methods made public for use outside of the package

[jira] [Updated] (HIVE-2098) Make couple of convenience methods in EximUtil public

2011-04-07 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2098: Status: Patch Available (was: Open) > Make couple of convenience methods in EximUtil pub

[jira] [Commented] (HIVE-2065) RCFile issues

2011-04-07 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017229#comment-13017229 ] Krishna Kumar commented on HIVE-2065: - The minor version is needed so that we

[jira] [Updated] (HIVE-956) Add support of columnar binary serde

2011-05-31 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-956: --- Attachment: HIVE.956.patch.0 Initial patch for lazy binary columnar serde. Reuses elements of columnar

[jira] [Commented] (HIVE-956) Add support of columnar binary serde

2011-05-31 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041502#comment-13041502 ] Krishna Kumar commented on HIVE-956: Initial patch for lazy binary columnar s

[jira] [Commented] (HIVE-956) Add support of columnar binary serde

2011-05-31 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041506#comment-13041506 ] Krishna Kumar commented on HIVE-956: Review board: https://reviews.apache.org/r

[jira] [Updated] (HIVE-956) Add support of columnar binary serde

2011-06-02 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-956: --- Status: Patch Available (was: Open) > Add support of columnar binary se

[jira] [Updated] (HIVE-956) Add support of columnar binary serde

2011-06-02 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-956: --- Attachment: HIVE.956.patch.1 Patch with a change on how null vs empty strings are handled. This patch

[jira] [Commented] (HIVE-956) Add support of columnar binary serde

2011-06-08 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046043#comment-13046043 ] Krishna Kumar commented on HIVE-956: Re-ordering my responses; bq.

[jira] [Updated] (HIVE-956) Add support of columnar binary serde

2011-06-08 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-956: --- Attachment: HIVE.956.patch.2 Re-ordering my responses; bq. can LazyBinaryColumnarSerDe share some

[jira] [Commented] (HIVE-956) Add support of columnar binary serde

2011-06-09 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046362#comment-13046362 ] Krishna Kumar commented on HIVE-956: bq. can warnedOnceNullMapKey be removed? I

[jira] [Created] (HIVE-2209) Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object

2011-06-09 Thread Krishna Kumar (JIRA)
URL: https://issues.apache.org/jira/browse/HIVE-2209 Project: Hive Issue Type: Sub-task Reporter: Krishna Kumar Assignee: Krishna Kumar Priority: Minor Now ObjectInspectorUtils.compare throws an exception if a map is contained

[jira] [Updated] (HIVE-2209) Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object

2011-06-09 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2209: Issue Type: Improvement (was: Sub-task) Parent: (was: HIVE-956) > Provide a way

[jira] [Updated] (HIVE-2209) Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object

2011-06-10 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2209: Attachment: HIVE-2209v0.patch Patch, with tests, added. > Provide a way by wh

[jira] [Updated] (HIVE-2209) Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object

2011-06-10 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2209: Status: Patch Available (was: Open) For review by He Yongqiang > Provide a way by wh

[jira] [Commented] (HIVE-2209) Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object

2011-06-14 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049297#comment-13049297 ] Krishna Kumar commented on HIVE-2209: - He Yongqiang, Can you take a look at this

[jira] [Updated] (HIVE-2209) Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object

2011-06-17 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2209: Attachment: HIVE2209v1.patch Added another implementation with sorting of the keys > Provide a

[jira] [Updated] (HIVE-2209) Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object

2011-06-20 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2209: Attachment: HIVE-2209v2.patch Fixed a lowercase/uppercase typo in the test classes > Provide a

[jira] [Updated] (HIVE-956) Add support of columnar binary serde

2011-06-20 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-956: --- Attachment: HIVE-956v3.patch After separating mapcomparer changes to its own jira/patch - HIVE-2209

[jira] [Updated] (HIVE-956) Add support of columnar binary serde

2011-06-20 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-956: --- Status: Patch Available (was: Open) I have tested the setting of rawDataSize with a "ANALYZE

[jira] [Updated] (HIVE-956) Add support of columnar binary serde

2011-07-19 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-956: --- Attachment: HIVE-956v4.patch Please note that HIVE-2209 is a pre-requisite for this patch. >

[jira] [Commented] (HIVE-956) Add support of columnar binary serde

2011-07-19 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067588#comment-13067588 ] Krishna Kumar commented on HIVE-956: Common code moved to a base class to

[jira] [Created] (HIVE-2384) import of multiple partitions from a partitioned table with external location overwrites files

2011-08-17 Thread Krishna Kumar (JIRA)
Project: Hive Issue Type: Bug Components: Query Processor Reporter: Krishna Kumar when we import multiple partitions from an exported partitioned table, if we import it with a specified external location, then the partitions end up overwriting. -- This

[jira] [Updated] (HIVE-2384) import of multiple partitions from a partitioned table with external location overwrites files

2011-08-17 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2384: Assignee: Krishna Kumar Status: Patch Available (was: Open) > import of multiple partiti

[jira] [Updated] (HIVE-2384) import of multiple partitions from a partitioned table with external location overwrites files

2011-08-17 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2384: Attachment: HIVE-2384.v0.patch Patch with tests > import of multiple partitions from a partitio

[jira] [Created] (HIVE-2413) BlockMergeTask ignores client-specified jars

2011-08-25 Thread Krishna Kumar (JIRA)
Reporter: Krishna Kumar Priority: Minor User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software

[jira] [Updated] (HIVE-2413) BlockMergeTask ignores client-specified jars

2011-08-25 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2413: Attachment: HIVE-2413.v0.patch Here is a simple fix which works for me, though there may be better

[jira] [Created] (HIVE-2417) Merging of compressed rcfiles fails to write the valuebuffer part correctly

2011-08-29 Thread Krishna Kumar (JIRA)
Type: Bug Components: Query Processor Reporter: Krishna Kumar Assignee: Krishna Kumar Attachments: HIVE-2417.v0.patch The blockmerge task does not create proper rc files when merging compressed rc files as the valuebuffer writing is incorrect. -- This

[jira] [Updated] (HIVE-2417) Merging of compressed rcfiles fails to write the valuebuffer part correctly

2011-08-29 Thread Krishna Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2417: Attachment: HIVE-2417.v0.patch Test added > Merging of compressed rcfiles fails to write

  1   2   >