Re: RCFile - some queries

2011-03-18 Thread Krishna Kumar
Hi yongqiang he, Have created a bug https://issues.apache.org/jira/browse/HIVE-2065 to carry on the discussion. Have attached the picture there too: https://issues.apache.org/jira/secure/attachment/12474055/Slide1.png. (looks like attachments are stripped from posts here?) Please comme

Re: RCFile - some queries

2011-03-18 Thread yongqiang he
>> but the recordLength is not the actual on-disk length of the record. It is acutal on-disk length. It is compressed key length plus the compressed value length >>Similarly, the next field - key length - is not the on-disk length of the >>compressed key. There are two keyLengths, one is compres

RCFile - some queries

2011-03-18 Thread Krishna Kumar
Hello, I was looking into the RCFile format, esp when used with compression; a picture of the file layout as I understand it in this case is attached. Some queries/potential issues: 1. RCFile makes a claim of being sequence file compatible; but the recordLength is not the actual on-d