Got it working using the columnar serde with the default seperators.
Steve
-Original Message-
From: yongqiang he [mailto:heyongqiang...@gmail.com]
Sent: Friday, March 18, 2011 3:50 PM
To: user@hive.apache.org
Subject: Re: Building Custom RCFiles
what's your table definition?
; Hive doesn't seem to want to take that. I have also tried using the ascii OCT
> codes.
>
> What do I need to setup to make this Map work?
>
> Thanks.
>
> Steve
>
> -Original Message-
> From: yongqiang he [mailto:heyongqiang...@gmail.com]
> Sent: Thu
rsday, March 17, 2011 5:09 PM
To: user@hive.apache.org
Subject: Re: Building Custom RCFiles
Yes. It is the same with normal hive tables.
thanks
yongqiang
On Thu, Mar 17, 2011 at 4:54 PM, Severance, Steve wrote:
> Thanks Yongqiang.
>
> So for more complex types like map do I just
ks.
>
> Steve
>
> -Original Message-
> From: yongqiang he [mailto:heyongqiang...@gmail.com]
> Sent: Thursday, March 17, 2011 4:35 PM
> To: user@hive.apache.org
> Subject: Re: Building Custom RCFiles
>
> A side note, in hive, we make all columns saved as Text interna
ubject: Re: Building Custom RCFiles
A side note, in hive, we make all columns saved as Text internally
(even the column's type is int or double etc). And with some
experiments, string is more friendly to compression. But it needs CPU
to decode to its original type.
Thanks
Yongqiang
On Thu, Ma
A side note, in hive, we make all columns saved as Text internally
(even the column's type is int or double etc). And with some
experiments, string is more friendly to compression. But it needs CPU
to decode to its original type.
Thanks
Yongqiang
On Thu, Mar 17, 2011 at 4:04 PM, yongqiang he wrot
You need to customize Hive's ColumnarSerde (maybe functions in
LazySerde)'s serde and deserialize function (depends you want to read
or write.). And the main thing is that you need to use your own type
def (not LazyInt/LazyLong).
If your type is int or long (not double/float), casting it to string
Hi,
I am working on building a MR job that generates RCFiles that will become
partitions of a hive table. I have most of it working however only strings
(Text) are being deserialized inside of Hive. The hive table is specified to
use a columnarserde which I thought should allow the writable typ
