Hi Raj,
UrlEncode It's a good way to encode data and be sure that you will encode
all special chars (as example \n will be encoded to %0A) It's not necessary
that the field be and url to encode these (you could use other encoder but,
we had very good results with UrlEncoder, ever the best way recover the
information correctly it's encode the data on create time).
If you use UrlEnode on an arbitrary text, In query time you could use
SELECT reflect("java.net.URLDecoder", "decode","to be decoded value)
(source: http://grokbase.com/t/hive/user/119f4qdsmh/urldecode-hive-column)
to read the field as it was in original time.
Example:
On save time URLEncoder.encode("Hello\nWorld","UTF-8")
On query time SELECT reflect("java.net.URLDecoder", "decode","my_field")
where reflect("java.net.URLDecoder", "decode","my_field") rlike "\\n"
Thanks,
Gabriel.
2013/9/20 Raj Hadoop <hadoop...@yahoo.com>
> Hi Gabo,
> Are you suggesting to use java.net.URLEncoder ? Can you be more specific ?
> I have lot of fields in the file which are not only URL related but some
> text fields which has new line characters.
>
> Thanks,
> Raj
>
> ------------------------------
> *From:* Gabriel Eisbruch <gabrieleisbr...@gmail.com>
>
> *To:* "user@hive.apache.org" <user@hive.apache.org>; Raj Hadoop <
> hadoop...@yahoo.com>
> *Sent:* Friday, September 20, 2013 4:43 PM
>
> *Subject:* Re: How to load /t /n file to Hive
>
> Hi
> One way that we used to solve that problem it's to transform the data
> when you are creating/loading it, for example we've applied UrlEncode to
> each field on create time.
>
> Thanks,
> Gabo.
>
>
> 2013/9/20 Raj Hadoop <hadoop...@yahoo.com>
>
> Hi Nitin,
>
> Thanks for the reply. I have a huge file in unix.
>
> As per the file definition, the file is a tab separated file of fields.
> But I am sure that within some field's I have some new line character.
>
> How should I find a record? It is a huge file. Is there some command?
>
> Thanks,
>
>
> *From:* Nitin Pawar <nitinpawar...@gmail.com>
> *To:* "user@hive.apache.org" <user@hive.apache.org>; Raj Hadoop <
> hadoop...@yahoo.com>
> *Sent:* Friday, September 20, 2013 3:15 PM
> *Subject:* Re: How to load /t /n file to Hive
>
> If your data contains new line chars, its better you write a custom map
> reduce job and convert the data into a single line removing all unwanted
> chars in column separator as well just having single new line char per line
>
>
> On Sat, Sep 21, 2013 at 12:38 AM, Raj Hadoop <hadoop...@yahoo.com> wrote:
>
> Please note that there is an escape chacter in the fields where the /t
> and /n are present.
>
> *From:* Raj Hadoop <hadoop...@yahoo.com>
> *To:* Hive <user@hive.apache.org>
> *Sent:* Friday, September 20, 2013 3:04 PM
> *Subject:* How to load /t /n file to Hive
>
> Hi,
>
> I have a file which is delimted by a tab. Also, there are some fields in
> the file which has a tab /t character and a new line /n character in some
> fields.
>
> Is there any way to load this file using Hive load command? Or do i have
> to use a Custom Map Reduce (custom) Input format with java ? Please advise.
>
> Thanks,
> Raj
>
>
>
>
>
> --
> Nitin Pawar
>
>
>
>
>
>
