Maybe i misread your original post. Didn't you say you were parsing the
hive client output?
You don't have to change the way you're writing the data - you only have to
change the output hive emits.
so for example when producing hive output i presume you do something like
this currently:
hive> select col1, col2 from table; -- where col1 or col2 have
embedded tabs and newlines?
so do this instead:
hive> select transform(col1, col2) using 'script' as col1, col2 from
table;
where you write 'script' and do your encoding as suggested in the previous
post.
NB. pay heed to the first warning on this page:
https://cwiki.apache.org/Hive/languagemanual-transform.html
On Mon, May 6, 2013 at 2:41 AM, Valluri, Sathish <sathish.vall...@emc.com>wrote:
> This is the idea which I have thought, But in our scenario we have less
> control on writing avro data with delimited TABS and NEWLINES.(encoding
> tabs and newlines with other characters).****
>
> Since avro data can be pumped on to the Warehouse system from many sources
> and if we have to implement this kind of logic we have handle this TABS and
> NEWLINES encoding on every data writing part.****
>
> Interested if this can be handled without delimiting avro data, like
> reading the AVRO data and transforming into other encoding format and
> sending to the cli output in this format.****
>
> And our app will decode the data and display.****
>
> ** **
>
> Regards****
>
> Sathish Valluri****
>
> ** **
>
> *From:* Sanjay Subramanian [mailto:sanjay.subraman...@wizecommerce.com]
> *Sent:* Saturday, May 04, 2013 12:08 AM
> *To:* user@hive.apache.org
>
> *Subject:* Re: hive cli escaping TAB and NEW LINE Characters.****
>
> ** **
>
> +1 to Stephens suggestion…****
>
> ** **
>
> *From: *Stephen Sprague <sprag...@gmail.com>
> *Reply-To: *"user@hive.apache.org" <user@hive.apache.org>
> *Date: *Friday, May 3, 2013 11:29 AM
> *To: *"user@hive.apache.org" <user@hive.apache.org>
> *Subject: *Re: hive cli escaping TAB and NEW LINE Characters.****
>
> ** **
>
> hate to sound like a broken record but when all else fails think about the
> transform() function. The notion here is of encoding your tabs and newlines
> to something like '\t' and '\n' (literally) for instance. If those aren't
> unique enough use '<<tab>>' and "<<newline>>' (you get the idea) then
> having your app decode those strings to real tabs and real newlines when
> reading it.****
>
> What do you think?****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> On Fri, May 3, 2013 at 2:07 AM, Valluri, Sathish <sathish.vall...@emc.com>
> wrote:****
>
> Hi All,****
>
> ****
>
> We have an application which parses hive cli output and displays results.*
> ***
>
> I have an external table with data in avro format, the contents in this
> avro file have TAB and NEW LINES in the Avro data part.****
>
> Since hive cli output rows are delimited by NEWLINES and columns are
> delimited by TABS, if the actual content have TABS and NEW LINE characters
> parsing the result set is giving wrong results.****
>
> Can anyone suggest some ideas regarding delimiting the TABS and NEW LINE
> characters in the hive cli output if the actual contents of the columns
> have TABS and NEW LINES.****
>
> ****
>
> Regards****
>
> Sathish Valluri****
>
> ** **
>
> ** **
>
> CONFIDENTIALITY NOTICE
> ======================
> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.*
> ***
>
