[
https://issues.apache.org/jira/browse/HIVE-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407374#comment-13407374
]
Thejas M Nair commented on HIVE-3168:
-------------------------------------
Yes, I tried the patch and it worked for me with hcatalog. I tested it using MR
job that uses hcat to read data.
You might want to make sure that the you have deployed the updated hive jars
and that hcatalog is using those.
hive build has some issues now, and I had to do "rm -rf
~/.ivy2/local/org.apache.hive/hive-* ; rm -rf
~/.ivy2/cache/org.apache.hive/hive-*" to clear the old jars. I am not sure if
this affects only unit tests, or if it affects the package creation as well.
But you might want to give that a try.
Hive(ie hcat as well) expects the binary column be base64 encoded (at least for
text format). The values you pasted don't have the padding with '==', and my
understanding is that base64 is expected to be padded to 4 bytes. Can you try
with valid base64 encoding of printable ascii strings ?
> LazyBinaryObjectInspector.getPrimitiveJavaObject copies beyond length of
> underlying BytesWritable
> -------------------------------------------------------------------------------------------------
>
> Key: HIVE-3168
> URL: https://issues.apache.org/jira/browse/HIVE-3168
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.9.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-3168.1.patch, HIVE-3168.2.patch
>
>
> LazyBinaryObjectInspector.getPrimitiveJavaObject copies the full capacity of
> the LazyBinary's underlying BytesWritable object, which can be greater than
> the size of the actual contents.
> This leads to additional characters at the end of the ByteArrayRef returned.
> When the LazyBinary object gets re-used, there can be remnants of the later
> portion of previous entry.
> This was not seen while reading through hive queries, which I think is
> because a copy elsewhere seems to create LazyBinary with length == capacity.
> (probably LazyBinary copy constructor). This was seen when MR or pig used
> Hcatalog to read the data.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
