[
https://issues.apache.org/jira/browse/HIVE-19726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495566#comment-16495566
]
Prasanth Jayachandran commented on HIVE-19726:
----------------------------------------------
Actually this patch might alone be not sufficient.. The same tests fail when
running on a different time zone (same test failed with
-Duser.timezone="Europe/Paris"). The reason is ORC PPD evaluation performs
Date.compareTo(Date) which compares millis offset. Date.compareTo(Date)
returned 0 in Los Angeles timezone whereas Date.compareTo(Date) return -1 in
Paris timezone for the same test case.
The fix for this would be in orc side where PPD evaluation has to happen on
DateWritables (instead of Date), as DateWritables compare days offset.
DateWritable.dateToDays is used for bloomfilter and date column vectors are
populated using DateWritable.getDays(). I will create ORC issue for the same.
> ORC date PPD is broken
> ----------------------
>
> Key: HIVE-19726
> URL: https://issues.apache.org/jira/browse/HIVE-19726
> Project: Hive
> Issue Type: Bug
> Affects Versions: 2.4.0, 3.1.0, 3.0.1, 4.0.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Major
> Attachments: HIVE-19726.1.patch
>
>
> When kryo was in version 2.22 we added a fix in HIVE-7222 and later in
> HIVE-10819. Now that we have updated kryo to 3.0.3 that old workaround fix
> was never removed. The issue was that kryo serialized Timestamp to Date type.
> So to recover the timestamp, during deserialization we deserialized *any*
> date instance to Timestamp object which is wrong (we don't know if date was
> serialized as date or timestamp serialized as date in first place). This
> breaks PPD on date time as kryo deserialization always converts Date to
> Timestamp breaking PPD because of type mismatch.
> Now that we have newer kryo version we can remove the code added in
> HIVE-10819.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
