[
https://issues.apache.org/jira/browse/HIVE-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969653#comment-16969653
]
Owen O'Malley commented on HIVE-22405:
--------------------------------------
It looks like the DateColumnVector is interpreting the longs as millis, while I
believe the correct interpretation is the days since 01-01-1970.
You can consider passing the number of values to update, because the number of
values set may be significantly smaller than the length. On the other hand,
this will be fine in the vast majority of cases.
I'd suggest that you calculate a upper bound on the date of divergence between
the calendars (eg. 1 Nov 1582) as a static and use that as a first pass where
no conversion is necessary. It would be great to have the vast majority of our
users not be slowed down by this code.
You also need to add tests that convert from proleptic calendar to
non-proleptic.
> Add ColumnVector support for ProlepticCalendar
> ----------------------------------------------
>
> Key: HIVE-22405
> URL: https://issues.apache.org/jira/browse/HIVE-22405
> Project: Hive
> Issue Type: Bug
> Components: storage-api
> Reporter: Owen O'Malley
> Assignee: László Bodor
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22405.01.patch, HIVE-22405.02.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
> * Change the calendar to or from proleptic. If the new and old values of
> the flag are the
> * same, nothing is done.
> * useProleptic - set the flag for the proleptic calendar
> * updateData - change the data to match the new value of the flag.
> */
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
> * Detect whether this data is using the proleptic calendar.
> */
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
