[
https://issues.apache.org/jira/browse/SPARK-57737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-57737:
-----------------------------------
Labels: pull-request-available (was: )
> Cache zone offset per task in date_trunc
> ----------------------------------------
>
> Key: SPARK-57737
> URL: https://issues.apache.org/jira/browse/SPARK-57737
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Rito Takeuchi
> Priority: Major
> Labels: pull-request-available
>
> h2. Background
> {{date_trunc}} (expression {{{}TruncTimestamp{}}}) truncates a timestamp in
> the session time zone. For every level above {{SECOND}} it resolves the zone
> offset for the row via {{{}ZoneRules.getOffset(Instant){}}}, a binary search
> over the zone's transition array. For non-fixed-offset zones this runs
> {*}twice per row{*}: once for the input instant and once for the candidate
> truncated instant (the DST-equality guard from SPARK-56663 / SPARK-56769).
> h2. Proposal
> Add a per-task {{ZoneOffsetCache}} that memoizes the resolved offset over the
> half-open epoch-second interval {{[lo, hi)}} on which it is provably
> constant, derived from the surrounding zone transitions
> ({{{}nextTransition{}}}/{{{}previousTransition{}}}). A lookup that falls
> inside the cached interval reduces to two comparisons instead of a binary
> search.
> h2. Benchmark
> {{DateTimeBenchmark}} Truncation, whole-stage codegen on, session zone
> {{{}America/Los_Angeles{}}}, OpenJDK 17 on a 12th Gen Intel i7-1260P, ns/row
> (lower is better):
> ||level||without cache||with cache||speedup||
> |date_trunc YEAR|98.2|56.8|1.73x|
> |date_trunc QUARTER|109.3|71.7|1.52x|
> |date_trunc MONTH|90.8|53.7|1.69x|
> |date_trunc WEEK|77.8|40.6|1.92x|
> |date_trunc DAY|64.8|33.0|1.96x|
> |date_trunc SECOND (control)|28.7|27.7|~1.0x|
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
