Spark doc refers to this Java 8 page https://docs.oracle.com/javase/8/docs/api/java/time/chrono/IsoChronology.html, which says > > This chronology defines the rules of the ISO calendar system. This > calendar system is based on the ISO-8601 standard, which is the *de facto* > world > calendar. > +1 to make it explicit in the spec.
Regards, Manu On Thu, Sep 12, 2024 at 2:03 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > At the moment, the specification is ambiguous on which calendar is used > for temporal conversion/writing [1]. Reading the java code it appears it is > using Java's OffsetDateTime which conforms to ISO8601 [2]. ISO8601 appears > to explicitly disallow the Julian calendar (but only says proleptic > gregorian can be used by mutual consent [3]). > > Therefore I'd propose: > 1. We make the ISO8601 + proleptic Gregorian + Gregorian calendars > explicit in the specification. > 2. Mention in an implementation note, that data migrated from other > systems or data written by older systems might follow the Julian calendar > (e.g. it looks like Spark only transitioned in 3.0 [4]). > * Does anybody know of metadata available for systems to make this > determination? > * Or a recommendation on how to handle these? > > Thoughts? > > Thanks, > Micah > > [1] This is esoteric but a few systems use 0001-01-01 as a sentinel value > for null so does have some wider applicability > [2] > https://docs.oracle.com/javase/8/docs/api/java/time/OffsetDateTime.html > [3] https://en.wikipedia.org/wiki/ISO_8601#Dates > [4] https://issues.apache.org/jira/browse/SPARK-26651 > >
