Re: [Python-Dev] PEP 495 accepted
On 22 September 2015 at 13:33, Nick Coghlan wrote: > On 22 September 2015 at 08:03, Guido van Rossum wrote: >> Just so people know, over at the datetime-sig I've accepted PEP 495, which >> adds a fold flag to datetime objects to distinguish ambiguous times. This >> enables roundripping of conversions for those times where the local clock is >> moved backward (creating ambiguous times that could not be distinguished >> before). > > Hurrah, and congratulations in particular on finding a name for the > flag which is memorable, meaningful and succinct. > >> I would like to thank Alexander and Tim for their unrelenting work on this. >> The idea seems simple, but the details were excruciatingly hard to get >> right, given the strict backwards compatibility requirements. > > I don't think I've seen a collision between mathematical and language > level invariants that complex since the first time I had to figure out > the conflict between container membership invariants and floating > point NaN values (and this one is even more subtle). > > I'm reading through the full PEP now, and really appreciating the > thorough write up. Thanks to Alexander and Tim, and to all the folks > involved in the extensive discussions! It turns out there's one aspect of the accepted proposal that I *think* I understand, but want to confirm: the datetime -> POSIX timestamp -> datetime roundtrip for missing times. If I'm reading the PEP correctly, the defined invariant for local times that actually exist is: dt == datetime.fromtimestamp(dt.timestamp()) No confusion there for the unambiguous times, or for times in a fold. In the latter case, the timestamps produced match the points where the UTC times match the local times in the "In the Fold" UTC/local diagram. The subtle part is the handling of the "timestamp()" method for the "missing" times where the given time doesn't actually correspond to a valid time in the applicable timezone (local time for a naive datetime object). Based on the UTC/local diagram from the "Mind the Gap" section, am I correct in thinking that the modified invariant that also covers times in a gap is: dt == datetime.fromtimestamp(dt.astimezone(utc).astimezone(dt.tzinfo).timestamp()) That is, for local times that exist, the invariant "dt == dt.astimezone(utc).astimezone(dt.tzinfo)" holds, but for times that don't exist, "dt.astimezone(utc).astimezone(dt.tzinfo)" will normalise them to be a time that actually exists in the original time zone, and that normalisation also effectively happens when calling "dt.timestamp()". Regards, Nick. P.S. Thanks to whoever drew the diagrams for the In the Fold/Mind the Gap sections - I found them incredibly helpful in understanding the change! -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 12:01 AM, Nick Coghlan wrote: > It turns out there's one aspect of the accepted proposal that I > *think* I understand, but want to confirm: the datetime -> POSIX > timestamp -> datetime roundtrip for missing times. > > If I'm reading the PEP correctly, the defined invariant for local > times that actually exist is: > > dt == datetime.fromtimestamp(dt.timestamp()) > Yup, except for floating point errors! Those have been fixed, finally: http://bugs.python.org/issue23517 is now closed (the fix didn't make 3.5.0; it will be in 3.5.1 though). > No confusion there for the unambiguous times, or for times in a fold. > In the latter case, the timestamps produced match the points where the > UTC times match the local times in the "In the Fold" UTC/local > diagram. > And this is where the fold flag is essential for the roundtripping. > The subtle part is the handling of the "timestamp()" method for the > "missing" times where the given time doesn't actually correspond to a > valid time in the applicable timezone (local time for a naive datetime > object). > > Based on the UTC/local diagram from the "Mind the Gap" section, am I > correct in thinking that the modified invariant that also covers times > in a gap is: > > dt == > datetime.fromtimestamp(dt.astimezone(utc).astimezone(dt.tzinfo).timestamp()) > > That is, for local times that exist, the invariant "dt == > dt.astimezone(utc).astimezone(dt.tzinfo)" holds, but for times that > don't exist, "dt.astimezone(utc).astimezone(dt.tzinfo)" will normalise > them to be a time that actually exists in the original time zone, and > that normalisation also effectively happens when calling > "dt.timestamp()". > That can't be right -- There is no way any fromtimestamp() call can return a time in the gap. I think about the only useful invariant here is dt.timestamp() == dt.astimezone(utc).timestamp() == dt.astimezone().timestamp() > Regards, > Nick. > > P.S. Thanks to whoever drew the diagrams for the In the Fold/Mind the > Gap sections - I found them incredibly helpful in understanding the > change! > You're welcome. It was a collaboration by myself and Alexander. I drew the first version by hand because I couldn't follow the math without a visual aid. :-) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 3:01 AM, Nick Coghlan wrote: > ... for times that > don't exist, "dt.astimezone(utc).astimezone(dt.tzinfo)" will normalise > them to be a time that actually exists in the original time zone, and > that normalisation also effectively happens when calling > "dt.timestamp()". > Yes. In fact, if you consider the canonical bijection between timestamps and datetimes (t = EPOCH + s * timedelta(0, 1); s = (t - EPOCH) / timedelta(0, 1)), t.astimezone(utc) and t.timestamp() become the same up to some annoying numerical details. The same logic applies to u.astimezone(tzinfo) and datetime.fromtimestamp(s). Note that I deliberately did not mark the units on the sketches: you can think of the UTC axis to be labeled by datetimes or by numeric timestamps. Note that dt != dt.astimezone(utc).astimezone(dt.tzinfo) is one way to detect that dt is in a gap, but I recommend (dt.replace(fold=0).utcoffset() > dt.replace(fold=1).utcoffset().) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 10:43 AM, Guido van Rossum wrote: > Based on the UTC/local diagram from the "Mind the Gap" section, am I >> correct in thinking that the modified invariant that also covers times >> in a gap is: >> >> dt == >> datetime.fromtimestamp(dt.astimezone(utc).astimezone(dt.tzinfo).timestamp()) >> >> That is, for local times that exist, the invariant "dt == >> dt.astimezone(utc).astimezone(dt.tzinfo)" holds, but for times that >> don't exist, "dt.astimezone(utc).astimezone(dt.tzinfo)" will normalise >> them to be a time that actually exists in the original time zone, and >> that normalisation also effectively happens when calling >> "dt.timestamp()". >> > > That can't be right -- There is no way any fromtimestamp() call can return > a time in the gap. > I don't think Nick said that. > I think about the only useful invariant here is > > dt.timestamp() == dt.astimezone(utc).timestamp() == dt.astimezone( other tz>).timestamp() > Yes, this is just another way to say that .astimezone() conversions are now "lossless." ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 10:55 AM, Alexander Belopolsky < [email protected]> wrote: > On Tue, Sep 22, 2015 at 10:43 AM, Guido van Rossum > wrote: > >> Based on the UTC/local diagram from the "Mind the Gap" section, am I >>> correct in thinking that the modified invariant that also covers times >>> in a gap is: >>> >>> dt == >>> datetime.fromtimestamp(dt.astimezone(utc).astimezone(dt.tzinfo).timestamp()) >>> >>> That is, for local times that exist, the invariant "dt == >>> dt.astimezone(utc).astimezone(dt.tzinfo)" holds, but for times that >>> don't exist, "dt.astimezone(utc).astimezone(dt.tzinfo)" will normalise >>> them to be a time that actually exists in the original time zone, and >>> that normalisation also effectively happens when calling >>> "dt.timestamp()". >>> >> >> That can't be right -- There is no way any fromtimestamp() call can >> return a time in the gap. >> > > I don't think Nick said that. > On the second reading, it looks like Nick's second sentence contradicts his first. Guido is right. Moreover, there is no way to get a time in the gap as a result of any conversion including astimezone() and fromutc() in addition to fromtimestamp(). Such datetimes may appear if you construct them explicitly, use .replace() to transplant a datetime to another timezone (or modify other components) and in the result of datetime + timedelta operation. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
[Nick Coghlan] >>> Based on the UTC/local diagram from the "Mind the Gap" section, am I >>> correct in thinking that the modified invariant that also covers times >>> in a gap is: >>> >>> dt == >>> datetime.fromtimestamp(dt.astimezone(utc).astimezone(dt.tzinfo).timestamp()) >>> >>> That is, for local times that exist, the invariant "dt == >>> dt.astimezone(utc).astimezone(dt.tzinfo)" holds, but for times that >>> don't exist, "dt.astimezone(utc).astimezone(dt.tzinfo)" will normalise >>> them to be a time that actually exists in the original time zone, and >>> that normalisation also effectively happens when calling >>> "dt.timestamp()". [Guido] >> That can't be right -- There is no way any fromtimestamp() call can return >> a time in the gap. [Alexander Belopolsky] > I don't think Nick said that. I do, except that he didn't ;-) Count the parens carefully. The top-level operation on the RHS is datetime.fromtimestamp(). However, it didn't pass a tzinfo, so it creates a naive datetime. Assuming dt was aware to begin with, the attempt to compare will always (gap or not) raise an exception. If it had passed dt.tzinfo, then Guido is right. >> I think about the only useful invariant here is >> >> dt.timestamp() == dt.astimezone(utc).timestamp() == >> dt.astimezone().timestamp() > Yes, this is just another way to say that .astimezone() conversions are now Nick, to be very clear, there are two scenarios here after PEP 495 is implemented in Python: 1. You're using a pre-495 tzinfo. Then nothing changes from what happens today. 2. You're using a new 495-conforming tzinfo. Then the discussion starts to apply. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
[Tim] > ... > The > top-level operation on the RHS is datetime.fromtimestamp(). However, > it didn't pass a tzinfo, so it creates a naive datetime. Assuming dt > was aware to begin with, the attempt to compare will always (gap or > not) raise an exception. Oops! In current Python, comparing naive and aware via `==` just returns False. That's even more confusing ;-) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 11:11 AM, Tim Peters wrote: > [Nick Coghlan] > ... > >>> dt == > >>> > datetime.fromtimestamp(dt.astimezone(utc).astimezone(dt.tzinfo).timestamp()) > ... > [Guido] > >> That can't be right -- There is no way any fromtimestamp() call can > return > >> a time in the gap. > > [Alexander Belopolsky] > > I don't think Nick said that. > > [Tim Peters] > I do, except that he didn't ;-) Count the parens carefully. > OK, it looks like Nick has managed to confuse both authors of the PEP, but not Guido. :-) The .astimezone() conversions in Nick's expression are a red herring. They don't change the value of the timestamp. That's the invariant Guido mentioned: dt.timestamp() == dt.astimezone(utc).timestamp() == dt.astimezone(utc).astimezone(dt.tzinfo).timestamp() Now, if dt is in its tzinfo gap, then dt != datetime.fromtimestamp(dt.timestamp(), dt.tzinfo) Instead, you get something like this: datetime.fromtimestamp(dt.timestamp(), dt.tzinfo) == dt + (1 - 2*dt.fold) * gap where gap is the size of the gap expressed as a timedelta (typically gap = timedelta(hours=1)). In words, when you ask for 2:40 AM, but the clock jumps from 01:59 AM to 03:00 AM, the round trip through timestamp gives you 03:40 AM if fold=0 and 01:40 AM if fold=1. This rule is somewhat arbitrary, but it has many nice mathematical and "human" properties. (There is an (imperfect) analogy with the roots of a quadratic equation here: when no real solutions exist, the two complex solutions are a ± i*b and "nice to have" real values are a ± b.) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 8:16 AM, Tim Peters wrote: > [Tim] > > ... > > The > > top-level operation on the RHS is datetime.fromtimestamp(). However, > > it didn't pass a tzinfo, so it creates a naive datetime. Assuming dt > > was aware to begin with, the attempt to compare will always (gap or > > not) raise an exception. > > Oops! In current Python, comparing naive and aware via `==` just > returns False. That's even more confusing ;-) > Hm, but that's in general how == is *supposed* to work between objects of incompatible types. < and > are supposed to fail but == is supposed to return False (the __eq__ should return NotImplemented). If == ever raises an exception, having two different objects as dict keys can cause random, hard-to-debug failures. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
[Tim] >>> ... >>> The >>> top-level operation on the RHS is datetime.fromtimestamp(). However, >>> it didn't pass a tzinfo, so it creates a naive datetime. Assuming dt >>> was aware to begin with, the attempt to compare will always (gap or >>> not) raise an exception. [Tim] >> Oops! In current Python, comparing naive and aware via `==` just >> returns False. That's even more confusing ;-) [Guido] > Hm, but that's in general how == is *supposed* to work between objects of > incompatible types. < and > are supposed to fail but == is supposed to > return False (the __eq__ should return NotImplemented). If == ever raises an > exception, having two different objects as dict keys can cause random, > hard-to-debug failures. Sure - no complaint. I was just saying that in the specific, complicated, contrived expression Nick presented, that it always returns False (no matter which aware datetime he starts with) would be more of a head-scratcher than if it raised a "can't compare naive and aware datetimes" exception instead. That's why, whenever anyone is confused by anything they see in a Python program, they should post all their code verbatim to Python-Dev, prefaced with a "Doesn't work! Fix it." comment ;-) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 9:47 AM, Tim Peters wrote: > [Tim] > >>> ... > >>> The > >>> top-level operation on the RHS is datetime.fromtimestamp(). However, > >>> it didn't pass a tzinfo, so it creates a naive datetime. Assuming dt > >>> was aware to begin with, the attempt to compare will always (gap or > >>> not) raise an exception. > > [Tim] > >> Oops! In current Python, comparing naive and aware via `==` just > >> returns False. That's even more confusing ;-) > > [Guido] > > Hm, but that's in general how == is *supposed* to work between objects of > > incompatible types. < and > are supposed to fail but == is supposed to > > return False (the __eq__ should return NotImplemented). If == ever > raises an > > exception, having two different objects as dict keys can cause random, > > hard-to-debug failures. > > Sure - no complaint. I was just saying that in the specific, > complicated, contrived expression Nick presented, that it always > returns False (no matter which aware datetime he starts with) would be > more of a head-scratcher than if it raised a "can't compare naive and > aware datetimes" exception instead. > And yet I think the desired behavior of == requires us to return False. I think we should change this in the PEP, except I can't find where the PEP says == should raise an exception in this case. > That's why, whenever anyone is confused by anything they see in a > Python program, they should post all their code verbatim to > Python-Dev, prefaced with a "Doesn't work! Fix it." comment ;-) > Oh, it would be so much better if they posted their code! -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 12:47 PM, Tim Peters wrote: > > I was just saying that in the specific, > complicated, contrived expression Nick presented, that it always > returns False (no matter which aware datetime he starts with) would be > more of a head-scratcher than if it raised a "can't compare naive and > aware datetimes" exception instead. The current behavior is no fault of yours. Guido, Antoine and I share all the blame and credit for it. [1,2] [1]: http://bugs.python.org/issue15006 [2]: https://mail.python.org/pipermail/python-dev/2012-June/119933.html ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
[Tim] >> Sure - no complaint. I was just saying that in the specific, >> complicated, contrived expression Nick presented, that it always >> returns False (no matter which aware datetime he starts with) would be >> more of a head-scratcher than if it raised a "can't compare naive and >> aware datetimes" exception instead. [Guido] > And yet I think the desired behavior of == requires us to return False. Yes - we remain in violent agreement on all points here. > I think we should change this in the PEP, except I can't find where > the PEP says == should raise an exception in this case. It doesn't - the only comparison behavior changed by the PEP is in case of interzone comparison when at least one comparand is a "problem time" (which can only happen with a post-495 tzinfo). Then "==" is always False. That hack is the ugliest part of the PEP, but was needed to preserve the hash invariant (d1 == d2 implies hash(d1) == hash(d2)). BTW, while the PEP doesn't spell this out, trichotomy can fail in some such cases (those where "==" would have returned True had it not been forced to return False - then "<" and ">" will also be False). In any case, nothing changes for any case of aware-vs-naive comparison. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 10:34 AM, Tim Peters wrote: > [Tim] > >> Sure - no complaint. I was just saying that in the specific, > >> complicated, contrived expression Nick presented, that it always > >> returns False (no matter which aware datetime he starts with) would be > >> more of a head-scratcher than if it raised a "can't compare naive and > >> aware datetimes" exception instead. > > [Guido] > > And yet I think the desired behavior of == requires us to return False. > > Yes - we remain in violent agreement on all points here. > > > > I think we should change this in the PEP, except I can't find where > > the PEP says == should raise an exception in this case. > > It doesn't - the only comparison behavior changed by the PEP is in > case of interzone comparison when at least one comparand is a "problem > time" (which can only happen with a post-495 tzinfo). Then "==" is > always False. That hack is the ugliest part of the PEP, but was > needed to preserve the hash invariant (d1 == d2 implies hash(d1) == > hash(d2)). > > BTW, while the PEP doesn't spell this out, trichotomy can fail in some > such cases (those where "==" would have returned True had it not been > forced to return False - then "<" and ">" will also be False). > > In any case, nothing changes for any case of aware-vs-naive comparison. > And I guess we can't make < and > raise an exception for backward compatibility reasons. :-( -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 1:57 PM, Guido van Rossum wrote: > BTW, while the PEP doesn't spell this out, trichotomy can fail in some >> such cases (those where "==" would have returned True had it not been >> forced to return False - then "<" and ">" will also be False). >> >> In any case, nothing changes for any case of aware-vs-naive comparison. >> > > And I guess we can't make < and > raise an exception for backward > compatibility reasons. :-( > Just to make it clear, naive to aware comparison is an error now and will still be an error: >>> from datetime import * >>> datetime.now() > datetime.now(timezone.utc) Traceback (most recent call last): File "", line 1, in TypeError: can't compare offset-naive and offset-aware datetimes What would be nice, was if datetime.now(tz1) > datetime.now(tz2) was an error whenever tz1 is not tz2, but this is not possible for backward compatibility reasons. I was toying with an idea to make t > s an error whenever the result depends on the value of t.fold or s.fold, but the resulting rules were even uglier than the hash invariant compromise. At the end of the day, this is the case of practicality beating purity. We overload > and - in datetime for convenience of interzone operations. (Want to know number of microseconds since epoch? Easy: (t - datetime(1970, 1, 1, tzinfo=timezone.utc))//timedelta.resolution). We pay for this convenience by a loss of some properties that we expect from mathematical operations (e.g. s - t != (s - u) - (t - u) is possible.) I think this is a fair price to pay for convenience of s > t and s - t over s.is_later(t) and s.timediff(t). Arguably, requiring s.timezone(utc) > t.astimezone(utc) would be "explicit is better than implicit," but you cannot deny the convenience of plain s > t. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
[Guido] >>> I think we should change this in the PEP, except I can't find where >>> the PEP says == should raise an exception in this case. [Tim] >> It doesn't - the only comparison behavior changed by the PEP is in >> case of interzone comparison when at least one comparand is a "problem >> time" (which can only happen with a post-495 tzinfo). Then "==" is >> always False. That hack is the ugliest part of the PEP, Correction: that's _only_ "ugly part" of the PEP. Except for that, it's quite elegant :-) >> but was needed to preserve the hash invariant (d1 == d2 implies >> hash(d1) == hash(d2)). >> >> BTW, while the PEP doesn't spell this out, trichotomy can fail in some >> such cases (those where "==" would have returned True had it not been >> forced to return False - then "<" and ">" will also be False). [Guido] > And I guess we can't make < and > raise an exception for backward > compatibility reasons. :-( Bingo. But, in its favor, that would be less incompatible than removing hash() and dicts from the language ;-) Another oddity is that interzone subtraction always gets "the right" result, even in interzone cases where "x == y" is forced to return False despite that "x - y == timedelta(0)". cmp-like comparison still enjoys trichotomy in all cases. Note that, for a week or two, we _tried_ to get around all that by making x != y for intrazone x and y differing only in `fold`. But that was so at odds with the naive time model that worse kinds of incompatibility snuck in. So we can make intrazone "naive time" work as expected in all cases, or we can make by-magic interzone subtraction and comparison work as expected in all cases. We started and ended with the former, with a painful abandoned attempt at the latter in between. If there's a sane(*) way to do both simultaneously, in a backward-compatible way, it eluded everyone. (*) An "insane" way would be to partition all aware datetimes into equivalence classes based on lumping together all possible spellings of a "problem time" in all zones, so that hash() could treat them all the same. But that requires knowing all possible tzinfos that can ever be used before the first time hash() is called. Even then it would be messy to do. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 11:26 AM, Alexander Belopolsky < [email protected]> wrote: > > On Tue, Sep 22, 2015 at 1:57 PM, Guido van Rossum > wrote: > >> BTW, while the PEP doesn't spell this out, trichotomy can fail in some >>> such cases (those where "==" would have returned True had it not been >>> forced to return False - then "<" and ">" will also be False). >>> >>> In any case, nothing changes for any case of aware-vs-naive comparison. >>> >> >> And I guess we can't make < and > raise an exception for backward >> compatibility reasons. :-( >> > > Just to make it clear, naive to aware comparison is an error now and will > still be an error: > Ah, I just realized one of the confusions here is the use of the word "comparison", since it could refer to == or to < and >. > >>> from datetime import * > >>> datetime.now() > datetime.now(timezone.utc) > Traceback (most recent call last): > File "", line 1, in > TypeError: can't compare offset-naive and offset-aware datetimes > IIUC the < and > operators raise here, and == returns False -- which is exactly as it should be. > What would be nice, was if datetime.now(tz1) > datetime.now(tz2) was an > error whenever tz1 is not tz2, but this is not possible for backward > compatibility reasons. > > I was toying with an idea to make t > s an error whenever the result > depends on the value of t.fold or s.fold, but the resulting rules were even > uglier than the hash invariant compromise. > > At the end of the day, this is the case of practicality beating purity. > We overload > and - in datetime for convenience of interzone operations. > (Want to know number of microseconds since epoch? Easy: (t - > datetime(1970, 1, 1, tzinfo=timezone.utc))//timedelta.resolution). We pay > for this convenience by a loss of some properties that we expect from > mathematical operations (e.g. s - t != (s - u) - (t - u) is possible.) I > think this is a fair price to pay for convenience of s > t and s - t over > s.is_later(t) and s.timediff(t). Arguably, requiring s.timezone(utc) > > t.astimezone(utc) would be "explicit is better than implicit," but you > cannot deny the convenience of plain s > t. > But the convenience is false -- it papers over important details. And it is broken, due to the confusion about classic vs. timeline arithmetic -- these have different needs but there's only one > operator. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 3:32 PM, Guido van Rossum wrote: > it is broken, due to the confusion about classic vs. timeline arithmetic > -- these have different needs but there's only one > operator. I feel silly trying to defend a design against its author. :-) Yes, a language with more than one > symbol would not have some of these problems. Similarly a language with a special symbol for string catenation would not have a non-commutative + and non-distributive *. All I am saying is that I can live with the choices made in datetime. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Sep 22, 2015 1:09 PM, "Alexander Belopolsky" < [email protected]> wrote: > > > On Tue, Sep 22, 2015 at 3:32 PM, Guido van Rossum wrote: >> >> it is broken, due to the confusion about classic vs. timeline arithmetic -- these have different needs but there's only one > operator. > > > I feel silly trying to defend a design against its author. :-) Yes, a language with more than one > symbol would not have some of these problems. Similarly a language with a special symbol for string catenation would not have a non-commutative + and non-distributive *. All I am saying is that I can live with the choices made in datetime. Is there a good argument against at least deprecating inequality comparisons and subtraction between mixed timezone datetimes? It seems like a warning that would be likely to catch real bugs. -n ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On Tue, Sep 22, 2015 at 4:14 PM, Nathaniel Smith wrote: > Is there a good argument against at least deprecating inequality > comparisons and subtraction between mixed timezone datetimes? That's a wrong question. The right question is: "Is current behavior sufficiently broken to justify a backward incompatible change?" We've historically been very conservative with datetime. I would say a proposal to change the way binary operators work with datetimes should face a similar scrutiny as a proposal to change that for a bultin type. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
[Guido] >> it is broken, due to the confusion about classic vs. timeline arithmetic >> -- these have different needs but there's only one > operator. [Alex] > I feel silly trying to defend a design against its author. :-) "Design" may be an overstatement in this specific case ;-) I remember implementing this stuff, getting to the comparison operators, and noting that the spec was silent about what to do in case the tzinfos differed. I looked at Guido and explained that, and asked "so whaddya wanna do?". One of us (I don't recall which) said "well, we could convert to UTC first - that would make sense". "Ya, sure," said the other. And I said "and then, of course, interzone subtraction should do the same." "Of course," said Guido, now annoyed that I was bothering him with the obvious ;-) Note that, near that time, Python blithely compared _any_ two objects, even if of wildly different types. Compared to that, doing _anything_ arguably sane with datetime objects seemed wildly desirable. Ironically, the datetime implementation was Python's first library type to _refuse_ to compare its objects to others of wildly different types. So, in all, I'd say well under a minute's thought - between us - went into this decision. And we've been living in Paradise ever since :-) > Yes, a language with more than one > symbol would not have some > of these problems. Similarly a language with a special symbol for > string catenation would not have a non-commutative + and non- > distributive *. All I am saying is that I can live with the choices made > in datetime. Given that the alternative is suicide, I approve of that life decision. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
Yeah, sadly the point where we *should* have made this clean break was in 3.0. That's where e.g. inequalities between a string and a number, or between either type and None, were changed from returning something "arbitrary but stable" into raising TypeError. It's much harder to break it now, even with endless deprecation warnings. We might want to try again in 4.0. :-) On Tue, Sep 22, 2015 at 1:25 PM, Tim Peters wrote: > [Guido] > >> it is broken, due to the confusion about classic vs. timeline arithmetic > >> -- these have different needs but there's only one > operator. > > [Alex] > > I feel silly trying to defend a design against its author. :-) > > "Design" may be an overstatement in this specific case ;-) > > I remember implementing this stuff, getting to the comparison > operators, and noting that the spec was silent about what to do in > case the tzinfos differed. I looked at Guido and explained that, and > asked "so whaddya wanna do?". > > One of us (I don't recall which) said "well, we could convert to UTC > first - that would make sense". "Ya, sure," said the other. And I > said "and then, of course, interzone subtraction should do the same." > "Of course," said Guido, now annoyed that I was bothering him with the > obvious ;-) > > Note that, near that time, Python blithely compared _any_ two objects, > even if of wildly different types. Compared to that, doing _anything_ > arguably sane with datetime objects seemed wildly desirable. > Ironically, the datetime implementation was Python's first library > type to _refuse_ to compare its objects to others of wildly different > types. > > So, in all, I'd say well under a minute's thought - between us - went > into this decision. And we've been living in Paradise ever since :-) > > > > Yes, a language with more than one > symbol would not have some > > of these problems. Similarly a language with a special symbol for > > string catenation would not have a non-commutative + and non- > > distributive *. All I am saying is that I can live with the choices made > > in datetime. > > Given that the alternative is suicide, I approve of that life decision. > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 495 accepted
On 23 September 2015 at 01:09, Alexander Belopolsky wrote: > > On Tue, Sep 22, 2015 at 10:55 AM, Alexander Belopolsky > wrote: >> >> On Tue, Sep 22, 2015 at 10:43 AM, Guido van Rossum >> wrote: Based on the UTC/local diagram from the "Mind the Gap" section, am I correct in thinking that the modified invariant that also covers times in a gap is: dt == datetime.fromtimestamp(dt.astimezone(utc).astimezone(dt.tzinfo).timestamp()) That is, for local times that exist, the invariant "dt == dt.astimezone(utc).astimezone(dt.tzinfo)" holds, but for times that don't exist, "dt.astimezone(utc).astimezone(dt.tzinfo)" will normalise them to be a time that actually exists in the original time zone, and that normalisation also effectively happens when calling "dt.timestamp()". >>> >>> >>> That can't be right -- There is no way any fromtimestamp() call can >>> return a time in the gap. >> >> >> I don't think Nick said that. > > On the second reading, it looks like Nick's second sentence contradicts his > first. Guido is right. Moreover, there is no way to get a time in the gap > as a result of any conversion including astimezone() and fromutc() in > addition to fromtimestamp(). Such datetimes may appear if you construct > them explicitly, use .replace() to transplant a datetime to another timezone > (or modify other components) and in the result of datetime + timedelta > operation. Sorry, what I wrote in the code wasn't what I wrote in the text, but I didn't notice until Guido pointed out the discrepancy. To get the right universal invariant, I should have normalised the LHS, not the RHS: dt.astimezone(utc).astimezone(dt.tzinfo) == datetime.fromtimestamp(dt.timestamp()) For unambiguous times and times in the fold, that's a subset of the stronger invariant: dt == dt.astimezone(utc).astimezone(dt.tzinfo) == datetime.fromtimestamp(dt.timestamp()) That stronger invariant is the one that *doesn't* hold for times in the gap, as with fold=0 they'll get normalised to use the right UTC offset (same UTC time, nominally an hour later local time), while with fold=1 they get mapped to an hour earlier in both UTC and local time. Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
