Re: [Python-Dev] [Python-checkins] cpython (3.2): Issue #14409: IDLE doesn't not execute commands from shell with default

[martin]

Is my suggested rephrasing correct? I don't know, as I'm not familiar
with either the original problem or what was done to fix it.


I don't know, either, and Andrew may not able to answer the question
as he may not see the fine difference between what he wrote and what
you wrote (your phrasing is grammatically fairly advanced).

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] peps: PEP 418: Fill the "Adjusted by NTP" column of the summary table

[Antoine Pitrou]

On Fri, 30 Mar 2012 04:21:22 +0200
victor.stinner  wrote:
> 
> diff --git a/pep-0418.txt b/pep-0418.txt
> --- a/pep-0418.txt
> +++ b/pep-0418.txt
> @@ -190,13 +190,13 @@
>  Name   Resolution   Adjusted by NTP?  Action on 
> suspend
>  =  ===    
> =

Is it supposed to be resolution or accuracy?

For example for QueryPerformanceCounter() and timeGetTime(), you are
giving an accuracy, but for CLOCK_* you are giving a resolution.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrating the PEP 397 launcher

[Vinay Sajip]

  v.loewis.de> writes:

> Now that we do have the PEP, I think that should be done properly.
> I thought you offered to rewrite it. Formally, I could accept the
> PEP being withdrawn, and the feature integrated anyway, but I still
> consider that bad style.
> 
> So: I can offer to rewrite the PEP to give a full specification
> of the feature. It might be that I still need some help to
> provide end-user prose in the PEP if people would want to see that as
> well.

+1 to both your and Brian's input. I will aim to keep the implementation in sync
with any changes required as a result of resolving open issues.

Regards,

Vinay Sajip



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime module and pytz with dateutil

[Serhiy Storchaka]

28.03.12 23:20, Andrew Svetlov написав(ла):

I figured out what pytz and dateutil are not mentioned in python docs
for datetime module.
It's clean why these libs is not a part of Python Libraries — but
that's not clean for Docs.


I don't understand why Python may not include the pytz. The Olson tz 
database is not part of pytz. Python can depend on a system tz database, 
as it depends on libssl or libbz2, which also can be updated (for 
security reasons) independently.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] peps: PEP 418: Fill the "Adjusted by NTP" column of the summary table

[Victor Stinner]

>> diff --git a/pep-0418.txt b/pep-0418.txt
>> --- a/pep-0418.txt
>> +++ b/pep-0418.txt
>> @@ -190,13 +190,13 @@
>>  Name                       Resolution       Adjusted by NTP?  Action on 
>> suspend
>>  =  ===    
>> =
>
> Is it supposed to be resolution or accuracy?

Oh, right. I changed the table to add an accuracy column.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bug in generator if the generator in created in a C thread

[Tim Lesher]

On Thu, Mar 29, 2012 at 17:26, Victor Stinner  wrote:
> The problem is not the frame, but the Python thread state referenced by the
> frame. It's a private attribute. My patch just updates this reference before
> running the generator (and it clears the reference when the generator
> excution is stopped or finished).

Right--my thought was the the situation we saw might be similarly
triggered because we were storing an exception with traceback (and
associated frames) generated by thread A, and then re-throwing it from
thread B some time after thread A has exited.  The frame attached to
the exception's traceback would still, then, be referencing a
nonexistent thread state, correct?

My concern was that this might one instance of a broader problem for
folks who embed Python and see the attractive PyGILState_Ensure() API.
It already prevents us from using subinterpreters (which for us might
have been a better solution than repeated initialize/finalize, with
its known issues).

We recently made a change to dispose of the traceback before storing
the exception, and that appears to eliminate the corruption we were
seeing, so that's making me more suspicious.

> You may leak memory if your threads have a short lifetime and you create
> many threads. For example if one thread is only used to process one request
> and then is destroyed.

Absolutely--this particular hack was only used for a thread created
outside Python that had to call into the VM. Their behavior is
well-defined in our case--two particular OS threads, with lifetimes
longer than those of the interpreters we create and finalize.

-- 
Tim Lesher 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

On 03/29/2012 06:07 PM, R. David Murray wrote:

On Thu, 29 Mar 2012 23:00:20 +0200, Stefan Behnel  wrote:

R. David Murray, 29.03.2012 22:31:

On Thu, 29 Mar 2012 13:09:17 -0700, Guido van Rossum wrote:

On Thu, Mar 29, 2012 at 12:58 PM, R. David Murray wrote:

Some of us have expressed uneasiness about the consequences of dict
raising an error on lookup if the dict has been modified, the fix Victor
made to solve one of the crashers.

I don't know if I speak for the others, but (assuming that I understand
the change correctly) my concern is that there is probably a significant
amount of threading code out there that assumes that dict *lookup* is
a thread-safe operation.  Much of that code will, if moved to Python
3.3, now be subject to random runtime errors for which it will not
be prepared.  Further, code which appears safe can suddenly become
unsafe if a refactoring of the code causes an object to be stored in
the dictionary that has a Python equality method.


My original assessment was that this only affects dicts whose keys
have a user-implemented __hash__ or __eq__ implementation, and that
the number of apps that use this *and* assume the threadsafe property
would be pretty small. This is just intuition, I don't have hard
facts. But I do want to stress that not all dict lookups automatically
become thread-unsafe, only those that need to run user code as part of
the key lookup.


You are probably correct, but the thing is that one still has to do the
code audit to be sure...and then make sure that no one later introduces
such an object type as a dict key.


The thing is: the assumption that arbitrary dict lookups are GIL-atomic has
*always* been false. Only those that do not involve Python code execution
for the hash key calculation or the object comparison are. That includes
the built-in strings and numbers (and tuples of them), which are by far the
most common dict keys. Looking up arbitrary user provided objects is
definitely not guaranteed to be atomic.


Well, I'm afraid I was using the term 'thread safety' rather too loosely
there.  What I mean is that if you do a dict lookup, the lookup either
returns a value or a KeyError, and that if you get back an object that
object has internally consistent state.  The problem this fix introduces
is that the lookup may fail with a RuntimeError rather than a KeyError,
which it has never done before.

I think that is what Guido means by code that uses objects with python
eq/hash *and* assumes threadsafe lookup.  If mutation of the objects
or dict during the lookup is a concern, then the code would use locks
and wouldn't have the problem.  But there are certainly situations
where it doesn't matter if the dictionary mutates during the lookup,
as long as you get either an object or a KeyError, and thus no locks are
(currently) needed.

Maybe I'm being paranoid about breakage here, but as with most backward
compatibility concerns, there are probably more bits of code that will
be affected than our intuition indicates.

--David
___


what this suppose to mean exactly? To "mutate" is a bit odd concept for 
a programming language I suppose. Also I suppose I must be missing 
something which makes you feel like this is an OT post when the problem 
seem most likely to be exclusively in python 3.3, another reason I guess 
to not upgrade yet all that massively using 2to3. :-)


cheers,
Etienne
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime module and pytz with dateutil

[R. David Murray]

On Fri, 30 Mar 2012 13:38:13 +0300, Serhiy Storchaka  
wrote:
> 28.03.12 23:20, Andrew Svetlov написав(ла):
> > I figured out what pytz and dateutil are not mentioned in python docs
> > for datetime module.
> > It's clean why these libs is not a part of Python Libraries — but
> > that's not clean for Docs.
> 
> I don't understand why Python may not include the pytz. The Olson tz 
> database is not part of pytz. Python can depend on a system tz database, 
> as it depends on libssl or libbz2, which also can be updated (for 
> security reasons) independently.

There is an extensive discussion of this somewhere in the archives of
this list.  If I remember correctly, it boils down to the fact that pytz
does bundle the database, and that Windows either does not have or does
not regularly update its own Olson database.  Rather than ship something
out-of-date, we choose to put the onus on the user to ensure that the
appropriate code+db exists on their system.

Hopefully someone will correct me if I'm wrong, and/or find a pointer
to the relevant thread.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime module and pytz with dateutil

[Andrew Svetlov]

I filed the http://bugs.python.org/issue14448 BTW.

On Fri, Mar 30, 2012 at 3:47 PM, R. David Murray  wrote:
> On Fri, 30 Mar 2012 13:38:13 +0300, Serhiy Storchaka  
> wrote:
>> 28.03.12 23:20, Andrew Svetlov написав(ла):
>> > I figured out what pytz and dateutil are not mentioned in python docs
>> > for datetime module.
>> > It's clean why these libs is not a part of Python Libraries — but
>> > that's not clean for Docs.
>>
>> I don't understand why Python may not include the pytz. The Olson tz
>> database is not part of pytz. Python can depend on a system tz database,
>> as it depends on libssl or libbz2, which also can be updated (for
>> security reasons) independently.
>
> There is an extensive discussion of this somewhere in the archives of
> this list.  If I remember correctly, it boils down to the fact that pytz
> does bundle the database, and that Windows either does not have or does
> not regularly update its own Olson database.  Rather than ship something
> out-of-date, we choose to put the onus on the user to ensure that the
> appropriate code+db exists on their system.
>
> Hopefully someone will correct me if I'm wrong, and/or find a pointer
> to the relevant thread.
>
> --David
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime module and pytz with dateutil

[Nick Coghlan]

On Fri, Mar 30, 2012 at 10:47 PM, R. David Murray  wrote:
> On Fri, 30 Mar 2012 13:38:13 +0300, Serhiy Storchaka  
> wrote:
>> 28.03.12 23:20, Andrew Svetlov написав(ла):
>> > I figured out what pytz and dateutil are not mentioned in python docs
>> > for datetime module.
>> > It's clean why these libs is not a part of Python Libraries — but
>> > that's not clean for Docs.
>>
>> I don't understand why Python may not include the pytz. The Olson tz
>> database is not part of pytz. Python can depend on a system tz database,
>> as it depends on libssl or libbz2, which also can be updated (for
>> security reasons) independently.
>
> There is an extensive discussion of this somewhere in the archives of
> this list.  If I remember correctly, it boils down to the fact that pytz
> does bundle the database, and that Windows either does not have or does
> not regularly update its own Olson database.  Rather than ship something
> out-of-date, we choose to put the onus on the user to ensure that the
> appropriate code+db exists on their system.
>
> Hopefully someone will correct me if I'm wrong, and/or find a pointer
> to the relevant thread.

That's my recollection as well. Because we don't want to take on the
task of providely timely updates in response to timezone database
changes, any named timezone support added to the stdlib would need to
be based on a system provided timezone database, rather than the
bundled database model used by pytz. This is straightforward on *nix
based systems that provide the zoneinfo structure in the filesystem,
but more complicated on Windows (which has its own custom scheme).

Before the idea of adding full timezone support to the standard
library could be seriously considered someone would have to, at the
very least, use the mapping data from the Unicode Consortium's CLDR
Supplementary data to map the standard Olsen database timezone names
to the correct values to look up through the Windows timezone APIs
(http://unicode.org/repos/cldr-tmp/trunk/diff/supplemental/zone_tzid.html)

Adding mappings for *new* timezones would still be controlled by our
release cycle (although I think it would be reasonable to permit such
additions in maintenance releases), but updates in response to things
like daylight savings dates changing would then be the responsibility
of the OS vendors. However, "pip install pytz" is easy enough that
there isn't a lot of motivation for anyone to do the work to switch
from a bundled copy of the timezone database to a bundled TZID ->
Windows API lookup mapping.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Guido van Rossum]

Etienne, I have not understood either of your messages in this thread. They
just did not make sense to me. Do you actually understand the issue at hand?

--Guido

On Friday, March 30, 2012, Etienne Robillard wrote:

> On 03/29/2012 06:07 PM, R. David Murray wrote:
>
>> On Thu, 29 Mar 2012 23:00:20 +0200, Stefan Behnel
>>  wrote:
>>
>>> R. David Murray, 29.03.2012 22:31:
>>>
 On Thu, 29 Mar 2012 13:09:17 -0700, Guido van Rossum wrote:

> On Thu, Mar 29, 2012 at 12:58 PM, R. David Murray wrote:
>
>> Some of us have expressed uneasiness about the consequences of dict
>> raising an error on lookup if the dict has been modified, the fix
>> Victor
>> made to solve one of the crashers.
>>
>> I don't know if I speak for the others, but (assuming that I
>> understand
>> the change correctly) my concern is that there is probably a
>> significant
>> amount of threading code out there that assumes that dict *lookup* is
>> a thread-safe operation.  Much of that code will, if moved to Python
>> 3.3, now be subject to random runtime errors for which it will not
>> be prepared.  Further, code which appears safe can suddenly become
>> unsafe if a refactoring of the code causes an object to be stored in
>> the dictionary that has a Python equality method.
>>
>
> My original assessment was that this only affects dicts whose keys
> have a user-implemented __hash__ or __eq__ implementation, and that
> the number of apps that use this *and* assume the threadsafe property
> would be pretty small. This is just intuition, I don't have hard
> facts. But I do want to stress that not all dict lookups automatically
> become thread-unsafe, only those that need to run user code as part of
> the key lookup.
>

 You are probably correct, but the thing is that one still has to do the
 code audit to be sure...and then make sure that no one later introduces
 such an object type as a dict key.

>>>
>>> The thing is: the assumption that arbitrary dict lookups are GIL-atomic
>>> has
>>> *always* been false. Only those that do not involve Python code execution
>>> for the hash key calculation or the object comparison are. That includes
>>> the built-in strings and numbers (and tuples of them), which are by far
>>> the
>>> most common dict keys. Looking up arbitrary user provided objects is
>>> definitely not guaranteed to be atomic.
>>>
>>
>> Well, I'm afraid I was using the term 'thread safety' rather too loosely
>> there.  What I mean is that if you do a dict lookup, the lookup either
>> returns a value or a KeyError, and that if you get back an object that
>> object has internally consistent state.  The problem this fix introduces
>> is that the lookup may fail with a RuntimeError rather than a KeyError,
>> which it has never done before.
>>
>> I think that is what Guido means by code that uses objects with python
>> eq/hash *and* assumes threadsafe lookup.  If mutation of the objects
>> or dict during the lookup is a concern, then the code would use locks
>> and wouldn't have the problem.  But there are certainly situations
>> where it doesn't matter if the dictionary mutates during the lookup,
>> as long as you get either an object or a KeyError, and thus no locks are
>> (currently) needed.
>>
>> Maybe I'm being paranoid about breakage here, but as with most backward
>> compatibility concerns, there are probably more bits of code that will
>> be affected than our intuition indicates.
>>
>> --David
>> __**_
>>
>
> what this suppose to mean exactly? To "mutate" is a bit odd concept for a
> programming language I suppose. Also I suppose I must be missing something
> which makes you feel like this is an OT post when the problem seem most
> likely to be exclusively in python 3.3, another reason I guess to not
> upgrade yet all that massively using 2to3. :-)
>
> cheers,
> Etienne
> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

Hi Guido,

I'm sorry for being unclear! I just try actually to learn what thoses 
consequences for theses 'unattended' mutations in dictionary key lookups 
could be, :-)


however, it seems now that I might have touch a nerve without realizing 
it. I would therefore appreciate more light on this "issue" if you like

to enlighten us all. :D

Regards,
Etienne

On 03/30/2012 10:47 AM, Guido van Rossum wrote:

Etienne, I have not understood either of your messages in this thread.
They just did not make sense to me. Do you actually understand the issue
at hand?

--Guido

On Friday, March 30, 2012, Etienne Robillard wrote:

On 03/29/2012 06:07 PM, R. David Murray wrote:

On Thu, 29 Mar 2012 23:00:20 +0200, Stefan
Behnel  wrote:

R. David Murray, 29.03.2012 22:31:

On Thu, 29 Mar 2012 13:09:17 -0700, Guido van Rossum wrote:

On Thu, Mar 29, 2012 at 12:58 PM, R. David Murray wrote:

Some of us have expressed uneasiness about the
consequences of dict
raising an error on lookup if the dict has been
modified, the fix Victor
made to solve one of the crashers.

I don't know if I speak for the others, but
(assuming that I understand
the change correctly) my concern is that there
is probably a significant
amount of threading code out there that assumes
that dict *lookup* is
a thread-safe operation.  Much of that code
will, if moved to Python
3.3, now be subject to random runtime errors for
which it will not
be prepared.  Further, code which appears safe
can suddenly become
unsafe if a refactoring of the code causes an
object to be stored in
the dictionary that has a Python equality method.


My original assessment was that this only affects
dicts whose keys
have a user-implemented __hash__ or __eq__
implementation, and that
the number of apps that use this *and* assume the
threadsafe property
would be pretty small. This is just intuition, I
don't have hard
facts. But I do want to stress that not all dict
lookups automatically
become thread-unsafe, only those that need to run
user code as part of
the key lookup.


You are probably correct, but the thing is that one
still has to do the
code audit to be sure...and then make sure that no one
later introduces
such an object type as a dict key.


The thing is: the assumption that arbitrary dict lookups are
GIL-atomic has
*always* been false. Only those that do not involve Python
code execution
for the hash key calculation or the object comparison are.
That includes
the built-in strings and numbers (and tuples of them), which
are by far the
most common dict keys. Looking up arbitrary user provided
objects is
definitely not guaranteed to be atomic.


Well, I'm afraid I was using the term 'thread safety' rather too
loosely
there.  What I mean is that if you do a dict lookup, the lookup
either
returns a value or a KeyError, and that if you get back an
object that
object has internally consistent state.  The problem this fix
introduces
is that the lookup may fail with a RuntimeError rather than a
KeyError,
which it has never done before.

I think that is what Guido means by code that uses objects with
python
eq/hash *and* assumes threadsafe lookup.  If mutation of the objects
or dict during the lookup is a concern, then the code would use
locks
and wouldn't have the problem.  But there are certainly situations
where it doesn't matter if the dictionary mutates during the lookup,
as long as you get either an object or a KeyError, and thus no
locks are
(currently) needed.

Maybe I'm being paranoid about breakage here, but as with most
backward
compatibility concerns, there are probably more bits of code
that will
be affected than our intuition indicates.

--David
__ _


what this suppose to mean exactly? To "mutate" 

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Nick Coghlan]

On Sat, Mar 31, 2012 at 1:27 AM, Etienne Robillard  wrote:
> Hi Guido,
>
> I'm sorry for being unclear! I just try actually to learn what thoses
> consequences for theses 'unattended' mutations in dictionary key lookups
> could be, :-)
>
> however, it seems now that I might have touch a nerve without realizing it.
> I would therefore appreciate more light on this "issue" if you like
> to enlighten us all. :D

Etienne,

For those that need to understand the issue in order to further
consider the consequences of the change, RDM has already explained the
problem quite clearly. If you'd like a more in-depth explanation,
please ask the question again over on core-mentors...@python.org. It's
not an appropriate topic for the main development list.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

"Multiple threads can agree by convention not to mutate a shared dict, 
there's no great need for enforcement. Multiple processes can't share 
dicts."


its not sure I get completely the meaning of "mutate"...  And if 
possible, I would like also the rational for the 2nd phrase while we're 
at it as it seem a little unclear too. Sorry also

if this is OT... :)

Regards,
Etienne

http://www.python.org/dev/peps/pep-0416/

On 03/30/2012 10:47 AM, Guido van Rossum wrote:

Etienne, I have not understood either of your messages in this thread.
They just did not make sense to me. Do you actually understand the issue
at hand?

--Guido

On Friday, March 30, 2012, Etienne Robillard wrote:

On 03/29/2012 06:07 PM, R. David Murray wrote:

On Thu, 29 Mar 2012 23:00:20 +0200, Stefan
Behnel  wrote:

R. David Murray, 29.03.2012 22:31:

On Thu, 29 Mar 2012 13:09:17 -0700, Guido van Rossum wrote:

On Thu, Mar 29, 2012 at 12:58 PM, R. David Murray wrote:

Some of us have expressed uneasiness about the
consequences of dict
raising an error on lookup if the dict has been
modified, the fix Victor
made to solve one of the crashers.

I don't know if I speak for the others, but
(assuming that I understand
the change correctly) my concern is that there
is probably a significant
amount of threading code out there that assumes
that dict *lookup* is
a thread-safe operation.  Much of that code
will, if moved to Python
3.3, now be subject to random runtime errors for
which it will not
be prepared.  Further, code which appears safe
can suddenly become
unsafe if a refactoring of the code causes an
object to be stored in
the dictionary that has a Python equality method.


My original assessment was that this only affects
dicts whose keys
have a user-implemented __hash__ or __eq__
implementation, and that
the number of apps that use this *and* assume the
threadsafe property
would be pretty small. This is just intuition, I
don't have hard
facts. But I do want to stress that not all dict
lookups automatically
become thread-unsafe, only those that need to run
user code as part of
the key lookup.


You are probably correct, but the thing is that one
still has to do the
code audit to be sure...and then make sure that no one
later introduces
such an object type as a dict key.


The thing is: the assumption that arbitrary dict lookups are
GIL-atomic has
*always* been false. Only those that do not involve Python
code execution
for the hash key calculation or the object comparison are.
That includes
the built-in strings and numbers (and tuples of them), which
are by far the
most common dict keys. Looking up arbitrary user provided
objects is
definitely not guaranteed to be atomic.


Well, I'm afraid I was using the term 'thread safety' rather too
loosely
there.  What I mean is that if you do a dict lookup, the lookup
either
returns a value or a KeyError, and that if you get back an
object that
object has internally consistent state.  The problem this fix
introduces
is that the lookup may fail with a RuntimeError rather than a
KeyError,
which it has never done before.

I think that is what Guido means by code that uses objects with
python
eq/hash *and* assumes threadsafe lookup.  If mutation of the objects
or dict during the lookup is a concern, then the code would use
locks
and wouldn't have the problem.  But there are certainly situations
where it doesn't matter if the dictionary mutates during the lookup,
as long as you get either an object or a KeyError, and thus no
locks are
(currently) needed.

Maybe I'm being paranoid about breakage here, but as with most
backward
compatibility concerns, there are probably more bits of code
that will
be affected than our intuition indicates.

--David
__ _

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Stefan Behnel]

Etienne Robillard, 30.03.2012 17:45:
> Sorry also if this is OT... :)

Yes, it is. Please do as Nick told you.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

you wish...are you also truth allergic or irritated by the consequences 
of free speech ? Please stop giving me orders. You don't even know me 
and this is at all not necessary and good netiquette if you want to 
bring a point to ponder.


Sorry for others who thinks this is not OT as I its probably related to 
pep-416 refusal.


Cheers!
Etienne

On 03/30/2012 11:54 AM, Stefan Behnel wrote:

Etienne Robillard, 30.03.2012 17:45:

Sorry also if this is OT... :)


Yes, it is. Please do as Nick told you.

Stefan


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2012-03-23 - 2012-03-30)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open3359 (+13)
  closed 22871 (+42)
  total  26230 (+55)

Open issues with patches: 1438 


Issues opened (35)
==

#14396: Popen wait() doesn't handle spurious wakeups
http://bugs.python.org/issue14396  opened by amscanne

#14397: Use GetTickCount/GetTickCount64 instead of QueryPerformanceCou
http://bugs.python.org/issue14397  opened by haypo

#14398: bz2.BZ2DEcompressor.decompress fail on large files
http://bugs.python.org/issue14398  opened by Laurent.Gautier

#14399: zipfile and creat/update comment
http://bugs.python.org/issue14399  opened by acassaigne

#14405: Some "Other Resources" in the sidebar are hopelessly out of da
http://bugs.python.org/issue14405  opened by nedbat

#14406: Race condition in concurrent.futures
http://bugs.python.org/issue14406  opened by anacrolix

#14407: concurrent.futures tests don't adher to test_cases protocol
http://bugs.python.org/issue14407  opened by anacrolix

#14408: Support ./python -m unittest in the stdlib tests
http://bugs.python.org/issue14408  opened by anacrolix

#14412: Sqlite Integer Fields
http://bugs.python.org/issue14412  opened by goatsofmendez

#14414: xmlrpclib leaves connection in broken state if server returns 
http://bugs.python.org/issue14414  opened by fancycode

#14417: dict RuntimeError workaround
http://bugs.python.org/issue14417  opened by Jim.Jewett

#14418: Document differences in SocketServer between Python 2.6 and 2.
http://bugs.python.org/issue14418  opened by gjb1002

#14419: Faster ascii decoding
http://bugs.python.org/issue14419  opened by storchaka

#14420: winreg SetValueEx DWord type incompatible with value argument
http://bugs.python.org/issue14420  opened by RoSanford

#14422: Pack PyASCIIObject fields to reduce memory consumption of pure
http://bugs.python.org/issue14422  opened by haypo

#14423: Getting the starting date of iso week from a week number and a
http://bugs.python.org/issue14423  opened by Esben.Agerbæk.Black

#14424: document PyType_GenericAlloc
http://bugs.python.org/issue14424  opened by eli.bendersky

#14425: Improve handling of 'timeout' parameter default in urllib.urlo
http://bugs.python.org/issue14425  opened by r.david.murray

#14426: date format problem in Cookie/http.cookies
http://bugs.python.org/issue14426  opened by Müte.Invert

#14427: urllib.request.Request get_header and header_items not documen
http://bugs.python.org/issue14427  opened by r.david.murray

#14428: Implementation of the PEP 418
http://bugs.python.org/issue14428  opened by haypo

#14432: Bug in generator if the generator in created in a C thread
http://bugs.python.org/issue14432  opened by haypo

#14433: Python 3 interpreter crash on windows when stdin closed in Pyt
http://bugs.python.org/issue14433  opened by alexis.d

#14434: Tutorial link in "help()" in Python3 points to Python2 tutoria
http://bugs.python.org/issue14434  opened by Dubslow

#14437: _io build fails on cygwin
http://bugs.python.org/issue14437  opened by luch

#14439: Easier error diagnosis when bootstrapping the runpy module in 
http://bugs.python.org/issue14439  opened by haypo

#14440: Close background process if IDLE closes abnormally.
http://bugs.python.org/issue14440  opened by asvetlov

#14443: Distutils test failure
http://bugs.python.org/issue14443  opened by rosslagerwall

#1: Virtualenv not portable from Python 2.7.2 to 2.7.3 (os.urandom
http://bugs.python.org/issue1  opened by jason.coombs

#14445: Providing more fine-grained control over assert statements
http://bugs.python.org/issue14445  opened by max

#14446: Remove deprecated tkinter functions
http://bugs.python.org/issue14446  opened by asvetlov

#14448: Mention pytz in datetime's docs
http://bugs.python.org/issue14448  opened by asvetlov

#14449: argparse optional arguments should follow getopt_long(3)
http://bugs.python.org/issue14449  opened by mamikonyan

#14450: Log rotate cant execute in Windows. (logging module)
http://bugs.python.org/issue14450  opened by shinta.nakayama

#14452: SysLogHandler sends invalid messages when using unicode
http://bugs.python.org/issue14452  opened by zmk



Most recent 15 issues with no replies (15)
==

#14452: SysLogHandler sends invalid messages when using unicode
http://bugs.python.org/issue14452

#14446: Remove deprecated tkinter functions
http://bugs.python.org/issue14446

#14440: Close background process if IDLE closes abnormally.
http://bugs.python.org/issue14440

#14427: urllib.request.Request get_header and header_items not documen
http://bugs.python.org/issue14427

#14424: document PyType_GenericAlloc
http://bugs.python.org/issue14424

#14418: Document differences in SocketServer between Python 2.6 and 2.
http://bugs.python.org/issue14418

#14414: xmlrpclib leaves c

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Stefan Behnel]

Etienne Robillard, 30.03.2012 18:08:
> are you also truth allergic or irritated by the consequences of
> free speech ?

Please note that "free speech" is a concept that is different from asking
beginner's computer science questions on the core developer mailing list of
a software development project. This is not the right forum to do so, and
you should therefore move your "free speech" to one that is more
appropriate. Nick has pointed you to one such forum and you would be well
advised to use it - that's all I was trying to say. I hope it's clearer now.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

your reasoning is pathetic at best. i pass... Thanks for the tip :-)

Cheers,
Etienne


On 03/30/2012 12:18 PM, Stefan Behnel wrote:

Etienne Robillard, 30.03.2012 18:08:

are you also truth allergic or irritated by the consequences of
free speech ?


Please note that "free speech" is a concept that is different from asking
beginner's computer science questions on the core developer mailing list of
a software development project. This is not the right forum to do so, and
you should therefore move your "free speech" to one that is more
appropriate. Nick has pointed you to one such forum and you would be well
advised to use it - that's all I was trying to say. I hope it's clearer now.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/animelovin%40gmail.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.2): Issue #14409: IDLE doesn't not execute commands from shell with default

[Andrew Svetlov]

Thank you for mentoring.

I will fix NEWS if you help me with better text.

The bug fixed is that commit is:
IDLE has 3 configs: user, system default and hardcoded in python code.
Last one had bad binding for  key.
Usually this config is never used: user or system ones overrides former.
But if IDLE cannot open config files by some reason it switches to
hardcoded configs and user got broken IDLE.

Can anybody guess me short and descriptive message describing what fix well?

On Fri, Mar 30, 2012 at 6:12 AM, Nick Coghlan  wrote:
> On Fri, Mar 30, 2012 at 2:01 AM, andrew.svetlov
>  wrote:
>> +- Issue #14409: IDLE doesn't not execute commands from shell,
>> +  error with default keybinding for Return. (Patch by Roger Serwy)
>
> The double negative here makes this impossible to understand. Could we
> please get an updated NEWS entry that explains what actually changed
> in IDLE to fix this?
>
> Perhaps something like "IDLE now always sets the default keybind for
> Return correctly, ensuring commands can be executed in the IDLE shell
> window"? (assuming that's what happened).
>
> This is important, folks: NEWS entries need to be comprehensible for
> people that *haven't* read the associated tracker issue. This means
> that issue titles (which generally describe a problem someone was
> having) are often inappropriate as NEWS items. NEWS items should be
> short descriptions that clearly describe *what changed*, perhaps with
> some additional information to explain a bit about why the change was
> made.
>
> Regards,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-checkins mailing list
> python-check...@python.org
> http://mail.python.org/mailman/listinfo/python-checkins



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] OT: refusal to listen and learn [was Re: Issue 14417: consequences of new dict runtime error]

[Ethan Furman]

Etienne Robillard wrote:

your reasoning is pathetic at best. i pass... Thanks for the tip :-)


The Python Developer list is for the discussion of developing Python, 
not for teaching basic programming.


You are being rude, and a smiley does not make you less rude.

I am adding you to my kill-file so I no longer see messages from you 
because you refuse to follow the advice you have been given.


~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

On 03/30/2012 02:23 PM, Ethan Furman wrote:

Etienne Robillard wrote:

your reasoning is pathetic at best. i pass... Thanks for the tip :-)


The Python Developer list is for the discussion of developing Python,
not for teaching basic programming.

You are being rude, and a smiley does not make you less rude.

I am adding you to my kill-file so I no longer see messages from you
because you refuse to follow the advice you have been given.

~Ethan~



Add me to whatever file you want, but I believe the consequences for the 
new dict runtime errors just won't be resolved this way, and neither by 
systematically blocking alternative opinions as OT will help, because 
thats typically oppression, not free speech. :-)


Cheers, :-)

Etienne

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Lennart Regebro]

The overview of the different monotonic clocks was interesting,
because only one of them is adjusted by NTP, and that's the unix
CLOCK_MONOTONIC. Hence we don't need a raw=False flag, which I
previously suggested, we only need to not use CLOCK_MONOTONIC (which
the PEP psuedo-code indeed also does not do, so that's all good).

That means I think the PEP is fine now, if we rename highres().
time.time() already gets the highest resolution clock it can. Hence a
highres() is confusing as the name implies that it returns a higher
resolution clock than time.time(). And the name does not in any way
indicate that the returned clock might be monotonic. try_monotonic()
seems the obvious choice, since that's what it actually does.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

On 03/30/2012 03:02 PM, Guido van Rossum wrote:

Hey Etienne, I am honestly trying to understand your contribution but
you seem to have started a discussion about free speech. Trust me that
we don't mind your contributions, we're just trying to understand what
you're saying, and the free speech discussion isn't helping with that.


I agree.


So if you have a comment on the dict mutation problem, please say so.


OK.


If you need help understanding the problem, python-dev is not the
place to ask questions; you could ask on the bug, or on the
core-mentorship list as Nick suggested. But please stop bringing up
free speech, that's not an issue.


Guido, thanks for the wisdom and clarity of your reasoning. I really 
appreciate a positive attitude towards questioning not so obvious problems.


So far I was only attempting to verify whether this is related to 
PEP-416 or not. If this is indeed related PEP 416, then I must obviously 
attest that I must still understand why a immutable dict would prevent 
this bug or not...


Either ways, I fail to see where this is OT or should be discussed on a 
more obscur forum than python-dev. :-)


Kind regards,
Etienne

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[R. David Murray]

On Fri, 30 Mar 2012 15:13:36 -0400, Etienne Robillard  
wrote:
> So far I was only attempting to verify whether this is related to 
> PEP-416 or not. If this is indeed related PEP 416, then I must obviously 
> attest that I must still understand why a immutable dict would prevent 
> this bug or not...

OK, that seems to be the source of your confusion, then.  This has
nothing to do with PEP-416.

We are talking about issue Issue 14417 (like it says in the subject),
which in turn is a reaction to the fix for issue 14205.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Guido van Rossum]

On Fri, Mar 30, 2012 at 12:13 PM, Etienne Robillard
 wrote:
> On 03/30/2012 03:02 PM, Guido van Rossum wrote:
>>
>> Hey Etienne, I am honestly trying to understand your contribution but
>> you seem to have started a discussion about free speech. Trust me that
>> we don't mind your contributions, we're just trying to understand what
>> you're saying, and the free speech discussion isn't helping with that.
>
>
> I agree.
>
>
>> So if you have a comment on the dict mutation problem, please say so.
>
>
> OK.
>
>
>> If you need help understanding the problem, python-dev is not the
>> place to ask questions; you could ask on the bug, or on the
>> core-mentorship list as Nick suggested. But please stop bringing up
>> free speech, that's not an issue.
>
>
> Guido, thanks for the wisdom and clarity of your reasoning. I really
> appreciate a positive attitude towards questioning not so obvious problems.
>
> So far I was only attempting to verify whether this is related to PEP-416 or
> not. If this is indeed related PEP 416, then I must obviously attest that I
> must still understand why a immutable dict would prevent this bug or not...

It's not related to PEP 416 (which was rejected). Please refer to
http://bugs.python.org/issue14417 for the issue being discussed.

> Either ways, I fail to see where this is OT or should be discussed on a more
> obscur forum than python-dev. :-)

We need to keep that list clear for important discussions. It is the
only channel that the core Python developers have. If it has too much
noise people will stop reading it and it stops functioning. Hence, we
try to keep questions from newbies to a minimum -- there are other
places where we welcome such questions though.

So, once more, if you don't understand the issue and cannot figure it
out by reading up, please ask somewhere else (or just accept that you
don't have anything to contribute to this particular issue). This
includes explaining basic terms like "mutate". On the other hand, if
you *do* understand the problem, by all means let us know what you
think of the question at hand (whether the change referred to in the
issue is going to break people's code or not). We don't need more
speculation though; that's how we got here in the first place (my
speculation that it's not going to be an issue vs. RDM's speculation
that it's going to cause widespread havoc :-).

I hope you understand.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14065: Added cyclic GC support to ET.Element

[Benjamin Peterson]

2012/3/30 eli.bendersky :
> http://hg.python.org/cpython/rev/0ca32013d77e
> changeset:   75997:0ca32013d77e
> parent:      75995:cf2e74e0b7d4
> user:        Eli Bendersky 
> date:        Fri Mar 30 16:38:33 2012 +0300
> summary:
>  Issue #14065: Added cyclic GC support to ET.Element
>
> files:
>  Lib/test/test_xml_etree.py |  27 ++-
>  Modules/_elementtree.c     |  63 +++--
>  2 files changed, 74 insertions(+), 16 deletions(-)
>
>
> diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py
> --- a/Lib/test/test_xml_etree.py
> +++ b/Lib/test/test_xml_etree.py
> @@ -14,9 +14,10 @@
>  # Don't re-import "xml.etree.ElementTree" module in the docstring,
>  # except if the test is specific to the Python implementation.
>
> -import sys
> +import gc
>  import html
>  import io
> +import sys
>  import unittest
>
>  from test import support
> @@ -1846,6 +1847,30 @@
>         self.assertRaises(TypeError, e.extend, [ET.Element('bar'), 'foo'])
>         self.assertRaises(TypeError, e.insert, 0, 'foo')
>
> +    def test_cyclic_gc(self):
> +        class ShowGC:
> +            def __init__(self, flaglist):
> +                self.flaglist = flaglist
> +            def __del__(self):
> +                self.flaglist.append(1)

I think a nicer way to check for cyclic collection is to take a
weakref to an object, call the GC, then check to make sure the weakref
is broken.

> +
> +        # Test the shortest cycle: lst->element->lst
> +        fl = []
> +        lst = [ShowGC(fl)]
> +        lst.append(ET.Element('joe', attr=lst))
> +        del lst
> +        gc.collect()

support.gc_collect() is preferable

> +        self.assertEqual(fl, [1])
> +
> +        # A longer cycle: lst->e->e2->lst
> +        fl = []
> +        e = ET.Element('joe')
> +        lst = [ShowGC(fl), e]
> +        e2 = ET.SubElement(e, 'foo', attr=lst)
> +        del lst, e, e2
> +        gc.collect()
> +        self.assertEqual(fl, [1])


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Guido van Rossum]

On Fri, Mar 30, 2012 at 12:01 PM, Lennart Regebro  wrote:
> The overview of the different monotonic clocks was interesting,
> because only one of them is adjusted by NTP, and that's the unix
> CLOCK_MONOTONIC. Hence we don't need a raw=False flag, which I
> previously suggested, we only need to not use CLOCK_MONOTONIC (which
> the PEP psuedo-code indeed also does not do, so that's all good).

Right on.

> That means I think the PEP is fine now, if we rename highres().
> time.time() already gets the highest resolution clock it can.

No, time.time() is the clock that can be mapped to and from "civil
time". (Adjustments by NTP and the user notwithstanding.) The other
clocks have a variable epoch and do not necessarily tick with a
constant rate (e.g. they may not tick at all while the system is
suspended).

> Hence a
> highres() is confusing as the name implies that it returns a higher
> resolution clock than time.time(). And the name does not in any way
> indicate that the returned clock might be monotonic. try_monotonic()
> seems the obvious choice, since that's what it actually does.

I am still unhappy with the two names, but I'm glad that we're this
close. We need two new names; one for an OS-provided clock that is
"monotonic" or "steady" or whatever you want to call it, but which may
not exist on all systems (some platforms don't have it, some host may
not have it even though the platform generally does have it). The
other name is for a clock that's one or the other; it should be the
OS-provided clock if it exists, otherwise time.time(). Most code
should probably use this one, so perhaps its name should be the
shorter one.

C++ calls these steady_clock and high_resolution_clock, respectively.
But it also calls the civil time clock system_clock, so perhaps we
shouldn't feel to bound by it (except that we *shouldn't* call
something steady if it isn't).

I still think the name "monotonic" give the wrong impression; I would
be happy calling it steady.

But for the other, I'm still at a loss, and that name is the most
important one. We can't call it steady because it isn't always.
highres or hires sounds awkward; try_monotonic or try_steady are even
more awkward. I looked in an online thesaurus and here's a list of
what it gave:

Big Ben, alarm, chroniker, chronograph, chronometer, digital watch,
hourglass, metronome, pendulum, stopwatch, sundial, tattler,
tick-tock, ticker, timekeeper, timemarker, timepiece, timer, turnip,
watch

I wonder if something with tick would work? (Even though it returns a float. :-)

If all else fails, I'd go with turnip.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Victor Stinner]

Hi,

Windows provides two main monotonic clocks: QueryPerformanceCounter()
and GetTickCount(). QueryPerformanceCounter() has a good accuracy (0.3
ns - 5 ns) but has various issues and is know to not have a steady
rate. GetTickCount() has a worse accuracy (1 ms - 15 ms) but is more
stable and behave better on system suspend/resume.

The glib library prefers GetTickCount() over QueryPerformanceCounter()
for its g_get_monotonic_time() because "The QPC timer has too many
issues to be used as is."
Ihttp://mail.gnome.org/archives/commits-list/2011-November/msg04589.html

The Qt library tries QueryPerformanceCounter() but may fallback to
GetTickCount() if it is not available.

python-monotonic-time only uses GetTickCount() or GetTickCount64().

It is important to decide which clock is used for the Python
time.monotonic() because it may change the design of the PEP 418. If
we use GetTickCount() for time.monotonic(), we should use
QueryPerformanceCounter() for time.highres(). But in this case, it
means that time.highres() is not a simple "try monotonic or falls back
to system time", but may use a different clock with an higher
resolution. So we might add a third function for the "try monotonic or
falls back to system time" requirement.

Python implements time.clock() using QueryPerformanceCounter() on Windows.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[R. David Murray]

On Fri, 30 Mar 2012 12:40:25 -0700, Guido van Rossum  wrote:
> But for the other, I'm still at a loss, and that name is the most
> important one. We can't call it steady because it isn't always.
> highres or hires sounds awkward; try_monotonic or try_steady are even
> more awkward. I looked in an online thesaurus and here's a list of
> what it gave:
> 
> Big Ben, alarm, chroniker, chronograph, chronometer, digital watch,
> hourglass, metronome, pendulum, stopwatch, sundial, tattler,
> tick-tock, ticker, timekeeper, timemarker, timepiece, timer, turnip,
> watch
> 
> I wonder if something with tick would work? (Even though it returns a float. 
> :-)
> 
> If all else fails, I'd go with turnip.

We could call it "alice"[*]: sometimes it goes fast, sometimes it goes slow,
sometimes it even goes backward, but it does try to tell you when you
are late.

--David

[*] 'whiterabbit' would be more descriptive, but that's longer than
turnip.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Yury Selivanov]

On 2012-03-30, at 3:40 PM, Guido van Rossum wrote:

> I still think the name "monotonic" give the wrong impression; I would
> be happy calling it steady.

Simple google search comparison shows that people ask about 'monotonic'
clock in python, not 'steady'.

How about following Nick's (if I recall correctly) proposal of calling the 
OS function - '_monotonic', and a python wrapper - 'monotonic'?

And one more question: what do you think about introducing a special check,
that will ensure that our python implementation of 'monotonic', in case of
fallback to 'time.time()', raises an exception if time suddenly goes 
backward?

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

On Fri, Mar 30, 2012 at 1:21 PM, Victor Stinner
 wrote:
> Windows provides two main monotonic clocks: QueryPerformanceCounter()
> and GetTickCount(). QueryPerformanceCounter() has a good accuracy (0.3
> ns - 5 ns) but has various issues and is know to not have a steady
> rate. GetTickCount() has a worse accuracy (1 ms - 15 ms) but is more
> stable and behave better on system suspend/resume.
>
> The glib library prefers GetTickCount() over QueryPerformanceCounter()
> for its g_get_monotonic_time() because "The QPC timer has too many
> issues to be used as is."
> Ihttp://mail.gnome.org/archives/commits-list/2011-November/msg04589.html
>
> The Qt library tries QueryPerformanceCounter() but may fallback to
> GetTickCount() if it is not available.
>
> python-monotonic-time only uses GetTickCount() or GetTickCount64().
>
> It is important to decide which clock is used for the Python
> time.monotonic() because it may change the design of the PEP 418. If
> we use GetTickCount() for time.monotonic(), we should use
> QueryPerformanceCounter() for time.highres(). But in this case, it
> means that time.highres() is not a simple "try monotonic or falls back
> to system time", but may use a different clock with an higher
> resolution. So we might add a third function for the "try monotonic or
> falls back to system time" requirement.
>
> Python implements time.clock() using QueryPerformanceCounter() on Windows.

Oh dear. I really want to say that 15 ms is good enough. Some possible
exceptions I can think of:

- Profiling. But this really wants to measure CPU time anyways, and it
already uses a variety of hacks and heuristics to pick the best timer,
so I don't really care.

- Certain algorithms for thread (or even process?) communication might
benefit from being able to set very short timeouts. But it seems you
can just specify a timeout when waiting for a lock and it will be
taken care of. So probably a non-use-case.

So I think I still prefer GetTickCount*() over QueryPerformanceCounter().

And this is another reason why I don't like highres as a name -- as I
said, I really prefer to have just two new functions, one that's a
decent OS-provided timer that isn't affected by wall clock
adjustments, if there is one, and the other strictly an alias for that
one with a fallback to time.time() if there isn't one. Having two
different OS-provided timers with different pros and cons will just
make it too hard for users to decide, and hence we'll see a higher
rate of using the wrong timer.

(In fact, this even argues against having both the timer with fallback
and the timer without fallback. So maybe we should just have a single
timer function, *with* fallback, and a separate mechanism to inquire
its properties.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

On Fri, Mar 30, 2012 at 1:52 PM, Guido van Rossum  wrote:
> On Fri, Mar 30, 2012 at 1:21 PM, Victor Stinner
>  wrote:
>> Windows provides two main monotonic clocks: QueryPerformanceCounter()
>> and GetTickCount(). QueryPerformanceCounter() has a good accuracy (0.3
>> ns - 5 ns) but has various issues and is know to not have a steady
>> rate. GetTickCount() has a worse accuracy (1 ms - 15 ms) but is more
>> stable and behave better on system suspend/resume.
>>
>> The glib library prefers GetTickCount() over QueryPerformanceCounter()
>> for its g_get_monotonic_time() because "The QPC timer has too many
>> issues to be used as is."
>> Ihttp://mail.gnome.org/archives/commits-list/2011-November/msg04589.html
>>
>> The Qt library tries QueryPerformanceCounter() but may fallback to
>> GetTickCount() if it is not available.
>>
>> python-monotonic-time only uses GetTickCount() or GetTickCount64().
>>
>> It is important to decide which clock is used for the Python
>> time.monotonic() because it may change the design of the PEP 418. If
>> we use GetTickCount() for time.monotonic(), we should use
>> QueryPerformanceCounter() for time.highres(). But in this case, it
>> means that time.highres() is not a simple "try monotonic or falls back
>> to system time", but may use a different clock with an higher
>> resolution. So we might add a third function for the "try monotonic or
>> falls back to system time" requirement.
>>
>> Python implements time.clock() using QueryPerformanceCounter() on Windows.
>
> Oh dear. I really want to say that 15 ms is good enough. Some possible
> exceptions I can think of:
>
> - Profiling. But this really wants to measure CPU time anyways, and it
> already uses a variety of hacks and heuristics to pick the best timer,
> so I don't really care.
>
> - Certain algorithms for thread (or even process?) communication might
> benefit from being able to set very short timeouts. But it seems you
> can just specify a timeout when waiting for a lock and it will be
> taken care of. So probably a non-use-case.
>
> So I think I still prefer GetTickCount*() over QueryPerformanceCounter().
>
> And this is another reason why I don't like highres as a name -- as I
> said, I really prefer to have just two new functions, one that's a
> decent OS-provided timer that isn't affected by wall clock
> adjustments, if there is one, and the other strictly an alias for that
> one with a fallback to time.time() if there isn't one. Having two
> different OS-provided timers with different pros and cons will just
> make it too hard for users to decide, and hence we'll see a higher
> rate of using the wrong timer.
>
> (In fact, this even argues against having both the timer with fallback
> and the timer without fallback. So maybe we should just have a single
> timer function, *with* fallback, and a separate mechanism to inquire
> its properties.)

And disagreeing with myself, I just found myself using a tool for RPC
analytics that displays times in msec, and the times are often between
1 and 20 msec. Here a 15 msec timer would be horrible, and I see no
alternative but to use QPC()... (Well, if the tool ran on Windows,
which it doesn't. :-)

Can we tell what the accuracy of GetTickCount*() is when we first
request the time, and decode then?

Can you go into more detail about QPC()'s issues? What is unsteady
about its rate?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

Possibly we really do need two timers, one for measuring short
intervals and one for measuring long intervals? Perhaps we can use
this to decide on the names?

Anyway, the more I think about it, the more I believe these functions
should have very loose guarantees, and instead just cater to common
use cases -- availability of a timer with minimal fuss is usually more
important than the guarantees. So forget the idea about one version
that falls back to time.time() and another that doesn't -- just always
fall back to time.time(), which is (almost) always better than
failing.

Then we can design a separate inquiry API (doesn't have to be complex
as long as it's extensible -- a dict or object with a few predefined
keys or attributes sounds good enough) for apps that want to know more
about how the timer they're using is actually implemented.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Cameron Simpson]

On 30Mar2012 13:52, Guido van Rossum  wrote:
| (In fact, this even argues against having both the timer with fallback
| and the timer without fallback. So maybe we should just have a single
| timer function, *with* fallback, and a separate mechanism to inquire
| its properties.)

Well, you should be able to know what kind of clock you're getting. But
what do to if you don't get a satisfactory one? There's then no way for
ask for an alternative; you don't get to control the fallback method,
for example.

I've come to the opinion that the chosen approach is wrong, and suggest
a better one.

There seem to be a few competing features for clocks that people want:

  - monotonic - never going backward at all
  - high resolution
  - no steps

and only a few, fortunately.

I think you're doing this wrong at the API level. People currently are
expecting to call (for example) time.monotonic() (or whatever) and get
back "now", a hopefully high resolution float.

Given the subtlety sought for various purposes, people should be
calling:

  T = time.getclock(flags)

and then later calling:

  T.now()

to get their float.

That way people can:
  - request a set of the three characteristics above
  - inspect what they get back (it should have all the requested flags,
but unrequested flags may be set or not depending on the underlying
facility)
Obviously this should return None or raise an exception if the flag
set can't be met.

Then users can request a less featured clock on failure, depending on
what aspects of the clock are most desirable to their use case. Or of
course fail if fallback is not satisfactory.

Of course you can provide some convenient-with-fallback function that
will let people do this in one hit without the need for "T", but it
should not be the base facility offered. The base should let people
request their feature set and inspect what is supplied.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

Time is nature's way of keeping everything from happening at once.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

On Fri, Mar 30, 2012 at 2:43 PM, Cameron Simpson  wrote:
> On 30Mar2012 13:52, Guido van Rossum  wrote:
> | (In fact, this even argues against having both the timer with fallback
> | and the timer without fallback. So maybe we should just have a single
> | timer function, *with* fallback, and a separate mechanism to inquire
> | its properties.)
>
> Well, you should be able to know what kind of clock you're getting. But
> what do to if you don't get a satisfactory one? There's then no way for
> ask for an alternative; you don't get to control the fallback method,
> for example.
>
> I've come to the opinion that the chosen approach is wrong, and suggest
> a better one.
>
> There seem to be a few competing features for clocks that people want:
>
>  - monotonic - never going backward at all
>  - high resolution
>  - no steps
>
> and only a few, fortunately.
>
> I think you're doing this wrong at the API level. People currently are
> expecting to call (for example) time.monotonic() (or whatever) and get
> back "now", a hopefully high resolution float.
>
> Given the subtlety sought for various purposes, people should be
> calling:
>
>  T = time.getclock(flags)
>
> and then later calling:
>
>  T.now()
>
> to get their float.
>
> That way people can:
>  - request a set of the three characteristics above
>  - inspect what they get back (it should have all the requested flags,
>    but unrequested flags may be set or not depending on the underlying
>    facility)
>    Obviously this should return None or raise an exception if the flag
>    set can't be met.
>
> Then users can request a less featured clock on failure, depending on
> what aspects of the clock are most desirable to their use case. Or of
> course fail if fallback is not satisfactory.
>
> Of course you can provide some convenient-with-fallback function that
> will let people do this in one hit without the need for "T", but it
> should not be the base facility offered. The base should let people
> request their feature set and inspect what is supplied.

I like this out-of-the-box thinking. But I'm still wondering if there
really are enough flags for this to be worth it. If there are, great,
the API is pretty. But if there are only 2 or 3 flag combinations that
make actual sense, let's forget it.

Another out-of-the-box idea, going back to simplicity: have just one
new function, time.hrtimer(), which is implemented using the
highest-resolution timer available on the platform, but with no strong
guarantees. It *may* jump, move back, drift, change its rate, or roll
over occasionally. We try to use the implementation that's got the
fewest problems, but we don't try to hide its deficiencies, and
nothing suitable exists, it may be equivalent to time.time(). If the
times you measure are too weird, measure again. For scheduling things
a day or more in the future, you should use time.time() instead.

One issue that hasn't had enough attention: *scope* of a timer. If two
processes running on the same machine ask for the time, do the values
they see use the same epoch, or is the epoch dependent on the process?
Some code I saw in timemodule.c for working around Windows clocks
rolling over seem to imply that two processes may not always see the
same timer value. Is there a use case where this matters?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

On 03/30/2012 03:27 PM, Guido van Rossum wrote:

On Fri, Mar 30, 2012 at 12:13 PM, Etienne Robillard
  wrote:

On 03/30/2012 03:02 PM, Guido van Rossum wrote:


Hey Etienne, I am honestly trying to understand your contribution but
you seem to have started a discussion about free speech. Trust me that
we don't mind your contributions, we're just trying to understand what
you're saying, and the free speech discussion isn't helping with that.



I agree.



So if you have a comment on the dict mutation problem, please say so.



OK.



If you need help understanding the problem, python-dev is not the
place to ask questions; you could ask on the bug, or on the
core-mentorship list as Nick suggested. But please stop bringing up
free speech, that's not an issue.



Guido, thanks for the wisdom and clarity of your reasoning. I really
appreciate a positive attitude towards questioning not so obvious problems.

So far I was only attempting to verify whether this is related to PEP-416 or
not. If this is indeed related PEP 416, then I must obviously attest that I
must still understand why a immutable dict would prevent this bug or not...


It's not related to PEP 416 (which was rejected). Please refer to
http://bugs.python.org/issue14417 for the issue being discussed.


Either ways, I fail to see where this is OT or should be discussed on a more
obscur forum than python-dev. :-)


We need to keep that list clear for important discussions. It is the
only channel that the core Python developers have. If it has too much
noise people will stop reading it and it stops functioning. Hence, we
try to keep questions from newbies to a minimum -- there are other
places where we welcome such questions though.

So, once more, if you don't understand the issue and cannot figure it
out by reading up, please ask somewhere else (or just accept that you
don't have anything to contribute to this particular issue). This
includes explaining basic terms like "mutate". On the other hand, if
you *do* understand the problem, by all means let us know what you
think of the question at hand (whether the change referred to in the
issue is going to break people's code or not). We don't need more
speculation though; that's how we got here in the first place (my
speculation that it's not going to be an issue vs. RDM's speculation
that it's going to cause widespread havoc :-).

I hope you understand.


No, not really. Anyways, I guess I'll have to further dig down why is
PEP-416 is really important to Python and why it was likewise rejected, 
supposing I confused the pep 416 and issue 14417 along the way.. :-)


CHeers,

Etienne


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Victor Stinner]

> Can you go into more detail about QPC()'s issues?

Yes, see the PEP:
http://www.python.org/dev/peps/pep-0418/#windows-queryperformancecounter

> What is unsteady about its rate?

Hum, I read that you can expect a drift between QPC and GetTickCount.
I don't have exact numbers. There are also issues with power-saving.
When the CPU frequency changes, the frequency of the TSC is also
impacted on some processors (others provide a TSC at a fixed
frequency).

2012/3/30 Guido van Rossum :
> Possibly we really do need two timers, one for measuring short
> intervals and one for measuring long intervals? Perhaps we can use
> this to decide on the names?

"short" and "long" interval is not how these clocks should be defined.
I prefer use cases:

 * time.highres() should be used for profiling and benchmarking
 * time.monotonic() should be used for a scheduler and timeout

time.monotonic() rate should be as steady as possible, its stability
is more important than its accuracy. time.highres() should provide the
clock with the best accuracy. Both clocks may have an undefined
starting point.

> Anyway, the more I think about it, the more I believe these functions
> should have very loose guarantees, and instead just cater to common
> use cases -- availability of a timer with minimal fuss is usually more
> important than the guarantees. So forget the idea about one version
> that falls back to time.time() and another that doesn't -- just always
> fall back to time.time(), which is (almost) always better than
> failing.

I disagree. Python must provide a truly monotonic clock, even it is
not truly monotonic by default.

If we want to provide a convinient "monotonic or falllback to system
time", there are different options: add a dedicated function (e.g.
time.try_monotonic()) or add a parameter to enable or disable the
fallback. Between these two options, I prefer the parameter because it
avoids the creation of a new function. Even if I now agree that a
truly monotonic clock is required, I also agree with Guido: the most
common case is to use a fallback.

I suggest this API: time.monotonic(fallback=True).

Another option is to not provide a "monotonic or falllback to system
time" clock and write an explicit try/except where you need a
fallback. I don't like this solution because time.monotonic() might
fail at runtime with OSError, and so catching AttributeError is not
enough...

> Then we can design a separate inquiry API (doesn't have to be complex
> as long as it's extensible -- a dict or object with a few predefined
> keys or attributes sounds good enough) for apps that want to know more
> about how the timer they're using is actually implemented.

Sometimes, it does matter which exact OS clock is used. The
QElapsedTime class of the Qt libraries has an attribute (an enum)
describing which clock is used. We can give the name of the function
and the resolution. In some specific cases, we are able to read the
accuracy (ex: GetTickCount). The accuracy may change at runtime (at
least for GetTickCount).

Should I include such new function in the PEP, or can it be discussed
(later) in a separated thread?

--

I don't want to only provide time.monotonic() which would fallback if
no monotonic clock is available, because we would be unable to tell if
the clock is monotonic or not. See the implementation in the PEP: the
"try monotonic or fallback" function may use a different clock at each
call, and so may be monotonic at startup and then becomes
non-monotonic.

An alternative is time.monotonic() returning (time, is_monotonic) but
this is an ugly API. You cannot simply write:
   a=time.monotonic(); benchmark(); b=time.monotonic(); print(b-a)
because it raises a TypeError on b-a.

--

I will update the PEP to describe my new proposition.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 14417: consequences of new dict runtime error

[Victor Stinner]

> No, not really. Anyways, I guess I'll have to further dig down why is
> PEP-416 is really important to Python and why it was likewise rejected,
> supposing I confused the pep 416 and issue 14417 along the way.. :-)

The frozendict builtin type was rejected, but we are going to add
types.MappingProxyType: see issue #14386.
types.MappingProxyType(mydict.copy()) is very close to the frozendict
builtin type.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Cameron Simpson]

On 30Mar2012 15:21, Guido van Rossum  wrote:
| On Fri, Mar 30, 2012 at 2:43 PM, Cameron Simpson  wrote:
| > Given the subtlety sought for various purposes, people should be
| > calling:
| >  T = time.getclock(flags)
| > and then later calling:
| >  T.now()
| > to get their float.
| >
| > That way people can:
| >  - request a set of the three characteristics above
| >  - inspect what they get back (it should have all the requested flags,
| >    but unrequested flags may be set or not depending on the underlying
| >    facility)
| >    Obviously this should return None or raise an exception if the flag
| >    set can't be met.
| >
| > Then users can request a less featured clock on failure, depending on
| > what aspects of the clock are most desirable to their use case. Or of
| > course fail if fallback is not satisfactory.
[...]
| I like this out-of-the-box thinking. But I'm still wondering if there
| really are enough flags for this to be worth it. If there are, great,
| the API is pretty. But if there are only 2 or 3 flag combinations that
| make actual sense, let's forget it.

There are at least three characteristics listed above. There are several
use cases described in the long discussions.

I don't think we should be saying "we will only support these 3
thought-of use cases and the others are too hard/weird/perverse/rare".
We should be saying: we can characterise clocks in these very few ways.
Ask for a clock of some kind. We will provide one (abitrarily, though
perhaps preferring higher precision) that satisfies the request if
available.

I can easily envisage calling code as simple as this:

  T = getclock(T_MONOTONIC|T_HIGHRES) or getclock(T_MONOTONIC)

to do my own fallback for my own use case if we went with returning None
if there is no satisfactory clock.

If I need monotonic (for example) my code is in NO WAY helped by being
handed something that sometimes is not monotonic; to make my code
reliable I would need to act as though I wasn't getting a montonic clock
in the first place. No win there at all.

This has the following advantages:

  - it removes _all_ policy from the PEP, at least at the core

  - it makes things totally clear and concise at the calling end;
the clock characterists are right there in front of the reader

  - policy is in the hands of the user where it belongs, and is easy to
specify into the bargain

  - it avoids the horrendous ideas mooted of asking for something called
monotonic and getting sometime only mostly monotonic, and the
equivalent flavours for the other flags

  - implementation of the PEP is in principle as easy as a table of
clock wrappers with the flags each supports. Loop over the table
until one matches all the requested flags, or return None.

  - for higher quality of implementation you embed policy in the
convenience function, not the getclock() function
To take your suggestion of:
  Another out-of-the-box idea, going back to simplicity: have just one
  new function, time.hrtimer(), which is implemented using the
  highest-resolution timer available on the platform, but with no strong
  guarantees.

Example:
  T = time.hrtimer()
which would provide the "best" hires timer. You'd give it an
optional flag argument for additional flags so that you could
request the most-hires timer matching the flags. This is like:
  T = time.getclock(T_HIRES)
except that is makes clear your preferred attribute.

So internally one would have:

  - clock wrappers for the platform specific facilities, with the
constructors being singletons or not as suits the facility

  - a list of the wrappers with their flag sets specifiying what
clock attributes they offer

  - for the .hrtimer([flags]) or .monotonic([flags]) calls they each
have their own list of the clock wrapper, ordered on most-hires
or best-monotonic, as the implementor sees fit

That gets you:
  - arbitrary user case support
  - convenience functions for common policies

| It *may* jump, move back, drift, change its rate, or roll
| over occasionally. We try to use the implementation that's got the
| fewest problems, but we don't try to hide its deficiencies, and
| nothing suitable exists, it may be equivalent to time.time(). If the
| times you measure are too weird, measure again. For scheduling things
| a day or more in the future, you should use time.time() instead.

Getting ugly. This is why I think we should not be offering only what
we think the user may want, especially when we offer something loose
and rubbery like this.

Instead, get the user to ask. The platform timers can all be well
characterised with respect to the flags suggested above; you only need
to tabulate what timers offer what behaviours.

By all means offer convenience functions providing common choices,
but don't make those the _only_ choices. Let the user ask for anything;
if the platform can't support it that's a pity, but leave the user the
choic

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

Given the amount of disagreement I sense, I think we'll need to wait
for more people to chime in.

On Fri, Mar 30, 2012 at 3:47 PM, Victor Stinner
 wrote:
>> Can you go into more detail about QPC()'s issues?
>
> Yes, see the PEP:
> http://www.python.org/dev/peps/pep-0418/#windows-queryperformancecounter
>
>> What is unsteady about its rate?
>
> Hum, I read that you can expect a drift between QPC and GetTickCount.
> I don't have exact numbers. There are also issues with power-saving.
> When the CPU frequency changes, the frequency of the TSC is also
> impacted on some processors (others provide a TSC at a fixed
> frequency).

And how does that impact teh various use cases? (There still isn't
enough about use cases in the PEP. Thank you very much though for
adding all those details about the various platform clocks.)

> 2012/3/30 Guido van Rossum :
>> Possibly we really do need two timers, one for measuring short
>> intervals and one for measuring long intervals? Perhaps we can use
>> this to decide on the names?
>
> "short" and "long" interval is not how these clocks should be defined.

Why not?

> I prefer use cases:
>
>  * time.highres() should be used for profiling and benchmarking
>  * time.monotonic() should be used for a scheduler and timeout

So if I want to schedule something a week in the future, what should I
use? monotonic() or time()? And can you explain the reason for your
answer? What about an hour or a day? Or a month or a year?

> time.monotonic() rate should be as steady as possible, its stability
> is more important than its accuracy. time.highres() should provide the
> clock with the best accuracy. Both clocks may have an undefined
> starting point.

So what kind of drift is acceptable for each?

>> Anyway, the more I think about it, the more I believe these functions
>> should have very loose guarantees, and instead just cater to common
>> use cases -- availability of a timer with minimal fuss is usually more
>> important than the guarantees. So forget the idea about one version
>> that falls back to time.time() and another that doesn't -- just always
>> fall back to time.time(), which is (almost) always better than
>> failing.
>
> I disagree. Python must provide a truly monotonic clock, even it is
> not truly monotonic by default.

Why?

> If we want to provide a convenient "monotonic or falllback to system
> time", there are different options: add a dedicated function (e.g.
> time.try_monotonic()) or add a parameter to enable or disable the
> fallback. Between these two options, I prefer the parameter because it
> avoids the creation of a new function. Even if I now agree that a
> truly monotonic clock is required, I also agree with Guido: the most
> common case is to use a fallback.
>
> I suggest this API: time.monotonic(fallback=True).

But this goes against the idea of "A keyword argument that gets passed
as a constant in the caller is usually poor API."

And it would encourage the creation of trivial lambdas just to call
the timer with this flag, since a universal API is a timer function
that takes no arguments.

> Another option is to not provide a "monotonic or falllback to system
> time" clock and write an explicit try/except where you need a
> fallback. I don't like this solution because time.monotonic() might
> fail at runtime with OSError, and so catching AttributeError is not
> enough...

Check out Cameron Simpson's proposal earlier in this thread. If we
really are going to have a variety of functions and fallback options,
his approach may be the best.

>> Then we can design a separate inquiry API (doesn't have to be complex
>> as long as it's extensible -- a dict or object with a few predefined
>> keys or attributes sounds good enough) for apps that want to know more
>> about how the timer they're using is actually implemented.
>
> Sometimes, it does matter which exact OS clock is used.

When is that?

> The QElapsedTime class of the Qt libraries has an attribute (an enum)
> describing which clock is used. We can give the name of the function
> and the resolution. In some specific cases, we are able to read the
> accuracy (ex: GetTickCount). The accuracy may change at runtime (at
> least for GetTickCount).

That sounds like a fine API.

> Should I include such new function in the PEP, or can it be discussed
> (later) in a separated thread?

FWIW, I prefer a single thread devoted to all aspects of this PEP.

> --
>
> I don't want to only provide time.monotonic() which would fallback if
> no monotonic clock is available, because we would be unable to tell if
> the clock is monotonic or not.

There could be a separate inquiry API.

> See the implementation in the PEP: the
> "try monotonic or fallback" function may use a different clock at each
> call, and so may be monotonic at startup and then becomes
> non-monotonic.

No, it should never switch implementations once it has chosen. That's
just asking for trouble. If it can become non-monotonic it just isn't
monotonic. Again, C

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

On Fri, Mar 30, 2012 at 4:44 PM, Cameron Simpson  wrote:
[Lots of good stuff]

Maybe you and Victor should try to merge your proposals off-line and
then get back with a new proposal here.

> Make the epoch available in the clock wrapper as a property. At least
> then there's a mechanism for reconciling things. Don't try to mandate
> something that possibly can't be mandated.

Ah, but if a clock drifts, the epoch may too -- and we may never know
it. I like knowing all sorts of things about a clock, but I'm not sure
that for clocks other than time.time() I'd ever want to know the epoch
-- ISTM that the only thing I could do with that information would be
shooting myself in the foot. If you really want the epoch, compute it
yourself by bracketing a timer() call in two time() calls, or vice
versa (not sure which is better :-).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Steven D'Aprano]

Guido van Rossum wrote:


But for the other, I'm still at a loss, and that name is the most
important one. We can't call it steady because it isn't always.
highres or hires sounds awkward; try_monotonic or try_steady are even
more awkward. I looked in an online thesaurus and here's a list of
what it gave:


"hires" is a real English word, the present tense verb for engaging the 
service or labour of someone or something in return for payment, as in "he 
hires a gardener to mow the lawn". Can we please eliminate it from 
consideration? It is driving me slowly crazy every time I see it used as an 
abbreviation for high resolution.




Big Ben, alarm, chroniker, chronograph, chronometer, digital watch,
hourglass, metronome, pendulum, stopwatch, sundial, tattler,
tick-tock, ticker, timekeeper, timemarker, timepiece, timer, turnip,
watch

I wonder if something with tick would work? (Even though it returns a float. :-)

If all else fails, I'd go with turnip.


I can't tell if you are being serious or not.

For the record, "turnip" in this sense is archaic slang for a thick pocket 
watch.



--
Steven

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Victor Stinner]

2012/3/31 Guido van Rossum :
> Given the amount of disagreement I sense, I think we'll need to wait
> for more people to chime in.

I hope that the PEP now gives enough data to help to choose the best API.

>> Hum, I read that you can expect a drift between QPC and GetTickCount.
>> I don't have exact numbers. There are also issues with power-saving.
>> When the CPU frequency changes, the frequency of the TSC is also
>> impacted on some processors (others provide a TSC at a fixed
>> frequency).
>
> And how does that impact teh various use cases? (There still isn't
> enough about use cases in the PEP. Thank you very much though for
> adding all those details about the various platform clocks.)

I didn't find a good source, but it looks like QueryPerformanceCounter
behaves badly on system suspend/resume, whereas GetTickCount is
reliable and has a well defined behaviour: "The elapsed time retrieved
by GetTickCount or GetTickCount64 includes time the system spends in
sleep or hibernation." So QPC is not really monotonic nor steady. It
may be even worse than the system clock (especially in virtual
machines).

Impact: A timeout may be 42 seconds shorter than the requested
duration if is uses QPC. For a scheduler, a task would be scheduled at
the right moment.

>> I prefer use cases:
>>
>>  * time.highres() should be used for profiling and benchmarking
>>  * time.monotonic() should be used for a scheduler and timeout
>
> So if I want to schedule something a week in the future, what should I
> use? monotonic() or time()? And can you explain the reason for your
> answer? What about an hour or a day? Or a month or a year?

You should use time.monotonic() because time.monotonic() has a steady
rate and so is reliable for long duration (at least more reliable than
time.highres()).

In practice, time.monotonic() and time.highres() are only different on
Windows. If I understood correctly, the Windows kernel uses something
like GetTickCount whch has an accuracy of 1 ms in the best case. So
Python only needs a clock with a similar accuracy for timeout and
schedulers.

time.highres() (QPC) rate is only steady during a short duration: it
is not an issue for a benchmark because you usually rerun the same
test at least 3 times and keep the minimum. It looks like QPC bugs are
only unexpected forward jumps (no backward jump), so taking the
minimum would workaround these issues. The hibernation issue should
not affect benchmarking/profiling.

>> time.monotonic() rate should be as steady as possible, its stability
>> is more important than its accuracy. time.highres() should provide the
>> clock with the best accuracy. Both clocks may have an undefined
>> starting point.
>
> So what kind of drift is acceptable for each?

I don't know.

>> I disagree. Python must provide a truly monotonic clock, even it is
>> not truly monotonic by default.
>
> Why?

See Zooko Wilcox-O'Hearn's email:
http://mail.python.org/pipermail/python-dev/2012-March/118147.html

>> I suggest this API: time.monotonic(fallback=True).
>
> But this goes against the idea of "A keyword argument that gets passed
> as a constant in the caller is usually poor API."

Right :-) After changing (again) the PEP, I realized that it
reintroduces the time.monotonic(fallback=False) raising
NotImplementedError case and I really don't like this exception here.

> And it would encourage the creation of trivial lambdas just to call
> the timer with this flag, since a universal API is a timer function
> that takes no arguments.

My bet is that fallback=True is an uncommon use case.

>> Sometimes, it does matter which exact OS clock is used.
>
> When is that?

If you need to know more properties of the clock,. For example, Python
may be unable to give the accuracy of the clock, but you may get it
differently if you know which clock function is used.

>> Should I include such new function in the PEP, or can it be discussed
>> (later) in a separated thread?
>
> FWIW, I prefer a single thread devoted to all aspects of this PEP.

Let's propose an API:

time.clock_info() ->
{'time': {'function': 'gettimeofday', 'resolution': 1e-6, 'monotonic': False},
 'monotonic': {'function': 'clock_gettime(CLOCK_MONOTONIC)',
'resolution': 1e-9, 'monotonic': True},
 'highres': {'function': 'clock_gettime(CLOCK_MONOTONIC)',
'resolution': 1e-9, 'monotonic': True}}

or

time.clock_info('time') -> {'function': 'gettimeofday', 'resolution':
1e-6, 'monotonic': False}
time.clock_info('monotonic') -> {'function':
'clock_gettime(CLOCK_MONOTONIC)', 'resolution': 1e-9, 'monotonic':
True}
time.clock_info( 'highres') -> {'function':
'clock_gettime(CLOCK_MONOTONIC)', 'resolution': 1e-9, 'monotonic':
True}

or

time.clock_info(time.time) -> {'function': 'gettimeofday',
'resolution': 1e-6, 'monotonic': False}
time.clock_info(time.monotonic) -> {'function':
'clock_gettime(CLOCK_MONOTONIC)', 'resolution': 1e-9, 'monotonic':
True}
time.clock_info( time.highres) -> {'function':
'clock_gettime(CLOCK_MONOTONIC)', 'resolution'

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Glyph]

On Mar 30, 2012, at 8:51 PM, Victor Stinner wrote:

> time.highres() (QPC) rate is only steady during a short duration

QPC is not even necessarily steady for a short duration, due to BIOS bugs, 
unless the code running your timer is bound to a single CPU core.  
 mentions 
SetThreadAffinityMask for this reason, despite the fact that it is usually 
steady for longer than that.

-glyph

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Victor Stinner]

> There seem to be a few competing features for clocks that people want:
>
>  - monotonic - never going backward at all
>  - high resolution

These features look to be exclusive on Windows. On other platforms, it
looks like monotonic clocks are always the most accurate clocks. So I
don't think that we need to be able to combine these two "flags".

>  - no steps

You mean "not adjusted by NTP"? Except CLOCK_MONOTONIC on Linux, no
monotonic clock is adjusted by NTP. On Linux, there is
CLOCK_MONOTONIC_RAW, but it is only available on recent Linux kernel
(2.6.28).

Do you think that it is important to be able to refuse a monotonic
clock adjusted by NTP? What would be the use case of such truly steady
clock?

--

The PEP 418 tries to expose various monotonic clocks in Python with a
simple API. If we fail to agree how to expose these clocks, e.g.
because there are major differences between these clocks, another
solution is to only expose low-level function. This is already what we
do with the os module, and there is the shutil module (and others) for
a higher level API. I already added new "low-level" functions to the
time module: time.clock_gettime() and time.clock_getres().

> Of course you can provide some convenient-with-fallback function that
> will let people do this in one hit without the need for "T", but it
> should not be the base facility offered. The base should let people
> request their feature set and inspect what is supplied.

The purpose of such function is to fix programs written with Python <
3.3 and using time.time() whereas a monotonic would be the right clock
(e.g. implement a timeout).

Another concern is to write portable code. Python should help
developers to write portable code, and time.monotonic(fallback=False)
is not always available (and may fail).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

On Fri, Mar 30, 2012 at 5:51 PM, Victor Stinner
 wrote:
> 2012/3/31 Guido van Rossum :
>> Given the amount of disagreement I sense, I think we'll need to wait
>> for more people to chime in.
>
> I hope that the PEP now gives enough data to help to choose the best API.

Lots of data, but not enough motivation, not enough about the use cases.

>>> Hum, I read that you can expect a drift between QPC and GetTickCount.
>>> I don't have exact numbers. There are also issues with power-saving.
>>> When the CPU frequency changes, the frequency of the TSC is also
>>> impacted on some processors (others provide a TSC at a fixed
>>> frequency).
>>
>> And how does that impact teh various use cases? (There still isn't
>> enough about use cases in the PEP. Thank you very much though for
>> adding all those details about the various platform clocks.)
>
> I didn't find a good source, but it looks like QueryPerformanceCounter
> behaves badly on system suspend/resume, whereas GetTickCount is
> reliable and has a well defined behaviour: "The elapsed time retrieved
> by GetTickCount or GetTickCount64 includes time the system spends in
> sleep or hibernation." So QPC is not really monotonic nor steady. It
> may be even worse than the system clock (especially in virtual
> machines).

I think you are in serious danger of overspecifying the problem. No
clock can fulfill all requirements. (That's why there are so many to
choose from of course!)

> Impact: A timeout may be 42 seconds shorter than the requested
> duration if is uses QPC. For a scheduler, a task would be scheduled at
> the right moment.

I don't understand this paragraph. And why is it always exactly a loss
of 42 seconds?

>>> I prefer use cases:
>>>
>>>  * time.highres() should be used for profiling and benchmarking
>>>  * time.monotonic() should be used for a scheduler and timeout
>>
>> So if I want to schedule something a week in the future, what should I
>> use? monotonic() or time()? And can you explain the reason for your
>> answer? What about an hour or a day? Or a month or a year?
>
> You should use time.monotonic() because time.monotonic() has a steady
> rate and so is reliable for long duration (at least more reliable than
> time.highres()).

Ah, but have you *thought* about the use case? *Why* would I want to
schedule something that far in the future? Maybe it's a recurring
birthday? Surely time() is better for that. (Etc. Think!) My question
is trying to tease out the "meaning" of the various clocks that might
exist.

> In practice, time.monotonic() and time.highres() are only different on
> Windows. If I understood correctly, the Windows kernel uses something
> like GetTickCount whch has an accuracy of 1 ms in the best case. So
> Python only needs a clock with a similar accuracy for timeout and
> schedulers.

So is it worth having two functions that are only different on
Windows? ISTM that the average non-Windows user will have a 50% chance
of picking the wrong timer from a portability perspective.

> time.highres() (QPC) rate is only steady during a short duration: it
> is not an issue for a benchmark because you usually rerun the same
> test at least 3 times and keep the minimum. It looks like QPC bugs are
> only unexpected forward jumps (no backward jump), so taking the
> minimum would workaround these issues. The hibernation issue should
> not affect benchmarking/profiling.

Now you're talking: indeed the application should work around the
limitations (if there are any) using end-to-end means, since only the
application developer knows what matters to them.

>>> time.monotonic() rate should be as steady as possible, its stability
>>> is more important than its accuracy. time.highres() should provide the
>>> clock with the best accuracy. Both clocks may have an undefined
>>> starting point.
>>
>> So what kind of drift is acceptable for each?
>
> I don't know.

Ok, let's just assume that we can't control drift so we'll have to let
the app deal with it.

>>> I disagree. Python must provide a truly monotonic clock, even it is
>>> not truly monotonic by default.
>>
>> Why?
>
> See Zooko Wilcox-O'Hearn's email:
> http://mail.python.org/pipermail/python-dev/2012-March/118147.html

He explains the basic difference between the two types of clocks, and
that's important. He doesn't say anything about a strict requirement
for monotonicity or steadiness. This is why I still balk at
"monotonic" for the name -- I don't think that monotonicity is the
most important property. But I don't know how the put the desired
requirement in words; "steady" seems to come closer for sure.

>>> I suggest this API: time.monotonic(fallback=True).
>>
>> But this goes against the idea of "A keyword argument that gets passed
>> as a constant in the caller is usually poor API."
>
> Right :-) After changing (again) the PEP, I realized that it
> reintroduces the time.monotonic(fallback=False) raising
> NotImplementedError case and I really don't like this exception here.

Me neither.

>> 

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Guido van Rossum]

On Fri, Mar 30, 2012 at 6:24 PM, Victor Stinner
 wrote:
>> There seem to be a few competing features for clocks that people want:
>>
>>  - monotonic - never going backward at all
>>  - high resolution
>
> These features look to be exclusive on Windows. On other platforms, it
> looks like monotonic clocks are always the most accurate clocks. So I
> don't think that we need to be able to combine these two "flags".
>
>>  - no steps
>
> You mean "not adjusted by NTP"? Except CLOCK_MONOTONIC on Linux, no
> monotonic clock is adjusted by NTP. On Linux, there is
> CLOCK_MONOTONIC_RAW, but it is only available on recent Linux kernel
> (2.6.28).
>
> Do you think that it is important to be able to refuse a monotonic
> clock adjusted by NTP? What would be the use case of such truly steady
> clock?

That depends on what NTP can do to the clock. If NTP makes the clock
tick *slightly* faster or slower in order to gradually adjust the wall
clock, that's fine. If NTP can make it jump wildly forward or even
backward, it's no better than time.time(), and we know why (for some
purposes) we don't want that.

> --
>
> The PEP 418 tries to expose various monotonic clocks in Python with a
> simple API. If we fail to agree how to expose these clocks, e.g.
> because there are major differences between these clocks, another
> solution is to only expose low-level function. This is already what we
> do with the os module, and there is the shutil module (and others) for
> a higher level API. I already added new "low-level" functions to the
> time module: time.clock_gettime() and time.clock_getres().
>
>> Of course you can provide some convenient-with-fallback function that
>> will let people do this in one hit without the need for "T", but it
>> should not be the base facility offered. The base should let people
>> request their feature set and inspect what is supplied.
>
> The purpose of such function is to fix programs written with Python <
> 3.3 and using time.time() whereas a monotonic would be the right clock
> (e.g. implement a timeout).
>
> Another concern is to write portable code. Python should help
> developers to write portable code, and time.monotonic(fallback=False)
> is not always available (and may fail).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Glyph]

On Mar 30, 2012, at 9:32 PM, Guido van Rossum wrote:

>>>  - no steps
>> 
>> You mean "not adjusted by NTP"? Except CLOCK_MONOTONIC on Linux, no
>> monotonic clock is adjusted by NTP. On Linux, there is
>> CLOCK_MONOTONIC_RAW, but it is only available on recent Linux kernel
>> (2.6.28).
>> 
>> Do you think that it is important to be able to refuse a monotonic
>> clock adjusted by NTP? What would be the use case of such truly steady
>> clock?
> 
> That depends on what NTP can do to the clock. If NTP makes the clock
> tick *slightly* faster or slower in order to gradually adjust the wall
> clock, that's fine. If NTP can make it jump wildly forward or even
> backward, it's no better than time.time(), and we know why (for some
> purposes) we don't want that.

"no steps" means something very specific.  It does not mean "not adjusted by 
NTP".

In NTP, changing the clock frequency to be slightly faster or slower is called 
"slewing" (which is done with adjtime()).  Jumping by a large amount in a 
single discrete step is called "stepping" (which is done with settimeofday()).  
This is sort-of explained by .

I think I'm agreeing with Guido here when I say that, personally, my 
understanding is that slewing is generally desirable (i.e. we should use 
CLOCK_MONOTONIC, not CLOCK_MONOTONIC_RAW) if one wishes to measure "real" time 
(and not a time-like object like CPU cycles).  This is because the clock on the 
other end of the NTP connection from you is probably better at keeping time: 
hopefully that thirty five thousand dollars of Cesium timekeeping goodness is 
doing something better than your PC's $3 quartz crystal, after all.

So, slew tends to correct for minor defects in your local timekeeping 
mechanism, and will compensate for its tendency to go too fast or too slow.  By 
contrast, stepping only happens if your local clock is just set incorrectly and 
the re-sync delta has more to do with administrative error or failed batteries 
than differences in timekeeping accuracy.

-glyph

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Victor Stinner]

>> Impact: A timeout may be 42 seconds shorter than the requested
>> duration if is uses QPC. For a scheduler, a task would be scheduled at
>> the right moment.
>
> I don't understand this paragraph. And why is it always exactly a loss
> of 42 seconds?

Sorry, it's late here, I'm too tired. A jump of 42 seconds is one of
QPC known bug (it was a bug in VirtualBox and it is now fixed).

> So is it worth having two functions that are only different on
> Windows? ISTM that the average non-Windows user will have a 50% chance
> of picking the wrong timer from a portability perspective.

Can we solve this by documenting correctly time.highres() issues (it's
not really monotonic) and when you need to use time.monotonic() or
time.highres()?

Well, there is already time.clock() that uses QPC on Windows, but
time.highres() and time.clock() give different results for an idle
process on UNIX. pybench, timeit and other programs have their own
heuristic getting the most accurate clock depending on the OS and on
which clocks are available. time.highres() is a portable function for
these programs.

>> See Zooko Wilcox-O'Hearn's email:
>> http://mail.python.org/pipermail/python-dev/2012-March/118147.html
>
> He explains the basic difference between the two types of clocks, and
> that's important. He doesn't say anything about a strict requirement
> for monotonicity or steadiness. This is why I still balk at
> "monotonic" for the name -- I don't think that monotonicity is the
> most important property. But I don't know how the put the desired
> requirement in words; "steady" seems to come closer for sure.

If we provide a way to check if the monotonic clock is monotonic (or
not), I agree to drop the flag from time.monotonic(fallback=True) and
always fallback. I was never a fan of the "truly monotonic clock".

time.clock_info('monotonic')['is_monotonic'] is a good candidate to
store this information.

"time.clock_info('monotonic')['is_monotonic']" looks like a bug: why
would I check if a *monotonic* clock is *monotonic*?
time.clock_info('steady')['monotonic'] looks more natural, but I don't
remember why we should not use the term "steady".

>> Well, the PEP mentions something like that in the "One function, no
>> flag" section.
>
> But you don't seem to like it. (In general the alternatives section
> could do with reasons for rejection or at least pros and cons for each
> alternative.)

I was concerned by the fact that time.monotonic() may become
not-monotonic between two calls, but I changed my mind. I agree that
it should not occur in practice. So always fallback and add a way to
know if the monotonic clock is monotonic now sounds like the best API
to me.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Victor Stinner]

> (...)
>  By contrast, stepping only happens if your local clock is just set
> incorrectly and the re-sync delta has more to do with administrative error
> or failed batteries than differences in timekeeping accuracy.

Are you talking about CLOCK_REALTIME or CLOCK_MONOTONIC?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Glyph]

On Mar 30, 2012, at 10:17 PM, Victor Stinner wrote:

>> (...)
>>  By contrast, stepping only happens if your local clock is just set
>> incorrectly and the re-sync delta has more to do with administrative error
>> or failed batteries than differences in timekeeping accuracy.
> 
> Are you talking about CLOCK_REALTIME or CLOCK_MONOTONIC?

My understanding is:

CLOCK_REALTIME is both stepped and slewed.

CLOCK_MONOTONIC is slewed, but not stepped.

CLOCK_MONOTONIC_RAW is neither slewed nor stepped.

-glyph

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

[Glyph]

On Mar 30, 2012, at 10:25 PM, Glyph wrote:

> 
> On Mar 30, 2012, at 10:17 PM, Victor Stinner wrote:
> 
>>> (...)
>>> By contrast, stepping only happens if your local clock is just set
>>> incorrectly and the re-sync delta has more to do with administrative error
>>> or failed batteries than differences in timekeeping accuracy.
>> 
>> Are you talking about CLOCK_REALTIME or CLOCK_MONOTONIC?
> 
> My understanding is:
> 
> CLOCK_REALTIME is both stepped and slewed.
> 
> CLOCK_MONOTONIC is slewed, but not stepped.
> 
> CLOCK_MONOTONIC_RAW is neither slewed nor stepped.

Sorry, I realize I should cite my source.

This mailing list post talks about all three together: 


Although the documentation one can find by searching around the web is really 
bad.  It looks like many of these time features were introduced, to Linux at 
least, with no documentation.

-glyph
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Terry Reedy]

On 3/30/2012 8:26 PM, Steven D'Aprano wrote:


"hires" is a real English word, the present tense verb for engaging the
service or labour of someone or something in return for payment, as in
"he hires a gardener to mow the lawn". Can we please eliminate it from
consideration


I agree. Heavy cognitive dissonance.
'Hires' is also a very famous brand of root beer.
Hi-res *really* needs the hyphen (or underscore equivalent).

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14065: Added cyclic GC support to ET.Element

[Eli Bendersky]

On Fri, Mar 30, 2012 at 21:30, Benjamin Peterson wrote:

>  > +def test_cyclic_gc(self):
> > +class ShowGC:
> > +def __init__(self, flaglist):
> > +self.flaglist = flaglist
> > +def __del__(self):
> > +self.flaglist.append(1)
>
>
> I think a nicer way to check for cyclic collection is to take a
> weakref to an object, call the GC, then check to make sure the weakref
> is broken.
>
> > +
> > +# Test the shortest cycle: lst->element->lst
> > +fl = []
> > +lst = [ShowGC(fl)]
> > +lst.append(ET.Element('joe', attr=lst))
> > +del lst
> > +gc.collect()
>
> support.gc_collect() is preferable
>
> > +self.assertEqual(fl, [1])
> > +
> > +# A longer cycle: lst->e->e2->lst
> > +fl = []
> > +e = ET.Element('joe')
> > +lst = [ShowGC(fl), e]
> > +e2 = ET.SubElement(e, 'foo', attr=lst)
> > +del lst, e, e2
> > +gc.collect()
> > +self.assertEqual(fl, [1])
>
>
Thanks for the insights, Benjamin. I'll explore these alternatives and will
submit a fix.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Lennart Regebro]

On Sat, Mar 31, 2012 at 02:26, Steven D'Aprano  wrote:
> Guido van Rossum wrote:
>> If all else fails, I'd go with turnip.
>
> I can't tell if you are being serious or not.
>
> For the record, "turnip" in this sense is archaic slang for a thick pocket
> watch.

If I understand this correctly, the most common use for this function
is when to time things. It will give you the best source available for
timers, but it doesn't guarantee that it is steady or monotonic or
high resolution or anything. It is also not the time, as it's not
reliable as a wall-clock.

So, how about time.timer()?

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime module and pytz with dateutil

[Lennart Regebro]

On Fri, Mar 30, 2012 at 12:38, Serhiy Storchaka  wrote:
> I don't understand why Python may not include the pytz. The Olson tz
> database is not part of pytz.

Yes it is.

> Python can depend on a system tz database

That works on Unix, but not on Windows, where there is no Olsen database.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com